Tesla recently published a highly detailed patent that explains the inner workings of its vision-based occupancy network. The patent is titled Artificial Intelligence Modeling Techniques For Vision-based Occupancy Determination and was officially published on March 12, 2026.
Credited to a team of engineers that includes Ashok Elluswamy, the document provides a deep dive into how Tesla uses artificial intelligence to understand the physical world without relying on radar or LiDAR.
Understanding the Voxel Grid
The core of Tesla's occupancy network revolves around voxels. A voxel is essentially a three-dimensional pixel that represents a specific point within a volumetric grid surrounding the vehicle. To build this grid, the artificial intelligence model ingests image data from the eight exterior cameras of the vehicle. The system then executes the model to predict whether each voxel is occupied by an object having mass.
Because labeling millions of 3D data points manually would be impossibly time-consuming, the patent notes that Tesla relies heavily on unsupervised training methods to train these models at scale.
Variable Resolution and Sub-Voxels
One of the most interesting details revealed in the patent is how Tesla manages computing power by dynamically adjusting the size of these voxels. The default size for a voxel is 33 centimeters on each vertex. This size is generally acceptable for objects located far away or outside of the immediate driving surface.
However, FSD can reduce the voxel size to 10 centimeters for areas that are occupied and within a threshold distance from the vehicle. This allows for much higher granularity where it matters most. The neural networks can even predict partial occupancy by dividing occupied spaces into smaller sub-voxels.
This allows FSD to identify the exact shape of a curved object accurately. The analytics server can also use trilinear interpolation to estimate the occupancy status of any specific point within a voxel.
Temporal Fusion and 3D Semantics
Tesla’s AI does not just look at static frames in isolation. The artificial intelligence model uses a transformer to aggregate the 2D image data into a unified 3D representation. It then fuses this current 3D space with representations from previous timestamps. This combination of spatial and temporal data allows the network to calculate occupancy flow. Occupancy flow indicates the exact velocity of moving voxels.
Finally, FSD applies 3D semantic data to identify what the object actually is. It can distinguish whether a group of occupied voxels represents a moving car, a static building, or a street curb. The system is designed to prioritize certain semantic shapes. For instance, a moving vehicle near the ego will be analyzed much more thoroughly than a static building located far off the roadway.
Powering Vehicles and Optimus
All of this data is continuously aggregated into a queryable dataset. FSD can constantly query this dataset to receive occupancy statuses and make real-time navigational decisions. Additionally, this same dataset is used to generate the 3D environmental map displayed on the user interface inside the vehicle.
While the patent is heavily focused on autonomous vehicles, it confirms that the underlying technology is highly adaptable. The document specifically notes that this exact same vision-based occupancy network can be utilized by a general-purpose, bipedal humanoid robot to navigate various terrains.
If you enjoyed this article, we recommend reading our full series on Tesla patents related to FSD:
