Tesla will support Bluetooth controllers in next update

By Nuno Cristovao

Tesla’s early FSD betas included driving visualizations that used simple wireframe boxes to represent vehicles, and lane markings were made up of individual dots.

The visualizations gave an excellent overview of some of the information provided to the autopilot, but even then only a fraction of the information used by the autopilot was actually displayed on the screen.

In effect, Autopilot creates a 3D representation of every object it tracks. Each detected object then has different attributes. For example, a detected vehicle will have attributes for its speed, distance, type of vehicle, predicted path, etc.

Visualizations of Tesla in early FSD betas

Car visualizations are an important part of FSD because they help us better understand what the car is able to see and react to. However, the information and visualization that the autopilot needs is radically different from what humans need.

For Tesla to achieve FSD, they essentially need to be able to create a very accurate video game that represents the real world, in real time.

The car wants to access as much information as possible about each object. Meanwhile, humans want a visualization that closely resembles the real world.

With the introduction of FSD Betas 9.x, Tesla released a more human-like consumables visualization. One that included proper 3D models of general vehicle types, road pylons, and solid lane markers.

Road edges and lane markings are more distinct lines, 3D models have working brake lights, and other objects such as speed bumps, bike lanes, and crosswalks are depicted using visualizations that correspond to the real world.

For Tesla to achieve FSD, they essentially need to be able to create a very accurate video game that represents the real world, in real time.

However, something that was missing were the visualizations, that is dynamic vehicle sizing. The 3D vehicle models used by Tesla have a static size. When the vehicle sees a bus, it calculates its length, width, and height in addition to a bunch of other measurements. However, the 3D model that appears on the screen is of a predefined size, which means that it does not actually correspond to what the vehicle saw.

That’s why you may have seen a tractor-trailer moving forward and backward or you may have seen two vehicles on top of each other. One signifies the start of the vehicle and since the vehicle is so much longer than the model, it adds another vehicle at the end to signify the end of the vehicle.

Scalable vehicle models

However, in the latest 10.10.2 FSD update, we now see individual vehicle models scaled to Tesla so that they represent the calculated size of the surrounding vehicles. Contextually, this could be useful to better understand the situation of our car in the world.

In version 10.10.2, car shrinks or stretches 3D vehicle models in each dimension so that the 3D model matches the calculated dimensions for each vehicle. This is particularly evident in longer vehicles such as buses, trucks and tractor-trailers, where vehicle lengths are more likely to vary, but you can also see this scaled to other vehicle models such than very small cars.

In this example below, you will see that Tesla is now able to accurately represent buses of different sizes. Tesla only has one model for a full-length bus, but in this case Tesla has detected that the length of one of the buses is considerably shorter than the vehicle model, so it has chosen to reduce the length of the bus to the length that the autopilot had calculated. In the image below you can see how the same bus model is displayed in two different sizes.

Tesla can now accurately render vehicles of different sizes

It is important to realize the difference between visualizations and what the autopilot uses. The visualizations are there simply to help us better understand what the autopilot can see. The FSD computer itself has always taken note of the size of surrounding objects and various other data points. The trajectory, approach speed, proximity, etc. are also part of it, but this update helps Tesla get visualizations that provide a more accurate representation of reality.

It’s not just buses and trucks that are being upsized or downsized. Tesla has downsized a bobcat into a vehicle that’s about half the length of its normal sedan model.

Tesla can now accurately render vehicles of different sizes

The models are adjusted in three dimensions. We have seen some truck models that have been stretched to become taller while reducing their length. It’s not perfect because you’re scaling all of the truck’s components at the same rate, but it does produce a much more accurate representation of the vehicle and how much space it takes up.

Vehicles are resized in three dimensions to better match the length, width and height of the vehicle

Tesla has come a long way in a short time with the number of objects they are able to detect, but obviously when you compare the environment the car sees today there is still a lot missing.

In the short term, we would like to see more visualized objects. Objects commonly encountered while driving, such as trailers and gates.

We’d also like to see other common objects added, such as additional traffic light configurations, crosswalks, mailboxes, and maybe even a generic object that lets us know the vehicle is seeing something that he needs to maneuver, but he may not know exactly what it is.

In the future, I think we’ll see Tesla display a richer, more complete 3D environment that will display static and moving objects that are important for the vehicle to avoid, objects such as barriers, buildings, trees, sidewalks, etc. Today, Tesla is one step closer to achieving that goal.

Be sure to check out our full list of all visualizations of Tesla’s latest FSD beta.

Follow us on Twitter and Facebook to follow our latest news