A big part of the presentation so far was an engineer describing the huge difficulty of stitching together the multiple cameras into one vector space that can be the input to the network, instead of treating each camera individually.

Seems like the biggest problem was each pixel from a camera does not tell you how far away it is, so even if you know the camera is X feet off the grounding pointing at Y degrees, you don't know if there's a wall in front of the camera or not. So you can easily reconstruct a 2D space from all the cameras just by knowing their positions, but you can't simply translate that back to 3D.

I could barely follow the solution to this problem, but it seems to approximate the distance somewhat okay... but it required yet another universe of features in the neural net pipeline just to achieve it.

The funniest part is... isn't this exactly what LIDAR is amazing at?? Or what about having each "camera" actually be two cameras that can achieve depth via parallax?

While taking two sequential images from the same camera in can provide this depth information also. But simple use of stereo cameras can solve a magnitude of problems (standing still and low parallax motions). Traditional stereo and even machine-learning based methods have have great success and accuracy for many years and could easily be an alternative to LIDAR also.

I really don't know why this isn't leveraged more (maybe it is and I am unaware?).

Tesla has multiple cameras looking out the front, with some offset. I believe that in past talks they did talk about using that parallax to achieve better depth.

He talks about the approach here. They actually used radar and LIDAR to train the depth sensing neural network: https://t.co/osmEEgkgtL?amp=1

That was basically what I was thinking as well. There's a reason that Mobileye's camera only solutions tend to have a lot more cameras than Tesla does. Stereoscopic vision would be a big help, and tesla only has it in some directions and with relatively low resolution.

Yes. This is exactly what LIDAR is for. This is the reason every single team in the 2007 DARPA Urban challenge that finished the race was equipped with a Velodyne laser. We knew it was a critical enabling technology 14 years ago, and it baffles me Tesla is eschewing it. Tesla's lack of this technology is the main reason I feel they will never achieve what they claim to without a huge breakthrough in AI.

Idk.. animals seem to do fine without a lidar

If Tesla switches to stereoscopic vision with fully articulated cameras that can move independently of the vehicle (not fully independent, but limited 3 degrees of motion), and then they manage to integrate the output into something meaningful then maybe it would be comparable.

Animals tend to use a combination of four tools to detect distance. Stereo vision, parallax, focus detection and scene context. Independent movement is not required for any of these (though it can help generate parallax.) Humans (and other animals) can learn to do pretty well with one eye and no head movement given the appropriate training regime.

It makes the software problem harder but it certainly is possible.

Maybe it's more like spider eyes that see a blended vectorspace of reality with less movement?

They already have multiple cameras; it's not clear why articulation of cameras would be required as they do not have foveas that require such articulation; they already do integrate the output of each camera into a view of the vehicle's surroundings.

> Idk.. animals seem to do fine without a lidar

Animals also have (non)-artificial general intelligence. That doesn't mean Tesla's making a reasonable design choice if they build something in a way that relies on having that kind of capability.

The way I imagine it, Elon made Andrej put in that slide about "We are effectively building a 'synthetic animal' from the ground up." It vaguely hints at a small-scale AGI. A little bit of Andrej's soul dies when he has to says those ridiculous words, but he views this as the price he has to pay to have his dream job. Anyhow, that's how it would have played out if i had Andrej's job.

I'm sure a rhino would appreciate one. They have bad depth vision due to their horn being in the way.

Stick an animal, even a human, into a room of non-realistic objects, misleading proportions etc., and see how well it orients itself: it will be very bad. That is because sight and orientation rely heavily on intelligence: we can tell how far something is because we recognize the object and can tell whether it is small and close or large and far away. Even if one object is misleading, we look at other objects around and can usually tell if we're looking at a toy car on a real road etc.

But betting your FSD on doing object reconition and physical intuition seems like a bad bet.

Are humans animals?

Animals do just fine with more complex navigation challenges than driving a car. While we can 'mimic' various animal capabilities, I don't believe we are anywhere close to AGI with either a dog's level of intelligence, motor control or sensory integration.

The theory of mind problems involved in driving are huge and are primarily why I don't think you could teach an non-human animal to drive on streets / in traffic.

But depth perception isn't really involved there so those theory of mind issues don't really have any bearing on the need for lidar for self driving cars. Clearly creating world models of sufficient accuracy for driving scale navigation challenges is possible (but potentially pretty hard) with just vision data.

Have you imagined building a birds nest using a helicopter or similar flying machine? Seems to me trickier than driving a car.

...with millions (billions?) of years of evolution.

Do you know how animals do it?

What was their reasoning for not using Lidar?

I think originally it was because of the cost of the equipment. Elon and team saw how expensive lidar was and thought they needed to be able to solve the problem without it. However lidar has dropped drastically in price, so at this point I think they aren't using it just for ego reasons (i.e., we claimed it was possible to get self driving without lidar in the past, so we have to continue down that path)

Lidar might have dropped in price, but please show me the Lidar that can produce that quality output, is incredibly cheap, easy to manufacture and integrate into the car and available millions of time by next year.

And it would be great if it something that almost never broke as things that move a lot tend to. As far as I know static lidar are nowhere near as good.

Then if the difference in quality is not actually that great, doing the development effort once will scale to millions of cars the next few years. So we are literally talking many, many years of millions of lidars produced and integrated (and not replacing other sensors) to maybe have a slightly better outcome.

However that depends on your sensor fusion and how the lidar handles weather conditions and other complexities.

But I am sure its all ego.

Just because not needing lidar would be great for Tesla, doesn't make it real. That's wishful thinking. Tesla is not entitled to a well working lidar-free solution just because they need to sell cars right now.

Can you point me to literally any evidence what so ever that the results fake? Is the FSD Beta regularly crashing into things it put at the wrong range?

Tesla has cars with Lidar on the road to check the models as well.

And who says anything about entitled, they have spent a huge amount of effort and therefore money into a solution that they think will be better.

Nobody with Lidar has delivered either so what is better is still an open question. Tesla has picked their strategy and working on it. Others do the same.

My point is simply, not that Tesla has the perfect solution, rather the idea that Tesla is operating based on 'ego' is idiotic. Elon Musk and his companies have repeatably shown that they are very willing, comically so, to throw away work and go back to the beginning if they have encountered a problem. E

lon is well known for not falling into the sunk cost fallacy. Consider the manufacturing of Model 3. Consider Starship being planned to be built from carbon fiber, and switching to stainless steel.

Do you really believe they know lidar would be better and they not changing because of what Musk said 4 years ago? That is literally the opposite of how his company have operated.

> Is the FSD Beta regularly crashing into things it put at the wrong range?

As a matter of fact, yes: self-driving Teslas have been crashing into emergency vehicles so often, they are being investigated[1]

1. https://news.ycombinator.com/item?id=28197355

I don't think that's a range thing though. It's confusing the vehicles with things like road signs or puddles that you can drive over/past.

That's exactly a range thing: LIDAR could tell you that there is an obstacle in front of you, it couldn't mistake a 3D object for a 2D picture on the road.

Like you say, FSD beta is new, so, no. The autopilot software has killed people though.

The sunk cost fallacy is stronger when it's not just R&D but thousands of people who paid 10k$ on the promise that their car had all the hardware required... Cars that shipped without a lidar.

And I believe waymo uses lidar and has, as far as I know, killed nobody. They also take a more cautious approach where bystanders are not turned into beta testers, of course :-)

> The autopilot software has killed people though.

Sorry, what’s your point? You need to prove that the autopilot software has killed more people than if the cars were driven by humans.

> And I believe waymo uses lidar and has, as far as I know, killed nobody.

With a footprint probably orders of magnitude smaller than Tesla. What do you want to bet is the difference in terms of miles driven by Waymo vs. miles driven by Tesla autopilot?

> And it would be great if it something that almost never broke as things that move a lot tend to. As far as I know static lidar are nowhere near as good.

Keep in mind you're saying this about something that attaches to a car, which already has dozens or hundreds of parts that rotate at hundreds or thousands of RPM.

> Then if the difference in quality is not actually that great, doing the development effort once will scale to millions of cars the next few years

The adage "do things that don't scale" comes to mind. If your options are "have tens of thousands of autonomous vehicles" or "have none", I'm not sure how the second is better. And we know that hundreds of thousands or even millions of lidar units are being produced (cruise and waymo and co have multiple units per vehicle and collectively hundreds of thousands of vehicles).

Adding more rotating parts is exactly what you are trying to avoid. For example, go look at how Tesla designed the heating and cooling system. The integrated all the different streams of hot and cold in one system duplicate these parts that can break easily.

In some cases it can simply not be avoided of course but you better make sure that you really need it if you are gone add it to millions of cars.

> cruise and waymo and co have multiple units per vehicle and collectively hundreds of thousands of vehicles

Waymo has vehicle count of a few 1000s as far as I know. I don't know about cruise but I don't think the have even 100k vehicles. Please show me a source.

And even if that were the case, the build those fleets of quite a long time and the cost per vehicle they currently have is nowhere near close to what would be possible for a car like the Model 3 or future cheaper models.

Margins in car industry are incredibly tight, car companies invest gigantic amount of capital to remove a few $ per car. These vehicles need to be able to be produced at a run rate of millions per year. If you are proposing to add something that cost 100-1000$ that is a absolute killer.

In addition its an additional part that can hold up your production. This current chip shortage is a perfect example where having a Lidar would just introduce a whole host of new chips that might be in a shortage.

> Waymo and cruise especially, but even a number of others, have demonstrated better autonomy than Tesla.

Have they? They have a different approach where they focus huge effort on individual Geo-fenced locations and they both are losing 100M of $ every year. Neither has given any timeline for general availability in all locations.

The race, as far as I am concerned is still open. It is not at all clear to me that Waymo is ahead at what actually matters.

> Musk has been claiming full self driving is 6 months away for 5 years now, and he hasn't gotten that much closer.

The claim they made no progress is objectively false.

> They have a different approach where they focus huge effort on individual Geo-fenced locations

Where they actually have driverless vehicles. That's the key difference. Waymo and cruise have demonstrated driverless vehicles. People get driverless taxi rides today from waymo, and I think cruise as well. Tesla doesn't. It's still sitting in driver assist land and isn't particularly better than other luxury vehicle driver assist.

> The claim they made no progress is objectively false.

Thankfully that's not what I said.

Wasn't another factor the physical size of the lidar units? Part of Tesla's schtick is making normal or even attractive looking cars (as opposed to the "alien bug" aesthetic EVs were synonymous with at the time) and that's a lot harder to do with a big lidar unit on top.

Also they marketed autonomous driving features as future software updates (including a discount/price hike for early/later adopters). Early adopters would be unhappy if future features required additional hardware.

They upgraded hardware in the past to support FSD customers, so I’m not sure they’re completely against it. If it was, say, just a $500 decision to make it completely viable, I don’t think Tesla would hesitate. But the $500 lidars can’t do what needs to be done.

Not just marketed, but sold, and for a very pretty penny.

See, for example, Andrej's talk starting at the 6-minute mark. Lidar requires a pre-rendered detailed map of the lanes, traffic lights and obstacles. Vision can operate in any novel environment the car is not preprogrammed for and is thus more scalable.

https://www.youtube.com/watch?v=a510m7s_SVI

I for the life of me cannot find the quote, but at some point I believe he said that if humans can judge distance/drive with nothing but vision, cars/computers will be able to as well.

It was around the time he made the comment that anyone using lidar is doomed, which is much easier to dig up as it was a headline everywhere:

https://arstechnica.com/cars/2019/08/elon-musk-says-driverle...

Hoping someone else has the link to the discussion of humans using nothing but vision, my google-fu is lacking this evening.

Too expensive. They also removed radar because of the global chip shortage so they're all in on vision.

> Too expensive. They also removed radar because of the global chip shortage so they're all in on vision.

That doesn't make much sense to me. Tesla also needs chips for computer vision.

Yeah, from their talk it seemed more that it was difficult to integrate the information from the radar and visual sensors.

Cost and reliability.

But Elon says it is because they don't need it.

Too expensive + Your eyes already do the distance estimation and they are basically cameras so we can do this just as well or better.

> Your eyes already do the distance estimation and they are basically cameras so we can do this just as well or better.

This is not really accurate. Our brains do the distance estimation, and they use all kinds of tricks and contextual clues to do it, not just parallax (~2.5 inch parallax for objects more than ~100ft away isn't all that helpful). And this is the whole problem with vision-only autonomous driving - ML capabilities are nowhere near the capabilities of a human brain.

Monocular depth estimation has gotten really good recently though[0]. Not saying this one paper/method is 100% sufficient, but we're closing the gap in this one capability (depth estimation from pure vision) quite rapidly.

[0] https://roxanneluo.github.io/Consistent-Video-Depth-Estimati...

That’s exactly what Tesla described tonight. 8 video cameras on a moving platform provide enough eyes, parallax, context, etc to build accurate 4D (!) models in real time.

> Our brains do the distance estimation, and they use all kinds of tricks and contextual clues to do it

So, first, this presentation we're talking about is literally about problems like that.

Second: your brain isn't nearly as good as you think it is, it's just constructing a coherent story to fool you into thinking it is. Try this on the highway sometime as a passenger: close your eyes and recite the distances to the vehicle in front of you and the one to either side. I bet you anything a Tesla is going to do that better.

Third: they pretty much cracked this already. They stopped shipping radar on US Model 3's and Y's in the spring, have shipped hundreds of thousands of them now, and there's not a hint of signal that something is off with distance measurements with the cars. My car doesn't get this perfectly (you can actually watch the animations on screen bounce around a bit as the estimates change) but I think it objectively does better than I do.

Distance/Lidar framing is old news, basically. Vision works fine. The worst bugs remaining with all the FSD Beta footage on Youtube are almost entirely pathing and planning issues. The car sees its environment just fine.

Yeah but some people suck at throwing darts. So some part of their mechanical system is bad at making judgements. Sure they can learn. But I want a 2 ton cyber truck hurtling down the freeway to KNOW how far things are away.

Telsa should pivot and innovate to make lidar cheaper.

My eyes don't get covered in salt and mud while driving down the highway.

But people in California probably don't get this.

So Elon just needs the neural acne to be working and hook up a brain in a jar to drive teslas

Maybe the government's Autopilot probe will force them to change their mind on Lidar.

I think Elon basically says that long term they will only need vision so they will spend all their time focusing on vision from the start. Maybe lidar will succeed first, but it will be a worse success than vision: redundant / expensive / intrusive.

No one here talking about the freaking humanoid bot?!?

Also interesting to see how focused they are still on pure vision input to construct the vector space instead of just using LIDAR, but I wonder how limited they are by existing models already being out there with set hardware.

What's to comment on? It's a piece of plastic with nothing inside it. You really think they'll have a humanoid robot by next year? Just look at Elon's face, he literally made up the timeline on the spot.

I was just about to post the same thing... like, I had to refresh multiple times and double check the comments if I were going crazy.

Of course it is. Elon has said that before. They use the human physics as a baseline of technology to mimic. If humans don't need lidar and can operate with vision, then they can do was the logic.

I might be completely wrong here, but hasn’t their collision incidents often been with weird things in the road such as sideways 18 wheelers and yellow lines heading into barricades? They’ve put lots of effort into tracking some pedestrian waaaaay over on the side passing behind trees, and that is really really impressive. But it seems to me the only reason it doesn’t hit a car in front on the road is because the neural net has tagged it as a “car”. I would like to see some system that also doesn’t collide into any sort of random object directly in front.

Along these lines, I read about the ditching of the radar, and I understand the technical limitations with the phantom stops under overpasses. But could someone help me understand why Teslas don’t just have the run of the mill Camry collision braking as a secondary measure independently?

> I would like to see some system that also doesn’t collide into any sort of random object directly in front.

The flip side of this is false positives: you don’t want a car slamming on the brakes because of a plastic shopping bag or other soft piece of trash lying in the road. This is a hard problem, regardless of technology employed.

On the other hand, you need the car to distinguish a mattress lying in the road from road markings. This is an easy problem if you have depth sensors, which brings me to my next point:

> But could someone help me understand why Teslas don’t just have the run of the mill Camry collision braking as a secondary measure independently?

Because for whatever reason (hubris?) Tesla insists on eschewing LIDAR/RADAR for 100% camera based systems. Not being able to explicitly sense depth is really kneecapping their system’s abilities.

>The flip side of this is false positives: you don’t want a car slamming on the brakes because of a plastic shopping bag or other soft piece of trash lying in the road. This is a hard problem, regardless of technology employed.

Is this really a problem with current vehicles? I've had ford, gm, and dodge/ram vehicles over the last ~8 years with forward collision sensors and automatic braking and never once had any of them slam on the brakes due to a plastic shopping bag or other piece of soft trash lying in the road. I honestly haven't even had them give warning from one of those items being in the road (or any item that wasn't a car to be honest).

Radar isn't going to signal on a shopping bag, but I bet you it would for a mylar baloon. Existing radar-based collision sensors really aren't that accurate anyway. They are quite late with detection in most cases, precisely because they have to be tuned not to brake on false positives. They want a good strong signal, at which point the best they can do is reduce the speed of an inevitable collision.

This is because those cars are using radar (a plastic bag probably would show up as a small blip), but Tesla is removing radar from their cars to rely solely on cameras.

> Because for whatever reason (hubris?) Tesla insists on eschewing LIDAR/RADAR for 100% camera based systems.

It's a retrofit problem. The moment you admit that you need LIDAR/RADAR, every Tesla on the road loses value unless you're willing to retrofit them all.

Comma.ai has sidetracked this issue by basically using the built-in sensors on the vehicles, and there's rumors of a camera-only version for those without radar coming in the future. This would allow any car that has fly by wire with steering control to use it for "level 2 autonomy"

> Comma.ai has sidetracked this issue by basically using the built-in sensors on the vehicles

Comma/OpenPilot doesn't work with Lidar anywhere I'm aware of...

I'm not sure about that, it's supposedly radar but I'm not aware of any cars on the road that use lidar.

> The flip side of this is false positives: you don’t want a car slamming on the brakes because of a plastic shopping bag or other soft piece of trash lying in the road. This is a hard problem, regardless of technology employed.

Driving is hard.

I totalled* a car trying to avoid what ended up being a black trash bag full of foam rubber on an on ramp at night.

* By insurance standards, it was perfectly drivable.

Thus far, this is all information Andrej has presented elsewhere.

Edit: Wow, Ashok's section was amazing. It does seems as though their ability to simulate (on top of their fleet's data collection advantage) could allow them to rapidly accelerate toward true FSD. I really like that they capture real world failures and simulate many realizations of them.

I like Andrej and I think he's a brilliant guy. At the end of the day though, there is no way for me or anyone to audit the neural networks or their training data. How can anyone trust them? I think this is really a fundamental problem with using neural networks in self-driving cars--I don't want to put my life in the hands of a fragile system that I, and no one alive, can understand. We are being asked to close our eyes, take a leap of faith, and give complete trust to this system, and just hope it's been trained on every edge case under the sun and nothing out of the ordinary will ever happen while we're in the car.

> I don't want to put my life in the hands of a fragile system that I, and no one alive, can understand. [...] just hope nothing out of the ordinary will ever happen while we're in the car.

This honestly sounds pretty similar to the current state of affairs.

Currently, there are 0 people that fully understand the human brain. If you're driving, and someone turns in front of you suddenly, you can ask "why did they do that", but neither you nor anyone else can truly understand the set of inputs and thoughts that went into that decision at the time. Even if the other driver says "I guess I didn't see you", they also don't understand their own brain. It's probably a post-hoc rationalization.

The argument you have applies just as well to letting humans drive as letting opaque neural networks drive... better in fact since we have a better understanding of neural nets than we do of brains.

I think this is a fundamental problem with driving cars, and it's exactly this reason that I would rather take trains and subways exclusively than drive. Even if the operator's brain has a weird malfunction on a train, we have well understood systems (tracks) that prevent them from turning into traffic.

The main difference is you have knowledge of how your own brain operates in reality, what kind of actions you might take in a given situation (and by extension the same can be said of other humans), whereas most (if not all?) people have no clue how neural nets operate, so all bets are off when it comes to predicting what the computer could/will do.

Contrast that with a human: you know the subset of possible actions a human would likely take. Even when you see a car swerving and acting like the driver is drunk, you have a mental model of the set of actions to expect from such a car (you might add more distance between you and that driver for example).

How can anyone trust any other complex system? Even something as seemingly trivial as a bridge. Picture driving on an unfamiliar road when suddenly there's a bridge ahead. How can anyone in that situation trust that bridge? Without knowing how it was built, whether it is in a state of disrepair, or whether the two sides of the bridge connect without a precipice inbetween[1]. The answer is that we don't need to completely trust a system as long as our experience suggests reasonable safety. Once unassisted Tesla proves to be safer than human drivers, we still won't trust it, but that won't prevent us from using it.

[1]: Not relevant to the above comment, but I once encountered such a bridge. It was while driving in the dark, at about 1am and with no traffic, when the navigation in my, iirc, Android 2.1 phone, suggested I cross an unfinished bridge. The only warning sign ahead of the precipice were a couple of traffic cones. The road was familiar, though, and I drove very carefully, curious about the new bridge.

> At the end of the day though, there is no way for me or anyone to audit the neural networks or their training data. How can anyone trust them?

By external measurements and statistics. You measure an ML engine by performance. Literally no one anywhere knows how to debug these things by looking inside the model.

If it's a "fragile system" then it will be shown to be so in data. At the end of the data, either cars crash or they don't. It doesn't matter what kind neurons made the decisions.

How can you measure the performance of an ML engine in untested conditions? Tesla's whole goal is to achieve L5 capabilities. That means anywhere in any conditions, which is clearly impossible to comprehensively validate with a finite test set. It's plausible that we could get a long period of acceptable safety before a black swan event suddenly causes deaths because it can't handle smoke, heavy rain/fog, or blizzards.

From the video, it looks like the team is trying to minimize the set of untested conditions through simulation and clips from the fleet.

I agree with you that those techniques won't eliminate errors, but I don't think that's the requirement for L5 - human drivers' errors cause lots of deaths today.

That's just ludditism. That logic works for any new technology. How do you know vaccines are safe under all conditions? How do you know planes won't crash under some "black swan" event? How do you know GM food won't poison you? How do you know your fuel tank won't explode? How do you know your ocean liner won't hit an iceberg and sink? There's literally no technology in the world that meets your standards.

You don't know. You measure and you decide, based on numbers and science and moral reasoning about risk. You don't get absolutes from anything else, why are you demanding it from cars?

I'm not sure what in my post gave you the impression that I'm demanding perfection, but it's incorrect. My interpretation of the GP's point is that we know these kinds of perception models have very real failure cases in the presence of noise and adversarial images in the environment, among other possible operating conditions. They're asking what safeguards exist to mitigate the risk of a system failure, which is something other SDC companies have put enormous amounts of work into.

It's a perfectly fair question and the general expectation of things we have as a society is that things generally shouldn't fail catastrophically at random. In a plane, that's achieved to socially acceptable levels by an incredible amount of systems engineering, redundancy, aerodynamics, and ultimately well trained pilots & ATC. With food it's achieved by government oversight, standards, and the general principle that most things won't harm you. With ships, we simply have them avoid icebergs.

What isn't a good answer is just handwaving about measuring model performance. That's the smallest of the many prerequisites to safety here.

It just has to be better than humans.

That's why I think Tesla's approach is wrong. It tries to replicate a human driver. The argument against lidar is "we don't need a lidar to drive, our eyes and brain are enough". Yes, enough to kill thousands of people. You can easily find pictures of involuntarily camouflaged obstacles that take way too long for us to notice, and as expected, vision based system fail just like us, but lidar have no problem.

It is hard for computers to be as good as humans when it comes to vision. That's something our brain is exceptionally good at, and a large part of it is dedicated to it. But here, for true self driving cars to be a reality, they need to be way better than we are. It means that we should throw every advantage machines can have at the problem, and extra sensors are part of it, don't be limited to visible light cameras. lidar, radar, sonar, IR, inter-car communication, satellite maps, use everything.

And BTW, focusing on seeing roads is nice, but don't forget that there is more to driving that seeing roads. People communicate with their cars, sometimes in subtle ways, current self driving tech doesn't, and that's a big reason why they feel so alien and other drivers hate them. Self driving cars should communicate too, and better than humans.

It doesn't need to be better than humans- slightly worse, but doesn't get drunk or tired still saves thousands of lives. Slightly worse but never drives crazy or gets distracted by someone in the passenger seat. Worse but is never surreptitiously texting.

Compare it to a horse. No one knows what's going on in the horse's brain. However, we learned to trust it and how to deal with it safely. E.g., blinkers (the devices to limit a horse's peripheral view).

Horse accidents are quite scary things. Way more deaths per mile than cars I'm sure.

That’s an amazing analogy.

> We are being asked to close our eyes, take a leap of faith

Actually, you're being asked to "Always watch the road in front of you and be prepared to take corrective action at all times. Failure to do so can result in serious injury or death.", until FSD gets released :)

Do you trust cab drivers to "be trained on every edge case under the sun"? Self driving only needs to beat humans, who have some serious flaws, not achieve perfection.

If all airplanes needed to do is "be safer than cars" then no one would ever fly.

> Self driving only needs to beat humans, who have some serious flaws, not achieve perfection.

In particular, though, it needs to beat good human drivers, not average ones.

I would argue the average human driver is actually a good driver. People argue that Tesla training their system to mimic a human driver is a bad idea because then it will replicate bad human driver behavior, but the fact is that human drivers don’t end up killing themselves or others the vast, vast majority of the time. George Hotz of Comma.ai brought up an interesting point about this on one of his Lex Fridman podcast appearances. He said from the driving data they’ve gathered, it seems that average drivers are good in the same ways, while especially bad drivers are bad in unique ways. The result is that the bad driver behavior in the data is pretty much overwhelmed and washed out by the abundance of good driver behavior in the data. Add to this the fact that self-driving systems still have things like automatic emergency braking, and I’d say you’ve got a pretty safe driver.

It depends. If it is equal to the average human driver then being on the road would be a benefit because below average drivers (drunk, elderly, tired, etc) could use FSD while above average drivers drove themselves.

How can you know that it beats humans? You would have to drive tens of trillions of miles to have any worthwhile data on this, all the while putting humans at risk.

Why do you need tens of trillions of miles?

"We trained a model, overfit on 1.4 trillion accident scenarios, and if behaved correctly on all of them."

Unless your model actually has trillions of parameters (and it doesn't, even gpt-3 only has 175 billion) it is not even possible to overfit on 1.4 trillion training inputs. You can't actually pigeonhole it.

Suppose that you train a neural network to predict the next number in an arithmetic sequence (a, a+b, a+2b, a+3b, a+4b, ...). As input it gets two numbers, the last number and the current number and has to predict the next one.

Suppose you had 1.4 trillion examples in the following test set (using a model with 175 billion parameters):

(1,2)->3

(2,3)->4

(3,4)->5

...

Do you think it is possible to overfit and score perfect on the test set, while failing to generalize?

I think you've specified this problem in a very strange way. But if you're saying that you're trying to train on the specific dataset where a = 1 and b = 1, then your model will fit the data perfectly with 175 billion parameters. It will also fit the data perfectly with, like, 15 parameters.

If you're trying to fit to some more complex space where a and b are unknown and you're given 3 numbers in the sequence, then what you're trying to fit is `f(a, b) = a + 2(b - a)` (or 2b - a, however you want to represent it), which is a swell function, but if you only give data that can be equally represented by `f(a, b) = b + 1`, you're mis-training your model.

But you could once again do that with a model with a dozen parameters. In both cases, the issue isn't overfitting, but misrepresentative data.

I didn't specify the training set, just the test set. It's possible that your model actually models an arithmetic series. Or that it simply overfits. The point is that it doesn't require trillions of parameters to overfit to a trillion-sized test set.

What you need are more parameters than the complexity of the underlying distribution. If you drop to a linear function you're modelling, you only need a couple of parameters.

"Overfitting" is memorizing the training data instead of generalizing. The example you're providing isn't overfitting, it's just generalizing to the wrong function. Overfitting would be if the validation set was, say, 30 random values that you got right, but didn't get other values along the same lines correct.

> I didn't specify the training set, just the test set

Then unless you constructed the training set with the intent of mistraining the model, I think a training set that got good accuracy on that validation set would generalize.

> The point is that it doesn't require trillions of parameters to overfit to a trillion-sized test set.

You can't "overfit" a validation set, unless you've done something wrong. Overfitting is, by definition, learning the training set too well such that you fail to generalize to a validation set.

Overfitting is, by definition, learning a model that doesn't generalize to the distribution of inputs you care about. If your validation set has the same distribution as the inputs you care about, then your definition holds. But that's definitely not true in practice. Usually the data you collect won't be exactly representative of the conditions you're looking to test, unless your problem is very simple.

> Overfitting is, by definition, learning a model that doesn't generalize to the distribution of inputs you care about.

No, that's just mis-modelling. Overfitting is specifically doing so in a way that learns the training data too well, at the cost of generalizing. If you try and have a single layer perception network classify a nonlinear function, it will fail to generalize. But it certainly isn't "overfitting".

Overfitting is not the only form of mistake when training a model. You've presented a different one, which is just like trying to train on misrepresentative data. But that isn't "overfitting", it's just having bad data. Your model isn't "failing to generalize", it has nothing to generalize over.

The classic demonstration of this is that overfitting usually results in a accuracy curve that "frowns" on validation data. Your accuracy peaks, but then decreases as you learn the structure of the test data instead of the general structure. In your example that won't happen.

Training a model in the wrong problem isn't overfitting. In fact, your example is more like underfitting than overfitting. The model in your example would fail to see the full complexity of the structure, instead of as in overfitting, make it more complicated than reality.

That isn't overfit, that's fit. Nothing can protect you if your training set just doesn't have any indication of the thing you want it to learn.

I didn't specify that the training set wasn't representative.

All this shows is that you don't need parameters anywhere close to the number of test examples to overfit.

And my point is that is not what overfit is. Overfit is a specific problem where the network fails to recognize a commonality in the training set and instead interprets the irrelevant details of some subset of training samples (in the extreme, individual samples) as distinct properties.

Your example training set is not filled with noise that the network is picking up on to its detriment. Your example training set is simply not representative of the function you are trying to teach.

I don't have an example training set. I don't have an example model.

My exact point is that if your test set isn't representative of the underlying distribution, then accuracy on the test set doesn't mean that your model isn't overfit.

What if the 1.4 trillion accident scenarios were the test set?

k-fold cross validation

It's a long tail that goes on to virtually infinity, yes, but the collective probability of the unhandled edge cases will become vanishingly small, and just an accepted part of living in our reality, which is not perfect and impossible to fully predict and control.

And each human is unique, making it is really difficult to QA/monitor/rollout updates to the fleet.

I think you're right that the bar of "average human on the road's driving skills" is very low. However, I think one of the reasons there's such an inherent distrust of Tesla's FSD is that when it fails, it fails in scenarios in which people can easily see themselves succeeding.

Say, on average, human drivers crash into barricades once out of every 500,000 miles driven. And say Tesla's FSD can beat that number by crashing into barricades once out of every 1,000,000 miles driven, it's still possible (probable?) that many people would trust their own control of the vehicle more than Tesla's FSD AI. And they might be justified in doing so!

Why? Because human-caused accidents are not uniformly distributed among the driving population. Insurance companies have employed actuarial analysts for decades to split the driving population up into buckets of greater and lesser risk. Thus, if you're a 17-year-old male, or an 85-year-old woman, or have several DUIs, you're likely to pay higher premiums than most other drivers.

If you're not in one of those high-risk categories though, it's possible your driving performance exceeds the average, and maybe you might only crash into barricades once out of every 2,000,000 miles driven. In that case, you'd be right to trust your skills ahead of the AI.

My example is contrived. It's possible that the FSD AI leapfrogs even the most skilled human drivers. But until it does, there may be rational reasons for many folks to continue distrusting it.

> Because human-caused accidents are not uniformly distributed among the driving population.

This is such a great point. Controlling for driving circumstances (e.g. weather, location, etc.), human caused accidents are not uniformly distributed across the population, but machine caused accidents are.

This, in my mind, is the fundamental reason people are distrustful of self-driving cars. Everyone thinks they’re far better than the median driver, even if this is mathematically impossible.

It's not just about skill. A lot of accidents are under the influence. That's a choice. While you can't control what other drivers are doing (and neither can the Tesla), you can at least significantly increase your odds of safety by some simple choices such as this.

> A lot of accidents are under the influence. That's a choice.

Only for one of the drivers! There are at least two cars involved in almost every fatal accident.

I made this point elsewhere but it bears repeating. Even if you think your driving is so perfect you can't benefit from a self-driving car, don't you want everyone else in one?

> My example is contrived.

Pretty much. I think if you're going to make an argument like "accidents by teenagers, senior citizens and drunks don't matter" you need to put some numbers behind that.

I mean, if you had a teenager or a family member with a substance problem, would you feel safer with them in a Tesla? If so, then I really don't think I understand your point.

Basically you're just saying that you, personally, as a young single man, feel like you're much better at driving than the median driver. So you want a car that is significantly better than even the median before you will "feel safe".

Alternate framing: it doesn't matter how safe you think your driving is, you're still sharing the road with grandparents. And you (if you're being rational about it) want them in a Tesla.

> And you (if you're being rational about it) want them in a Tesla

Right. Rationally, we all want folks who are worse than the state-of-the-art FSD AI to be using FSD. But thinking that the "very low bar" that has to be cleared is drunk drivers misses the point that most drivers are not drunk. So it's much less impressive to a "safe" human driver to clear the low bar.

> But thinking that the "very low bar" that has to be cleared is drunk drivers

I said median, not drunk driver. Existing shipping autopilot is already saving DUI fatalities. This feels like a strawman to me...

> I said median, not drunk driver.

Right, and I originally said:

> I think one of the reasons there's such an inherent distrust of Tesla's FSD is that when it fails, it fails in scenarios in which people can easily see themselves succeeding.

The above-the-median driver is rationally able to see themselves beating Tesla's FSD. But importantly, even below-the-median drivers are likely to think they can beat Tesla's FSD for as long as the mistakes it makes seem naive and trivially-avoided by them. Human trust will require a higher bar than "beats the median human on 5 KPIs, but still on rare occasions decapitates someone because it doesn't know what a semi truck looks like from the side".

FSD advocates seem to want to focus on the "beats the median human" bit. I'm saying it's unlikely they'll gain much public trust until the "can't recognize a sideways semi" stories die down.

That's ridiculous. Humans are quite good at driving, along with many other complex tasks. The fact that we see about 1 death for every 100,000,000 miles is pretty fantastic, especially when you consider how heavily that is tilted towards impaired drivers.

Exclude the worst 1% of drivers and the average goes up considerably.

I don't know if anyone else caught this, but their "optimal trajectory" planned path is illegally changing lanes in an intersection, at -1:26:00: https://www.youtube.com/watch?v=j0z4FweCy4M. Pretty funny.

I remember reading something about the legality of changing lanes in an intersection in different jurisdictions, and a quick google gives me this [0].

The consensus seems to be that it's legal in many places, however a dangerous lane change is always illegal and changing lanes in an intersection is almost always dangerous.

[0] https://www.reddit.com/r/NoStupidQuestions/comments/25kbg3/i...

At least in California, it is not illegal to change lanes in an intersection, given that the maneuver does not pose any other risk

I find it amusing (and maybe even ironic?) that tools and methods (GPUs and game engines, etc.) whose development was mainly driven by gaming and entertainment (rather frivolous use cases) now find applications in a much more "serious" domain with a potential to revolutionize our lives.

A lot of games in general allow the development of skills for use in more serious areas of life.

C++ is mentioned many times in the live presentation today. I guess they felt it is important to mention that.

The whole presentation is basically a recruiting event.

Honest Q for anyone here, how is Tesla's software-eng culture?

I've of course heard of people on the ground floor of Tesla (factory workers) not being treated well, I've heard of SpaceX employees having high churn as they get "burnt through", and of Elon being kind of an ass.

But having said that I have a Tesla and am one of those types that like to work on things that bring me value or that I can have some sort of tangible connection to.

I did some contract work for Tesla last year over the course of ~8 months. If I were to pick a few key words it would be: High pressure, somewhat disorganized, and results-driven.

It was high pressure in that there were hard deadlines, and these deadlines needed to be met. If they weren't met, you needed to have a damn good explanation for as to why they weren't met.

Somewhat disorganized in that they did have a complete dev-ops lifecycle, but it felt kind of patched together. The team I was on did not do any formal standups or sprint planning. There was a very robust QA, validation, and homologation process though. They expected developers to do most of the QA, though. The point of the actual QA team was to ensure that there were 'no surprises'. I happened to be paired with a dedicated QA rep, though, which was nice.

Results-driven in that you were expected to do what you needed to do to get your job done. "Nobody told me what to do" was not an excuse. If you needed something done by someone else, you were expected to take initiative and ask and bug people until it got done. Most employees knew this, however, and were happy to help.

Engineers somewhat idolized Elon but also legitimately did not want to be the one to tell him bad news (such as they would prefer a different company rep do it). There are a lot of really smart people at Tesla, and you really do get to work on things that have never been done before (I got plenty of news articles written about what I worked on), but you really need to be dedicated to the company.

When I was there, there was a lot of crunch towards the end of my contract, and they said it wasn't usual, but I'd say expect crunch at least once or twice a year.

Thanks for the insight. At this point in my life I'm interested in WLB, sounds like engineers there may be more likely to be on the younger (in their 20s?) side of things.

Sorry, didn't see this, but actually the ones I was around were often late 20's to mid 30's. Often had kids and a family. They were just very dedicated to Tesla.

"Human-level hands"

Do sensible people really take this seriously?

How can sensible people not? Really the problem would be that anyone that has not increased their threshold of the absurd are blind to the future that is here.

Is there a way to start watching at the beginning if I missed the start?

Open it in YouTube and scroll back.

Maybe it changed but the live stream link is an embeded youtube player, which you can rewind on normally. You can also increase playback speed and watch at 1.5x while it's still live if you're catching up to the current time.

Link here: https://www.youtube.com/watch?v=j0z4FweCy4M

Bottom line, robots coming soon.

He's slowly building out the roadmap for the labor pool, transportation, communications system and so on for the colonization of Mars.

They were supposed to do coast to coast self driving years ago according to these hype videos

https://www.digitaltrends.com/cars/tesla-coast-to-coast-auto...

Hard to take them seriously given how much hype they generate each year.

Accurate hardware/software development timelines are an unsolved problem.

To me, the question is: is the technology getting better over time? And from what I've seen I would answer that question with a resounding yes.

> Accurate hardware/software development timelines are an unsolved problem

Then why is it sold as a $10,000 full self driving package?

Tesla is selling a $10k hardware package, with the promise that eventually they'll have the software to match. Musk makes lots of optimistic predictions which are not hard deadlines. There is no actual guaranteed timeline in the terms that come along with the FSD package, AFAIK. It is made explicit that the software required to utilize the hardware you are purchasing is not ready yet.

And what are the chances that the $5k hardware packages they sold in 2016 are up to the challenge, when these $10k hardware packages they have are a few orders of magnitude faster?

Same thing come 2026: Tesla will almost surely have new hardware for that time, and I'm willing to bet that FSD still will be a vaporware product and incomplete. If it happened over the past 5 years, history tends to repeat and I can make the same bet will happen in 5 years into the future.

When you are doing innovative stuff that's never been done before like FSD it must be near impossible to get an accurate timeline. I guess they maybe need to do a bit of hype for the investors.

When you're selling "Full Self Driving" in 2016 for $5000, I think its reasonable for people to want results in 2021. Especially when there are people getting beheaded on the road running into trucks and emergency vehicles.

How much would you be willing to bet that any of the roadmaps you listed succeed and within which timescale? Because I'm nearly certain none of them will succeed, given they're run by Elon Musk.

which manufacturer tesla using for their dojo chips?

Presumably TSMC https://electrek.co/2020/08/18/tesla-hw-4-0-self-driving-chi...

Seems like delivering cars is not the only thing they delay...

Really? In 2013, Tesla projected 500,000 car deliveries per year by 2020. They missed by what, a few dozens?

A sufficiently advanced technologist is never late, nor is he early.

A sufficiently advanced technologist is also indistinguishable from a magician.

Elon is finally on stage @ 5:48 PM pacific.

Par for the course. Ever the optimist…

“ As a kid Elon Musk had so much trouble being on time, his younger brother developed a trick to make sure Musk didn’t miss the bus to school. Kimbal Musk — who currently sits on Tesla’s board — would lie to his older brother about the time, telling the future billionaire that the bus would be arriving, several minutes ahead of schedule. The slight manipulation ensured Musk wouldn’t be left behind.” https://www.washingtonpost.com/news/innovations/wp/2018/06/0...

He must have been the one who laid out the schedule for the CyberTruck.

TL;DW?