it's been a long time since I filmed outside or in my car though according to some people I won't have to be sitting here for long which reminds me I recently picked up a large British audience because of a few videos I made so let me uh this should make them feel a little more at home and is oddly appropriate since this is what many people think that a driver's seat of a car might look like soon no steering wheel just a HUD it's pronounced HUD heads-up display not hood and I won't have to do anything more than just sit here and sleep or sing really we don't have an embarrassing shot of me singing but there are many problems and challenges that driverless cars must overcome first when artificial intelligence first started the designers thought to themselves hmm what's the most difficult thing we can get a computer to do what answer did they come up with chess and checkers and go board games but not just any board games these are games for super nerds if they can beat super nerds they must be pretty smart turns out all these games may be hard for you they're really easy for a computer like laughably easy deep blue beat the best human chess master in 1997 that was before most of you had internet and if you were lucky enough to have internet it was dial-up and you hoarded those AOL CDs with thousands of hours the game checkers was officially declared dead in 2007 because the computer had memorized every possible game situation all five times 10 to the 20 positions in 2015 chess was likewise salton in 2016 a computer beat the second best international go player board games are easy you input some rules maybe let it watch a few thousand games and that's it it doesn't require any motor functions or sensory input so what people didn't see coming was how difficult it is to get a computer to see seeing requires much more than just photon receptors that's why you have an entire lobe or about 20% of your brain dedicated to it colors are relatively simple and not all that difficult for a computer to figure out so they won't be the focus of this video but I'm I'm building to it shapes are also relatively easy but edge detection is not so when a computer see like this it sees a bunch of colors and shapes but it has no idea what any of it means and can't tell the objects from the background is this an object is this an object is this all one object with multiple colors but you can see it easily and without even having to think about it because one of the primary purposes of area v1 of your brain also known as ba 17 is edge detection it's one of the first things that your brain does with visual information this problem has been mostly figured out but I'll get to how in a bit a problem that will never be solved though is how to teach a computer to see depth it has no idea what is closer this or this again you can with little to no effort when you see an optical illusion like this you can tell that something isn't right but a computer has no idea first let's go through the monocular cues that is one eye for depth perception when you look at this picture you see the horizon your brain uses this as a major clue the closer it is to the horizon the further away it usually is and the higher up or down the closer to you it is we also look for parallel lines which converge the farther away they are also the further away you look the more atmospheric haze there is which makes things have less contrast and appear more blue than they should but a more obvious monocular cue is relative size which is how big or small something is compared to something that should be similar like in this illusion they're all the same size by the way but because of their positions they look hilariously miss proportioned or my hands one of them is bigger than the other not because I'm a freak but because one is closer to the camera another obvious clue is occlusion since my right hand is obscuring my left hand it must be closer which was the case with those cars but that's just how you do it with one eye you don't actually see this because you have two eyes giving you by ocular vision you see this the focus of your two eyes gives you depth perception because of a fun little bit of math that your eyes and brain calculate without your conscious awareness the angle of your eyes is calculated when you're looking here and when you're looking here and your brain essentially does triangulation to figure out that this point is further away than this point granted you don't actually see this blurry mess unless you take off your glasses during a 3d movie but in real life your brain cleans up that image a lot while you may think you see in 3d you actually only see in 2d your brain creates a 3d space using that 2d information and we're not even going to get into the motion cues because we've already complicated things enough and I mean come on maybe that'll be another video all of these are things that your brain does without even thinking about it many of these monocular cues can be programmed into a computer but not all of them and just like how your brain can be fooled by various illusions so can a computer so a computer cannot only rely on vision to determine depth or distance it needs some sort of outside help many of you might have jumped to the idea of GPS or satellite imagery unfortunately GPS doesn't provide you with any images and it doesn't even tell you where you are it tells you where it is and your computer makes its best guess on your location which is usually only accurate with NS euphy which doesn't really work when you're talking about piloting a one Tong hunk of metal 60 miles an hour satellite imagery also doesn't work because it isn't live which is pretty much what you need when navigating a busy street and even if you somehow manage to fix that problem you'll never overcome weather buildings and trees and stuff so a driverless car needs to use localized vision along with something else to perceive distance local to the car and we've actually had the solution to this longer than we've had satellites echo location such as sonar or radar basically it sends out a signal and based on how long it takes to reflect off of the object in return it can figure out how far away it is the problem is that this information is limited and not quite how you see it in the movies for an environment like this radar will return an image like this just like with visual information it's 2-dimensional and will only tell you how far something is from the transceiver at the level of the transceiver so if it's on the roof of your car that's not very useful when trying to figure out how far away someone's bumper is so have one on the bumper obviously that's on top of the fact that it only tells you how far away something is in that moment it doesn't tell you what direction is traveling or how fast it's traveling which is made even more complicated when you add in the fact that you are likewise moving the technology to do that does exist and has been in use for C and air travel for decades but those are long range with far fewer vehicles and little environmental obstructions so okay all of these are challenges that have been or can be solved so let's talk about some issues with vision that have not yet been solved what is this right it's a bicycle what is this it's it's still a bicycle come on okay but what is this it's still a bicycle this isn't rocket science it's pretty easy yeah for you for a computer it's infinitely difficult object recognition is by far the hardest thing to get a computer to do especially when you consider that 3d objects look completely different in a 2d image from different angles even in perfect conditions you may remember this video from a few years ago of a robot navigating around the world and interacting with objects you probably mostly remember it because of the jerk with the hockey stick it's all pretty impressive as long as you aren't aware of the difficulties of getting a computer to see you know what a cognitive psychologist sees when they watch that video a robot interacting with a bunch of QR codes those QR codes tell their all about what the object is its distance and its orientation whether it's a box or a door if they want the robot to interact with it there are QR codes stamped all over it sorry for ruining the magic what am I talking about computers are we good at recognizing things but how Facebook knows I'm in all those pictures or all those snapchat filters yeah I mean they're good at faces I'll give you that there's a pretty standard pattern for those I mean your brain is wired to think this is a face it's not it's a chair and it's not really like recognizing faces is all that useful for driverless cars computers are getting better at recognizing objects though and you're helping whether you're playing Google quick-draw or doing one of those new captured images where you select all the squares with a stop sign you're teaching computers had a seat you're helping our future machine overlords recognize objects however even with all of this learning it still has to match what it sees with a previously learned template it's not like you able to figure out what new objects are and what they mean on the fly for example say you come across this sign it might only take you a second or two to skim over it and realize that it doesn't apply to you and continue driving but a computer the first thing it will recognize is the shape it sure looks like a stop sign it's red and white just like a stop sign it has a little more to it than a stop sign but just to be sure I better stop in the middle of this street that could cause an issue or perhaps this situation clearly that's a stop sign but it's covered with a trash bag whatever Internet traffic uplink it's using says that there should be a stop sign here and there it is but it's it's partially covered with a trash bag again you can look at this situation and quickly figure it out you're supposed to follow the temporary green light and ignore the stop sign but a computer especially one that's never encountered this before won't be able to react as easily say the temporary light wasn't there is a trash bag in some duct tape all it takes to fool a self-driving car into just flying through the intersection if so you're going to see a lot more teenage pranks and YouTube videos like this show up and what about this detour sign your GPS says that you should go straight you might know that you can safely go straight but a computer sees this sign saying it must go right maybe there's an obstruction ahead these are all things that you can quickly figure out but a computer has to obey a set of rules and when presented with something outside of its rules that it may not know how to react which is why I'm not very impressed when people bring up Google's self-driving car that drove up and down the highway on its own a few years ago driving on a highway is easy all you have to do is stay between the lines there are very few dynamic situations very few unique situations and relatively few challenges it's so easy a monkey could do it actually it's so easy a dog could do it this isn't a prank this isn't a joke this is an actual dog driving a car there's an entire channel dedicated to showing various dogs learning how to drive cars it's not that hard obviously they're on a track by themselves heavily supervised but they are staying in the lines it's not that hard and it's not that impressive highway driving is so easy you regularly completely zone out and stop paying attention and things usually turn out ok not like in a city where you're on constant guard and see dynamic and unique situations all the time which brings us to the last major hurdle that driverless cars must face once you're able to get it to see and understand what it sees you then have to tell it what to do with that information let's say you're driving along and come across this situation again you as a human can quickly figure out the context of this situation and probably wouldn't stop a computer on the other hand wouldn't know that this person was just about to turn right and get into the driver's side door according to this person's current trajectory if the car doesn't stop now it's going to hit them does it assume that the person is fully aware and is acting safely or does it stop possibly causing an axe with the car behind it that's a simple situation a very simple situation let's say that the car is driving along on a two-lane road and it realizes that its brakes are out I don't know maybe a line that severed or a wire shorted out it doesn't matter it's rare but it's not unheard of coming towards the car are two motorcyclists who are dangerously riding side by side in both lanes your car must now choose who to hit you as a human can freeze up and yell jesus take the wheel and let physics decide who lives and dies a computer on the other hand camps not making a decision is a decision to do nothing which means that the car will hit one or both of them which means that the car decided to hit one or both of them there is no scenario where a computer can claim to have been so flustered it couldn't make a decision it could decide to follow the law and strike the person who is travelling in the incorrect lane that's one way to do it or we can make the situation even more interesting by pointing out that one rider is wearing a helmet and all of their protective clothing while the other is simply wearing a t-shirt and shorts your car may decide that this person is more likely to survive a collision although very slimly more likely and therefore steer the car into that person which would paradoxically make it less safe to be wearing a helmet or we could point out this tree to the side it could avoid hitting both riders and instead I like to crash itself into the tree just injuring you you likely survive while the riders probably wouldn't but who is the car supposed to protect you the owner and operator or some bonehead Harley riders who weren't obeying the law some might say that as the owner the car's main directive should be to protect you and your passengers while others might say that it should protect as many lives as possible but given the choice if there were cars on the market that safeguarded all lives or cars on the market that just protected you and your passengers you might be more inclined to buy the one that places you and your family above others let's pose another situation say you're at an intersection and your car wants to make a right turn but there's a line of school children currently crossing the street all holding hands single-file so your patiently waiting but another car coming down the road has hit a patch of ice or has its brakes and steering go out whatever it doesn't matter the point is that the car cannot stop and no longer has control it's also a self-driving car and using magic is alerting all other cars in the area about its situation if your car is designed to only protect you it'll probably sit tight and force you to watch something so horrifying you'll never see the end of the therapy bills if your car is designed to protect as many lives as possible it might pull forward into the intersection stopping the car from plowing through all those kids but you'll be t-boned and your possibility of walking away from this accident is pretty low these are situations that driverless cars will be forced to make decisions on and they're incredibly tough decisions not to mention the fact that I've only given you a handful of the literal infinite amount of possible situations I certainly don't want to be the one writing moral and ethical codes for driverless cars but someone has to especially if we ever went intersections would look like this where there are no traffic lights all the cars are driverless and are simply communicating with each other with hyper efficiency and it's absolutely impossible first of all it requires that every single car on the road be self-driving if there's even one manually driven car game over which then also means that your car must be self-driving at all times if you're able to manually switch it on and off an intersection like that will never work which means that Old Man River out on his dirt road would have to be using a self-driving car we can get around this situation by saying that only on certain roads autopilot must be enabled fine but let's say you're on one of these autopilot only roads in you're late for work when this happens your HUD tells you that an emergency is occurring on the road so all travel is currently halted never mind how furious you'll be over the fact that the government can just seize control of your car your lane so you flip the manual override and decide to proceed anyway and congratulations you just caused the collision don't act like this situation is impossible how many people do you know who just drove around a closed highway sign because the weather was bad if people can break the law in order to save themselves some time they will but let's go back to this intersection and assume that all cars will always be self-driving with no possible manual override this intersection is a disaster waiting to happen let's completely set aside the idea that anyone would ever go to this intersection with malicious intent even though those people have always and will always exist and we'll assume that all of these cars are completely unhackable again we're assuming perfect conditions imagine a tree branch falls in this intersection or a tire blows out or a trucks unsecured cargo falls off you're looking at a several car pileup even with AI they can respond instantly also having traffic flow like this renders the intersection completely useless the pedestrians and bicyclists there's an easy solution to this of course a four-way footbridge which likewise dramatically increases the likelihood of something or someone falling into the intersection accidentally or not but again in order to achieve this perfect flow of traffic everyone needs to have a driverless car cars aren't like phones where people get a new one every year or two cars last a long time like 15 to 20 years most people don't go out and get a new one until their current one breaks beyond repair so even if by some miracle all technological and ethical hurdles are overcome in the next 10 years which is extremely generous they totally won't be and they stopped selling manually driven cars the same day without government intervention it would still take 15 to 20 years to phase out all of the manually driven cars excluding antiques of course because you're never going to take those away from people on the topic of government intervention we also have several legal issues that need to be ironed out just to throw a few out there who's at fault when a car decides to hit someone when you're the only person riding in a self-driving car are you allowed to be on your phone what about sleeping or drunk if you're required to be awake and attentive the entire time doesn't that kind of ruin the point of it being self-driving self-driving cars will happen don't get me wrong they are coming but if you think they're going to take over the roads in the next 10 20 or even 30 years hopefully now you know better ah why you watching me stand with the end me I've got a remedy oh why are you watching me dance with the enemy here is my remedy Oh