Homepage / Technology / Car companies' vision of a gradual transition to self-driving cars has a big problem
Amazon says this Prime Day was its biggest shopping event ever Kudlow says President Trump is 'so dissatisfied' with China trade talks that he is keeping the pressure on As stocks regain their footing, an ominous warning looms Goldman Sachs downgrades Clorox to sell, says valuation is 'unsustainably high' How Satya Nadella has spurred a tripling of Microsoft's stock price in just over four years Kudlow says economic growth could top 4% for 'a quarter or two,' more tax cuts could be coming The one chart that explains Netflix’s stunning comeback US housing starts plunge 12% in June to a nine-month low Aerospace titans Boeing and Airbus top $110 billion in orders at Farnborough Target uses Prime Day to its advantage, logging its 'biggest online shopping day' so far this year Billionaire Marc Lasry sees bitcoin reaching up to $40,000 as it becomes more mainstream and easier to trade These are the 10 US airports where you're most likely to be hacked Amazon shares slightly higher as investors await Prime Day results Wreck of Russian warship found, believed to hold gold worth $130 billion A bullish ‘phenomenon’ in bond market is weeks away from fading, top credit strategist says Stocks making the biggest moves premarket: MS, GOOGL, TXN, UAL, NFLX & more Twitter shares up 50% since late April means most upside priced in, analyst says in downgrade EU fines Google $5 billion over Android antitrust abuse Mortgage applications fall 2.5% as buyers struggle to find affordable homes America may not have the tools to counter the next financial crisis, warn Bernanke, Geithner and Paulson Investors are getting spooked as the risk of a no-deal Brexit rises EU expected to fine Google $5 billion over Android antitrust abuse Ex-FBI chief James Comey urges Americans to vote for Democrats in midterm elections Elon Musk apologizes to British cave diver following baseless 'pedo guy' claim Disney, Comcast and Fox: All you need to know about one of the biggest media battles ever Xiaomi shares notch new high after Hong Kong, mainland China stock exchanges reach agreement The trade war is complicating China's efforts to fix its economy European markets set for a strong open amid earnings; Google in focus Hedge fund billionaire Einhorn places sixth in major poker tournament The biggest spender of political ads on Facebook? President Trump Asian stocks poised to gain after Fed's Powell gives upbeat comments; dollar firmer Stocks are setting up to break to new highs Not all FAANG stocks are created equal EU ruling may be too little, too late to stop Google's mobile dominance Cramer explains how Netflix's stock managed to taper its drop after disappointing on earnings Airbnb condemns New York City's 'bellhop politics,' threatens legal retaliation Amazon sellers say they were unfairly suspended right before Prime Day, and now have two bad choices Investor explains why 'duller' tech stocks can have better returns than 'high-flying' tech names Elon Musk is 'thin-skinned and short-tempered,' says tech VC Texas Instruments CEO Brian Crutcher resigns for violating code of conduct Google Cloud Platform fixes issues that took down Spotify, Snapchat and other popular sites Uber exec: We want to become the 'one stop' transportation app 'What a dumb hearing,' says Democrat as Congress grills tech companies on conservative bias Amazon shares rebound, report says Prime Day sales jumped 89 percent in first 12 hours of the event How to put your medical history on your iPhone in less than 5 minutes Investment chief: Watch these two big events in 2018 Even with Netflix slowing, the market rally is likely not over Cramer: Netflix subscriber weakness debunks the 'sky's the limit' theory on the stock Netflix is looking at watch time as a new area of growth, but the competition is stiff Why Nobel laureate Richard Thaler follows Warren Buffett's advice to avoid bitcoin Rolls-Royce is developing tiny 'cockroach' robots to crawl in and fix airplane engines After Netflix plunge, Wall Street analysts forecast just tame returns ahead for the once high-flying FANG group Roku shares rise after analyst raises streaming video company's price target due to customer growth China is investing 9 times more into Europe than into North America, report reveals Amazon says US Prime Day sales 'so far bigger than ever' as glitch is resolved Netflix is on pace for its worst day in two years US lumber producers see huge opportunity, rush to expand San Francisco to consider tax on companies to help homeless Homebuilder sentiment, still high, stalls as tariffs, labor and land drive up costs Powell backs more rate hikes as economy growing 'considerably stronger' Netflix history is filled with big stock declines – like today – followed by bigger rebounds Intel shares get downgraded by Evercore ISI due to rising competition from Nvidia, AMD Petco aims to reinvent the pet store with something you can't buy online Genetic testing is coming of age, but for consumers it's buyer beware Tech 'FAANG' was the most-crowded trade in the world heading into the Netflix implosion, survey shows Netflix weak subscriber growth may indicate a 'maturity wall' that could whack the stock even more: Analyst This chart may be predicting the bull market's demise Wall Street says Netflix's stock plunge is a ‘compelling’ buying opportunity because the streaming giant ‘never misses twice’ Tesla sinks after Musk tweets, again Boeing announces new division devoted to flying taxis Stocks making the biggest move premarket: NFLX, UNH, GS, AMZN, WMT & more Deutsche Bank downgrades Netflix, but says big subscriber miss is not 'thesis changing' IBM is experimenting with a cryptocurrency that’s pegged to the US dollar North Korea and Zimbabwe: A friendship explained Virgin Galactic spinoff Orbit to launch rockets from the UK with space deal Artificial intelligence will create more jobs than it destroys? That’s what PwC says ‘Treasonous’ Trump and ‘Putin’s poodle:' Scathing headlines follow the Trump-Putin summit China’s fintech companies offer ‘enormous’ opportunity, investment manager says Trump's performance at summit with Putin was 'unprecedented,' experts say Walmart and Microsoft link up on cloud technology as they both battle Amazon European stocks seen mixed amid earnings; Fed’s Powell to address Congress How I knew I should quit my day job and run my start-up full-time: Viral website founder China's stocks have been trounced, but the trade war may ultimately be good news for those shares Billionaire tech investor Peter Thiel bets on crypto start-up Block.one Asian shares subdued open after mixed close on Wall Street; energy stocks under pressure Amazon cloud hits snags after Amazon Prime Day downtime Netflix isn't doomed by one quarter unless people start questioning the long-term investor thesis Tech stocks set to sink on Tuesday after rough evening for ‘FANG’ Netflix plummets after missing big on subscriber growth This wristband lets humans control machines with their minds The U.S. has a rocky history convincing Russia to extradite computer criminals Amazon suffers glitches at the start of Prime Day Jeff Bezos is now the richest man in modern history 'The United States has been foolish': Read Trump and Putin's full exchange Goldman Sachs recommends these 5 highly profitable companies — including Nvidia — to combat rising inflation Goldman Sachs releases 'tactical' stock picks for this earnings season Three red flags for Netflix ahead of its earnings report The bond market may be raising recession fears, but don't expect one anytime soon Cramer: Banks are 'making fortunes' but are still as hated as they were during the financial crisis Putin told Trump at summit: Russia never meddled in US election


Car companies' vision of a gradual transition to self-driving cars has a big problem

An important moment in the self-diving car debate came on May 7, 2016, when Joshua Brown lost his life after his Tesla vehicle crashed into a semi-truck trailer. Brown had engaged Tesla’s Autopilot feature, and the software didn’t detect the white side of the trailer against the daytime sky. The car slammed into the truck at full speed — 74 miles per hour — shearing off the top of the car and killing Brown.

The accident — characterized by many as the first of self-driving car accident — caught the attention of the National Transportation Safety Board, which released hundreds of pages of new details about the incident last month.

But the industry that’s trying to make self-driving a reality takes exception with this characterization. Tesla argued that autopilot is not a self-driving technology, but more like an advanced form of cruise control. Drivers need to keep hands on the wheel and their eyes on the road at all times when Autopilot is engaged, the company says. Computer logs released by the NTSB last month show that Brown’s car warned him seven times to keep his hands on the wheel. Each time, Brown put his hands on the wheel for a few seconds — long enough to make the warnings go away — before taking them off again.

More from Vox:
The Maine event
The Senate is out, but its health care bill is still getting slammed
Donald Trump Jr. is fanning the flames of the #CNNBlackmail story

In January, the National Highway Traffic Safety Administration sided with Tesla in the case, concluding that the Autopilot system wasn’t defective. “Not all systems can do all things,” said agency spokesman Bryan Thomas. “There are driving scenarios that automatic emergency braking systems are not designed to address.”

Still, Brown’s death illustrates one of the biggest challenges carmakers will face as they work to bring self-driving cars to market. Most car companies envision a gradual path to self-driving capabilities, selling cars with increasingly sophisticated driver-assistance features over several years before ultimately introducing fully self-driving cars with no pedals or steering wheel.

But there’s a danger that people will trust the technology too quickly, as Brown did. After hundreds or even thousands of miles of flawless driving, people could stop paying attention to the road, with deadly consequences. Industry’s answers to what makes driverless cars safer to drive reveal a lot about how we already drive — and how easy it us to become dependent on a new technology.

Tesla’s approach to self-driving — one the company has doubled down on since Brown’s death — is for cars with partial self-driving capabilities to pester drivers to pay attention to the road.

Earlier this month, I visited Tesla’s Palo Alto headquarters for my own test drive of the Model S. Before she let me drive the car on my own, a Tesla employee gave me a lecture where she emphasized that Autopilot is not a full self-driving system. She pointed out that before enabling the Autopilot feature for the first time, drivers must read and accept a disclaimer, displayed on the 17-inch screen in Tesla’s center console, that stresses the need to keep hands on the wheel at all times.

And this isn’t just an idle suggestion. The Model S had sensors that that allowed it to detect if I had my hands on the steering wheel. If I didn’t, the dashboard would eventually begin to flash until I put my hands back on the wheel.

Joshua Brown’s Tesla gave him these warnings too — seven of them — but he didn’t pay enough attention to them. Since his death, Tesla has established a stricter “three strikes and you’re out” rule: If the driver ignores three consecutive warnings, he gets locked out of Autopilot for the rest of that trip. If the driver still doesn’t grab the wheel, the car will assume the driver is incapacitated and come to a gradual stop with the hazard lights flashing.

Other car companies are working on similar technologies. Audi, for example, will soon be introducing a product called traffic jam pilot that will allow hands-free freeway driving up to 35 miles per hour (self-driving at full highway speeds is four to five years away, Audi says). During a recent test drive, Audi engineer Kaushik Raghu told me that Traffic Jam Pilot will include a “driver-availability monitoring system” that makes sure the driver isn’t sleeping or looking backward for an extended period of time.

Cadillac recently announced a freeway-driving technology called Super Cruise that does this. An infrared camera mounted in the steering wheel can tell if the driver is looking out at the road or down at a smartphone. If the driver’s eyes are off the road for too long, the car will start beeping until the driver gets his eyes back on the road.

I got an idea of what this might look like when I visited Nauto, a startup that is developing sophisticated driver-monitoring technology. Nauto makes a windshield-mounted device that looks back at the driver and can tell if the driver isn’t looking at the road. If integrated into a self-driving car — or a conventional car, for that matter — this kind of technology could prevent a lot of distracted driving and highway deaths.

Of course, plenty of people drive regular cars without their hands on the wheel, meaning that driver-assistance cars could ironically wind up asking drivers to pay more attention than their all-human-driven counterparts.

Chris Urmson, who led the team of Google’s self-driving car engineers until he left to found his own startup last year, describes the decision over whether to assist drivers or take the drivers out of the equation as “one of the big open debates in the space.”

Google — which recently renamed its self-driving car project Waymo — has been thinking about this problem for years. Back in 2014, Google self-driving car engineer Nathaniel Fairfield gave a talk at a computer vision conference describing how Google had built self-driving technology for freeway driving, then decided that the approach was too dangerous. Waymo customers will never be allowed to touch the steering wheel on its cars.

Early versions of Google’s self-driving car technology worked a lot like Tesla’s Autopilot or the Audi prototype I rode in a few weeks ago. The driver would be responsible for navigating surface streets at the beginning and end of a trip, but would be able to activate self-driving mode during freeway driving. Clearly marked roads and a lack of pedestrians and other obstacles make freeway driving a relatively easy computer science problem.

The problem, Fairfield said, was that people started trusting the system too quickly. “People have this curve where they go from somewhat unreasonable but plausible distrust to way, way overconfidence,” Fairfield said. After a few hours of seeing Google’s freeway driving technology in action, people came to have “complete and utter trust” in its efficacy.

“In-car cameras recorded employees climbing into the back seat, climbing out of an open car window, and even smooching while the car was in motion,” John Markoff reported in the New York Times.

A technology that drives perfectly for 100 or even 1,000 miles might still make a catastrophic mistake once in a while. But after mile after mile of flawless driving, it’s unlikely the driver will be paying close attention to the road. Which means that if the car suddenly encounters a problem it can’t handle, handing off control to the human driver could make things worse rather than better.

This is a problem that the aviation industry has grappled with for decades. In 2009, Air France flight 447 crashed into the ocean on the way from Rio de Janeiro to Paris. The problem: ice on the plane’s sensors caused the autopilot to disengage. The pilot, with little experience handling the plane without computer assistance, did exactly the wrong thing, pitching the plane’s nose up when they should have pushed it down. The plane stalled, and the resulting crash killed all 228 people on board.

“It’s clear that automation played a role in this accident, though there is some disagreement about what kind of role it played,” a 2015 Slate article on the crash argues. “Maybe it was a badly designed system that confused the pilots, or maybe years of depending on automation had left the pilots unprepared to take over the controls.”

This is the paradox of automated systems: The better an autopilot system is, the more human beings will come to depend on it and the worse-prepared they’ll be if they’re forced to suddenly take over control.

It’s easy to imagine something similar happening with self-driving cars — especially to younger drivers. As driver assistance technologies become more common, there will be a growing cohort of teenage drivers who have never driven without computer assistance. But if these cars are designed to fall back on a human driver in unexpected situations, there’s a danger that human drivers won’t be prepared, and will do exactly the wrong thing at a life-or-death moment.

And the closer self-driving cars get to full autonomy, the harder it will be to get drivers to pay attention. Even if cars can force drivers to keep their hands on the wheel and their eyes on the road, they might still zone out if they don’t have to actually make any decisions for thousands of miles in a row. Beyond a certain point, it might become safer for the car to just make its best guess about what to do rather than turn over control to a driver who is likely to be disoriented and out of practice.

Google ultimately concluded that partially self-driving cars were a technological dead end. Instead, the company set itself a goal to build a fully self-driving vehicle — one that was so reliable it would never need intervention from a human driver.

Urmson, the former Google engineer, believes that driver assistance and full self-driving are “actually two distinct technologies.”

“In a driver-assistance system, most of the time it’s better to do nothing than to do something,” Urmson said in an April talk. “Only when you’re really, really, really sure that you’re going to prevent a bad event, that’s when you should trigger. That will guide you down a selection of technologies that will limit your ability to bridge over to the other side.”

In contrast, Waymo is trying to build cars that never hand off control to a human passenger. That means the software has to be able to choose a reasonable, safe response to every conceivable situation. And it means building in redundancies so that the car can respond gracefully even if key components fail.

“Each of our self-driving vehicles is equipped with a secondary computer to act as a backup in the rare event that the primary computer goes offline,” Google wrote in a 2016 report. “Its sole responsibility is to monitor the main computer and, if needed, safely pull over or come to a complete stop. It can even account for things like not stopping in the middle of an intersection where it could cause a hazard to other drivers.”

Building software that can gracefully handle any contingency — and redundant, bulletproof hardware — is a much harder technical problem than building driver-assistance technology that counts on humans to handle tricky situations. But it also has a big upside: If the car never transfers control to a human driver, then it never has to worry about the driver being caught unprepared.

For now, Waymo does still have drivers behind the wheel, but these drivers are Waymo employees with special training on handling self-driving vehicles, who are presumably paid to continue paying attention to the road no matter how boring it gets. And in recent months, the job has been getting pretty boring. According to a regulatory filing, Waymo’s self-driving cars drove more than 600,000 miles in California and only had to hand over control to a human driver 124 times.

That works out to one disengagement every 5,000 miles, a four-fold improvement over 2015, and by far the best showing of any company testing on California roads. At that rate of progress, it’ll take a few more years for Waymo to surpass human levels of driving safety. If the California data is any indication, rivals like GM, Ford, BMW, and Mercedes have a lot of catching up to do.

While Waymo seems to be the clear leader in self-driving technology right now, the company is taking a big risk by trying to jump straight to full self-driving technology. Reaching the necessarily level of reliability — going from 99.99 percent reliability to 99.9999 percent, say — might take several more years. During that time, companies taking a gradualist approach could steadily gain ground.

One big advantage of the gradualist approach is that it can allow car companies to collect a lot of data, and many experts believe that having a lot of data is crucial for building successful self-driving cars. Waymo’s cars have driven more than 3 million miles on public road, providing the company with the raw data they use to tune their software algorithms. in contrast, Tesla has collected more than 1 billion miles of real-world sensor data from its customers’ cars. All that extra data could allow Tesla to make more rapid progress toward a goal of full autonomy, allowing it to eventually catch and surpass Waymo’s early lead.

Disclosure: My brother works at Google.

Source: Tech CNBC
Car companies' vision of a gradual transition to self-driving cars has a big problem

Comments are closed.