The Road to a
Graphic Source: NZ Story Group
The spectrum of automated driving
Before delving any further, it’s useful to understand the six levels of autonomy. As defined by SAE International, they are:
Why autonomous vehicles will be electric
“The fate of AVs and EVs will only grow more intertwined in the coming decade.”
That’s the confident prediction of a recent report by Bloomberg Philanthropies and the Aspen Institute, titled ‘Taming the Autonomous Vehicle: a Primer for Cities’.
To date, a healthy proportion of early-stage AVs are either fully electric or hybrids – for example, in 2017 General Motors rolled out a test fleet of autonomous Chevy Bolts.
“A lot of brands are reasonably comfortable with the notion that there won’t be a fully autonomous vehicle that’s not also an electric vehicle,” says Volvo New Zealand General Manager Coby Duggan.
Why this symbiosis of autonomy and electricity? One reason is technical: it’s simply easier for computers to drive electric vehicles, which roll with a lot fewer moving parts on board.
It’s also a matter of timing.
“We’re in the middle of three big paradigm shifts,” says Audi NZ General Manager Dean Sheed. “One is electric; the second is autonomous drive; and the third is the connected car. All three shifts are being developed in parallel.”
“There’s no need for an autonomously driven car to be electric, but there’s a deep connection,” continues Sheed, a Drive Electric board member. “From an OEM manufacturer’s point of view, most of the R&D expenditure is going into electric drives and autonomous vehicles. These things will merge, and the cars that they’re designing today for market launch in three years time will likely be electric to some degree, and have elements of autonomous drive.”
There are other factors pushing the electric cause. For example, the fact that most AVs will be deployed in dense urban environments, where emissions restrictions will be in place. As well, the predicted growth of ride-sharing will suit that AV/EV combination – recharging an unmanned car will always be easier, safer and cheaper than filling up the tank.
(While on that subject, autonomous technology could help optimise the use of electric charging infastructure, by planning routes and stops.)
The Bloomberg/Aspen report notes that electric AVs will be cheaper to operate for early adopters like taxi fleets, with fewer parts that must be maintained, and lower energy costs.
“For consumers,” it concludes, “these two cutting-edge technologies will go hand in hand – all electric and all-autonomous will be a powerful package for marketing next-generation vehicles.”
Why the future vehicle will be autonomous
For anyone who gets a kick out of driving, the autonomous vehicle must seem like a solution in search of a problem. But you don’t have to be a raging petrol head to struggle with the idea of ceding control of your car. Surveys of public opinion tend to report high levels of concern at riding in self-driving vehicles.
So why are we heading down this road?
The case for autonomy starts with safety. According to the World Health Organisation, there are 1.34 million road deaths every year, and the vast majority are caused by human error. (In New Zealand, it’s more than 90 per cent.) By putting computers at the wheel, and enabling cars to ‘talk’ to each other and the road environment, it’s argued that accidents will become a rarity.
New Zealand’s Ministry of Transport is certainly convinced, judging by a recent MOT document on road safety.
“These vehicles will be ‘connected’ to all other vehicles on the road and the number of collisions will plummet dramatically, even while technology enables them to drive closer together,” it asserts. “They will follow the rules of the road and have real-time information on road conditions. They won’t get tired or distracted. The human factor in accidents will be reduced to nearly zero.”
The ‘Taming the Autonomous Vehicle’ report notes that distracted driving is responsible for a growing number of road deaths. It also suggests that AVs can respond six times faster than a human in the event of an emergency, eliminating collisions caused by inadequate braking.
But the report highlights potential unintended consquences of AVs, and raises ethical questions. In cars with partial autonomy, might drivers be lulled into complacency? Will over-confident automakers start removing existing safety features? And what about the scenario of a car swerving to protect its occupants and driving into a crowd of bystanders? How does a computer make that call?
Arup’s Dr David Young, a Road Safety Expert, notes that “Autonomous vehicles have the potential to lead to a step change in the way the New Zealand transport system performs and road trauma reduction is one of the areas with the most potential. Fully realising these gains will require careful planning and a recognition of the potential dichotomy of benefits for vehicle occupants and other road users. Furthermore it requires a realisation that the new type of vehicles that we will see on our roads may be unlike anything we have seen before and to embrace their true potential we, as professionals, must ensure that they are fit for purpose.”
Young points out that pedestrians, cyclists and drivers in non-autonomous vehicles will continue to make mistakes and that there needs to be consideration for developing a more resilient transport system that is forgiving and sensitive to human error.
Coby Duggan says that Volvo’s decision to not pursue Level 3 partial automation is all about safety – a traditional focus for the Swedish carmaker. “Volvo’s concern is that, because the driver is ultimately still in control but is now far more distracted, will they be able to react and intervene fast enough?”
“Volvo has said it’s going to get Level 2 as advanced as possible, then make the jump to Level 4. At that stage, it will accept full liability for any incidents that occur when a car is in autopilot mode or fully autonomous mode. Volvo was the first manufacturer to make a statement as bold as that about liability.”
It’s expected that autonomous technology will improve mobility for groups such as the elderly, the disabled and the young.
The trade association for the UK motor industry recently canvassed the views of more than 3,600 respondents. Its report, ‘Connected and Autonomous Vehicles: Revolutionising Mobility in Society’, argues that CAVs “offer the chance to improve the quality of life for many sections of our society currently restricted from travelling”, citing better training and employment opportunities, among benefits.
The Bloomberg/Aspen report suggests that AVs will be a vital technology for enabling populations to age in place. “By 2030, the world will be home to more than 1.4 billion people aged 60 or over. As the spread of AVs reaches peak intensity in the 2030s, this group will shape the AV market.”
The report cites a 2016 study that predicted an even more telling effect on the lives of the disabled. “The study found that this group, while much smaller than the healthy elderly population, would have an even greater impact on travel demand – more than 55 billion vehicle miles travelled.”
For anyone living in New Zealand’s larger cities, that last statement should sound a small alarm about the potential downside of all this enhanced mobility: congestion. What good is improved accessibility, you might ask, if a surge in people travelling results in everyone going nowhere fast?
Efficient road use
Yet if managed well, a rise in numbers using the roads needn’t produce worsening gridlock.
Firstly, there’s an expectation that AVs will use roads more efficiently, partly because they’ll be equipped to safely maintain closer following distances. Highway ‘platooning’ could increase vehicle throughput by 250-500 per cent, according to some estimates.
But platooning is only part of the answer; we will also need to improve vehicle occupany.
In theory, autonomous technology could hasten the advent of widespread car sharing. You can visualise the scenario: driverless cars or ‘taxibots’ circulating a city, picking up and depositing commuters when requested.
William McGill, of Arup New Zealand, says this is one reason why organisations such as Arup that are involved in the planning of cities get so excited by the potential of AVs.
“The uptake of AVs will be a great enabler for companies that offer car-sharing services, and may provide a solution to some of our current transport challenges, such as the first and last mile,” says McGill.
“With the right incentives in place, and if operated alongside an effective high-capacity public transport system, it could significantly improve the transport offering. It provides an opportunity to develop an integrated transport network, linking different modes of transport and a seamless journey for the customer.”
He points to an OECD investigation into the potential impact of shared self-driving cars on city traffic. The report found that the combination of an effective public transport network with self-driving cars capable of being shared simultaneously by several passengers could remove nine out of every 10 cars in a mid-sized European city.
In a similar vein, the Bloomberg/Aspen report cites five major simulation studies from around the world. “They consistently show that automation can reduce the size of taxi fleets, and AV ride-sharing is likely to substantially reduce the need for private vehicle ownership,” it states.
The commercial service providers are connecting the dots between ride-sharing and the advent of AVs. Uber, for example, has invested heavily in the self-driving economy. Solis describes it as “among the more aggressive of the bunch in accelerating the arrival and implementation of autonomous vehicles in commercial applications”.
If the potential surge in traffic can be managed, AVs promise to deliver “enormous energy savings”, according to the Bloomberg/Aspen report. It cites European research indicating that even in the case of just two vehicles, truck platooning could reduce fuel use by as much as 15 per cent on long trips.“For cities, however, there are more important gains to be had from AVs,” the report suggests. “According to a 2014 analysis by the US National Renewable Energy Laboratory several other advantages – accelerated adoption of electric vehicles, reduction in vehicle weight, smoothing of peak demands for electricity, higher occupancy and more efficient driving – will all make a greater positive contribution to energy demand than platooning alone.”
Where’s the technology at?
Pundits searching for a symbol to illustrate the speed of technological change routinely cite the iPhone, an everyday handheld device that packs more punch than all of NASA’s combined computing at the end of the 1960s. Well here’s another: Tesla’s Autopilot relies on a graphics unit more powerful than the US Government’s fastest supercomputer at the turn of this century, just a teenager’s lifespan ago.
We are experiencing the opening scene of the autonomous vehicle revolution, with most auto manufacturers now offering semi-autonomous features in particular models. In Race to 2021, Solis talks about the path to ‘automotive 2.0’ as being a series of developments, with each new driver assistance feature forcing consumers to think about cars differently – “to tame fears and humanise the technologies, before fully autonomous vehicle release and adoption is feasible.”
What are those semi-autonomous features and how do they operate?
Today, if you were sitting behind the wheel of a car with existing Advanced Driver Assistance Systems (ADAS), you could expect the following:
- Traffic sign recognition. Detects, reads and displays road signs, including speed limits, ‘Do Not Enter’ signs and stop signs.
- Adaptive cruise control. Automatically adapts speed to maintain a safe distance to the vehicle in front and to a predefined maximum cruising speed.
- Lane departure warning. Provides visual, audible and/or steering vibration warnings when the vehicle appears to be crossing line markings without indication.
- Lane keep assist. Provides active steering assistance to stay within lane markings, typically used in conjunction with adaptive cruise control in ‘pilot’ mode. Driver is required to keep their hands on the wheel and to be prepared to take over at any time.
- Auto lane change. Helps the car change lanes on its own when the driver indicates on a multi-lane road.
- Blind spot information and lane change assist. Alerts when a vehicle is detected in the driver’s blind spot or if vehicles are fast approaching.
- Cross traffic alerts. Alerts when vehicles are detected while reversing.
- Automatic emergency braking. Detects slow or stopped traffic ahead. To prevent or reduce the severity of an imminent crash, alerts the driver to take corrective action and urgently applies the brakes if he fails to respond.
There’s 30 years of safety development work in that list, a project that gave us anti-lock braking and early cruise control. Meanwhile, the rapid evolution of AVs has been made possible by a series of more recent technological advances.
So what’s onboard an autonomous vehicle?
- LIDAR (for Light Detection and Ranging) essentially allows the car to ‘see’ where it’s going, using a pulse laser light and sensors to measure distance to an object. Born during the Space Race of the 1960s, the technology creates a 360-degree, 3D map of your car’s surrounding environment, with near perfect precision. LIDAR has been an important factor in the rapid evolution of autonomous vehicles in the last five years.
- GPS allows the car to ‘know’ where it’s going. Next-generation GPS will locate the car to within 5cm.
- Radar. Bumper-mounted radars supplement LIDAR for close-in control and can accurately monitor the speed of surrounding vehicles.
- Ultrasonic sensors to measure the position of objects close to the vehicle – for example, roadside curbs when the car is parking itself.
- Cameras. Detect traffic lights, read lane markings and road signs and keep track of other vehicles, pedestrians, etc. Image capture and interpretation is a fertile area for development.
- Central computer. Captures all this data to make decisions about braking, speed and route guidance. The software algorithms are by far the most complex aspect of AVs.
How fast is all this coming together?
Brian Solis titled his stocktake ‘The Race to 2021’ for good reason. “If you’re not thinking about this or implementing it, then you are way behind the market,” says Audi’s Dean Sheed.
Among car makers, unsurpringly much of the early innovation is happening at the premier end – Audi, Mercedes-Benz, BMW, Volvo, and others. But Toyota, Nissan, Honda and Hyundai are now also clearly in the hunt, with some massive recent investments in developing and testing.
Yet the traditional auto industry giants aren’t the only players – Silicon Valley has a massive stake in the outcome of what is really a revolution of the digital age.
As hinted at above, some unlikely partnerships are being formed, with car makers aquiring artificial intelligence firms and digital mapping ventures, or breaking bread with Uber. They’ve even set aside old rivalries to collaborate with each other.
Who is at the heart of the AV industry, and what’s their vision?
In 20 years, according to Tesla CEO Elon Musk, having a vehicle in your garage that lacks full autonomy will be like owning a horse: “You will only be owning it for sentimental reasons.”
Solis describes Tesla as one of the more aggressive pursuers of an autonomous vehicle future, and as the company that has done the most to introduce self-driving technology.
Its flagship development is Autopilot, a driver assist technology that offers features such as autosteer, automatic braking, lane changing, blind spot warnings, self-parking and the ability to be summoned remotely. In 2016, the company announced that all model S, X and 3 cars will ship with hardware and software capable of enabling fully autonomous mode, once conditions permit.
Why is Tesla so interested in AVs? “It’s about accelerating the move to sustainable energy and creating the safest cars for Tesla’s customers,” says the company’s spokesperson for New Zealand, Heath Walker, who firmly believes we won’t see a fully autonomous petrol car.
Google began working on self-driving vehicles as early as 2009, unveiling a prototype driverless ‘pod’ car in 2014 that had no steering wheel, accelerator or brake pedal – a clear point of difference with the auto companies. The following year it conducted the first test of a driverless car on a public road, in Austin, Texas.
In 2016, it shifted its self-driving R&D into a spin-off company called Waymo (short for ‘Way forward in mobility’), and announced a partnership with Fiat Chrysler. The automaker will supply plug-in hybrid Pacifica minivans for Waymo engineers to fit with self-driving technology.
Does the partnership signal that Google has abandoned the idea of producing its own vehicle? Waymo CEO John Krafcik said as much: “We’re not in the business of making better cars; we’re in the business of making better drivers.”
Audi’s motto translates as “progression through technology”, so it’s no surprise that the German carmaker has a well-advanced AV programme, with Level 2 vehicles already on the roads. According to Dean Sheed, the technology will eventually be standard across the model range.
Next year, the company intends to launch a range of Level 3 vehicles, including the A8 and the e-tron Quattro, an all-electric luxury SUV.
In an example of the kind of partnerships that are being formed, Audi has expanded its collaboration with chipmaker Nvidia to bring a Level 4 vehicle to market by 2020. It’s been suggested that if the venture is successful, a number of auto-manufacturers currently working on their own self-driving technologies may opt instead to install Nvidia’s.
Volvo’s highly safety-focused AV programme began in 2014 with the Pilot Assist semi-autonomous technology installed in its flagship 90 Series. A second generation of Pilot Assist was launched last year (this was the technology on display in that Tauranga road trial), capable of handling braking, acceleration and steering at speeds of up 130kmh, but still requiring a degree of driver interaction.
As mentioned, the company has decided to skip Level 3, and focus instead on getting a Level 4 car to market in 2021. It has partnered with Autoliv, supplier of car safety systems, and Nividia, to develop the hardware and software.
This year, Volvo launched the world’s most ambitious autonomous driving trial. As part of the ‘DriveMe’ project, 100 hybrid XC90 SUVs equipped with Level 4 features are being tested on Gothenburg roads by members of the public. Further trials are slated for London and Shanghai.
Some pundits have been surprised by the alacrity with which US automakers Ford and GM have embraced the AV revolution. “Detroit is stomping Silicon Valley in the self-driving car race,” suggested Wired magazine, citing a ‘leaderboard’-style report that ranked Ford and GM as leaders in their ability to bring a driverless car to market.
The magazine acknowledged that the rankings could easily shift with smart partnerships or acquisitions. “But they serve as a reminder that it takes more than clever tech to change the world – and that muscle still matters.”
In Ford’s case, it’s now calling itself a ‘mobility company’, and has made some massive investments, including $US 1 billion in artificial intelligence firm Argo AI, shares in a 3D mapping startup, the purchase of a machine learning firm and a stake in Uber. By 2021, it plans to launch a fleet of commercial Level 4 vehicles for use in ride-hailing services.
Meanwhile, GM has invested half a billion in Uber rival Lyft to develop a driverless ride-hailing service. In good news for electric vehicle proponents, it has focused on making the fully electric Chevy Bolt its first publicly available self-driving car, and plans to launch thousands of Bolts with Lyft from next year. Among safety wins, GM’s ‘Super Cruise’ semi-autonomous technology includes facial recognition to alert distracted or sleepy drivers.
“For BMW, autonomous driving is one of the key pillars of our strategy for the future,” says BMW Group New Zealand Managing Director Florian Renndorfer, who points to the creation of a dedicated AV development and testing centre near Munich with 2000 employees, as well as BMW partnering with Mobileye and Intel to create an open standards-based platform.
The BMW iNEXT, touted for release in 2021, is intended to be electric, connected and Level 3 autonomous, as well as theoretically capable of an upgrade to Levels 4 or 5. However, even a Level 5 BMW would include a steering wheel and pedals, so you could drive it if you wanted. “That fits with our heritage,” says Renndorfer.
After a slow start on autonomous technology, the Asian carmakers are now at full throttle. Honda is focusing on the mainstream, unveiling an affordable semi-autonomous version of its Civic LX sedan in 2016. It is targeting 2020 for the release of an autonomous car, as is Mitsubishi. Hyundai is collaborating with Google to accelerate its programme (note that the electric Hyundai Ioniq is equipped with Level 2), and Nissan is also rolling out semi-autonomous features. Meanwhile, Toyota has invested $US 1 billion in the Toyota Research Institute to develop autonomous technology. According to Solis, it has pledged another billion dollars to a university-led programme dedicated to “studying artificial intelligence, robotics and reducing driver fatalities”.
The technical pathway to fully autonomous vehicles
The self-driving car is no longer the stuff of sci-fi fantasy. Whether the world is ready for it is another matter, and there remains a raft of legislative, social and infrastructural impediments.
Leaving those obstacles aside for now, what kind of technology is needed to usher in a driverless future?
We’re at least a decade away from a car that can drive solo, according to computer-vision scientist Michael Jones, of the Mitsubishi Electric Research Laboratories. “We can get 95 per cent of the way there, but that last five per cent is really difficult,” he has said. “You almost have to solve all of A.I. to have a car that drives with no driver.”
Nvidia, which is one of the major technology companies involved the development of driverless vehicles, has stated that achieving full autonomy before 2025 is unrealistic. In contrast, the carmakers are assuming a shorter timeframe. Whoever is right, delivering full autonomy will require advances on almost every front.
“The companies working on self-driving technology disagree on the computing capacity needed to achieve autonomous driving,” notes a 2017 article on the Electrek website. One thing everyone agrees on, however, is that fully autonomous vehicles will need a bigger brain.
Last year, Tesla began equipping its vehicles with a supercomputer as part of its Autopilot 2.0 hardware. It believes there is probably enough capacity there to enable Level 5 autonomous driving when conditions permit, simply by using an over-the-air software update.
Not everyone is convinced. The Electrek article points out that Tesla’s original Autopilot 1.0 achieved Level 2 with 0.256 trillion of operations per second (or TOPs). However, it continues, “Level 3 to 5 are expected to need significantly greater capacity – anything from 2 to 20 TOPs, depending on who you ask”.
Meanwhile, Intel entered the AV race this year when it aquired Mobileye, which makes vision technology for self-driving cars. An article on the Intel website argues that semiconductor companies capable of “delivering leading-edge silicon” will play a critical role in pushing AV development.
“Automakers have determined that all the computing muscle needs to be located in the safest location in the car – under the driver’s seat,” it states. “In order to meet this requirement, they will have to use semiconductors, which provide very high processing capabilities and use very little power.”
LIDAR and other ‘kit’, and the cost issue
There’s plenty of room to improve the accuracy and scope of the sensors, cameras and other onboard technology that helps an AV navigate through the world. Driverless cars still have trouble recognising traffic signs, for example, let alone finding their way safely through highly complex urban roading environments – even graffiti on a stop sign can cause confusion. There is also a problem of sensors being handicapped by extreme weather.
At the same time, the industry is focused on driving down the cost of technology to broaden the market for AVs. It can take heart from the success in cutting the size, weight and price of LIDAR, the ‘eyes’ of an autonomous vehicle. A few years ago, this component cost around $80,000; today, it will set you back just a few thousand dollars, and the price is falling.
As Brian Solis observes, accurate 3D terrain mapping is critical to the effectiveness and safety of self-driving cars. “The modern roadway is a complex system,” he writes. “Maps provide autonomous vehicles with necessary information upfront, so the onboard computing systems can focus on detecting and reacting to other vehicles, obstacles, pedestrians, cyclists, and so on.”
One of the more significant developments in this area was the acquisition by BMW, Daimler and Volkswagen of Nokia’s HERE mapping technology. The system will compile transportation data from each of the company’s vehicles to be used by a consumer-facing app.
“Mapping is a very big part of the autonomous future,” says BMW New Zealand spokesman Paul Sherley. “Getting that mapping down to ten centimetre accuracy, or even less, is absolutely critical.”
Image Source: NVIDIA
Accurate and highly detailed mapping data becomes particularly important in extreme weather, which tends to limit the effectiveness of AV sensors and cameras. Last year, Ford tested driverless cars on winter roads, where the road markings were obscured by snow. Ford’s solution was to programme LIDAR sensors to detect landmarks above the ground, such as buildings. The cars could compare the feedback against digital maps generated during better weather.
Live video data from hundreds of thousands of BMW, Daimler and Volkswagen vehicles will be fed into the HERE system. It’s an example of Vehicle to Vehicle (V2V) technology, in which, as Solis puts it, “cars link to one another to map roadways, become more intelligent about road conditions, and also communicate with other vehicles so that all cars in the fleet become smarter”.
But V2V is just part of it. The future is also about Vehicle to Infrastructure (V2I) technology, which will enable travelling cars and the roadway infrastructure to ‘talk’ to one another, to improve road safety for all. Most of the main players’ AV development programmes acknowledge this – although Tesla appears to be pursuing a different path.
However, increased connectivity also brings security and privacy concerns – your car can now be hacked. “For self-driving vehicles it remains critical that the growing volumes of data transmitted to, from, and within the vehicle are safe,” states the Intel article. “The vehicle will need to rely on its data and make quick, accurate decisions – and to prevent, identify and isolate malicious threats.”
The human touch
It’s the great AV paradox: robots may ultimately prove safer drivers, but first they need to learn from humans to become better citizens of the road.
Audi has been teaching its vehicles to drive more like people – moving closer to lane markings before signalling a lane change, for example, or slowing down to let another vehicle merge safely.
Meanwhile, Nissan is taking a highly human-centred approach to its AV programme. By closely studying driver behaviour, it hopes to teach autonomous vehicles how to interpret and act on those subtle human cues we use on the roads – the nod of the head at an intersection, for instance.
“Machines may become pretty good at identifying similar and different actions and even anticipating patterns,” says Nissan Research Principal Scientist Melissa Cefkin, in an interview with design agency Experientia. “But they don’t have a means to understand and judge in social terms about the contingencies and meanings of the patterns.”
What about ethical decisions? Can a driverless car be programmed to weigh the ethical implications of its actions – to make a ‘judgement call’? Among other developments in this area, MIT’s ‘Moral Machine’ is using crowdsourced survey results to create ethical guidelines for AVs.
The road to a driverless future.
In a sense, we have already entered the autonomous age. This year in Christchurch, a driverless, electric-powered ‘Smart Shuttle’ was deployed at the airport campus, a local version of a scenario being played out in many other countries. That formula, involving a controlled environment and vehicles travelling at low speeds for short distances, will likely be the public’s first glimpse of driverless technology in action.
When will we see AVs ‘out in the wild’? Many pundits are picking that the first to hit our roads won’t be private cars. More likely, predicts the Bloomberg/Aspen report, is that the technology will be rolled out initially in long-haul trucks and taxi fleets. Already, a self-driving truck has delivered commercial freight in Colorado, and AV taxis are plying trade in Singapore’s ‘one-north’ high tech business district.
Private vehicles won’t be too far behind, however, with some predicting that by 2030 we could be entering a transition period on our roads, with driverless and non-driverless autos in a potentially awkward co-existence for a couple of decades. By then, proponents hope, the increasing ubiquity of driver assistance features and the demonstrable safety benefits of AVs will have melted residual consumer unease.
We will see. One certainty: if forecasts of a driverless future are realised, it will be due as much to social, legislative and commercial accommodations as to technological advances. In the next paper, we’ll take a look at what life might look like in the age of the AV, and what must change to make it happen.