The History of the Future

The idea of automated vehicles is already over 100 years old but was initially explored only in fiction. The first technical experiments started in the 1950s but research with environment sensors did not start until the 1980s. Advanced driver assistance systems finally entered production vehicles in the late 1990s. Then, the DARPA Grand and Urban Challenges sparked the transition to technical development of fully autonomous vehicles, which was originally spearheaded by Google. Recent years now saw the formation of a whole industry around automated vehicles, made up of traditional automotive players as well as new ones, including many startups. In the last few months, the landscape has changed even more dramatically with many partnerships forming and acquisitions in the multibillion dollar range, all betting on the unprecedented success of automated vehicles in the next few years.

The dream of self-driving vehicles was already approached in fiction not long after the development of the automobile itself in 1886. In 1915, a magazine depicted a self-driving street car, “a motorist’s dream: a car that is controlled by a set of push buttons”. The 1950s saw a range of vehicle automation in fiction, leading to Herbie in the 1960s and KITT from the 1980s TV series Knight Rider, which was able to drive itself and to “see” the environment with the help of a scanner.

Early research

The late 1950s marked the starting point for research on self-driving or robotic vehicles. GM and the Radio Corporation of America (RCA) cotested automated highway prototypes in 1958, using radio control for velocity and steering. Vehicles were equipped with magnets, thus, were able to track a steel cable buried in the road, while the overall traffic flow was managed by control towers. In 1979, Stanford University’s Cart marked a milestone: initially remote controlled, it successfully crossed a room full of chairs in five hours, without any human intervention. Carnegie Mellon University’s (CMU) Rover followed soon after, as did Shakey from the Stanford Research Institute, the first general-purpose  mobile robot that was able to reason about its own actions. The Tsukuba Mechanical Engineering Lab in Japan developed the first automated street vehicle with environment perception in 1977, it was capable of following lane markings for up to 50 m, with velocities reaching 30 km/h. The worldwide first actual robotic street vehicles were built at the Bundeswehr University Munich in Germany by Prof. Ernst Dickmanns and his research group in the 1980s. Operating with computer vision and probabilistic algorithms, they drove 20 km on an empty highway at speeds of up to 96 km/h. In 1995, Prof. Dickmanns’ team drove from Munich to Denmark and back, traveling a distance of 1758 km. Both longitudinal and lateral guidance were achieved by computer vision, reaching velocities as high as 175 km/h. The longest part driven autonomously on that trip was 158 km. That same year, CMU conducted the “No hands across America” drive. Their research vehicle covered the entire distance from Washington, D.C., to San Diego, California, with automated steering while longitude was controlled manually. San Diego was also the venue for a technology demonstration of the U.S. Department of Transportation’s National Automated Highway System Research Program (NAHS) in 1997. Automated cars, buses, and trucks drove on the I-15 with lateral lane keeping enabled by computer vision and magnets embedded in the road, and longitudinal distance by radar or lidar sensors.

The DARPA Challenges set the stage

DARPA, the Defense Advanced Research Projects Agency, funded a series of autonomous vehicle competitions. 15 teams qualified for the final competition of the first event in 2004. Their unmanned self-driving vehicles had to attempt a 150-mile course in Nevada’s Mojave Desert across dirt roads, flats, and mountain passes. A set of GPS waypoints was distributed to the teams only 24 hours before the event. Not a single robot vehicle was able to finish the race, the best team completed only seven miles. The following year, DARPA repeated the event; five teams completed the 132-mile course, providing evidence of the remarkable progress in this field in a very short time span. First one to cross the finish line after six hours and 54 minutes was the robot car Stanley, which was developed by Stanford University’s Sebastian Thrun and his team. The 2007 DARPA Urban Challenge then moved the competition to an urban scenario at an abandoned military base. The teams had to negotiate
fourway intersections, blocked roads, or parking lots with other selfdriving as well as human-driven vehicles. The first autonomous vehicle traffic jam occurred at this event as did the first minor collision of two robot cars. The 60-mile race was won by CMU’s vehicle “Boss”; Stanford’s “Junior” came in second. For the first time, researchers heavily relied on high-resolution lidar sensors and maps.

Industrial development starts

The Urban Challenge also marked the transition from academic research to industrial development; in 2008, Google started working on self-driving cars and announced its self-driving car programme two years later. The programme was directed by Sebastian Thrun. Several car manufacturers announced similar projects, such as Daimler, BMW, Audi, Volkswagen, GM, Nissan, Honda, Toyota, Volvo, Ford, Tesla, Hyundai, Jaguar Land Rover, and Faraday Future. Automotive suppliers like Bosch, Delphi, Continental, and Mobileye followed suit. Also, ride share companies, e.g., Uber, IT companies, e.g., Baidu, and chip manufacturers, e.g., Nvidia or Intel joined the field with their own self-driving car projects as did startup companies like Zoox, Cruise Automation, drive.ai, comma.ai, nutonomy, or Nauto. In 2013, a VisLab research vehicle drove autonomously for 13 km in public traffic of Parma, Italy, partially without a safety driver in the driver seat. That same year, Daimler and the Karlsruhe Institute of Technology built the autonomous research vehicle “Bertha” which drove the 100 km long Bertha Benz Memorial Route from Mannheim to Pforzheim, Germany, in public traffic. In 2014, Google presented a car that was specifically designed for self-driving and by 2016, Google’s fleet had completed over 1.5 million miles. Significant advances in machine learning over recent years enhanced an enormous progress in the area of environment understanding for automated driving, one of the key enablers for developing driverless cars driving in real traffic.

Automated trucks are potential beneficiaries

Lately, also automated trucks have gained attention. In 2015, Daimler Trucks North America demonstrated a self-driving truck in Nevada. Volvo and other truck manufacturers have also held autonomous freeway driving demonstrations in Europe. The Silicon Valley startup Otto started working on autopilot retrofit kits for trucks in 2016. Another Silicon Valley startup, Peloton Technology, has already been working on truck convoys for fuel efficiency since 2011.

The industry changes

In the last years, specifically in the past few months, unprecedented partnerships have formed in the industry. E.g., the competitors Audi, BMW, and Mercedes allied to buy the mapping company here in 2015 for over €2.8 billion. In 2016, GM bought the small startup Cruise for supposedly $1b; BMW, Intel, and Mobileye teamed up; Delphi and Mobileye announced a partnership, Uber acquired Otto for reportedly $680m; Delphi, Mobileye, and Intel partnered; Google spun out Waymo. In 2017, Intel invested in here and then bought Mobileye for over $15b. The automotive industry has realised that the technical challenges of automated driving are tremendous and require significant investments. At the same time, new and more agile players enter the field with new technologies and, as a consequence, development speed has increased. As a result, new partnerships form in areas that do not contribute to product differentiation, such as maps, sensors or perception.

Seven key challenges remain

In over a century, autonomous driving has developed from fiction to research to industrial development and first partially automated production systems. Several challenges remain though:

(1) Technology and functional safety

Algorithms are still under development and being optimised. Suppliers are working on new and better sensors, and companies are testing various different sensor sets. Remaining challenges are, for example, how to deal with unforeseen weather and road conditions? Or with objects the sensors have never seen before? Or temporary construction zones or changes to a prerecorded map. Automated vehicle technology has made tremendous advances in the past years, but how reliable and safe does the technology need to be in order to be ready for introduction?

(2) Cyber security

Automated vehicles will likely also be connected vehicles in order to download map and traffic updates, and connecting to other vehicles or infrastructure, such as traffic lights and highway signage. However, connected vehicles have recently become targets for attacks. Cyber security may never be perfect. How can manufacturers build vehicles that are safe, nevertheless?

(3) Human factors

What is the role of the human inside partially, highly, and fully automated vehicles, respectively? Who is in charge and responsible? How does the handoff transition of responsibility from driver to vehicle and back work? And how will traffic participants and persons outside of the automated vehicle react to it?

(4) Legality, regulations, liability

The 1949 Geneva and 1968 Vienna United Nations Conventions on Road Traffic require a driver who is “at all times […] able to control” the vehicle. However, the interpretation of this requirement differs from country to country. Whereas the predominant opinion in the United States is that automated vehicles are legal, for example, the German interpretation is that highly automated vehicles are not [yet legal]. In 2016, the United Nations approved amendments to the Vienna Convention as well as to UN Regulation No. 79, explicitly allowing automated driving technologies transferring driving tasks to the vehicle, provided that these technologies are in conformity with the United Nations vehicle regulations or can be overridden or switched off by the driver. In the meantime, several US states have introduced, passed, or rejected various non-uniform state bills and regulations. Uncertainty also remains in the area of liability: Who will be liable when an automated vehicle is  involved in an accident? Will liability shift from driver to manufacturer or even supplier as vehicle control shifts from the human operator to the automated vehicle?

(5) Ethics

Should automated vehicles be as safe or safer than human drivers? How safe is safe enough? What if automated vehicles reduce fatalities significantly, but at the same time cause new, different types of accidents? Arguably, there may never be concrete answers to many ethical questions pointed out by philosophers dealing with the matter.

(6) Infrastructure

Traffic depends on road infrastructure, and automated vehicles depend on digital and analog infrastructure as well. But how much infrastructure in the form of digital maps, infrastructure based communication systems, dedicated lanes is really required? Can, should, or must automated vehicles rely on the infrastructure? And how often do maps need to be updated?

(7) Urban planning and road planning

Automated vehicles will have a huge effect on mobility demand and traffic but the concrete changes on the amount of traffic, future traffic patterns, and urbanisation are highly uncertain. Will there be more total vehicle miles travelled? But will automated vehicles need less road space than today? And will there be fewer vehicles overall that need less parking? Significant challenges towards vehicles that can drive themselves in all locations and under all conditions still lie ahead of us. Given the tremendous progress in the field over the past ten years, the next ten years shall be even more exciting.