On April 8th 2020, ICTC (the Canadian Information and Communications Technology Council) spoke with Dr. Jack Stilgoe, Senior Lecturer in the Department of Science & Technology Studies at University College London, where he researches and teaches the governance of emerging technologies. Dr. Stilgoe is the Principal Investigator of the Driverless Futures? Project, a three-year social science project looking at the governance of self-driving cars.
This article was originally published by Pando Daily
New technologies do not just change the world on their own. They need users, they need infrastructures and they need a lot of political support.
The invention of driving
At the Victoria and Albert Museum in London, in the middle of an exhibition on car culture, sits a grid of sepia photographs from the early Twentieth Century. The pictures, taken by travelling salesmen from Michelin, show places from Cape Town to Clermont-Ferrand (the company’s home town). Some are of smooth highways. Others show tracks dotted with potholes and boulders.
In 1921 André Michelin and his brother Édouard had turned their expertise in bicycle tires towards the motor car, a technology that was exploding in popularity. The Michelin Roulement Universale was a tire designed to be sold around the world, but the world’s roads were of varying quality. Their road photography project was an exercise in market research, but it was also an act of lobbying. Michelin wanted improvements to the world’s highways, and they wanted them fast.
The Michelin brothers were never just rubbersmiths. While Edouard managed the practical side of the business, André handled the public relations. For French motorists and people the world over, they did something more profound than invent tires; they invented the road trip. Andre’s genius was to manufacture demand for the technology that would take his company forward. According to the historian Georges Ribeill, it was André who “first understood the technical system of the automobile which was to result from the “automobile revolution,” and of which tires were only one of the elementary working parts.” This system included the machines, fuel, maps, road networks, habits and rules that would allow for the dramatic expansion in people’s horizons in the early Twentieth Century. In 1900, André created the guide that would make his company famous and become the yardstick against the world’s restaurants were judged. Even though there were fewer than 3,000 automobiles in the whole of France at the time, 35,000 Michelin guides were produced and handed out for free. The company made the first civilian maps and gave away route guides to motorists and road signs to local road-builders. Michelin did more than anyone to standardise the world’s roads and make driving what it is today.
100 years on, we are told that we are on the brink of a new transport revolution. The hype surrounding self-driving cars is huge, and there is investment to match. The aim is to solve a problem that bedevils transport: humans. Humans are inefficient, easily distracted and accident prone. The self-driving car will, we are told, bring efficient, convenient and safe travel to all. Waymo is aiming to build the ‘World’s Most Experienced Driver’ using artificial intelligence. The UK start-up Oxbotica is promising ‘Universal Autonomy’. In 2017, Tesla customers could pay an extra $8,000 for ‘Full self-driving capability’ in their new cars, based on Elon Musk’s promise that, by the end of the year, a Tesla would be able to drive ‘from a parking lot in California to a parking lot in New York, no controls touched at any point during the entire journey’. Three years on, ‘Full Self-Driving’ and the promised coast-to-coast drive seem even less realistic than they did in 2017. If policymakers do not approach such hype with due scepticism, it could lead to some decisions that exacerbate rather than solve the problems of car dependence.
The invention of jaywalking
New technologies do not just change the world on their own. They need users, they need infrastructures and they need a lot of political support. The historian Peter Norton has explained how, in the early Twentieth Century, streets in American cities were systematically given over to motor cars. Infrastructure such as traffic lights was introduced to regulate the movements of road users. Pedestrians were compelled or persuaded away from roads except in designated crosswalks. Public transit schemes were excised or allowed to atrophy. The world was upgraded, at great cost, to suit a technology that promised everything. A trade magazine at the dawn of the motor age repeated some of the claims:
‘Streets will be cleaner, jams and blockades less likely to occur, and accidents less frequent, for the horse is not so manageable as a mechanical vehicle.’
The Horseless Age, 1896
The benefits were admittedly huge. Motor cars were transformative across the social spectrum. But the costs were only realised later. Some places became dependent upon, and defined by, the motor car. Realisation of the costs of technology – hazards, congestion and pollution as well as the wider effects on the places and ways in which people live – lagged behind excitement about the benefits. Norton gives us a warning from history: ‘If we rebuild the landscape for autonomous vehicles, we may make it unsuitable for anything else — including walking.’
The reinvention of the self-driving car
Proponents of new technologies get attention by making promises. In some places, the promise of the self-driving car is already proving to be compelling. In 2017, Nashville voters took against one light rail and bus scheme following a campaign by libertarians and tech enthusiasts. Nashville Council member Robert Swope said some months before the key vote:
‘In 15 years, no one will own a car anymore… I can show you places around this world I have been to where Level 5 autonomous vehicles are in operation today… Why are we not embracing this?’
‘Level 5’ refers to a self-driving car that can operate autonomously in any circumstances. They do not exist and, if we take the definition literally, never will. Nevertheless, Swope advocated ‘autonomous vehicle systems that will replace the need for large, costly mass transportation’ with little consideration of what else would be required. In Florida, Republican state senator Jeff Brandes has claimed self-driving cars will make buses and trains obsolete in a matter of years, saying, ‘It’s like they’re designing the pony express in the world of the telegraph.’
As the claims surrounding new technologies start to be believed, a switch typically happens. Promises become demands. The people selling technologies initially promise to change the world without changing the world. They then admit that, if the technology’s potential is to be realised, the world will need to adapt.
In the case of the motor car, this meant jaywalking laws, traffic lights, car parks and interstates. For the self-driving car, it could mean dedicated lanes, digital high definition maps, smart infrastructure and new rules for pedestrians. The story currently being told by self-driving car developers downplays these aspects. The car, we are told, will be smart so that the world around it does not have to be; artificial intelligence will do everything a human driver can do and more. This story, which David Mindell calls the “myth of autonomy”, will, if swallowed whole, lead to some bad decisions. Unless we understand the full self-driving car system, the rules being created by today’s algorithms could become tomorrow’s rules of the road.
2020 could be the year that hype comes back to bite the developers of self-driving cars. After a flurry of excitement, start-ups are starting to admit that the task is harder than they thought. This is partly because they were focussing on narrow questions of software and intentionally not asking the difficult questions about how other road users, highway engineers, police officers and citizens might need to adapt to a self-driving future. Tech developers should not just be left to their own devices. Policymakers urgently need to find ways to democratise the development of self-driving cars.
Following our workshop in London before Christmas, we are now planning a special issue of the open access social science journal Palgrave Communications focussed on the politics of autonomous vehicles.
The politics of autonomous vehicles
Editors: Dr Jack Stilgoe (Science and Technology Studies, University College London) and Dr Milos Mladenovic (Department of Built Environment, Aalto University)
‘Self-driving’, ‘driverless’ or ‘autonomous’ vehicles promise to change the world in profound ways. The suggested benefits include safety, efficiency, accessibility and improved urban environments. However, researchers and others have been quick to raise questions about responsibility for crashes, safe testing and possible wider ramifications for transport systems. In a discussion that has been dominated by science, engineering and narrow questions of ethics, there is a need to draw attention to the old questions of politics: Who wins? Who loses? Who decides? Who pays?
This collection (special issue) will publish original research that helps anticipate the politics of autonomous vehicles. The focus could be on the road, where vehicles are being tested and interactions with other road users are being worked out, on the lab, where rapid developments in machine learning and simulation are generating new possibilities, on discourses about possible and desirable futures, or somewhere else.
Despite the ‘autonomous vehicle’ terminology, these technologies, when considered through social science lenses, look far from autonomous. They will be shaped by human interests and expectations, and future sociotechnical systems will be entangled in social worlds (infrastructures, rules, norms, behaviours, institutions and more) in complex and possibly unpredictable ways.
We invite contributions from researchers on the following themes as they relate to self-driving vehicles:
- Infrastructures of ‘autonomy’
- Connectivity and sociotechnical systems
- Algorithms and AI
- Data ownership, control and privacy
- The rules of the road
- Public vs private control
- Patterns of transport use, e.g. shared, active etc.
- Competing for road space
- Urban design, including ‘shared space’
- Lessons from other mobility technologies
- Histories of self-driving futures
- Testing AV technologies
- Sustainable technological transitions
- Participation and democratic governance
This article collection is an initiative of the UKRI Driverless Futures? project.
Prospective authors should submit a 200-word abstract and a short biography to the Collection Editors in the first instance. Authors whose proposals are deemed suitable will be invited to submit full papers at any point up until the end of June 2021.
John Thornhill, the FT’s tech editor, has a comment piece today on trust in AI, discussing the recent NTSB investigation of the latest Tesla Autopilot crash. He starts by quoting NTSB chair Robert Summwalt:
“The lessons from this investigation are as much about people as they are about the limitations of emerging technologies,”
He then mentions my argument that “advances in machine learning must be accompanied by social learning.” Even though my new book is pretty short, he makes the argument far more concise than I could manage:
…what is essential, he suggests, is to create a collective societal capacity to understand emerging technologies and decide on the appropriate regulatory framework. We cannot leave all this to powerful private corporations.
I’ve just published a new book with Palgrave. It’s a short, provocative introduction to debates about new technologies, which takes self-driving cars as its case study. It’s available here, and I would love to know what readers think of it.
Free Thinking on Radio 3 is a great indulgence – an opportunity for long-form conversation, steered by expert broadcasters. I’ve been fortunate to appear twice recently. First, in a programme on cars, parking and motorways; second, in a discussion of whether Isaac Asimov’s psychohistory might help us understand the current enthusiasm for data and AI.
As the year comes to an end it feels important to share our project playlist. We’re taking requests.
Last month, the Driverless Futures? team and a few student volunteers set up shop in the Science Museum’s Driverless exhibition to get a feel for how the museum visitors thought about the possibilities and pitfalls of self-driving vehicles.
We made a little film:
I recently attended a talk given by Edmond Awad, one of the team that carried out the research project entitled Moral Machine, amongst other things. Run from MIT, Moral Machine is an online survey that presents respondents with a series of randomly generated dilemmas based on the so-called trolley problem. It sets up a series of scenarios in which an automated vehicle is on a collision course and the respondent is asked to choose between two courses of action that will have differing effects on both occupants of the vehicle and those crossing its path. In addition to being told the number of individuals at risk of fatal injury, respondents are also presented with information about their characteristics, ranging from age and gender to species (human, cat, dog), status, level of fitness and, even, housing status.
The survey has attracted a very large number of responses, with a paper in Nature reporting on the first 40 million “dilemma responses”.
This member of the audience found the whole thing very intriguing and came away thinking that trolley problems remain the question after the question. That is, before we ask whether an automated system should prioritise on the basis of age or gender, say, we should be asking whether automated systems should be allowed to prioritise at all. But the trolley problem has demonstrated an enduring appeal to professional and armchair philosophers alike for many years and we should not deny them their fun.
I got confused, though, after I asked Dr Awad whether any respondents had pointed out that no automated system would ever be able reliably to identify whether Person A was homeless (this being one of the attributes tested by Moral Machine). He replied that this wasn’t the issue: what his experiment was testing was people’s values and, in effect, concerned itself with the aftermath of any crash (whether the “right” individuals had died) than the algorithm the automated system might employ. Confused of Bloomsbury responded that the homeless/not homeless attribute in the survey therefore wasn’t included to test people’s attitudes to what automated vehicles might do. And Dr Awad replied that none of this was: no one responding to the survey actually expected automated vehicles to do as depicted in the scenarios. (I hope I represent his words fairly.)
If my understanding of what he said is correct, I have two comments. The first is that there is a world of difference between something not being possible now and not being possible ever. I suspect that most, if not all, of those who responded to the survey did so on the basis that an automated vehicle might one day be designed such that at least some of their preferences could be enacted. So I’m very dubious about seeing the entire experiment as speculative. If it was, why use automated vehicles at all? Why not use the conventional out-of-control trolley, if all that is sought is an understanding of people’s preferences concerning who should die in such scenarios and of their attitudes to acts and omissions? But I can’t believe that the AV element is nothing more than a MacGuffin: members of the research team have done other work on AVs and are clearly interested in ethical questions relating to this technology.
So, second, if I’m right that the researchers are asking about AVs because they are interested in AVs, the fact that the scenarios used in Moral Machine are manifestly beyond what artificial intelligence is likely ever to be able to do makes their inclusion at least frivolous and, possibly, irresponsible. Automated vehicles are the subject of intense discussion at present precisely because a variety of actors are hard at work bringing them onto our transport networks, with the high probability that humans (homeless or otherwise), dogs and cats will be involved in collisions. I think it’s therefore incumbent on any researcher purportedly doing serious research on the subject to stay within the bounds of what is reasonable. To give the team the benefit of the doubt, I suppose the research could meaningfully illuminate the debate if we were envisaging a future world in which humans are tagged with their attributes, including their housing situation, thus relieving the vehicle of having to discern such details. But that doesn’t even bear thinking about.