What can we learn from Little Roady?

Warren Buffet once said that it’s only when the tide goes out that you see who’s swimming naked. It’s a phrase that is being repeated a lot at the moment, despite it being nonsensical: who swims until the tide goes out around them? Regardless, the gist is that a company that may be able to survive while the economy is rising will have its flaws revealed when a recession hits. Self-driving car companies have benefited from huge investments in the last five years or so. Covid and its accompanying recession are likely to lead to bankruptcies, consolidations and corporate fire sales. However, I wonder if Buffet’s maxim is holding. Maybe the companies that survive will not be those with the best swimwear, but those who are most self-confident, regardless of their nudity. (Sorry. The analogy may have already snapped). This could be bad for innovation and bad for local transport.

We have already seen the demise of Starsky Robotics, a company that was trying to make self-driving technology safe and effective in a well-defined context: trucks on big roads. Other companies are now feeling the heat. A recent Venturebeat piece describes some of the troubles faced by May Mobility, whose approach to AV innovation I have admired for a while.

A year ago, I took a train from Boston to Providence, Rhode Island to ride in one of May Mobility’s prototype self-driving shuttles. ‘Little Roady’ was a year-long collaboration with the Rhode Island Department of Transportation to provide a free shuttle running a five-mile, ten-stop lap of the town. The body of the vehicle was a sort of low-speed, electric stretch golf cart. The brain and the sensors were added by May Mobility with the hope of eventually enabling driverless transport.

The publicity describes this as an autonomous vehicle, with the person in the driving seat there to monitor things, talk to passengers and take over if needs be. The reality is more complicated. The two shuttles I tried – one out, one back – revealed plenty about the possibilities and the limits of self-driving technology.

Station to station

The shuttle stop is right outside the train station. I only have to wait a couple of minutes. The first shuttle is far from autonomous. Our driver stays in manual mode, steering the thing with its handlebars. She seems uninterested in pushing the envelope and is keen to talk about what the vehicle can’t yet do. It can’t turn across traffic and it can’t enter a road at a junction without traffic lights. A few hundred yards away from the train station, we have to stop because there’s a street parade. Self-driving car engineers call such things ‘edge cases’ – new experiences, unaccounted for in their model.

An interruption from a street parade
Shuttle stop and route map
Web site with real-time shuttle info

I hop off near Olneyville Square, get some lunch and take a wander. I check the web site to see when the next shuttle is on its way.

Shuttle two is a bit closer to the technological sublime. This driver lets me ride up front with him. He puts the car in ‘auto’ and shows me the little green ‘A’ lighting up on his screen. He says the aim is to keep it in auto as much as possible, to train as well as test the machine. The company is hungry for data, not least to show the Rhode Island department of transportation how well they’re doing.

A is for Auto

When we come to a four-way stop junction, he doesn’t have to take over. He tells me that the sensors can see the other cars’ indicators. The computer identifies a gap and the vehicle takes its opportunity with aplomb. He admits that the system finds things easier in light Saturday traffic. Maybe the four-way stop would have been trickier in rush hour.

He says the system is getting better all the time. It used to overreact when someone was tailgating or be disabled when a road sign was partly obscured by a tree branch. He says it still isn’t great with bicycles. The last bit of the loop was covered in roadworks, so the driver switches to manual before we get there.

Just as we approach the train station, which marks the end of the loop, a picture of red traffic light appears on screen. I ask whether the vehicle sees the colour or detects it in some other way. The driver points to a box on a lamppost that transmits the signal from the light to the vehicle. ‘Autonomous’ vehicles, we should remember, are never completely autonomous.

A traffic light signal transmitter

Learning by doing

The Little Roady pilot has been cut short by Covid. This is a shame, as it seemed to be a genuine public experiment. In many cases, ‘trials’ of self-driving vehicles are not real tests of the technology; they are tests of public acceptance or just public displays of a technology. If failure is not an option, then little can be learnt. Other companies have tightly choreographed their ‘trials’, protecting them from technical and social complexity with safety marshals for pedestrians or non-disclosure agreements for passengers. (Noortje Marres and Declan Mcdowell-Naylor have been studying the sociology of self-driving car trials).

May Mobility were trying something different. They were trying to see if they could make an early technology work in the real world, for a real use case, with real people. No good deed, it seems, goes unpunished. The company found that some of its most frustrating glitches were with old rather than new tech. In hot weather the shuttles’ air conditioning had problems and in cold weather their batteries failed. Venturebeat criticised May for targetting “fixed-route transportation needs in geofenced, easily mappable business districts, campuses, and closed residential communities.” The criticism that “not a single one of the company’s commercial routes approached full autonomy” prompts us to ask what would count as “full autonomy”. May was attempting to serve two masters: a local transportation service with a set of requirements about air conditioning and a Wizard of Oz public discourse about autonomous vehicles. At some point, other companies will face similar reckonings. For now, however, I worry that companies will take the wrong lesson from May Mobility and keep telling grand stories rather than trying modest experiments.

It’s not a race

I was once in a band called ‘It’s a Race!’ The daft the name seemed to fit. It captured the carefree pointlessness of our music. We were guilty of some of the worst jam-band excesses. Our improvised, 10-minute songs could not cover for our lack of rehearsal or musical mediocrity. We didn’t know where we were going. To say we split up due to artistic differences is to give us too much credit.

A lot of self-driving car journalism uses the metaphor of a ‘race’. In a recent episode of the Autonocast, Ed Niedermeyer (a recent and welcome addition to Partners for Automated Vehicle Education) calls it horse-race journalism. He is rightly critical of this style. There is no point talking about a race if we don’t know where the finish line is.

Niedermeyer’s target is a recent Bloomberg piece on the “The State of the Self-Driving Car Race 2020”. This gets off to a bad start, with an analogy to the space race and a quote from a consultant: 

I mean, it’s literally like putting somebody on the moon. It’s that complex

It literally isn’t like putting somebody on the moon. Putting somebody on the moon, ironically, was not rocket science. That problem was hard, but it was not complex. According to Brenda Zimmerman and colleagues, some problems are simple, like following a recipe; some are complicated, like rocketry; others, like raising a child, are complex. The easy things about putting a person on the moon were that most people agreed on what success would look like and that there was nothing in the way. The hard thing – how to take an object containing a person such a vast distance and bring it back – could be solved with enough brains and enough money. Many terrestrial problems, including climate change, obesity and mobility, are wicked. Their is little agreement on the definition of problems, approaches or metrics for success and there are lots of organisations, interests and structures standing in the way. 

For systems comprising self-driving cars to work effectively, safely and fairly in a range of different places, much of what needs to happen lies beyond the control of the competitors in the self-driving car ‘race’. This is one reason why the technology’s success will take far longer than the hype currently suggests

The trouble with the Bloomberg piece, as with so much self-driving car journalism, is that it presumes the finish line is clear. If we are to make good decisions about this technology, we must first recognise that there are many different possible directions. We need to start talking about the desirable directions and working out how to bend current approaches to fit.

Self-Driving Cars and the Politics of Innovation – Interview with ICTC

On April 8th 2020, ICTC (the Canadian Information and Communications Technology Council) spoke with Dr. Jack Stilgoe, Senior Lecturer in the Department of Science & Technology Studies at University College London, where he researches and teaches the governance of emerging technologies. Dr. Stilgoe is the Principal Investigator of the Driverless Futures? Project, a three-year social science project looking at the governance of self-driving cars.

Full interview here

How to resist self-driving car hype

This article was originally published by Pando Daily

New technologies do not just change the world on their own. They need users, they need infrastructures and they need a lot of political support.

The invention of driving

At the Victoria and Albert Museum in London, in the middle of an exhibition on car culture, sits a grid of sepia photographs from the early Twentieth Century. The pictures, taken by travelling salesmen from Michelin, show places from Cape Town to Clermont-Ferrand (the company’s home town). Some are of smooth highways. Others show tracks dotted with potholes and boulders. 

In 1921 André Michelin and his brother Édouard had turned their expertise in bicycle tires towards the motor car, a technology that was exploding in popularity. The Michelin Roulement Universale was a tire designed to be sold around the world, but the world’s roads were of varying quality. Their road photography project was an exercise in market research, but it was also an act of lobbying. Michelin wanted improvements to the world’s highways, and they wanted them fast.

The Michelin brothers were never just rubbersmiths. While Edouard managed the practical side of the business, André handled the public relations. For French motorists and people the world over, they did something more profound than invent tires; they invented the road trip. Andre’s genius was to manufacture demand for the technology that would take his company forward. According to the historian Georges Ribeill, it was André who “first understood the technical system of the automobile which was to result from the “automobile revolution,” and of which tires were only one of the elementary working parts.” This system included the machines, fuel, maps, road networks, habits and rules that would allow for the dramatic expansion in people’s horizons in the early Twentieth Century. In 1900, André created the guide that would make his company famous and become the yardstick against the world’s restaurants were judged. Even though there were fewer than 3,000 automobiles in the whole of France at the time, 35,000 Michelin guides were produced and handed out for free. The company made the first civilian maps and gave away route guides to motorists and road signs to local road-builders. Michelin did more than anyone to standardise the world’s roads and make driving what it is today.

100 years on, we are told that we are on the brink of a new transport revolution. The hype surrounding self-driving cars is huge, and there is investment to match. The aim is to solve a problem that bedevils transport: humans. Humans are inefficient, easily distracted and accident prone. The self-driving car will, we are told, bring efficient, convenient and safe travel to all. Waymo is aiming to build the ‘World’s Most Experienced Driver’ using artificial intelligence. The UK start-up Oxbotica is promising ‘Universal Autonomy’. In 2017, Tesla customers could pay an extra $8,000 for ‘Full self-driving capability’ in their new cars, based on Elon Musk’s promise that, by the end of the year, a Tesla would be able to drive ‘from a parking lot in California to a parking lot in New York, no controls touched at any point during the entire journey’. Three years on, ‘Full Self-Driving’ and the promised coast-to-coast drive seem even less realistic than they did in 2017. If policymakers do not approach such hype with due scepticism, it could lead to some decisions that exacerbate rather than solve the problems of car dependence. 

The invention of jaywalking

New technologies do not just change the world on their own. They need users, they need infrastructures and they need a lot of political support. The historian Peter Norton has explained how, in the early Twentieth Century, streets in American cities were systematically given over to motor cars. Infrastructure such as traffic lights was introduced to regulate the movements of road users. Pedestrians were compelled or persuaded away from roads except in designated crosswalks. Public transit schemes were excised or allowed to atrophy. The world was upgraded, at great cost, to suit a technology that promised everything. A trade magazine at the dawn of the motor age repeated some of the claims:

‘Streets will be cleaner, jams and blockades less likely to occur, and accidents less frequent, for the horse is not so manageable as a mechanical vehicle.’
The Horseless Age, 1896

The benefits were admittedly huge. Motor cars were transformative across the social spectrum. But the costs were only realised later. Some places became dependent upon, and defined by, the motor car. Realisation of the costs of technology – hazards, congestion and pollution as well as the wider effects on the places and ways in which people live – lagged behind excitement about the benefits. Norton gives us a warning from history: ‘If we rebuild the landscape for autonomous vehicles, we may make it unsuitable for anything else — including walking.’

The reinvention of the self-driving car

Proponents of new technologies get attention by making promises. In some places, the promise of the self-driving car is already proving to be compelling. In 2017, Nashville voters took against one light rail and bus scheme following a campaign by libertarians and tech enthusiasts. Nashville Council member Robert Swope said some months before the key vote: 

‘In 15 years, no one will own a car anymore… I can show you places around this world I have been to where Level 5 autonomous vehicles are in operation today… Why are we not embracing this?’

‘Level 5’ refers to a self-driving car that can operate autonomously in any circumstances. They do not exist and, if we take the definition literally, never will. Nevertheless, Swope advocated ‘autonomous vehicle systems that will replace the need for large, costly mass transportation’ with little consideration of what else would be required. In Florida, Republican state senator Jeff Brandes has claimed self-driving cars will make buses and trains obsolete in a matter of years, saying, ‘It’s like they’re designing the pony express in the world of the telegraph.’

As the claims surrounding new technologies start to be believed, a switch typically happens. Promises become demands. The people selling technologies initially promise to change the world without changing the world. They then admit that, if the technology’s potential is to be realised, the world will need to adapt. 

In the case of the motor car, this meant jaywalking laws, traffic lights, car parks and interstates. For the self-driving car, it could mean dedicated lanes, digital high definition maps, smart infrastructure and new rules for pedestrians. The story currently being told by self-driving car developers downplays these aspects. The car, we are told, will be smart so that the world around it does not have to be; artificial intelligence will do everything a human driver can do and more. This story, which David Mindell calls the “myth of autonomy”, will, if swallowed whole, lead to some bad decisions. Unless we understand the full self-driving car system, the rules being created by today’s algorithms could become tomorrow’s rules of the road.

2020 could be the year that hype comes back to bite the developers of self-driving cars. After a flurry of excitement, start-ups are starting to admit that the task is harder than they thought. This is partly because they were focussing on narrow questions of software and intentionally not asking the difficult questions about how other road users, highway engineers, police officers and citizens might need to adapt to a self-driving future. Tech developers should not just be left to their own devices. Policymakers urgently need to find ways to democratise the development of self-driving cars.

Call for papers: The politics of autonomous vehicles

Following our workshop in London before Christmas, we are now planning a special issue of the open access social science journal Palgrave Communications focussed on the politics of autonomous vehicles.

The politics of autonomous vehicles

Editors: Dr Jack Stilgoe (Science and Technology Studies, University College London) and Dr Milos Mladenovic (Department of Built Environment, Aalto University) 

‘Self-driving’, ‘driverless’ or ‘autonomous’ vehicles promise to change the world in profound ways. The suggested benefits include safety, efficiency, accessibility and improved urban environments. However, researchers and others have been quick to raise questions about responsibility for crashes, safe testing and possible wider ramifications for transport systems. In a discussion that has been dominated by science, engineering and narrow questions of ethics, there is a need to draw attention to the old questions of politics: Who wins? Who loses? Who decides? Who pays?  

This collection (special issue) will publish original research that helps anticipate the politics of autonomous vehicles. The focus could be on the road, where vehicles are being tested and interactions with other road users are being worked out, on the lab, where rapid developments in machine learning and simulation are generating new possibilities, on discourses about possible and desirable futures, or somewhere else. 

Despite the ‘autonomous vehicle’ terminology, these technologies, when considered through social science lenses, look far from autonomous. They will be shaped by human interests and expectations, and future sociotechnical systems will be entangled in social worlds (infrastructures, rules, norms, behaviours, institutions and more) in complex and possibly unpredictable ways.   

We invite contributions from researchers on the following themes as they relate to self-driving vehicles:

  • Infrastructures of ‘autonomy’
  • Connectivity and sociotechnical systems
  • Algorithms and AI
  • Data ownership, control and privacy
  • The rules of the road
  • Public vs private control
  • Patterns of transport use, e.g. shared, active etc.
  • Competing for road space
  • Urban design, including ‘shared space’
  • Lessons from other mobility technologies
  • Histories of self-driving futures
  • Testing AV technologies
  • Sustainable technological transitions
  • Participation and democratic governance 

This article collection is an initiative of the UKRI Driverless Futures? project.  

Prospective authors should submit a 200-word abstract and a short biography to the Collection Editors in the first instance. Authors whose proposals are deemed suitable will be invited to submit full papers at any point up until the end of June 2021.

Trusting AI too much can turn out to be fatal – FT Comment

John Thornhill, the FT’s tech editor, has a comment piece today on trust in AI, discussing the recent NTSB investigation of the latest Tesla Autopilot crash. He starts by quoting NTSB chair Robert Summwalt:

“The lessons from this investigation are as much about people as they are about the limitations of emerging technologies,”

He then mentions my argument that “advances in machine learning must be accompanied by social learning.” Even though my new book is pretty short, he makes the argument far more concise than I could manage:

…what is essential, he suggests, is to create a collective societal capacity to understand emerging technologies and decide on the appropriate regulatory framework. We cannot leave all this to powerful private corporations.