A power outage in San Francisco resulted in traffic lights at an intersection going out and Waymo cars were left stranded in the middle of the intersection without knowing how to navigate their way out of the situation. As a result, emergency vehicles were blocked.
Waymo has said that while the autonomous vehicles deal with dark traffic signals as four way stops, its vehicles are designed to perform a confirmation check to ensure it makes the safest decision. Given the scale of the blackout which affected roughly 130 000 residents, this created an unusually massive amount of checks that Waymo’s servers could not handle, resulting in idle cars.
When AI fails, who or what is accountable? Should programmers be held liable or be absolved of all responsibility, saying that it was not their fault and rather a technical failure of a machine, or it was the user’s mistake in operating the system. When aeroplane crashes happen, they can be attributed to either human error or system failures. Should such a view be applied as well to AI systems? When AI infrastructure fails, there are consequences – these can span from a minor hallucination of data or in the case of the San Francisco blackout incident, life threatening.
When considering accountability of AI systems, we have to look at 4 aspects.
Micro: At the individual level, did programmers design the tool responsibly or did the user operate the tool in an appropriate manner? Casting blame on the individual level may seem a little harsh. The development of AI systems are not perfect and will never be, there will be bound to be some sort of oversight or programming bug introduced by human error. Unless it is found that there was indeed gross negligence in the development process, perhaps criminal proceedings against individual programmers or groups of programmers would be a bit too heavy a consequence. If criminal proceedings were handed out, it would perhaps deter people from entering the AI industry and innovation gets stifled as a process.
Meso: Should corporate institutions be liable and face charges or lawsuits against them? Shifting the blame from an individual to the corporation or business entity itself could perhaps provide a greater sense of accountability. Did the company adhere to appropriate guard rails? Additionally, corporations can bear the burden of criminal lawsuits against them, providing compensation for the injured parties.
Macro: Have government regulations passed adequate oversight of the technology on a national level? Did legislators balance protection of rights vs profits? For instance, in Singapore, the Infocomm Media Development Authority(IMDA) provides oversight regulations on the AI industry in Singapore, developing the Model AI Governance framework giving practical guidance on issues like fairness, explainability, transparency, and accountability.
Meta: Here, we look at global institutions like the UN or the Vatican and their frameworks for ethical AI development. For instance, the UNESCO Global Forum on the ethics of AI and the Vatican’s Note on the Relationship Between Artificial Intelligence and Human Intelligence provides a framework for developing AI systems for the common good of humanity and that which protects the dignity of the individual.
Accountability of AI systems spans across these 4 aspects and there is an amalgamation of responsibility shared among the 4 tiers of society. Many stakeholders are involved, with as many differing interests. But if we can get these 4 tiers of society to work together and establish a framework that is just, equitable and transparent, it would definitely pave the way for a world with safer AI.
In Waymo’s case, it was a lesson they had to learn and they are already working towards implementing future solutions. Waymo is currently rolling out fleet-wide updates that give their vehicles more information on regional outages and how to handle such situations more intelligently. Waymo is also updating its first responder training, training emergency responders how to interact with Waymo. They have already trained 25000 emergency responders and are looking to expand their reach.