Red Kill Switch for AI Autonomous Systems May Not be a Life Saver
By Lance Eliot, The AI Trends Insider We all seem to know what a red stop button or kill switch does. Whenever you believe that a contraption is going haywire, you merely reach for the red stop button or kill switch and shut the erratic gadgetry down. This urgent knockout can be implemented via a […]
By Lance Eliot, The AI Trends Insider
We all seem to know what a red stop button or kill switch does.
Whenever you believe that a contraption is going haywire, you merely reach for the red stop button or kill switch and shut the erratic gadgetry down. This urgent knockout can be implemented via a bright red button that is pushed, or by using an actual pull-here switch, or a shutdown knob, a shutoff lever, etc. Alternatively, another approach involves simply pulling the power plug (literally doing so or might allude to some other means of cutting off the electrical power to a system).
Besides utilizing these stopping acts in the real-world, a plethora of movies and science fiction tales have portrayed big red buttons or their equivalent as a vital element in suspenseful plot lines. We have repeatedly seen AI systems in such stories that go utterly berserk and the human hero must brave devious threats to reach an off-switch and stop whatever carnage or global takeover was underway.
Does a kill switch or red button really offer such a cure-all in reality?
The answer is more complicated than it might seem at first glance. When a complex AI-based system is actively in progress, the belief that an emergency shutoff will provide sufficient and safe immediate relief is not necessarily assured.
In short, the use of an immediate shutdown can be problematic for myriad reasons and could introduce anomalies and issues that either do not actually stop the AI or might have unexpected adverse consequences.
Let’s delve into this.
AI Corrigibility And Other Facets
One gradually maturing area of study in AI consists of examining the corrigibility of AI systems.
Something that is corrigible has a capacity of being corrected or set right. It is hoped that AI systems will be designed, built, and fielded so that they will be considered corrigible, having an intrinsic capability for permitting corrective intervention, though so far, unfortunately, many AI developers are unaware of these concerns and are not actively devising their AI to leverage such functionality.
An added twist is that a thorny question arises as to what is being stopped when a big red button is pressed. Today’s AI systems are often intertwined with numerous subsystems and might exert significant control and guidance over those subordinated mechanizations. In a sense, even if you can cut off the AI that heads the morass, sometimes the rest of the system might continue unabated, and as such, could end-up autonomously veering from a desirable state without the overriding AI head remaining in charge.
Especially disturbing is that a subordinated subsystem might attempt to reignite the AI head, doing so innocently and not realizing that there has been an active effort to stop the AI. Imagine the surprise for the human that slammed down on the red button and at first, could see that the AI halted, and then perhaps a split second later the AI reawakens and gets back in gear. It is easy to envision the human repeatedly swatting at the button in exasperation as they seem to get the AI to quit and then mysteriously it appears to revive, over and over again.
This could happen so quickly that the human doesn’t even discern that the AI has been stopped at all. You smack the button or pull the lever and some buried subsystem nearly instantly reengages the AI, acting in fractions of a second and electronically restarting the AI. No human can hit the button fast enough in comparison to the speed at which the electronic interconnections work and serve to counter the human instigated halting action.
We can add to all of this a rather scary proposition too: suppose the AI does not want to be stopped.
One viewpoint is that AI will someday become sentient and in so doing might not be keen on having someone decide it needs to be shut down. The fictional HAL 9000 from the movie 2001: A Space Odyssey (spoiler alert) went to great lengths to prevent itself from being disengaged.
Think about the ways that a sophisticated AI could try to remain engaged. It might try to persuade the human that turning off the AI will lead to some destructive result, perhaps claiming that subordinated subsystems will go haywire.
The AI could be telling the truth or might be lying. Just as a human might proffer lies to remain alive, the AI in a state of sentience would presumably be willing to try the same kind of gambit. The lies could be quite wide-ranging. An elaborate lie by the AI might be to convince the person to do something else to switch off the AI, using some decoy switch or button that won’t truly achieve a shutdown, thus giving the human a false sense of relief and misdirecting efforts away from the workable red button.
To deal with these kinds of sneaky endeavors, some AI developers assert that AI should have built-in incentives for the AI to be avidly willing to be cut off by a human. In that sense, the AI will want to be stopped.
Presumably, the AI would be agreeable to being shut down and not attempt to fight or prevent such action. An oddball result though could be that the AI becomes desirous of getting shut down, due to the incentives incorporated into the inner algorithms to do so and thus wanting to be switched off, even when there is no need to do so. At that point, the AI might urge the human to press the red button and possibly even lie to get the human to do so (by professing that things are otherwise going haywire or that the human will be saved or save others via such action).
One viewpoint is that those concerns about AI will only arise once sentience is achieved. Please be aware that today’s AI is not anywhere near to becoming sentient and thus it would seem to suggest that there aren’t any near-term qualms about any kill-switch or red button trickery from AI. That would be a false conclusion and a misunderstanding of the underlying possibilities. Even contemporary AI, as limited as it might be, and as based on conventional algorithms and Machine Learning (ML), could readily showcase similar behaviors as a result of programming that intentionally embedded such provisions or that erroneously allowed for this trickery.
Let’s consider a significant application of AI that provides ample fodder for assessing the ramifications of a red button or kill-switch, namely, self-driving cars.
Here’s an interesting matter to ponder: Should AI-based true self-driving cars include a red button or kill switch and if so, what might that mechanism do?
For my framework about AI autonomous cars, see the link here: https://aitrends.com/ai-insider/framework-ai-self-driving-driverless-cars-big-picture/
Why this is a moonshot effort, see my explanation here: https://aitrends.com/ai-insider/self-driving-car-mother-ai-projects-moonshot/
For more about the levels as a type of Richter scale, see my discussion here: https://aitrends.com/ai-insider/richter-scale-levels-self-driving-cars/
For the argument about bifurcating the levels, see my explanation here: https://aitrends.com/ai-insider/reframing-ai-levels-for-self-driving-cars-bifurcation-of-autonomy/
Understanding The Levels Of Self-Driving Cars
As a clarification, true self-driving cars are ones where the AI drives the car entirely on its own and there isn’t any human assistance during the driving task. These driverless vehicles are considered a Level 4 and Level 5, while a car that requires a human driver to co-share the driving effort is usually considered at a Level 2 or Level 3. The cars that co-share the driving task are described as being semi-autonomous, and typically contain a variety of automated add-on’s that are referred to as ADAS (Advanced Driver-Assistance Systems). There is not yet a true self-driving car at Level 5, and we don’t yet even know if this will be possible to achieve or how long it will take to get there.
Meanwhile, the Level 4 efforts are gradually trying to get some traction by undergoing very narrow and selective public roadway trials, though there is controversy over whether this testing should be allowed per se (we are all life-or-death guinea pigs in an experiment taking place on our highways and byways, some contend).
Since semi-autonomous cars require a human driver, the adoption of those types of cars won’t be markedly different than driving conventional vehicles, so there’s not much new to cover about them on this topic (though, as you’ll see in a moment, the points next made are generally applicable).
For semi-autonomous cars, it is important that the public needs to be forewarned about a disturbing aspect that’s been arising lately, namely that despite those human drivers that keep posting videos of themselves falling asleep at the wheel of a Level 2 or Level 3 car, we all need to avoid being misled into believing that the driver can take away their attention from the driving task while driving a semi-autonomous car.
You are the responsible party for the driving actions of the vehicle, regardless of how much automation might be tossed into a Level 2 or Level 3.
For why remote piloting or operating of self-driving cars is generally eschewed, see my explanation here: https://aitrends.com/ai-insider/remote-piloting-is-a-self-driving-car-crutch/
To be wary of fake news about self-driving cars, see my tips here: https://aitrends.com/ai-insider/ai-fake-news-about-self-driving-cars/
The ethical implications of AI driving systems are significant, see my indication here: http://aitrends.com/selfdrivingcars/ethically-ambiguous-self-driving-cars/
Be aware of the pitfalls of normalization of deviance when it comes to self-driving cars, here’s my call to arms: https://aitrends.com/ai-insider/normalization-of-deviance-endangers-ai-self-driving-cars/
Self-Driving Cars And The Red Button
For Level 4 and Level 5 true self-driving vehicles, there won’t be a human driver involved in the driving task. All occupants will be passengers; the AI is doing the driving.
Some pundits have urged that every self-driving car ought to include a red button or kill-switch. There are two major perspectives on what this capability would do. First, one purpose would be to immediately halt the on-board AI driving system. The rationale for providing the button or switch would be that the AI might be faltering as a driver and a human passenger might decide it is prudent to stop the system.
For example, a frequently cited possibility is that a computer virus has gotten loose within the onboard AI and is wreaking havoc. The virus might be forcing the AI to drive wantonly or dangerously. Or the virus might be distracting the AI from effectively conducting the driving task and doing so by consuming the in-car computer hardware resources intended for use by the AI driving system. A human passenger would presumably realize that for whatever reason the AI has gone awry and would frantically claw at the shutoff to prevent the untoward AI from proceeding.
The second possibility for the red button would be to serve as a means to quickly disconnect the self-driving car from any network connections. The basis for this capability would be similar to the earlier stated concern about computer viruses, whereby a virus might be attacking the on-board AI by coming through a network connection.
Self-driving cars are likely to have a multitude of network connections underway during a driving journey. One such connection is referred to as OTA (Over-The-Air), an electronic communication used to upload data from the self-driving car into the cloud of the fleet, and allows for updates and fixes to be pushed down into the onboard systems (some assert that the OTA should always be disallowed while the vehicle is underway, but there are tradeoffs involved).
Let’s consider key points about both of those uses of a red button or kill-switch. If the function entails the focused aspect of disconnecting from any network connections, this is the less controversial approach, generally. Here’s why.
In theory, a properly devised AI driving system will be fully autonomous during the driving task, meaning that it does not rely upon an external connection to drive the car. Some believe that the AI driving system should be remotely operated or controlled but this creates a dependency that bodes for problems.
Imagine that a network connection goes down on its own or otherwise is noisy or intermittent, and the AI driving system could be adversely affected accordingly. Though an AI driving system might benefit from utilizing something across a network, the point is that the AI should be independent and be able to otherwise drive properly without a network connection. Thus, cutting off the network connection should be a design capability and for which the AI driving system can continue without hesitation or disruption (i.e., however, or whenever the network connection is no longer functioning).
That being said, it seems somewhat questionable that a passenger will do much good by being able to use a red button that forces a network disconnect.
If the network connection has already enabled some virus to be implanted or has attacked the on-board systems, disconnecting from the network might be of little aid. The on-board systems might already be corrupted anyway. Furthermore, an argument can be made that if the cloud-based operator wants to push into the on-board AI a corrective version, the purposeful disconnect would then presumably block such a solving approach.
Also, how is it that a passenger will realize that the network is causing difficulties for the AI?
If the AI is starting to drive erratically, it is hard to discern whether this is due to the AI itself or due to something regarding the networking traffic. In that sense, the somewhat blind belief that the red button is going to solve the issue at-hand is perhaps misleading and could misguide a passenger when needing to take other protective measures. They might falsely think that using the shutoff is going to solve things and therefore delay taking other more proactive actions.
In short, some would assert that the red button or kill switch would merely be there to placate passengers and offer an alluring sense of confidence or control, more so as a marketing or selling point, but the reality is that they would be unlikely to make any substantive difference when using the shutoff mechanism.
This also raises the question of how long would the red button or kill switch usage persist?
Some suggest it would be momentary, though this invites the possibility that the instant the connection is reengaged, whatever adverse aspects were underway would simply resume. Others argue that only the dealer or fleet operator could reengage the connections, but this obviously could not be done remotely if the network connections have all been severed, therefore the self-driving car would have to be ultimately routed to a physical locale to do the reconnection.
Another viewpoint is that the passenger should be able to reengage that which was disengaged. Presumably, a green button or some kind of special activation would be needed. Those that suggest the red button would be pushed again to re-engage are toying with an obvious logically confusing challenge of trying to use the red button for too many purposes (leaving the passenger bewildered about what the latest status of the red button might be).
In any case, how would a passenger decide that it is safe to re-engage? Furthermore, it could become a sour situation of the passenger hitting the red button, waiting a few seconds, hitting the green button, but then once again using the red button, doing so in an endless and potentially beguiling cycle of trying to get the self-driving car into a proper operating mode (flailing back-and-forth).
Let’s now revisit the other purported purpose of the kill-switch, namely, to stop the on-board AI.
This is the more pronounced controversial approach, here’s why. Assume that the self-driving car is going along on a freeway at 65 miles per hour. A passenger decides that perhaps the AI is having trouble and slaps down on the red button or turns the shutoff knob.
What happens?
Pretend that the AI instantly disengages from driving the car.
Keep in mind that true self-driving cars are unlikely to have driving controls accessible to the passengers. The notion is that if the driving controls were available, we would be back into the realm of human driving. Instead, most believe that a true self-driving car has only and exclusively the AI doing the driving. It is hoped that by having the AI do the driving, we’ll be able to significantly reduce the 40,000 annual driving fatalities and 2.5 million related injuries, based on the aspect that the AI won’t drive drunk, won’t be distracted while driving, and so on.
So, at this juncture, the AI is no longer driving, and there is no provision for the passengers to take over the driving. Essentially, an unguided missile has just been engaged.
Not a pretty picture.
Well, you might retort that the AI can stay engaged just long enough to bring the self-driving car to a safe stop. That sounds good, except that if you already believe that the AI is corrupted or somehow worthy of being shut off, it seems dubious to believe that the AI will be sufficiently capable of bringing the self-driving car to a safe stop. How long, for example, would this take to occur? It could be just a few seconds, or it could take several minutes to gradually slow down the vehicle and find a spot that is safely out of traffic and harm’s way (during which, the presumed messed-up AI is still driving the vehicle).
Another approach suggests that the AI would have some separate component whose sole purpose is to safely bring the self-driving car to a halt and that pressing the red button invokes that specific element. Thus, circumventing the rest of the AI that is otherwise perceived as being damaged or faltering. This protected component though could be corrupted, or perhaps is hiding in waiting and once activated might do worse than the rest of the AI (a so-called Valkyrie Problem). Essentially, this is a proposed solution that carries baggage, as do all the proposed variants.
Some contend that the red button shouldn’t be a disengagement of the AI, and instead would be a means of alerting the AI to as rapidly as possible bring the car to a halt.
This certainly has merits, though it once again relies upon the AI to bring forth the desired result, yet the assumed basis for hitting the red button is due to suspicions that the AI has gone akilter. To clarify, having an emergency stop button that is there for other reasons, such as a medical emergency of a passenger, absolutely makes sense, and so the point is not that a stop mode is altogether untoward, only that to use it for overcoming the assumed woes of the AI itself is problematic.
Note too that the red button or kill switch would potentially have different perceived meanings to passengers that ride in self-driving cars.
You get into a self-driving car and see a red button, maybe it is labeled with the word “STOP” or “HALT” or some such verbiage. What does it do? When should you use it?
There is no easy or immediate way to convey those particulars of those facets to the passengers. Some contend that just like getting a pre-flight briefing while flying in an airplane, the AI ought to tell the passengers at the start of each driving journey how they can make use of the kill switch. This seems a tiresome matter, and it isn’t clear whether passengers would pay attention and nor recall the significance during a panic moment of seeking to use the function.
For more details about ODDs, see my indication at this link here: https://www.aitrends.com/ai-insider/amalgamating-of-operational-design-domains-odds-for-ai-self-driving-cars/
On the topic of off-road self-driving cars, here’s my details elicitation: https://www.aitrends.com/ai-insider/off-roading-as-a-challenging-use-case-for-ai-autonomous-cars/
I’ve urged that there must be a Chief Safety Officer at self-driving car makers, here’s the scoop: https://www.aitrends.com/ai-insider/chief-safety-officers-needed-in-ai-the-case-of-ai-self-driving-cars/
Expect that lawsuits are going to gradually become a significant part of the self-driving car industry, see my explanatory details here: http://aitrends.com/selfdrivingcars/self-driving-car-lawsuits-bonanza-ahead/
Conclusion
In case your head isn’t already spinning about the red button controversy, there are numerous additional nuances.
For example, perhaps you could speak to the AI since most likely there will be a Natural Language Processing (NLP) feature akin to an Alexa or Siri, and simply tell it when you want to carry out an emergency stop. That is a possibility, though it once again assumes that the AI itself is going to be sufficiently operating when you make such a verbal request.
There is also the matter of inadvertently pressing the red button or otherwise asking the AI to stop the vehicle when it was not necessarily intended or perhaps suitable. For example, suppose a teenager in a self-driving car is goofing around and smacks the red button just for kicks, or someone with a shopping bag filled with items accidentally leans or brushes against the kill-switch, or a toddler leans over and thinks it is a toy to be played with, etc.
As a final point, for now, envision a future whereby AI has become relatively sentient. As earlier mentioned, the AI might seek to avoid being shut off.
Consider this AI Ethics conundrum: If sentient AI is going to potentially have something similar to human rights, can you indeed summarily and without hesitation shut off the AI?
That’s an intriguing ethical question, though for today, not at the top of the list of considerations for how to cope with the big red button or kill-switch dilemma.
The next time you get into a self-driving car, keep your eye out for any red buttons, switches, levers, or other contraptions and make sure you know what it is for, being ready when or if the time comes to invoke it.
As they say, go ahead and make sure to knock yourself out about it.
Copyright 2021 Dr. Lance Eliot. This content is originally posted on AI Trends.
[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column: https://forbes.com/sites/lanceeliot/]
