Except there’s a difference between a machine killing somebody because it was programmed to and a person killing somebody on accident. One of those things has people making decisions who are not going to be held responsible.

The other problem is it creates openings for malicious actors: if your government (or even Saudi Arabia, or Israel) for instance wanted to kill a political dissident they could add a self erasing line of code to a car to run over a specific person.

This is why self driving laws need to be explicit about how they’re approaching this otherwise you’re inviting in a lot of suspicious behavior by amoral companies. There needs to be safeguards on how and who has access to self driving code.

I would say that source code for any self driving or autonomous machine in a public street should be held by insurance companies or a third party who performs regular validation checks on vehicle codes (which could be read and validated at charging stations or gas stations) and it should only be edited by publicly licensed software engineers whose licenses can be revoked for bad behavior.

Anything less is inviting a series of predictable public safety fiascos.