Post

When I say "outputs of the machine" I don't mean every intermediate step it takes, I mean the overall outcomes it produces in the way it changes the world around it. I wrote about the SQL case specifically too - acknowledging that yes, sometimes it writes the SQL wrong:

tilmonedwards.comAgents will delete your dataThe problem with giving an agent direct access to your data is that sometimes it will accidentally delete all of it.

Tilmon Edwards (Dirigo)

@tilmonedwards.com

This is systems resiliency stuff, which is very much my way of working and so maybe that's why this seems very clear to me. Engineering around an actor who makes mistakes is not new. A resilient system can catch those mistakes before they cause a problem in the outputs.

Tilmon Edwards (Dirigo)

@tilmonedwards.com

The argument I'm making is almost tautological, to be fair. If you give an agent direct access to query a database, and in the course of its queries the agent deletes your data, then you gave bad inputs. The database access tools you choose to give it are part of the I/O loop.

Tilmon Edwards (Dirigo)

@tilmonedwards.com

The machine cannot be blamed or held accountable. It doesn't even operate unless someone chooses to operate it, or someone chooses to allow it to operate itself. That choice is the root of accountability for the outcomes of the machines. If it's doing the wrong thing, fix it, that's the whole job.

15/10/26, 6:31 pm0 reposts1 like

Tilmon Edwards (Dirigo)

@tilmonedwards.com

Actually, revising my previous statement: The machine does not operate unless someone chooses to operate it, or someone builds additional machinery that enables it to operate itself. There's no "allow" in the sense of granting permission. These are constructions.