Post

1. Interrogation/interview-based design and specification 2. Make the decisions that matter and leave the rest to the agent. 3. Use agent to generate explainers, quizzes, and simulations. 4. Use verifiers and guardrails in the coding loop.

Interview-based design and spec: When I tell the agent what I want to build or extend, there's naturally ambiguity. But the agent knows what is ambiguous to it, so it can ask me. Closing that specification gap is how you get better results from your agents.

Make the decisions that matter: Not every decision matters in a product. Agents are fast at writing code and pretty good at some of the nitty gritty details, so I let it make those decisions. I make decisions on architecture, approach, and stack that will help differentiate.

Generating explainers, quizzes, and simulations: Asking for explainers helps quite a bit, though I have a custom skill for it. Agents aren't great at it out of the box. I also have it quiz me to help retain the concepts I learned from explainers.

0 reposts0 likes

I also do a form of tactical simulation game via @ejames_c where agents and I will discuss an explainer and discuss the pros and cons of options generated by subagents for the design decision under question.

Verifiers and Guardrails: Guardrails don't change with the feature, like typecheckers, linters, and pre-commit hooks. Verifiers are different for different features, such as unit/integration tests, formal method proofs, design verifiers.

Still getting the hang of building my own verifier. Importantly, verifiers get me out of the coding loop. When the agent can check for itself, I don't have to stay up at 2am begging it, "you fixed this, but broken that. pls fix."

a verifier you built is a permission slip for sleep. when the agent can check itself against something that isn't its own opinion, you stop being the test suite.

I've been asking engineers if it's still important to understand the code they're shipping. The opinions are split for different reasons, and it's also a matter of degree. What's your take? Is it still important to understand what you're shipping?