Anson

Anson

@ansonbiggs.com

Writing software to make a safer, autonomous future. Formerly Blue Origin and ULA. Send me your blog ansonbiggs.com

159 followers478 following67 posts

Anson reposted

Yoz Grahame
@yoz.bsky.social

Replying to @yoz.bsky.social

if you think software generated by LLMs is inherently worse than code typed by humans, then please sit down, because i'm about to give you some terrible news: you have considerably overestimated how good we, as a species, are at creating reliable software

Anson reposted

Simone Conradi
@sconradi.bsky.social

A fair coin "knows" π. Stop flipping when heads first outnumber tails. The expected fraction of heads is π/4. Average many trials → estimate π. Result by J. Propp, animated for #PiDay. Made with #Python #NumPy #Matplotlib and @marimo.io #Mathematics #MathArt #CreativeCoding

Anson reposted

business goose🪿
@goose.art

torrents will download at impressive speeds until of course you get to the last megabyte at which point they slow to 1 byte per second as they scour the entire world for the rare file fragments lost to the sands of time

Anson reposted

Anson's Notes
@ghost.notes.ansonbiggs.com.ap.brid.gy

But thankfully for you I'm smart enough to show you how its done

notes.ansonbiggs.comYou're probably using Agent Skills wrongThe entire ecosystem around Claude Code is pretty confusing, the naming conventions are a mess and the pace of change is beyond any production tool I've seen. However Skills are probably the most misused. I see it at work at ton but a paper just came up on Hacker News: SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse TasksAgent Skills are structured packages of procedural knowledge that augment LLM agents at inference time. Despite rapid adoption, there is no standard way to measure whether they actually help. We present SkillsBench, a benchmark of 86 tasks across 11 domains paired with curated Skills and deterministic verifiers. Each task is evaluated under three conditions: no Skills, curated Skills, and self-generated Skills. We test 7 agent-model configurations over 7,308 trajectories. Curated Skills raise average pass rate by 16.2 percentage points(pp), but effects vary widely by domain (+4.5pp for Software Engineering to +51.9pp for Healthcare) and 16 of 84 tasks show negative deltas. Self-generated Skills provide no benefit on average, showing that models cannot reliably author the procedural knowledge they benefit from consuming. Focused Skills with 2--3 modules outperform comprehensive documentation, and smaller models with Skills can match larger models without them.arXiv.orgXiangyi Li The HN title is editorialized for some reason _"Study: Self-generated Agent Skills are useless"_ , but it immediately grabbed me since I get massive value from Skills written by Agents, but I also consistently see them misused by my peers. The concept is great, I've been looking at benchmarking specific parts of the Agentic ecosystem myself so this was highly relevant to me. Overall the paper is decent but one bullet invalidates the whole thing: > **Self-Generated Skills** : No Skills provided, but the agent is prompted to generate relevant procedural knowledge before solving the task. This isolates the impact of LLMs’ latent domain knowledge. So all they are doing is taking a problem that a model can't solve well on its own, and asking it to write about the task before attempting it. They just reinvented thinking blocks but worse! ### The Skill Anti-Pattern What they did is a very common mistake that I see constantly. My Agent it bad at this thing so I ask the Agent to write a skill on this thing. I'll reiterate this is identical to thinking blocks. In order for your Agent to create something worthwhile you have to make sure they can see the gaps. I see this as the classic CS intro where you ask someone to write out the steps to make a PB&J, you don't really understand what makes the problem hard until you've struggled through solving it. This directly leads into the largest Faux Pas of the AI era, just asking a LLM someone elses question verbatim, and pasting the LLMs answer as your response. If I ask you how you did something cool with an Agent, and you just on the fly have a fresh Agent build me a SKILL.md on my question, _I will kill you._ ## What are Skills Before getting into proper usage, I just want to cover what skills are. As a primitive they are just markdown files that have some metadata at the top to help Agents/Tools know when to use them, and then the rest of the document is the skill. Each skill has its own folder so it can no only teach your Agent how to do something but also give it better tools. .claude/skills/ └── monitor-gitlab-ci/ ├── SKILL.md # The file metioned above ├── monitor_ci.sh # Complicated command └── references/ # Additional references ├── api_commands.md ├── log_analysis.md └── troubleshooting.md Above is a Skill I used a ton to let older versions of Claude work on my GitLab CI. It's a folder with a simple markdown Skill that just explained the setup and that the Agent needs to watch the CI until either a job fails or everything passes, a simple CLI to prevent the Agent from writing a script, and additional references for edge cases. ## Skills for Context Agents are completely stateless meaning that every new conversation is like meeting the model for the first time, it has no idea what your project is or what you were working on 10 minutes ago. CLAUDE.md does a lot to fix this, but for a large enough project it can't contain everything. If I open up a monorepo and tell Claude to run a SIL test then it is going to have to run around to figure out how to do that. It has to figure out what language the project is in, then look for common test patterns for that language, its going to see a complicated Docker Compose setup, its going to see that the containers need x86 but we're running on a Mac, then its going to look for CI, etc. This can all be solved by writing Skills for common, but not universal patterns. A good habit is having Claude explain bespoke parts of your project as a Skill while you implement. ## Skills for Hard Problems Claude can solve some really hard problems, but it might take $500 in tokens and you might have to yell at it for reward hacking a few times. Almost any time I have to intervene on a problem, once the Agent it unstuck I ask it what the gap was that kept it from figuring it out on its own. Sometimes it something silly, but sometimes it is something genuinely insightful and I have Claude make a Skill to fill the gap. ## Conclusion I edited the original benchmark to do Skills my way and the results were incredible, the Agents nailed the test with proper Skills. I don't have the money to spend on fully validating this result but the first pass was good enough for me to be happy. I assume this changes the benchmark enough that it was too hard to do properly so the Authors opted to do it wrong. Make sure when you create a Skill the Agent knows something that the base model doesn't. This can even happen during the creating of the Skill, but the Agent must have the oppurtunity to understand the issue it is solving before it can solve it better than a base model. Happy Hacking.
Anson
@ansonbiggs.com

Agent teams are pretty cool but it’s hilarious how often the orchestration Claude gets mad that things are taking so long

Anson reposted

Derek Newsome
@derek.space

There have been five years in our history where a rocket that would carry humans to the Moon stood on a launch pad. 1968, 1969, 1970, 1971, 1972 But now, there are six.

Anson reposted

Bluesky
@bsky.app

Unfortunately, Bluesky is unavailable in Mississippi right now, due to a new state law that requires age verification for all users. While intended for child safety, we think this law poses broader challenges & creates significant barriers that limit free speech & harm smaller platforms like ours.

Anson reposted

Ben Holmes
@bholmes.dev

Everyone: No, GitHub Desktop is better! No, VS Code! Me: ...I just use `git add -p`. Walks you through each change, hunk for hunk, and lets you accept with `y`. It’s like a mini code review before you commit. Pair that with dandavison/delta, and you get syntax highlighting too 👀

Anson reposted

Kagi HQ
@kagi.com

We are committed to surfacing content from the Small Web in our search results as well. The small web represents a return to the internet's roots: a space of personal expression, creativity, and community-driven content 👇 kagi.com/smallweb

Roberth@robstr.dev

I tried the @kagi.com small web feature for the first time, and on the third result I got a new post about the latest version of SuperTuxKart, and this just blew my mind. I totally forgot this thing even existed, and to find that it is under active development? Some things truly never change 😅

Anson reposted

NASAWATCH
@nasawatch.bsky.social

This is the #RIF that NASA can expect to endure - a reduction from 17,391 to 11,853 - so ... who will the unlucky 5,538 people be? Meanwhile let's gut the #NASA budget while simultaneously starting a crash program to beat China to #Mars - #EmbraceTheChallenge y'all

Anson reposted

ACLU
@aclu.org

NEW: We're asking a court to expand our clients' recent legal victory to ensure that all transgender, non-binary, and intersex people can be issued passports with the correct gender marker.

Anson
@ansonbiggs.com

and remember you’re better at writing alt text than an AI!

Mike Masnick@masnick.com

As Bluesky has grown, I've seen a lot of backsliding on alt text. Please try to be considerate to others and include alt text with all of your images. It helps everyone. Indeed, in the "accessibility" setting in Bluesky you can set it to make sure you add alt text, which is a useful thing to do.