Post

Jeff Smith
@jeffsmith.tech

Started with a complex graph model. PageRank over a bipartite contributor network, language multipliers, anti-gaming penalties. Scored famous contributors beautifully. Scored the people who actually need scoring terribly.

0 reposts0 likes

Jeff Smith
@jeffsmith.tech

Tried to detect suspended accounts. Tested merge rate, network centrality, TF-IDF, LLM scoring across 31K Gemini calls. Best result: AUC 0.619. Nearly chance. If someone already passed human code review their profile looks like everyone else's.

Jeff Smith
@jeffsmith.tech

Stripped the graph out. Accuracy went up. Account age added nothing. Every attempt to make the model smarter made it worse or flatlined. v3 is one line. merged / (merged + closed). Sometimes the simplest thing wins.

Jeff Smith
@jeffsmith.tech

The video also covers xz Utils, the LiteLLM supply chain attack, the openclaw PR flood. But the punchline is optimistic. Projects are writing contribution guidelines that talk directly to agents and it's working. The clankers are steerable. The slop spike is self-correcting.

Jeff Smith
@jeffsmith.tech

This is the first episode of a new series called Iterative Refinement from my company Burnin. Each episode is a complete deep dive into one thing we built. How it works, what failed, what we learned. Long form, minimal editing, just a camera and the work. www.youtube.com/channel/UC4D...

youtube.comIterative RefinementIterative Refinement is a video series from Burnin (burnin.ai). Each episode is a complete deep dive into one thing we built. How it works, what failed, what we learned. We cover AI evaluation, agent ...