Post

It's hard for me to fully comprehend that there are people who believe LLMs are strictly useless in programming and/or strictly harm the quality of code/product. Makes me want to write a blog post about it... Various scattered ideas and notes off the top of my head (*not fact checked*):

I myself have never vibe-coded, my biggest ever LLM usage in relation to code is generating a body of a small function. Vibe-coding from scratch gives shit results for obvious reasons - LLMs just predict the next most likely symbol (simplified) based on their training data. Garbage in - garbage out.

So no matter how long and hard you train an LLM, the average output quality is going to hover around the average quality of the training data, maybe slightly above. Model weights are based on frequency (read "popularity") of input text (token chains) (simplified).

That is why LLMs can for example reproduce popular Wikipedia articles pretty accurately, sometimes exactly. Wikipedia is frequently quoted and copied, so it occurs pretty frequently in training data, which carves it into the LLM's memory more and more accurately, gives it more "weight" (simplified).

Average quality of all code ever? Even just all code on GitHub? It's shit. Not extremely bad but pretty damn bad. An LLM can only reproduce that, so the output is obviously low quality code too. Power law, few popular repositories with high quality code vs many many low quality repositories, etc...

And model context plays into the quality too. Asking it to continue low quality text obviously makes low quality output more likely because in training data low quality text is usually surrounded with more low quality text. Having a bad codebase nerfs a model too, no matter how hard they "align" it.

The value proposition for vibe-coding from a developer's POV just doesn't make sense. Vibe-coding discourages ever going in to do something manually, basically locking them into vibe-coding the codebase, unless a developer wants to waste enormous amounts of time to understand garbage code (hard).

Of course, if the only thing that matters is shipping that code, to maximize shareholder value, and the business is pushing for "vibe-coded now, fix later/never", then it does make sense to vibe-code. Sacrifice ever achieving good for quickly achieving a working meh most of the time.