Post

I still haven't worked out how to get LLMs to avoid adding increasingly more complexity to projects. The projects I tend to want them to work on are like multi threaded C++ projects, so maybe I should just stick to getting them to write little python tools to make my life simpler.

I don't have any relevant expertise with multi-threaded C++, I would be surprised to hear they're much less effective at that The most important trick I know is making sure they have a test suite they can run - most of my Claude Code sessions start with me saying "first run the tests like this:"

27/11/25, 8:54 pm0 reposts25 likes

Tim Fish

@timfish.dev

Mael (Slava Ukraini🇺🇦) (🫶 women in 🇮🇷)

My experience is that they're quite shit for c++. Test cannot really confirm that you don't get UB in all circumstances due to threading issues so it's difficult to confirm it hasn't created you something that only definitely works for the test cases

@maelp.bsky.social

I find it perfectly believable that they're much less suited at that. When you do multi-threading and concurrency, you're basically trying to have your programs "encode proofs" (that there will be no race-conditions, etc). And LLMs are notoriously bad at maths

simonwillison.netdeepseek-ai/DeepSeek-Math-V2New on Hugging Face, a specialist mathematical reasoning LLM from DeepSeek. This is their entry in the space previously dominated by proprietary models from OpenAI and Google DeepMind, both of …

Not any more! bsky.app/profile/simo...

Simon Willison@simonwillison.net

DeepSeek-Math-V2 means we now have an open weights (Apache 2) model that can achieve gold medal performance on this year's International Mathematical Olympiad - previously proprietary models from OpenAI and Google DeepMind had achieved that 689GB from Hugging Face! simonwillison.net/2025/Nov/27/...

@petervandijck.bsky.social

I asked Google (no quotes): "how good is ai at writing c++ multithread code". Google's reply included limitations like this: "Multithreaded programming [involves]... issues like data races, deadlocks, and race conditions. AI models may struggle to identify and prevent these subtle bugs..."

Peter Van Dijck

That’s not a question an LLM will have a good answer to, even just simply because of the training cutoff date

simonwillison.netGoogle search hallucinates Encanto 2Jason Schreier on Bluesky: I was excited to tell my kids that there's a sequel to Encanto, only to scroll down and learn that Google's AI just completely made this …

Google's AI overviews are still the worst widely available AI models! I just checked and it's STILL saying Encanto 2 is a real movie, that bug is nearly a year old now simonwillison.net/2024/Dec/29/...

Pekka Lund

@pekka.bsky.social

@stevepowell99.bsky.social

It says "There is no officially confirmed Encanto 2 movie" for me. Even though all the top Google results below and next to it are YouTube videos and Reddit posts etc. indicating it exists or is confirmed to be coming.

Steve Powell

29 Nov 2025

Yes why is that? I use LLMs all day, for everything, but those AI summaries on Google search are verifiably absolute pants

29 Nov 2025

My guess I'd it's a cost thing - the Google SERP is one of the highest traffic pages on the internet and needs to respond really fast, so they have to use a small, cheap but weak model for it

29 Nov 2025

Well, Google's "AI Overview" is apparently good enough to get this right: 😉 '"Absolute pants" is British slang meaning something is complete and utter rubbish, nonsense, or very bad. It's a strong way to express that something is worthless, bad, or of poor quality.'

Are you saying you disagree with: "Multithreaded programming [involves]... issues like data races, deadlocks, and race conditions. AI models may struggle to identify and prevent these subtle bugs..."? I wouldn't think that was a controversial statement.

I think "AI models may struggle to identify and prevent these subtle bugs" isn't a particularly useful statement - it's safe, but it's avoiding the more interesting question of how capable the latest generation of models actually are

Karsten Konrad 🇪🇺

@zabong69.bsky.social

Google‘s search LLM is the village idiot of current models.

Google says this about it's current search LLM: "Google Search uses a version of the Gemini 3 model optimized for search queries. This is Google's most advanced model to date. It has enhanced reasoning, coding abilities, and multimodal understanding." (Gemini 3 just released last week.)

Karsten Konrad 🇪🇺

@zabong69.bsky.social

Models come in different sizes, with the smaller ones being far more economical. I suspect a fast, downsized model with heavy quantisation to handle the traffic. It certainly doesn’t use thinking mode.

In theory, I can envision a "code-understanding" system that could prevent race conditions (which are notoriously hard for humans to see). But I'm skeptical we'll see that delivered, in reliable form, any time soon.

I have a strong hunch that the best current "reasoning" models - GPT-5 Thinking, Gemini 3, Claude Opus 4.5 - could be very effective at identifying potential race conditions But I'm not enough of a threading programmer myself to credibly put them to the test!

OK, hmm. I'd be interested to hear from people who have tried them for that fairly demanding task.

Alan Paxton

@alanpx.bsky.social

30 Nov 2025

I can see how 100000000 working react programs on GitHub can guide LLMs to generate another one, but maintaining all the highly implicit invariants in something like RocksDB when adding an enhancement ? Different worlds.