Adi Mukherjee

Adi Mukherjee

@adim.in

Software Engineer, currently on sabbatical in Japan. Prev: Apple SRE. Working on something new.

41 followers53 following10 posts

Adi Mukherjee reposted

fuckin cool

Bryan Boatman@bryanboatman.bsky.social

Direct hit on the Sunset Fire in Hollywood. 🎯 Nice work, @lafd.bsky.social #LAFD c/o @nbcla.bsky.social 👨🏼‍🚒

Great blog covering the progress this year. “Asking o1 to complete proofs in creative ways is effectively asking it to be a research colleague. The model doesn't have to get proofs right to be useful, it just has to help us be better researchers.” Good example of utility that evals fail to capture.

Ethan Mollick@emollick.bsky.social

Bluesky can be a fraught place to post about AI but it is worth noting that the buzz over o1 (& now o3) is not “hype.” We know o1 can actually do some very hard tasks (see my post) & o3 appears to represent a big further leap. They aren’t AGI, but will matter. www.oneusefulthing.org/p/what-just-...

Adi Mukherjee reposted

Benchmarks are flawed but a way to trace AI over the last year is GPQA Diamond. This is a Google-proof question set that experts get 81% right in their fields & highly skilled non-experts with 30 minutes per question and Google use get 22% GPT-4 got 37% at the start of 2024. o1 got 78%. o3 is 87.7%

Adi Mukherjee reposted

I wish people would post more links to interesting things I feel like Twitter and LinkedIn and Instagram and TikTok have pushed a lot of people out of the habit of doing that, by penalizing shared links in the various "algorithms" Bluesky doesn't have that misfeature, thankfully!

View full thread

In the ARC AGI eval (linked article in the first post), the ‘high compute’ mode results came from spending ~$350K in total on inference, giving the model more compute to search the solution tree.

These models excel at reasoning-heavy tasks like coding, summarisation, and can work through PhD-level problems with sufficient test time compute. Unlike their predecessors (4o/3.5-sonnet), these reasoning models get ‘smarter’ with inference compute.

You are up to date.