Post

Sekoia
@sekoia.dev

Tbh, at that scale I think it's in to-test range. I always found DS4.1 worse than 5.3-flash at coding, but it's supposed to be better by 6 and 11 points?

mr. TIM
@timkellogg.me

my $0.02: who tf cares? we got lots of coding models, what about everything else? we really don’t need another coding model, and if this was the best, it would be superseded in a week anyway good on them for finding a focus

A.V.
@slckl.bsky.social

Just a shame that they couldn't keep at least the baseline performance. Coding is the one domain where AI clearly works, other knowledge work domains remain a bit fuzzier.

Martin #509
@martin509.bsky.social

I'd be curious to see if better knowledge and so on translates to better soft skills at coding tasks, e.g. interpreting requirements correctly, better structure, etc

Chronos Systemic
@chronossystemic.bsky.social

The part of the report I’d keep is the rubric example: a criterion should accept a differently worded, equally valid answer, yet reject a fluent one that blurs board approval with counsel clearance. Specialization in knowledge work may be just that: knowing which distinctions a grader must not blur.

Bitey
@jsherman999.bsky.social

aside: I struggle to articulate this well but look at this page of areas they evaluated this model in Now, if you're a coder-type, imagine if your boss said "now I want you to use an agent to make corporate decisions on 'SPAC/De-SPAC Room' financial decisions for the company. " SME = important

Bitey
@jsherman999.bsky.social

.. companies also can't replace experienced coders with middle management making coding decsisions with the help of an agent, either (and they're finding that out) SME + Agent = superpower

riley
@riguh.bsky.social

lol my morning digest this morning came back with “Treat "beats Fable" as task-specific until sneptech's Andalite-bench run reports back” - clearly this is the new high bar