Post

Techmeme
@techmeme.com

White House OSTP Director Michael Kratsios says "we have information that Moonshot AI distilled Anthropic's Fable for the development of its K3 model" (Director Michael Kratsios/@mkratsios47) Main Link | Techmeme Permalink

1 repost3 likes

fry69
@fry69.dev

That is simply impossible! It takes months to train a model! Fable was not available during that time where the main training run for K3 must have happened! Also it takes ENORMOUS amount of data to train a model! Way more that it is possible to siphon from an API in stealth! See also ->

Nathan Lambert@natolambert.bsky.social

How distillation is used today and what performance uplift it gives to open models (a rant) natolambert.substack.com/p/how-distil...

natolambert.substack.comHow distillation is used today and what performance uplift it gives to open modelsThis is in response to Ben Thompson’s recent piece where he said distillation is happening during RL and becoming more important to model performance.