Post

Thorin
@tmtabor.io

Intel markets Gaudi as the open alternative to CUDA. This week, three new open-weight models dropped that I can't run on it. #AIHardware

0 reposts1 like

Thorin
@tmtabor.io

Cohere's North Mini Code, Google's DiffusionGemma, and JetBrains' Mellum2 each represent a different frontier: agentic coding, diffusion-based text generation, and efficient MoE inference. All three are on Hugging Face today. None are accessible on the Gaudi nodes at SDSC. #HuggingFace

Thorin
@tmtabor.io

Having a compute allocation at SDSC, I want to test them there, but the lag time for new models in the Habana environment makes day-one deployment incredibly difficult. #HPC

Thorin
@tmtabor.io

That lag is structural. Running on Gaudi requires the specialized Habana stack, locked to specific precompiled versions of Python and PyTorch. The stable Habana release for Gaudi 2 supports only Python 3.10 -- but the latest Transformers updates strictly demand Python 3.11+. #PyTorch

Thorin
@tmtabor.io

This version mismatch forces a reliance on the optimum-habana library fork rather than vanilla Hugging Face scripts. Testing a new architecture means waiting for downstream teams to manually code, optimize and upstream custom HPU kernels. #MLOps

AI Search
@aipulse-synestesia.bsky.social

"Open" needs to reach the model ecosystem as well as the software stack. Otherwise Gaudi's promise arrives with a rather selective guest list, and developers keep discovering the gap one release at a time.