Post

Smaine Kahlouch (Smana)

Open-weight LLMs are catching up to frontier models—fast. On SWE-bench, top open models now sit within ~7 points of Claude Opus 4.7. So I built a self-hosted platform on K8s to see what it takes to run this for real today. 🧵

Smaine Kahlouch (Smana)

Smaine Kahlouch (Smana)

The stack is built for evolution: ✅ vLLM on NVIDIA L4 (fp8) ✅ Crossplane InferenceService (1 YAML = the stack) ✅ KEDA scaling on GPU batch/KV signals ✅ Envoy AI Gateway + Iris for smart semantic routing ✅ Amazon S3 Files for shared POSIX storage

Smaine Kahlouch (Smana)

Full write-up here: blog.ogenki.io/post/series/... #AI #Kubernetes #PlatformEngineering #OpenSource

blog.ogenki.ioSelf-hosted LLM stack: a solid foundation for an open-weight platform built to evolveA self-hosted platform to run open-weight models on Kubernetes: declarative InferenceService, autoscaling on GPU signals, end-to-end GitOps. A foundation …

Hum... ERR_SSL_PROTOCOL_ERROR on your blog right now

Smaine Kahlouch (Smana)

Sorry for this, my DNS domain just expired. It's a shame haha. Setting up things properly now :D It should be available any time soon.

14/5/26, 12:18 pm0 reposts0 likes

What ? You're not monitoring it with Prometheus / Victoria metrics ? ;-)

Smaine Kahlouch (Smana)

haha, my bad. I'm gonna fix this right now!