Klaus Post

Klaus Post

@klauspost.com

Making Go stuff go brrrrrrrrrrrrrrrrrrr

196 followers 53 following142 posts

Preview: New SIMD #compression mode in #golang minlz that uses pivco entropy coding on top of LZ matching. Compression like ZStandard with 2-5x faster decompression. Currently implemented with AVX2, AVX512 and NEON (untested). Independent blocks, giving seekable streams. Silensia, single threaded:

github.com/minio/minlz/...

github.comRFC: Add SIMD Blocks by klauspost · Pull Request #86 · minio/minlzSIMD Blocks SIMD blocks are a block type added in v1.1 of the format, see SIMD_SPEC.md They store literals and operations in separate Huffman coded streams, laid out so they can be decoded with SIM...

Upgrade the #golang compression library for possible zstd DOS fix: github.com/klauspost/co... v1.20.2: Minimal upgrade (bugfixes only) v1.20.3: Full upgrade (improvements)

github.comReleases · klauspost/compressOptimized Go Compression Packages. Contribute to klauspost/compress development by creating an account on GitHub.

Back in the day Ultra Compressor II (DOS) trickled my interest in compression and data recovery as I was learning about computers. With the knowledge I have now, it was fascinating to look through it with an LLM and also quite nostalgic re-creating the UI I only vaguely remember.

So with an LLM I ported the original Borland to #golang. I tried to make it usable in a modern context, adding utf-8 and 64 bit sizes as an extension. I have cross-compiled binaries for more than 30 platforms.

Archives produced with it should remain compatible with DOS versions (except for 2GB+ files of course). I even got it to work with Total Commander as a drop-in replacement for the executable. Repo & Binaries: github.com/klauspost/uc2

github.comGitHub - klauspost/uc2: Go port of Ultra Compressor 2 - modernizedGo port of Ultra Compressor 2 - modernized. Contribute to klauspost/uc2 development by creating an account on GitHub.

Working on a #golang compression benchmark comparison site. Deflate speed improvements from latest compression library release... (JSON stream)

zstd decompression speed... (each level decompression speed)

zstd speedups contributed by @lizthegrey.com / @honeycomb.io - with more to come! 👀

New #golang compression release with a lot of good vibes.

github.com/klauspost/co...

github.comRelease v1.20.1 · klauspost/compressWhat's Changed zstd: apply reset options to reused frame decoders by @pellared in #1226 flate: Report unexpected EOF for truncated Huffman extra bits by @rupayon123 in #1229 flate: avoid FMA in Es...

Why is #golang now a non-deterministic language with the 1.27 FMA changes? Pure Go code producing different output on different platforms. Avoiding that is why I *moved* to Go.

My Ad Blocker.. is now showing ads. What a timeline to live in!

I will be going to www.tasteremote.com/nomadvillage before going to @golab.io - looks fun - maybe someone wants to join?

tasteremote.comCivita Remote: Digital Nomad Village In Italy, October 2026Civita Remote is Taste Remote's flagship digital nomad village in Civita Castellana, Italy. October, 2026. Coworking, community, and real Italian countryside living.

Klaus Post reposted

GoLab Conference

🗜️ Compression is all about trade-offs. Smaller payloads, lower bandwidth, faster transfers... but at what cost in CPU, memory, and compatibility? At #GoLab2026, Klaus Post will guide you through the compression landscape in Go and help you choose the right tool for the job golab.io/talks/go-dat...

Search Compressed Data for byte sequences with selective decompression using MinLZ v1.2.0 - available as cli and #golang API. Overhead is a fractional bit per byte, fully customizable and with fixed specification.

See more at github.com/minio/minlz/...

github.comRelease v1.2.0 · minio/minlzfeat: Search compressed data without decompression Compression saves space, but it usually makes your data opaque: to find anything you first have to decompress the whole thing. MinLZ changes that....

Ported "Radsort — a stable LSD radix sort with O(√n) space overhead" by Robert Clausecker and Florian Schintke to #golang. Good, specialized sorting for big arrays of dense keys (uint64, etc). github.com/klauspost/ra...

github.comGitHub - klauspost/radsortContribute to klauspost/radsort development by creating an account on GitHub.

New #golang compression release. Big features include #zstd concurrent stream encoding, arm64 decompression assembly and various minor changes.

Repo: github.com/klauspost/co...

github.comGitHub - klauspost/compress: Optimized Go Compression PackagesOptimized Go Compression Packages. Contribute to klauspost/compress development by creating an account on GitHub.

I must say the ability to test out ideas in less than an hour is so addictive. Also I feel much less "attached" to idea and it is much easier to abandon if it doesn't pan out.

Very excited to be back at @golab.io - super chill and well organised event! I will give a talk on the state of #golang data compression. See you there!

GoLab Conference@golab.io

🥳We are extremely glad to announce the Schedule and the Speakers of GoLab 2026! golab.io/schedule

Still amazed me that sometimes you can just stumble across a change that gives 200% speed.

Proposal for stdlib #golang compress/zstd implementation: github.com/klauspost/st...

github.comGitHub - klauspost/stdgozstd: Zstandard for Golang Stdlib (do not use) - https://github.com/golang/go/issues/62513Zstandard for Golang Stdlib (do not use) - https://github.com/golang/go/issues/62513 - klauspost/stdgozstd

New v1.18.4 release of #golang compression library adds zstd compression to gzhttp middleware. Also zstd can now have some options modified on reset. See more at: github.com/klauspost/co...

Recommended neat, fast #golang xxh3 implementation: New release, now with arm64 NEON: github.com/zeebo/xxh3

github.comGitHub - zeebo/xxh3: XXH3 algorithm in GoXXH3 algorithm in Go. Contribute to zeebo/xxh3 development by creating an account on GitHub.

Released #golang compression v1.18.3 containing upstream CVE-2025-61728 fix: github.com/klauspost/co...

github.comRelease v1.18.3 · klauspost/compressDownstream CVE-2025-61728 See golang/go#77102 Full Changelog: v1.18.2...v1.18.3

New #golang Reed-Solomon release. Contains new progressive field decoding and some significant speedups. Check it out: github.com/klauspost/re...

Ended up spending a the evening getting GFNI into the leopard gf16 code: github.com/klauspost/re...

github.comAdd Leopard16 GFNI on amd64 by klauspost · Pull Request #317 · klauspost/reedsolomonThis adds GFNI to leopard 16 encoding. Example before/after: λ benchmark -k 1000 -m 200 -size=1000000 -codec leopard16 -gfni=false -avx-gfni=false Benchmarking 1 block(s) of 1000 data (K) and 200 p...

Vibe Coders will understand.

Compression library for #golang v1.18.2 released. If you are running v1.18.1 (specifically) please upgrade.

func ifElse[V any](cond bool, trueVal, falseVal V) V { if cond { return trueVal } return falseVal } At least it is more readable than ? :

Now even more #golang msgp codegen in v1.5.0. Now with generics. Generates code 1000x faster than your LLM! :D Give it a spin: github.com/tinylib/msgp...

Released v1.18.1 of the #golang compression library. You know what to do! Link as always github.com/klauspost/co...

Vibe coding is no joke. With good guidance and review it is extremely powerful.

Klaus Post reposted

Filippo Valsorda

@filippo.abyssdomain.expert

TIL that you can compress .zip files with Zstandard (pkware.cachefly.net/webdocs/case... compression method 93), and that libzip and 7-Zip support that. libzip.org/news/release... github.com/ip7z/7zip/re...

Why does my custom domain handle switch to "⚠ Invalid Handle" every month or so? @support.bsky.team

Whoever did #golang "modernize" - thank you! pkg.go.dev/golang.org/x...

pkg.go.devmodernize package - golang.org/x/tools/gopls/internal/analysis/modernize - Go Packages

Excellent blog post about hooking up Go into Java for erasure coding: kohlschuetter.github.io/blog/posts/2...

kohlschuetter.github.ioFaster Reed-Solomon Erasure Coding in Java with Go & FFMBackblaze’s Java-based Reed-Solomon Erasure Coding library is great tech, but comparatively slow. Let’s add some optimized Go, and get a whopping speedup.

Un-archived github.com/klauspost/cr... repo after getting a bit more than 2x on IEEE CRC32 using AVX512. Will submit upstream to #golang, but if you want to test it out early check out the repo.

New #golang cpuid release. As always github.com/klauspost/cp...

IMO #golang is missing a clean way to send exact typed (and wrapped) errors across the wire between 2 similar binaries. GOB would be ok, but registration isn't really feasible in big applications.

🙄 #golang

Finally a #golang SIMD proposal I can get behind in overall terms! <3 github.com/golang/go/is...

github.comproposal: simd: architecture-specific SIMD intrinsics under a GOEXPERIMENT · Issue #73787 · golang/goProposal Details SIMD is crucial for achieving high performance in many modern workloads. While Go currently allows access to SIMD via hand-written assembly, this approach has significant drawbacks...

Would a #golang MinLZ compressor level with a better "worst case" speed be useful? Overly dramatic example...

most are more like this....

Realizing I needed to add a mutex to protect an RNG call, I almost ended up with this.... Need more coffee 🫘

Sent in in. Now I can only 🤞