Bernd Wachter

About a year ago I was wondering if the "no matrix client with encryption for emacs" could be solved by emulating telegas architecture. I've used matrix rust sdk to get encryption bits working, and eventually built a full client on top. Anonymised screenshot here. I've been using it as my main

client for a while, and in the last few days switched to moving it as my only client on most devices. Problem now is just that I didn't really expect it to become useful, so I developed it inside .emacs.d, and called it matrix.el - so next up would be renaming that thing, and stripping out any hard

I've been trying to guide LLMs into interactive Emacs usage for a while, but so far they always refused. Yesterday evening during an unrelated experiment it realised that tmux is in its container, and can be used to instrument Emacs, cutting me out of the feedback loop. Asked it to document it.

I've been using someone else's page for my daily feeds for more than two decades now, but recently had a need to add more custom stuff. While I do use elfeed it sucks for ephemeral news - where I only care if they're in the feed while I'm checking. So here's my easy to consume feed in emacs:

Seems feeds nowadays are also reasonably well formed - last time I was doing manual feed parsing in perl about 20 years ago more than 50% of the feeds needed special handling as they contained bad data. This thing now just pulls headlines from atom or rss, displays up to 12, allows showing

So what I was doing there was trying to have a way to do inference from Emacs, with multiple backends. I've started with llama.cpp, and initially added sd.cpp for diffusion to proof that my approach can handle multiple different inference engines. For a lot of things sd.cpp and llama.cpp are

rust bit I was originally playing with a segment loaded via the .so, and using remote objects to where the actual work happens to emulate in-process behavior while protecting Emacs from crashes and memory ballooning. So far it looks more practical to have a simpler "daemon to talk to via sockets"

So what I was doing there was trying to have a way to do inference from Emacs, with multiple backends. I've started with llama.cpp, and initially added sd.cpp for diffusion to proof that my approach can handle multiple different inference engines. For a lot of things sd.cpp and llama.cpp are

probably fine - very good performance, decent model support, so we can do a lot of things with minimal effort. For testing new stuff or more obscure models that's not sufficient, though. So the next addition would be a rust bit exposing a bunch of candle methods as elisp. Adding that lead to a

This turned out to be way more interesting problem than originally anticipated.

I'm guessing that by offering number ranges for it to judge, and then pretty much doing binary search we should be able to reliable get to the correct number pretty fast - so that'll be the next experiment.

This turned out to be way more interesting problem than originally anticipated.

we should listen to the word judge, and where we should trust the generated result.

This turned out to be way more interesting problem than originally anticipated.

flags the result as incorrect in the end - but re-try will just generate 9 again. So we now clearly need different strategies depending on what we want to be generated - which again is where it shines. So before we do anything we give it a bunch of options what type of content it thinks is wanted.

This turned out to be way more interesting problem than originally anticipated.

judging the sentence so far including backtracking, and adding in mechanisms to pretty much feed in complete dictionaries by letting it judge letter groups the next word should start with. While that improves text output a lot it ends up breaking some other stuff, like, the math bits.

This turned out to be way more interesting problem than originally anticipated.

fallback to char level generation (I can give it at most 255 candidates for judging). That indeed does produce better results, and we can improve it a lot by adding judges in again: it doesn't have high confidence when it is forced to generate stuff, but it has very high confidence when judging it.

This turned out to be way more interesting problem than originally anticipated.

First a short explanation what I'm doing there: yev just judges probabilities. So for the above screenshot it just judged the probability of the next character (it has a-z and 0-9 as inputs) - and got the math right that way, but messed up with the "is". Everything pretty much expected so far.

And that'd be Anthropics Sonnet 4.6. Extremely fast, but costs $2.71, and is not very impressive, so unless @anthropic.com is in the business of sponsoring tokens for weird stuff I'll stick to other models.

Your stable diffusion toolchain delivers too good results in too little time? I can help. unhinged diffusion provides the harness needed to make any vision enabled, tool using LLM cosplay as stable diffusion in a canvas enabled Emacs.

Now it entered refining phase. As it needs to rely on text, and do image recognition in each step (unlike sd, which only does conversion at the very end) we allow it to draw circles, rectangles and lines to compensate a bit for the disadvantage.

Your stable diffusion toolchain delivers too good results in too little time? I can help. unhinged diffusion provides the harness needed to make any vision enabled, tool using LLM cosplay as stable diffusion in a canvas enabled Emacs.

It starts getting confident in the overall composition.

Your stable diffusion toolchain delivers too good results in too little time? I can help. unhinged diffusion provides the harness needed to make any vision enabled, tool using LLM cosplay as stable diffusion in a canvas enabled Emacs.

For the first few steps it gets told to focus on the overall composition.

Your stable diffusion toolchain delivers too good results in too little time? I can help. unhinged diffusion provides the harness needed to make any vision enabled, tool using LLM cosplay as stable diffusion in a canvas enabled Emacs.

You too can have it draw cat pictures like this in as little as 30 minutes. That is, 30 minutes model runtime, not setup time. Head over to github for code and more details: github.com/aard-fi/unhi...

I guess being referred to as "beautifully unhinged" by google AI when using it to research something for an idea I've been working on for a few days is a compliment?

In case somebody here is using opensuse tumbleweed or microos, and wants a quick go at emacs with canvas, but still would like the stable one - I've packaged emacs 31 with canvas patch added: build.opensuse.org/package/show... Also contains Fedora 43 packages for Arm (targetting asahi)