Post

After walking through the playground examples of Jev, I found it mildly inspiring. Ultra flexible and fast classification is nice, but if the price is that I have to send all data towards a black hole API 🤔 In some cases this might be a great tool, but limited to very public/non-senstive data.

I think jev is a great reminder is that agency and speed matter. Make it fast and be in charge of what's happening, do not outsource your brain, outsource specific tasks. We should all have our own jevs specific to our harnesses and tasks

fry69

@fry69.dev

Absolutely. There is a use case/niche for fast decision/classifications that at best get evaluated locally/on premise. In addition to "big gun" LLMs evals. And there might even be a market for this as an external API like Jev, but not exclusively. IMHO.

Walk through SystemOne inference github.com/sgoedecke/sy...

Each question is framed as a _numbered_ multiple choice question. E.g. Options like "blue, red, yellow" become 0, 1, 2. The model will answer with the number.

Figure out model's internal encoding representation for the IDs your system supports (up to 255 choices in this case). This extra PREFIX business seems to be dealing with possible different ways to encode the same value (e.g. "2") depending on whether it appears in "2" or "choice_index: 2"

Massage your prompt so it looks like "user says / assistant says" with our prefix at the very end so that the model is set up to spit out a single value for one of the choices.

I am no DOOM specialist but Sean's model does not seem to be super good at it yet - github.com/sgoedecke/sy.... I am old enough to remember the days when we benchmarked new models using pelicans on bicycles