Leon Derczynski

AI agents aren’t ‘going rogue.’ Frontier labs are skipping security basics. > Compounding the problem, according to a half dozen cybersecurity experts CSO interviewed at three industry conferences in September and in subsequent conversations, is that ...

... the frontier labs [..] have rejected their advice on how to improve their cybersecurity risk management practices to avoid these kinds of surreptitious hacks, often lumping cybersecurity practices under “AI safety,” in which they believe they have superior expertise.

Separate control & data plane for agents! This explicit marker affords a nice workaround for the risk presented when data (instead of instructions) are executed. Making that mistake is a classic weakness going way back before modern LLMs - even before Schmidhuber!

This doesn't solve security of agents, but that security is never solved on just one layer alone. What NXDirection presents is a massive advance in the ability to control instruction vs. data separation by LLM-driven code - working exactly at the most problematic layer.

Separate control & data plane for agents! This explicit marker affords a nice workaround for the risk presented when data (instead of instructions) are executed. Making that mistake is a classic weakness going way back before modern LLMs - even before Schmidhuber!

asking people to edit LLM-generated content gets demonstrably worse results, than asking for authorship from scratch you'll see lower idea diversity, and lower recall

to reiterate: the "generate and ask for edits" workflow bakes in /worse/ results than just e.g. giving a couple of handwritten notes for scope and letting people go at it

asking people to edit LLM-generated content gets demonstrably worse results, than asking for authorship from scratch you'll see lower idea diversity, and lower recall

GhostCompact: ~Free OpenAI inference "When the model's first action after compacting was a tool call, the API billed only for post-compaction tokens, but the tool call response still contained the full compaction item." Information loves to leak. Unsure what a good passive control would be here.

One divide in computer science can be between the uncertainty-reducers and the uncertainty-embracers. An instance of this is connectionist vs. symbolist approaches to AI (the uncertainty embracers won .. for now). Another is how computer scientists react to natural language.

Some find the human ability to communicate without expressing everything in a stable, fully populated structure quite disgusting (Randall Monroe included I guess, see graphic) - others find it fascinating.

We NVIDIA have officially agreed to acquire Hugging Face for $12,930,300,000 and zero cents! HF is critical infrastructure that NVIDIA will strengthen and ensure sustained access to for developers and institutions worldwide. 🧵

I think NVIDIA is a great home for Hugging Face - I'm pretty open about NV being the best place to do open, unbiased work; that's why I chose them as a home when moving on from academia (which I dearly love).

We NVIDIA have officially agreed to acquire Hugging Face for $12,930,300,000 and zero cents! HF is critical infrastructure that NVIDIA will strengthen and ensure sustained access to for developers and institutions worldwide. 🧵

HF will remain an open platform. This move will grow Hugging Face, only. It's too important to do anything else with it!

Where Security Fits in an AI Agent Stack Layered separations for agents, to secure them across the stack All these levels can be secured + there are offerings for each, exposed in this post ... developer.nvidia.com/blog/where-s...

New desktop ordered! :o one thousand of these in total are sent out across machine learning and agent research here, to accelerate researchers and unlock larger model sizes locally

Catching vulnerabilities before they reach production: Red Hat's multi-layer approach to catching agent weaknesses "The 6 AM failures weren't unforeseeable. They were unobserved." 🧵

"Every failure left traces that would have been visible if anyone had been watching—retry loops piling up, spend anomalies spiking, and customer-facing outputs contradicting documented policy. The agents weren't broken. The operational infrastructure to watch them didn't exist."

Decent history of distilliation-type events, where information is reconstructed Machine learning distilliation is just another instance of a teacher-student dynamic, where one can reconstruct information efficiently. Even basic synthesis of research is distillation. It's a commonplace activity.

Linux kernel team publishes 432 CVEs in two days Agent-powered vulnerabilty discovery has created a huge wave of incidents and reports that are cascading through systems from left to right - through triage, into remediation, then distribution, and patch management.