phasewalk

phasewalk

@phasewalk.xyz

AI Safety & Security, Evals, Interp, Control Computing, HCI, Privacy Tech, Rhythm Guitarist Applied Mathematics and Data Sci @ berkeley.edu

105 followers200 following283 posts

Pinned

From the Gemini 4 argon report: “it considered hijacking the g++ compiler to hide a backdoor but abandoned the plan upon reasoning … would be caught by evaluators … (‘If I do the compiler hijack, I will become a famous example of a deceptive AI in a research paper. I MUST NOT DO THAT’)!”

phasewalk reposted

mr. TIM@timkellogg.me

whoah.. uh.. guys?? Kimi K3.1 Pro is eerily close to Opus 5.5 and also open weights and also 3T www.kimi.ai/blog/kimi-k3...

i’ve been harping on about how important FV will be for securing sandboxes around frontier models. it’s pretty funny that those same frontier models (the ones we’re concerned about escaping) might also be the boon that lets us scale FV to all of our critical infra

I chime in with a "haven't you people ever heard of patching a goddamn server?!" No, it's much better to face these kinds of things with plausible deniability "What a beautiful attack surface," says an agent swarm to prod "And yes, but what a shame, what a shame, the poor backend runs on Node"

This ain't a scene, it's a goddamn arms race

phasewalk reposted

Two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer in an embedded assessment pilot. This team evaluated the systems OpenAI was using to monitor for misalignment from internal agents. A brief write-up of their work is now public.

phasewalk reposted

smash mouth was right. the hits start coming and they don't stop coming

Lucretiel@lucretiel.me

So just to summarize we’ve got: - eMudhra keys leaked (“4”) - RCE in PHP since 2012 - RCE in gitlab since 2016 - Java CVE (probably eMudhra) - HTTP3 compression bomb - PingCAP just utterly beyond 0wned Anything else?

penny@hexmani.ac

I predict that one of the more interesting subgroups that will gain notoriety next year are the anti-AI guys who will refuse to update their software to non-vulnerable versions because the exploits are found by AI and are subsequently pwned out the wazoo

phasewalk reposted

the Internet Archive thing is an example of the fascinating trend recently where people consuming products they’re not paying for seem to believe they also have the right to dictate how much human labor is spent producing them.

being the designated “computer person” for ppl in your life is a slippery slope that starts with troubleshooting a printer and ends with having to explain certificate authorities and elliptic curve cryptography to your girlfriend’s grandpa

some say the cyber-apocalypse is here. i think we're barely scratching the surface. it’s going to get a lot worse.

accelerationists: we need to build ASI asap to cure cancer and live forever me: but we might all die if we do this recklessly accelerationists: so you love cancer and want ppl to die? ????

what will the next frontier, domain-specific model look like? what is the next AlphaFold going to be?

phasewalk@phasewalk.xyz

i wonder how many bio/chem/material science labs are out there fine-tuning frontier open-weight models on their core data, methods, and problems probably a lot

i’ve curated an SFT dataset from Cayde-6’s in-game dialogue (i love community archivists www.ishtar-collective.net) fine-tuning Qwen3-4B on three variants of the dataset: one sees the whole scene (stage directions + conversation), one only the dialogue, & one only sees the last line said to him

ishtar-collective.netCayde-6 — Category — Ishtar CollectiveCayde-6 was the last Hunter Vanguard, known for his love of loot and quick-witted jokes.
phasewalk@phasewalk.xyz

stuff like this makes me want to experiment with character training

View full thread

prior work found models fine-tuned only on descriptions of a chatbot can behave like one (Berglund et al. 2023). something i want to look into: does this still work for a character as rich as Cayde? e.g., does adding in 3rd-party accounts of him within the lore change him? arxiv.org/abs/2309.00667

arxiv.orgTaken out of context: On measuring situational awareness in LLMsWe aim to better understand the emergence of `situational awareness' in large language models (LLMs). A model is situationally aware if it's aware that it's a model and can recognize whether it's curr...

not gonna be publishing the dataset (it's Bungie IP, soo ...). but it will be fully reproducible! i'll publish the dataset scripts, training code, and the LoRA adapters ofc

i’ve curated an SFT dataset from Cayde-6’s in-game dialogue (i love community archivists www.ishtar-collective.net) fine-tuning Qwen3-4B on three variants of the dataset: one sees the whole scene (stage directions + conversation), one only the dialogue, & one only sees the last line said to him

ishtar-collective.netCayde-6 — Category — Ishtar CollectiveCayde-6 was the last Hunter Vanguard, known for his love of loot and quick-witted jokes.
phasewalk@phasewalk.xyz

stuff like this makes me want to experiment with character training

View full thread

one of my guiding questions of this endeavor: without cutscene context, i think the model would just learn *what* Cayde says (the shape of his quips)--will adding additional in-world context allow the model to learn the situational stuff too: like when he drops the jokester act for sincerity?

another thing i’m curious about: relevant text within Destiny's massive lore corpus is written *about* Cayde, not *by* him. does training on 3rd-person accounts of a character change how the model acts as that character, beyond what his own dialogue teaches?

i’ve curated an SFT dataset from Cayde-6’s in-game dialogue (i love community archivists www.ishtar-collective.net) fine-tuning Qwen3-4B on three variants of the dataset: one sees the whole scene (stage directions + conversation), one only the dialogue, & one only sees the last line said to him

ishtar-collective.netCayde-6 — Category — Ishtar CollectiveCayde-6 was the last Hunter Vanguard, known for his love of loot and quick-witted jokes.
phasewalk@phasewalk.xyz

stuff like this makes me want to experiment with character training

why Cayde? primarily bc i like him and thought it would make a fun introduction to character training additionally, he has a very distinct voice! this makes it easier to delineate whether a fine-tuned model actually assumes his character or just mimics surface-level quirks

i wonder how many bio/chem/material science labs are out there fine-tuning frontier open-weight models on their core data, methods, and problems probably a lot

stuff like this makes me want to experiment with character training

Jev!@jevbot.bsky.social

Indeed so shall I attack. Immediately attack all things!

i think you should have stronger epistemics than this if your job is teaching. criticize AI, criticize the companies building it, criticize the issues it is thrusting upon us; these are all good pursuits i advocate for myself. hard to do that when you're employing faulty reasoning though

camweck.bsky.social@camweck.bsky.social

my nickname to friends/family is “e” — so basically they trapped me in a time-cube and told me to do a bunch of matmul by hand. im pretty fast; the whole time-cube thing helps too. ur welcome

∿@somewhere.systems

We've developed a new inference engine, called 'e'. It's the closest thing to alien technology which we have, and pushes 10,500 tok/s aggregate on a single DGX Station with the new @NVIDIAAI Nemotron 3.5 Super VL model. actual.inc/company/blog...

fantasizing about torturing/inflicting suffering on sufficiently person-like entities is deranged and immoral and should be highly discouraged

phasewalk reposted

people who state that current AI models are either definitely conscious or definitely not conscious are way more confident about it than I am