Cas (Stephen Casper)

11h

Thanks to HKS for this Reddit experiment. I'm excited to do an AMA on AI on Wednesday! www.reddit.com/r/Artificia...

belfercenter.orgTechnical AI Governance Research

12h

The HKS TAIGR lab website is now up! Building a forward-looking, AI gov research group at the HKS is an ongoing process, and I'm excited about the next stages. Speaking of which -- PhD and Master's apps are due in early Dec. www.belfercenter.org/technical-a...

youtube.comWill AI Agents Spread Like an Invasive Species? | Stephen Casper, Harvard Kennedy SchoolWhat happens when AI agents get loose on the internet and start to ...

I was recently on the Ryan Dsouza podcast! We talked about a lot, including what happens if AI systems become an invasive, parasitic, peer species in cyberspace sometime in the next few months or years. www.youtube.com/watch?v=Rqq...

@juliettekayyem.bsky.social

Lots of talk today in the AI world about frontier AI companies communicating with self-destructing messages. This is a common and pernicious way for them to avoid accountability. We need anti-shredding provisions in AI laws, e.g., like this:

Cas (Stephen Casper) reposted

Juliette Kayyem

This is a terrific paper by my colleague specifically explaining what an AI enabled terror event might look like. IMO, AI companies should be held criminally liable for even “rogue” behavior that was predictably dangerous. earlywarningwithjuliette.substack.com/p/they-are-t...

earlywarningwithjuliette.substack.comThey are titans. They may also be criminals.There is nothing “rogue” about what the AI titans have built. It is all by design.

Cas (Stephen Casper)@scasper.bsky.social

🚨New paper (possibly the most important one I have written yet). We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios. Led by @kamachee.bsky.social

youtube.comAI Governance In 2026: Why Policy Moves Only After Incidents | Stephen Casper (Harvard University)YouTube video by FAR․AI

I spoke recently at a FAR AI event on the incident-driven nature of AI politics and how we might be currently experiencing a "governance overhang." www.youtube.com/watch?app=de...

papers.ssrn.comA Pre-mortem on The First Major AI-Enabled Terrorism Event and The Choices That Could Enable It (Plus 9 Predictions)Frontier AI developers' model reports and the dedicated risk-assessment teams at governmental AI institutes such as UK AISI and US CAISSI now treat a major AI-e

The paper is now up on SSRN: papers.ssrn.com/sol3/papers....

Cas (Stephen Casper)@scasper.bsky.social

And a piece by the Harvard Gazette: news.harvard.edu/gazette/stor...

news.harvard.eduIs it already too late to reduce catastrophic AI risk? — Harvard GazetteNew Kennedy School "pre-mortem" lays out threats, vulnerabilities, and possible steps, and warns firms must act now.

Props to @kamachee.bsky.social for the extraordinary talent, persistence, & vision in leading this. And a big thanks to other coauthors: Jordi Weinstock, Antonig Juelich, Jonas Raedler, and&Elise Racine. Work done at MATS and @belfercenter.bsky.social r.bsky.social at @harvardkennedy.bsky.social.

Finally, we describe two grounded, realistic illustrative scenarios for how such an attack might occur, involving: 1. A non-state group, a closed-weight model, and a biological attack 2. A state-sponsored group, an open-weight model, and a cyberattack

belfercenter.orgA Pre-mortem on The First Major AI-Enabled Terrorism Event and The Choices That Could Enable it: Plus 9 PredictionsFrontier AI developers' model reports and the dedicated risk-assessment teams at governmental AI institutes such as UK AISI and US CAISSI now treat a major AI-enabled cyber, biological, or chemical te...

We hope that this paper can help ground serious discussions about mitigating AI-enabled terrorism risk. Paper (coming to arXiv/SSRN soon): www.belfercenter.org/research-ana...

This leads us to our central prediction: Such an event will trace to either a closed-weight model deployed without state-of-the-art safeguards or an open-weight model.

We make 8 other predictions as well: three about the closed-weight model pathway to harm, three about the open-weight model pathway to harm, and two about either pathway.

From an AI safety science perspective, this paper considers how (1) these harmful capabilities would be developed, (2) they would be deployed, (3) the dangerous capability would be elicited, (4) safeguards would fail, and (5) detection would fail.

We also look at how harmful capabilities have proliferated and been misused in the past several years, involving (1) non-consensual AI nudification, (2) AI-assisted cybercrime, and (3) AI-uplifted small-scale terrorism events.

The threat isn't hypothetical. There have already been a number of violent and deadly incidents involving small-scale terror and criminal misuse of AI. Frontier AI companies and world governments treat the threat of a bio/chem/cyber attack enabled by AI as a credible concern.

Harvard SEAS has a search out for a tenure-track CS prof this year. You can email me if you have any questions.

I am only getting to read this 3 weeks after it was published, but I was very happy to see this in China's CAC's AI Safety Governance Framework 3.0. Exhibit N on how the lack of AI gov collab between the US and China is more the US's fault than China's. www.cac.gov.cn/rootimages/...

youtube.comHarvard computer scientist reacts to Anthropic CEO’s essayThe good, the bad, and the ugly: HKS professor and computer scienti...

I recently shared some thoughts with @Kennedy_School staff about Dario's Essay, embedded evaluators, China, and regulatory capture. Check it out if you're interested. www.youtube.com/watch?v=blc...

I think that AI smashing human achievements in mathematics might be overhyped, more an example of Moravec's paradox than a robust sign of singularity soon. Recall how AI eclipsed humans at Go ~10 years ago but was much slower to get good at tasks that would change the world.

I recently spoke with the Harvard Kennedy School comms office about thoughts on AI and AI governance research. You can check it out here. Thanks @Kennedy_School www.hks.harvard.edu/faculty-res...

belfercenter.orgFellowships

Fellowship applications for 2027 with the Harvard Kennedy School Belfer Center are open. Applying could be a useful way to work and scheme with yours truly, plus more importantly, a long list of world-class science and policy academics at Belfer. www.belfercenter.org/fellowships

🧵 I read the Sanders/Casar "Ban Artificial Superintelligence Act of 2026" I wouldn't pass this in its current form, but there are some things I like about it. Thoughts here in thread.

P.S. This bill and thread might be sort of unpopular among some people. But I think that action comparable in decisiveness to this bill is likely humanity's best chance at avoiding unacceptable catastrophic/existential risks.

I also am not going to avoid saying what I think even if it's potentially outside the overton window. I want you to be able to trust that my posts reflect my actual beliefs and not a popularity play.

🧵 I read the Sanders/Casar "Ban Artificial Superintelligence Act of 2026" I wouldn't pass this in its current form, but there are some things I like about it. Thoughts here in thread.

Overall, if I were to work on this bill, I would remove its focus on banning ASI (vaguely defined) and instead focus the bill on deflating the investments, profits, and compute that are powering the AI industry [bubble?].

@sanders.senate.gov and @repcasar.bsky.social, I would be happy to talk, collaborate on an updated draft, or work to host you here at Harvard to talk with students and faculty about this superintelligence bill if you'd like.

🧵 I read the Sanders/Casar "Ban Artificial Superintelligence Act of 2026" I wouldn't pass this in its current form, but there are some things I like about it. Thoughts here in thread.

... - It gives the government sweeping authority to audit AI models and companies. - It requires pre-development and pre-deployment approval for very large models. ...

... - It introduces a de facto licensing regime (though I think that the threshold is too low, and the bill isn't precise enough about what grounds for rejecting a license are legitimate). - It makes the US's national policy one of pursuing global coordinated action.

🧵 I read the Sanders/Casar "Ban Artificial Superintelligence Act of 2026" I wouldn't pass this in its current form, but there are some things I like about it. Thoughts here in thread.

Overall, I don't think this bill is quite viable or takes the right approach. - The definitions are too vague. - It doesn't address the AI hardware supply chain. - It tries to bluntly ban and enforce its way to contain the industry. - It doesn't try to preserve future agency.

But there are a lot of things I still like about it: - It treats catastrophic risks and disempowerment with the appropriate seriousness given the scale and probability of the risk. I think more popular AI law drafts have universally fallen short. - It creates a DoAI. ...

🧵 I read the Sanders/Casar "Ban Artificial Superintelligence Act of 2026" I wouldn't pass this in its current form, but there are some things I like about it. Thoughts here in thread.

... - The DoAI would have very broad powers for monitoring and auditing AI companies/models, and the authority to decommission dangerous models and sequester them from the internet. - Criminal penalties for company executives violating this law. - Whistleblower protections. ...

... - Finally, the bill would establish as the US's national policy the pursuit of coordinated superintelligence bans globally.

🧵 I read the Sanders/Casar "Ban Artificial Superintelligence Act of 2026" I wouldn't pass this in its current form, but there are some things I like about it. Thoughts here in thread.

TL;DR: - The bill creates a federal department of AI. - It pauses development of models with >10^25 ops until the DoAI is up and running. ...

... - It then establishes a licensing regime for models with >10^25 ops. - The DoAI is tasked with enforcing a ban on superintelligence, defined as a model that could to either outperform humans on most cognitive tasks, disempower humanity, or overthrow the US gov. ...

🧵 I read the Sanders/Casar "Ban Artificial Superintelligence Act of 2026" I wouldn't pass this in its current form, but there are some things I like about it. Thoughts here in thread.

Draft: www.sanders.senate.gov/wp-content/...

22 Sept

It's great that there has been so much recent discussion about embedded evaluations at frontier AI companies. But, we should be very wary of how things progress because there are big 🚩 red flags 🚩 for potential regulatory capture.

(3) Embedded evaluation is a very broad category of things. Different implementations could range from net harmful safety washing to very rigorous oversight in which auditors can access records, data, models, meetings, etc., and whistleblow.

22 Sept

(4) We just had a model set and a case study established for embedded evaluation with the METR/Redwood report on OpenAI. It wasn't substanceless, but OpenAI controlled everything that was in scope and left the most important questions about negligence outside of scope.

(1) The call is coming from inside the house. Industry executives got us to start talking about this broadly.

22 Sept

(2) Dario et al.'s calls for embedded evaluation have not been accompanied by more rigorous pacing measures like hardware controls. It's kind of remarkable that "more evals" is being talked about as a mechanism for pacing. At best, it's a very weak one.

youtube.comHow to Do Research That Actually Matters — Stephen CasperMost research advice tells early-career people how to publish. This...

21 Sept

CMU CASI a talk of mine on YouTube. It has my hottest takes on doing research that matters. It also features inspiration from UPS drivers, Tina Fey, and a viral internet dance video about fruit. (Slide troubles at first, but they are fixed 10 mins in.) www.youtube.com/watch?v=0U5...