Post

O3 sounds very interesting but I think the other announcement by openai deserves some discussion as well: deliberative alignment. Today, prompt injections are a huge problem for many use cases, especially agentic ones. openai.com/index/delibe...

openai.comDeliberative alignment: reasoning enables safer language modelsIntroducing our new alignment strategy for o1 models, which are directly taught safety specifications and how to reason over them.

21/12/24, 1:56 pm0 reposts3 likes

Daniel Bachler

@danielbachler.de

21 Dec 2024

If DA could be augmented by instructions in the system prompt then this would have the potential of reducing prompt injection attacks a lot.

Daniel Bachler

@danielbachler.de

21 Dec 2024

A lot of use cases could be unlocked if prompt injections and jailbreaking could be prevented entirely, but even making them very very hard to pull off would already be super valuable.