Post

Tim Duffy
@timfduffy.com

I think it's difficult to tell how much Astra's rumored recurrent depth approach will reduce CoT monitorability without more info, which unfortunately don't expect OpenAI to provide. My understanding is that recurrent depth makes monitorability worse mainly by adding more thinking in between ...

0 reposts14 likes

Tim Duffy
@timfduffy.com

generated tokens, so just how much they're looping is important to judge the impact. Larger/deeper models reduce monitorability even without recurrent depth, as shown in the attached image from OpenAI's CoT monitorability piece from last year. openai.com/index/evalua...

Tim Duffy
@timfduffy.com

The post from The Information also says that Astra's approach is similar to the one laid out in this paper: arxiv.org/pdf/2502.05171 In the paper, in addition to having a recurrent block within one token's generation, they also suggest using the final hidden state from one token as the ...

Tim Duffy
@timfduffy.com

starting point for the next like in Chain of Continuous Thought. If that were used as well it would probably further reduce monitorability.