Post

That is not how any of this works. This is not even close to why its hard to make a machine like that (and very much possible to do so, in theory).

The model does not produce statistically indistinguishable from human outputs, but the outputs that satisfy its training rewards system most strongly. Those are very different things. And the rest of the argument is less logically.

How would a machine capable of detecting that difference be usable to train a 'better' model? Only _if_ the goal was to make it indistinguishable from human, which it is not, the goal is to make it succeed at a set of training tasks! Tasks that in fact don't lead to it being all that human like.

And if we assume that was possible, and that such a detector could be used to train better models, which in turn would lead to better detectors, that would not break any laws of information theory of thermodynamics either! That would still require immersive amounts of compute!

And compute, famously, cost energy! Which is why this doesn't break thermodynamics. It also doesn't break information theory, because we already know what human text looks like, and are just extracting out the patterns that make it look like so via computation.

Also, the thing is that, a text or any other thing, that has all the average values of a large diverse population, is very strange! A text that is statistically closest to the overall dataset average of all human text, is not very human look at all!

Humans do not write text that is the exact average. Humans write texts with statistically significant differences. In fact, you can compare texts to find if they have the same author! This is not a new thing either!

One example is that you can analyze the usage of location specific slang or terminology. People have accents, and not just in their speech! As a result, with enough text, it is very much possible to specifically identify the author of something!

The thing that makes LLM detection hard now, is not some inherent impossibility, but the models tendencies to change this linguistic 'profile' based on user input. So if you throw a prompt at it, like 'write in x style' it will often adopt another, distinct 'voice.'

The actual technical question with LLM detection feasibility now is if those profiles share certain properties that human profiles do not. The answer is, probably yes for old models, and absolutely for most new ones, because AI labs intentionally fine tune them for it.

Again, they are NOT made to be human indistinguishable, but achieve some human made reward metric. This is, in big part, because a whole lot of human text is not a good base! A LOT of the raw text is nonsensical arguments, for example. Or just people saying hello, or posting about their vacation.

In general, things the LLM doesn't need to do very well, or doesn't need to focus on all that much. And well, the people training these models do actually have a self interest in keeping them distinguishable. Feeding older AI slop is, in fact, not great for making a newer slopinator.

Sure, most of those companies are just in for a quick buck and don't give a fuck if their tools are accurate, but to claim that some law of nature were violated by such a machine existing is just nonsense.

Especially if your argument is so unsound that it basically implies you can't train a neural network at all. Like, that whole 'going to infinity' thing makes no sense. Nothing is going to infinite, the models error rate (in the example of meaning output text not looking human) goes to zero.

And hey, we happen to provably know that neural networks are in fact capable universal function approximation, if just made big enough. If you make it big enough, and by extension stupidly expensive to run, it can do the task. The cost is just _slightly_ non linear, though.