I'm curious to hear someone on the frontline of research give an estimate of the tractability here. This problem seems pretty robust against all the known methods of improving LLMs. It's gonna take some major hyperpolation on the part of the humans to finally pass that ability on to the silicon generation.
1) Useful novelty is not necessarily dependent on hyperpolation - both the recent Erdős problem proof that impressed terrance tao and mythos surfacing high-fidelity bugs are arguably both non-hyperpolative but still novel to humans
2) Evolutionary models in domains like go and chess are able to come up with solutions that are interpolative in the underlying games’ positional space but arguably hyperpolative in the games’ strategical idea space - similar to Ord’s comparison of genotype and phenotype. Better explanation and discussion of RL-based exploratory directions can be found here: https://arxiv.org/pdf/2406.04268
This reminds me of something I always get pulled back to thinking about which is the way Peirce understood the distinction between deduction, induction, and abduction. For him, abduction wasn't anything like IBE. The way I understand Peirce's carving, he thinks deduction is non-ampliative reasoning to a conclusion, induction is ampliative reasoning to a conclusion, and abduction is something like reasoning to a hypothesis space or generating candidate explanations. So we use abduction to generate hypotheses, we use deduction to work out the consequences of hypotheses, and we use induction to confirm and disconfirm hypotheses. Might be that AI is also bad at abduction in Peirce's sense.
My theory is that I've become great at hyperpolation because I have pretty bad memory. That (and other factors) forced me to form a single consistent world model as opposed to many tiny disconnected models, because it was still easier to do all that in advance and then rederive everything I was supposed to have memorized from my model than it was to memorize. So for decades I had to confront every inconsistency and try to find some way to make every new observation fit with everything else.
If I extrapolate from the one big model it looks like hyperpolation from the perspective of any one tiny model.
The LLMs have enormous memory, so they're not forced to world-model very hard during training.
I have an article on this on my blog titled Modelers and Indexers.
> For example, it isn’t funny. I asked all the leading models to come up with ten original jokes.
Have you tried doing this to humans? It's not that easy to come up with new funny jokes. Most jokes comedians come up with flop — you just hear battle tested ones.
On this specific prompt? “Come up with ten original jokes”?
“Would still do better” or “still do better” — have you actually tried this prompt?
I do stand up comedy (91 performances on stage), and every now and then I talk to people who seriously think about trying stand up. They have trouble with coming up with ten jokes. No surprise — coming with jokes are hard, I also had this problem before starting.
I also occasionally brainstorm jokes with AI. And it's not completely terrible. As in, I'd rather brainstorm jokes with another comedian, but I'd also rather brainstorm jokes with an LLM than a regular person. Basically, Comedians > AI > untrained people.
2. With context it's much much easier. You gave AI a hard task and you didn't give a human an equivalent task. Can you actually build a funny stand up routine out of jokes you hear over the course of a week?
Yeah this is a good point. I still think there's some important qualitative sense in which AI funniness lags behind ability in other domains, but may have been hasty.
Wouldn’t you say, though, that AI is actually considered a TRAINED comedian and an average person is not? That is, a trained comedian will produce much better comedy in quality and consistency than any AI. I can tell you I noticed AI is terrible in poetry. It can analyze a poem with utmost skill, better than most professional poets, but its poetry is close to gibberish.
most schools teach kids to interpolate and extrapolate, not to hyperpolate, and it shows. To be frank, I feel it. I find it much easier to read and interpolate and extrapolate than to hyperpolate.
(to do some interpolating/extrapolating) This reminded me a lot of the Qualia Research Institute and their thoughts on neural annealing and high-dimensional experiences, especially on dmt, and how that can lead to seeing things in a new, better way. It could be that it's easier to hyperpolate during a high-dimensional experience, which would explain why people tend to come away from psychedelic experiences either with a new, healthier perspective on life, or sometimes with a totally outlandish view.
The hyperpolation concept is the interesting one. Models that confidently fill in a data point that isn't even in the distribution they were trained on. That's not really interpolation or extrapolation, it's something else entirely.
Weird, LLMs to me seem extremely good at hyperlation. You just take an existing paper and say hey can you translate it into other {field, method, etc.}.
I've had some LLM's make passable-to-good, and apparently original, jokes to me, but only unprompted. Every 10,000 words or so they'll throw in a bon mot and you think "hey, this isn't bad".
Can you give an example of a situation where creativity is accurately described as inferring the state of a third variable like this? I'm not immediately convinced the two ideas share more than an "out of nowhere" theme.
Also, this description of hyperbolation doesn't give me a clear impression of the sort of reasoning that's going on. Would hyperbolation on a firm basis have to be latent variable statistics, more or less?
I'm curious to hear someone on the frontline of research give an estimate of the tractability here. This problem seems pretty robust against all the known methods of improving LLMs. It's gonna take some major hyperpolation on the part of the humans to finally pass that ability on to the silicon generation.
Two thoughts:
1) Useful novelty is not necessarily dependent on hyperpolation - both the recent Erdős problem proof that impressed terrance tao and mythos surfacing high-fidelity bugs are arguably both non-hyperpolative but still novel to humans
2) Evolutionary models in domains like go and chess are able to come up with solutions that are interpolative in the underlying games’ positional space but arguably hyperpolative in the games’ strategical idea space - similar to Ord’s comparison of genotype and phenotype. Better explanation and discussion of RL-based exploratory directions can be found here: https://arxiv.org/pdf/2406.04268
This reminds me of something I always get pulled back to thinking about which is the way Peirce understood the distinction between deduction, induction, and abduction. For him, abduction wasn't anything like IBE. The way I understand Peirce's carving, he thinks deduction is non-ampliative reasoning to a conclusion, induction is ampliative reasoning to a conclusion, and abduction is something like reasoning to a hypothesis space or generating candidate explanations. So we use abduction to generate hypotheses, we use deduction to work out the consequences of hypotheses, and we use induction to confirm and disconfirm hypotheses. Might be that AI is also bad at abduction in Peirce's sense.
My theory is that I've become great at hyperpolation because I have pretty bad memory. That (and other factors) forced me to form a single consistent world model as opposed to many tiny disconnected models, because it was still easier to do all that in advance and then rederive everything I was supposed to have memorized from my model than it was to memorize. So for decades I had to confront every inconsistency and try to find some way to make every new observation fit with everything else.
If I extrapolate from the one big model it looks like hyperpolation from the perspective of any one tiny model.
The LLMs have enormous memory, so they're not forced to world-model very hard during training.
I have an article on this on my blog titled Modelers and Indexers.
> For example, it isn’t funny. I asked all the leading models to come up with ten original jokes.
Have you tried doing this to humans? It's not that easy to come up with new funny jokes. Most jokes comedians come up with flop — you just hear battle tested ones.
Good point but I think people still do better.
> I think people still do better
On this specific prompt? “Come up with ten original jokes”?
“Would still do better” or “still do better” — have you actually tried this prompt?
I do stand up comedy (91 performances on stage), and every now and then I talk to people who seriously think about trying stand up. They have trouble with coming up with ten jokes. No surprise — coming with jokes are hard, I also had this problem before starting.
I also occasionally brainstorm jokes with AI. And it's not completely terrible. As in, I'd rather brainstorm jokes with another comedian, but I'd also rather brainstorm jokes with an LLM than a regular person. Basically, Comedians > AI > untrained people.
But normal people make jokes all the time in context.
1. So do AIs nowadays. Do you not notice this?
2. With context it's much much easier. You gave AI a hard task and you didn't give a human an equivalent task. Can you actually build a funny stand up routine out of jokes you hear over the course of a week?
Yeah this is a good point. I still think there's some important qualitative sense in which AI funniness lags behind ability in other domains, but may have been hasty.
Wouldn’t you say, though, that AI is actually considered a TRAINED comedian and an average person is not? That is, a trained comedian will produce much better comedy in quality and consistency than any AI. I can tell you I noticed AI is terrible in poetry. It can analyze a poem with utmost skill, better than most professional poets, but its poetry is close to gibberish.
A couple assorted thoughts:
most schools teach kids to interpolate and extrapolate, not to hyperpolate, and it shows. To be frank, I feel it. I find it much easier to read and interpolate and extrapolate than to hyperpolate.
(to do some interpolating/extrapolating) This reminded me a lot of the Qualia Research Institute and their thoughts on neural annealing and high-dimensional experiences, especially on dmt, and how that can lead to seeing things in a new, better way. It could be that it's easier to hyperpolate during a high-dimensional experience, which would explain why people tend to come away from psychedelic experiences either with a new, healthier perspective on life, or sometimes with a totally outlandish view.
The hyperpolation concept is the interesting one. Models that confidently fill in a data point that isn't even in the distribution they were trained on. That's not really interpolation or extrapolation, it's something else entirely.
Really interesting post!
The lost pen is actually pretty funny.
Weird, LLMs to me seem extremely good at hyperlation. You just take an existing paper and say hey can you translate it into other {field, method, etc.}.
Mine’s pretty funny
Man, telling the AI researchers exactly what to target next is a philosophical violation of Seerow’s Kindness
I've had some LLM's make passable-to-good, and apparently original, jokes to me, but only unprompted. Every 10,000 words or so they'll throw in a bon mot and you think "hey, this isn't bad".
Can you give an example of a situation where creativity is accurately described as inferring the state of a third variable like this? I'm not immediately convinced the two ideas share more than an "out of nowhere" theme.
Also, this description of hyperbolation doesn't give me a clear impression of the sort of reasoning that's going on. Would hyperbolation on a firm basis have to be latent variable statistics, more or less?
Maybe we should think of hyperpolation as The Work of Humans. That is part of what I was trying to get at here: https://bobm858524.substack.com/p/the-work-of-humans?r=bi9a&utm_medium=ios
For example, Claude was very helpful in my attempt to locate New Wye in Nabokov’s novel Pale Fire, but it needed my hyperpolation of using the location and timing of cicada broods to home in on the location. https://bobm858524.substack.com/p/where-is-new-wye-claude-leaps-in?r=bi9a&utm_medium=ios