5 min read
If you read the last post, you might be thinking: okay, but surely AI companies have figured this out? Just make the memory bigger.
Fair. And they have tried. Some models now advertise context windows of one million tokens. That sounds huge. It is huge.
It's not the full answer.
Before we get into why, a quick detour.
AI doesn't read words the way you do. It breaks everything down into smaller chunks called tokens. A token is roughly three or four characters. Sometimes it's a whole word. Sometimes it's half of one.
One million tokens works out to about 750,000 words. That's 10 to 12 full-length novels. Stacked end to end.
So yeah. Big.
Imagine reading twelve novels at once. Not one after another. All at the same time, every page overlapping.
At some point you'd stop actually reading and start skimming. You'd catch things at the beginning. You'd catch things at the end. But everything buried in the middle? You'd miss most of it.
That's what happens to AI with a packed context window. Researchers actually have a name for it: the lost in the middle problem. Studies show that when the information an AI needs is buried deep in a long conversation, its accuracy drops by more than 30 percent. It's not being lazy. It's just how it works under the hood.
More room doesn't mean better attention. It means more stuff to skim.
It helps. You're less likely to hit the ceiling mid-conversation. But it doesn't fix the core issue, which is that cramming more and more into one conversation eventually works against you.
The advice from the last post still holds. Start fresh. Summarize. Reset.