ChatGPT’s memory upgrade risks poisoning all its answers
▼ Summary
– ChatGPT’s Dreaming V3 feature builds a profile from past chats, but it can create inaccurate assumptions, such as claiming the user uses Home Assistant when they do not.
– Turning off memory does not delete stored data, and users must delete individual chats to fully purge information from the AI’s system.
– The feature retains outdated or irrelevant facts from old conversations, which can distort future AI responses and require manual curation to correct.
– OpenAI reduced compute costs for Dreaming V3 by 5X, making it scalable for mass access, but the author criticizes it as an irresponsible feature due to privacy and accuracy concerns.
– The AI’s automated profiling may skew answers based on a limited, inaccurate view of the user, increasing cognitive load and reducing trust in responses.
OpenAI recently touted upgrades to how ChatGPT manages user memories, but the so-called improvements raise serious concerns. While the company frames this as progress, the reality is that ChatGPT’s enhanced memory feature risks distorting future answers by building an incomplete, often inaccurate profile from past chats. Turning the feature off may not fully solve the problem, leaving users in a difficult position.
The concept of ChatGPT memories started in 2024 as a simple list of facts the AI could reference. Today, it has evolved dramatically. The system now pulls from your entire chat history, explicit instructions, personal constraints, and even implicit preferences it infers from your behavior and casual remarks. This expanded scope means old or irrelevant details can easily poison the AI’s responses.
OpenAI’s Dreaming V3 feature, available to Plus and Pro subscribers and rolling out to all users, takes this further. It performs background data synthesis, effectively composing a dossier about you. While the company claims factual recall has jumped from 41% to 82% and preference adherence from 31% to 71%, the system still makes glaring errors. For example, I asked ChatGPT about my experience with Kasa smart plugs. It claimed I had moved monitoring into Home Assistant,a software I have never installed. The AI had synthesized a false narrative from scattered conversations.
The interface for managing these memories is also problematic. You can disable memory, but your stored data and chat history remain. Deleting saved memories doesn’t purge the chat logs themselves. And OpenAI’s FAQ notes that turning off memory doesn’t disable safety features that may retain context in high-risk situations. This creates a privacy concern that feels invasive and opaque.
The most troubling aspect is how ChatGPT now filters its worldview through a personal lens derived solely from your interactions. It doesn’t know who you really are,only what you’ve shown it. When I asked, “What is the single most important thing I care about?” the AI answered, “Preserving your ability to keep doing meaningful, independent work.” It completely missed my wife and my little dog, the true priorities in my life. The AI’s assumptions are skewed, outdated, and potentially dangerous if used to limit informational responses.
Dreaming V3 is a technical achievement in efficiency and scalability, but it’s an irresponsible feature. It processes old conversations without users’ full consent, makes it nearly impossible to prune what the AI recalls, and cannot keep up with real-life changes. Users now face the added cognitive burden of filtering out AI bias and hallucinations from every answer. Will the AI conveniently leave out information because it thinks you’re not interested? Will it skew its presentation to match a flawed internal model?
The fundamental trust issue remains: we could never fully trust an AI’s answers, but this selective memory capability makes it worse. Every response may now be colored by an aggregate representation of who the AI thinks you are, what it thinks you care about, and how it thinks you want to be communicated with. That’s a risk worth thinking hard about.
(Source: ZDNet)
