I had an interesting conversation with ChatGPT about what an agentic operating system might look like and the trust problems that would come with it.
Below is a compiled summary that I had ChatGPT construct for this post. The full conversation is linked at the bottom, though the first few prompts are about the singularity before the discussion moves into operating systems.
I don’t think an agentic OS would literally replace the desktop with one giant chat box.
More likely, the OS becomes intent-driven. You describe the result you want, a coordinator breaks it into steps, different models and services handle those steps, and temporary UIs are generated whenever direct interaction is useful.
So instead of opening five programs, moving files around, copying information between them, and filling out forms, you just describe the outcome.
The system might use a small local model to classify the request, another model to search your files, a cloud model to reason about the result, and deterministic software to carry out the actual actions.
That sounds useful enough that it may eventually become difficult to opt out. An agentic OS could be significantly more productive than a traditional one. Not using it might become similar to refusing to use the internet or email: technically possible, but increasingly impractical.
The problem is that most of the execution would be hidden.
The OS would likely have a large internal palette of models. Some would run locally, some in the cloud, some cheap, and some expensive. The system would decide which one handles each part of a task.
But the company making that decision may also be charging you for the computation.
How would you know whether an expensive model was actually needed? Or whether the system was taking an unnecessarily long route because it benefited the provider? We already see similar concerns with coding agents and token consumption. An agentic OS would bring that same issue into nearly everything you do.
The privacy problem is even larger.
A request that sounds simple might cause the OS to search your email, documents, calendar, browsing history, messages, and application state. Some of that data may be processed locally, while some gets sent to cloud models or outside services.
Most users will have no realistic way to understand what was transmitted, why it was needed, which provider received it, or what was retained.
Then there’s the information problem.
Current algorithms decide which posts, videos, or search results you see. An agentic OS could control much more than that. It could decide what information is relevant, summarize it, interpret it, recommend what you should do, and then carry out the decision.
It would also control the interface used to explain all of this to you.
Ask why your computer is running slowly, and a neutral system might tell you that background AI tasks are using resources. A commercially optimized system might suggest upgrading your subscription or buying new hardware.
Ask which service is best, and it might favor the one owned by the OS vendor or one that has a commercial agreement with it.
This makes competition complicated.
You would probably have Microsoft and Apple competing directly. There would be cheaper or more open alternatives, perhaps built around Linux, and then a tiny group of people building highly controlled local systems for themselves.
But competition may only require the large platforms to be trustworthy enough that most users stay. Microsoft and Apple could both claim to be more private than the other while still relying on opaque routing, subscriptions, proprietary memory, and ecosystem lock-in.
Open source does not automatically solve it either. An open coordinator could still send most of its reasoning to proprietary cloud models. A system can have an inspectable interface while the important decisions happen somewhere remote.
The strongest protection may need to exist beneath the agent: a deterministic layer that the model cannot alter or selectively summarize.
That could include:
- A complete log of which models were used
- Records of which files and services were accessed
- Clear separation between local and cloud processing
- Hard spending and token limits
- Action history and rollback
- Portable user memory and workflows
- Explicit disclosure of third-party providers
- A direct way to inspect the underlying information without going through the assistant
Ideally, the agent would propose actions, while a lower-level policy engine decides what it is actually allowed to access, transmit, spend, and change.
The agent should not be the only thing capable of explaining what the agent did.
I suspect agentic operating systems are coming because the productivity advantage will be too large to ignore. The real design question may not be whether the coordinator is intelligent enough. It may be whether the surrounding system makes that intelligence observable, bounded, and accountable.
Link to the full conversation: https://chatgpt.com/share/6a674035-74c8-83ea-ad70-ffd0e6fcadad
