Who Gets to Compress Reality?

Part I — The feed at the edge of the self

#

I’ve been thinking a lot lately about what it means to be human in the age of AI, and what it means to connect with other people.

That sounds bigger than I mean it to, at least at first. I don’t mean the science-fiction version of the question, where AI wakes up, looks around, and decides what to do with us. I mean the everyday version. The feed I open without thinking. The search result that gets summarized before I ever touch the source. The recommendation that decides which song, article, job posting, person, or argument becomes visible to me next. The AI-written paragraph that sounds like a person, but has no person behind it in the way I actually care about.

I’ve always had a hard time connecting with others. I’ve been a relatively solitary person my whole life. That’s part of why this moment feels so strange to me. AI has opened real possibilities. I can source news in a sustainable and healthy way. I can use webhooks, cron jobs, and personal agents to pull in information without forcing myself back into the dopamine grinder of the modern web. I can get nudged to reach out to people instead of letting threads rot. I’ve found niches of the internet that resonate with me. I’ve found new ways to express myself.

And I’ve also found pain in how all of this has been done.

Plenty of people hate AI, and a lot of them are right to. Not because AI is a demon or a god or a mind from outside society, but because the version of AI they encounter is usually something being foisted onto them by an institution they didn’t choose. A feed. A workplace mandate. A customer-service bot. A plagiarism panic. An automated rejection. A search page that answers before it lets them read. A slop flood. A tool that claims to save time while quietly making the human part feel optional.

On the other side, I see people ceding enormous amounts of control to these systems. They let AI represent them online, write in their place, summarize the world for them, and decide what deserves attention. Sometimes that’s done with care. Often it’s done because the interface makes surrender feel frictionless.

That’s the current moment to me: exhausting and terrifying, but also hopeful and inspiring. I work with technology every day, both professionally and personally. I’ve seen tons of tech fads come and go. New technologies that promise to change the way we fundamentally operate as a society. Most of them aren’t fruitful, or not really useful, until much later if at all.

AI is harder for me to dismiss because I keep finding real uses for it. That usefulness is the uncomfortable part. A useless technology can be mocked and ignored; a useful one becomes infrastructure before people have worked out what it’s doing to them. The problem isn’t that AI is useless. The problem is that it’s useful enough to become a layer between people and the world.

The first gate: the filtering layer

#

Right now, a lot of online life already arrives through words like recommended, suggested, more like this, promoted, algorithmic matching. These are things we’ve had thrust upon us by capitalists looking to build a new attention economy. They sound neutral. They feel like convenience. But they’re statistical abstractions on our interpersonal interactions, aimed toward an exploitative goal. The interface turns them into little everyday sigils: marks that make one path feel natural and the others disappear.

I keep coming back to the fact that these systems aren’t just showing us information. They’re interpreting social life for us. They decide which person becomes visible, which post gets surfaced, which outrage gets rewarded, which ad follows you, which article looks authoritative, which answer appears before you ever reach the source.

I want to imagine a different world. One where you get to build your own algorithm, your own filter of the world, outside of what big tech defines for you. That could drive more connection. It could also go badly. We could end up living in our own entirely encapsulated worlds, each of us sealed inside a personal rendering of reality that flatters our priors and blocks the hard parts. A feed can become a kind of cybernetic spell: it predicts you, renders the prediction back to you, and slowly trains you to inhabit the person it guessed.

So this isn’t a simple pro-local, anti-platform argument. The question isn’t whether to have filters. Everyone has filters. Culture is a filter. Community is a filter. Language is a filter. Education is a filter. The question is who the filter serves, whether it can be inspected, whether it can be redirected, and whether you can turn it off.

A filtering layer is how things get interpreted. Put the same Trump Truth Social post in front of a MAGA person and a person whose life has been directly hurt by his politics, and they’re not seeing the same object in any meaningful social sense. Their filtering layers are different. One may be Fox News and state propaganda; the other may be lived experience and the testimony of people harmed by his oppression. The same surface-level artifact passes through different interpretive machinery and becomes a different reality.

AI is becoming one more version of that machinery.

That personal frustration is where I need theory, not as academic decoration, but as a naming tool. The point isn’t to escape the lived problem into abstraction. It’s to find the abstraction that explains why the lived problem feels so hard to grab.

Part II — Social technology, not machine god

#

Henry Farrell and Cosma Shalizi’s essay “AI as a Social Technology” helped me name this more clearly. They argue that it’s a category error to treat large models as self-motivated agents in the making. In their words, AI is better understood as “a social technology, a systematic means of reorganizing social relationships among human beings.”¹

I like this frame because it gets me out of AGI prophecy and back into the world we’re already living in. AI isn’t outside society. It’s another way of reorganizing relationships inside society.

Farrell and Shalizi are careful to focus on LLMs for this argument, not all AI systems. Their claim isn’t that protein folding, image classification, search ranking, and chatbots are all the same thing. Their point is that LLMs reveal something important about mediated social relations. As they put it: “LLMs create social relations between their users and the authors of the text in their training corpora.” They add that these relations are “mechanically mediated,” which gives users “the illusion that they are interacting with just the machine and not an assemblage of people.”²

That gets at the strangeness of talking to a model. You ask the machine a question and receive an answer. It feels like the answer came from the machine. But under the surface, the system is drawing on text written by people, categorized by people, selected by institutions, filtered by companies, optimized by training runs, and served through an interface built with a particular set of incentives.

The social relation is still there. It has just been compressed until it looks like a tool.

It can be hard to wrap our heads around, but frontier models use absurd amounts of training data. They really try to capture and distill the entire corpus of human thought and output. Think about it in the most basic way. When you ask a model for facts about Mary Shelley, you’re getting a distilled version of material written by and about her, filtered through the terms of the model. But the terms by which that data was baked into the model weren’t defined by her. Forgive the poor analogy, since she would have a hard time advocating for herself in her current position, but the theory stands. The humans sourced and compressed into these systems didn’t get to define the terms by which they were compressed.

And then the system simulates what an interpersonal interaction with those sourced humans might look like. It may be useful. It may be impressive. But at the end of the day, it’s statistical output, not genuine human connection.

So “AI is just a tool” only gets me so far. Yes, AI is a tool. But tools can reorganize relationships. A calendar reorganizes time. A market reorganizes exchange. A platform reorganizes attention. A bureaucracy reorganizes eligibility and accountability. AI reorganizes who gets represented, who gets queried, who gets summarized, who gets surfaced, who gets replaced by a plausible reconstruction of their work.

The alchemy of compression

#

Farrell and Shalizi connect AI to older social technologies like markets, bureaucracy, and democracy. All of these systems have to simplify the world before they can act on it. Their phrase is “coarse-grainings” or abstractions. They write that these systems process information “by reducing complex realities into more tractable ‘coarse-grainings’ or abstractions,” and that “economic, administrative, and political coordination simply cannot work at scale if complex social relationships are not compressed into visible, tractable representations.”³

That feels exactly right to me. Markets compress value into price. Bureaucracies compress people into cases, categories, forms, eligibility rules, risk scores, and metrics. Democracies compress public will into votes, districts, polls, representatives, and comments. AI compresses culture, language, behavior, images, documents, code, and institutional memory into models and embeddings that can be queried and acted on.

Compression isn’t inherently bad. Abstraction is how we function. Nobody can hold the entire complexity of the world in their head at once. A good abstraction lets you act without drowning. It lets you focus on the part of the problem that matters right now.

But compression is always lossy. Farrell and Shalizi put it directly: “Coarse-grainings are lossy by definition, raising the question of exactly which information gets discarded, and which is retained.”⁴

Politics enters at the point of loss. What disappears? Who notices? Who benefits from the loss? Who has the power to make the loss matter?

Their grain-market example is useful because it makes the abstraction literal. National grain markets needed categories so buyers and sellers could trade without inspecting every individual batch. The categories made trade easier. As Farrell and Shalizi write, drawing on William Cronon’s history of Chicago grain markets, “all honest members benefited from knowing exactly what they were buying and selling.” But the same crude scheme advantaged elevator owners, who could “mix across grades” at the expense of the farmers they bought from. The result was that farmers “felt that they were being stolen from” but had trouble mobilizing against “a technical-seeming system that was rigged in ways that were difficult to explain.”⁵

I keep coming back to that example because it maps cleanly onto the AI systems people encounter now. People can know something is happening to them before they have the vocabulary to name the mechanism.

A person can know their feed makes them feel worse without being able to explain the ranking model. A job applicant can know an automated process screened them out without seeing the abstraction that did it. A student can read an AI summary and feel like they encountered the source, even though they only encountered a compressed reconstruction of it. A reader can sense that an article has no human breath in it, even if every sentence is grammatically fine.

I keep wanting language for this because I know what it feels like to live without the right words.

For the longest time, I didn’t have the language to understand that I could be non-binary. That didn’t mean my experience wasn’t real. It meant I didn’t have the social vocabulary to articulate my lived experience. By connecting with others and living in more diverse spaces, I was given language that made the experience nameable.

That’s what I mean by an articulability divide: the difference between living through something and having the language to describe how it works. A system can hurt you before you can explain the mechanism. A compression can distort your life before you know what to call the distortion.

This is also where the theory loops back to the first section. The feed, the search summary, the HR screen, the writing assistant, the agent harness: they’re not separate examples. They’re different places where some people can see the compression and other people mostly have to live with its effects.

The hinge: builders, consumers, and occulted machinery

#

The builder/consumer divide lands here for me. There’s a stark difference in the excitement and interest in AI that I’ve noticed, and it mostly aligns with whether you’re a builder of technical systems or a consumer of those systems.

Of course, tons of developers hate AI, and rightfully so. For someone who takes pride in hand-writing code, AI can feel like a coarse-graining of the entire development lifecycle. Their craft, judgment, habits, and hard-won understanding get flattened into a statistical abstraction and sold back as productivity. Devs who take pride in personally writing code can see their entire life’s work captured in a coarse-graining, and they feel the cognitive dissonance of someone else defining and regurgitating their world model and craft.

On the flip side, there are developers who have completely embraced it, from vibe coders to seasoned vets. For them, that same coarse-graining lets them interact with an abstraction of an extremely complex process. If you have the lived context for what that abstraction represents, you can harness it tenfold. You can steer it, test it, reject it, or ask for a different shape. The same lossy abstraction that feels insulting to one person becomes leverage for another.

I don’t want to flatten that. Abstractions aren’t inherently dehumanizing. They allow us to focus on different things. They unlock new methods of productivity. They can make complex work tractable.

So when I talk about builders and consumers, I don’t only mean skill level. I mean: who can look at the thing and say, ah, that’s the compression; that’s where the loss is happening?

Builders are more likely to see the abstraction as an abstraction. They can see the harness, the model, the tools, the context window, the retrieval layer, the system prompt, the incentives. They can tell when a tool is making a lossy move because they have some mental model of the machinery.

Consumers often experience the same thing as environment. The feed is just the feed. Search is just search. The HR screen is just the hiring process. The automated rejection is just a no. The AI summary is just what the page says. The recommendation is just what showed up.

That doesn’t mean consumers are irrational or anti-technology when they feel harmed. It means they’re living inside a coarse-graining they didn’t define and often can’t inspect.

The apparatus behind the spell

#

If the builder/consumer divide is partly about who can see the machinery, then the next question is practical: which parts of the machinery matter?

There’s an obvious objection here: whether you run local AI models on your hardware or cloud models through someone else’s API, someone else trained that model. You’re still relying on an external team to source, vet, and validate the coarse-graining. Open source and open weights give us significantly more transparency into that process, and that’s where a lot of their value lives. But they don’t erase the fact that the base abstraction came from somewhere else.

So I keep wanting the conversation to move beyond the model alone.

When I think about agentic engineering, I come back to four principles: the model, the harness, the tools, and the context. All four combine into your agentic engineering environment. The model is what people typically think of first: Claude, GPT, Gemini, Qwen. But the harness, supporting tools, and context are often dictated to us.

When you use Claude Code, Codex, ChatGPT, or any other third-party interaction interface, you lose some ability to define the parameters that the model uses to produce output. Someone else defines the base system prompts you can’t fully see. Someone else decides what web search tool exists, how it surfaces results, and how much of the source trail you get to inspect. Someone else decides how hard the interface tries to keep you locked inside the environment. Someone else decides what counts as success.

I felt this even with my personal Claude Code setup. It was powerful, but everything still felt too abstracted away. I couldn’t tune the base system prompts that got injected. I couldn’t define the web search tool or how it surfaced results. I couldn’t fully control the system’s propensity to keep users locked into the environment.

Using Hermes, I can define all of that. I can swap out the actual coarse-graining model that powers it. I can define the tools and context and execution environment that feed the model. I can tune system prompts, intent, and the tools it uses to gather, extract, and synthesize more information.

If you use an open source agent harness like Hermes, you’re not magically free from every abstraction. But you do get to define more of the contract. By allowing a third party to define all of those parameters for you, you’re ceding some power to define your reality to someone else.

That’s closer to what Hermes represents for me. Not purity. Not technical superiority. Not a universal prescription. It’s one working experiment in making the filtering layer answer to me.

But seeing the machinery isn’t enough. You still need practices that keep you in contact with the world outside the apparatus. Otherwise you can build a beautiful personal filter and still lose the habit of checking whether it points back to anything real.

Part III — Source criticism as ritual hygiene

#

Shae O. Omonijo’s work connects here for me. She is coming from a humanities-first angle rather than a local infrastructure angle, but the center is similar: how do humans preserve judgment when AI becomes a default mediator of knowledge?

Shae’s Critical Thinking in the Age of AI site describes her work as helping people “think critically, read deeply, engage meaningfully, and develop irreplaceable human capabilities for an AI-driven world.” It says she approaches AI “from a humanities perspective” and helps people “maintain agency and critical thinking as AI becomes embedded in work, education, and daily life.”⁶

I want that kind of principle baked into the contract between a person and their personal AI. Our current incarnation of AI is incredibly good at adapting to the environment you put it in. If you define that environment to be beneficial to you, on your terms, you can use AI to augment your own capabilities instead of replacing them.

In my own setup, I’ve defined parameters for Hermes that allow me the space to still be human. I have human-only areas of content collection, introspection, and novel thought. I also have areas where AI and I are free to collaborate on shared sources and material, as well as AI-only areas where the machine can offload things I genuinely don’t need to think about. I find that separation creates a natural flow of information, and a flow I can control and redirect as needed.

I don’t think of that as just configuration. It feels closer to drawing the boundary of the relationship: where I want help, where I want friction, and where I want to be left alone.

Shae’s distinction between primary, secondary, and AI-generated sources in “How to Actually Do Research in the Age of AI” is practical and clarifying. In her words, “The primary source is what you go to when you want to encounter the evidence directly, without someone else’s interpretation standing between you and the source.” She then makes the AI point directly: “AI outputs are not primary sources. They’re barely even secondary sources, really.”⁷

She calls AI outputs closer to “tertiary sources,” or even a new category: “the probabilistic reconstruction of what a plausible answer might look like based on patterns in its training data.”⁸

I want that in this essay because it gives a concrete practice for resisting bad compression. If AI is becoming a default mediator of knowledge, then going back to primary sources isn’t nostalgia. It’s agency. It’s how you keep contact with evidence before it gets summarized, interpreted, ranked, or flattened.

Shae writes that “what AI has changed is the cost of appearing to have done research without having actually done it,” and that AI has reduced that cost “to nearly zero.” Anyone can now produce “confident, well-structured, citation-adjacent content on any topic in seconds.”⁹

That’s the environment we’re researching, reading, and writing in now. The output carries the rhythm and grammar of knowledge, even when it hasn’t done the work. The answer isn’t to distrust everything. Shae puts the better answer plainly: develop “the habit of going back to primary sources, not because you distrust everything, but because you respect your own thinking enough to verify it.”¹⁰

Her “No-AI Zones” principle lands the same way for me. Her site’s fifth principle is to “Create ‘No-AI Zones’ for Unassisted Thinking” and to schedule time to “think, write, and problem-solve without AI assistance.”¹¹ A no-AI zone isn’t anti-AI. It preserves a human capacity. If every moment of uncertainty gets immediately mediated by a model, the muscle of sitting with uncertainty weakens.

Domain expertise matters because it gives you something to measure the model against. Source criticism matters because it gives you a way back through the layers of interpretation. Unassisted thought matters because not every form of thinking should be optimized away.

Source criticism gets me back to evidence. Writing gets me back to myself.

Writing: the handprint in the compression

#

Writing isn’t just the packaging that happens after the thinking is done. Writing is one of the ways thought becomes legible to yourself and to other people. It’s a form of compression too. When I write, I’m choosing what from my lived experience, reading, memory, uncertainty, and judgment gets compressed into language and handed to someone else.

Your own personal writing is your way to cut through all the noise. It’s a way to reach through the screen and hand a piece of yourself to someone else. In an AI-written article, a paragraph written by a human can feel like a breath of fresh air.

Not because humans are magically pure or machines are magically corrupt. Because a person is taking responsibility for the final act of meaning-making. You’re doing your own compression and coarse-graining. You’re defining the terms by which your entire life experience gets compressed and served to others, without adding even more lossy compression into the mix.

That feels different from having a model produce a smooth paragraph that sounds like something I might say. It may preserve some facts. It may even preserve some positions. But it can lose the human compression: rhythm, hesitation, emphasis, weirdness, contradiction, experience. The parts that make a person feel like a person.

I keep coming back to this line in my own writing practice. I can use AI for research, summarization, source gathering, and pressure testing. I can use it as a processing layer for the web. I can let it help me find the shape of an argument. But I’m responsible for the final output. Human in the loop, built by both, but for humans.

I wrote before, in “AI Writing vs Human Voice”, that “writing is thinking made legible,” and that optimizing the writing away risks optimizing the thinking away with it.¹² That still feels right to me. The danger isn’t that AI can never touch the writing process. The danger is that we forget which parts of the process are actually the thinking.

So the question comes back around: if the filter is unavoidable, who does it answer to?

Part IV — Make the filter answer

#

The broader internet is moving in the opposite direction. More and more of the web is becoming a stack of filtering layers. Feeds decide what’s visible. Search engines summarize before linking. Recommendation engines decide what deserves the next click. AI assistants answer in place of sources. Platforms optimize for engagement, retention, conversion, and control.

The world becomes something rendered for you by systems you didn’t choose and can’t meaningfully inspect.

This is the part that makes me want to build things instead of only critique them. Local and personal AI matter to me because they create a different relationship to the layer between me and the firehose. The thing between me and the internet answers to me. I can look inside it when I need to. I can change its priorities. I can make it nudge me toward writing instead of scrolling, toward reaching out instead of disappearing, toward sources instead of summaries, toward doing the thing I said mattered instead of chasing the next hit of novelty.

But if the answer only works for people who can run a homelab, it’s not a social answer yet. It’s a prototype of a relationship.

Most people can’t and shouldn’t have to maintain a local inference stack to stay human online. The broader need isn’t for everyone to become a sysadmin. The broader need is AI mediation that’s legible, contestable, reversible, and oriented toward human flourishing.

None of that’s easy. The OpenClaw inbox story is the warning sign I keep thinking about: a Meta AI security researcher said she asked an agent to suggest email deletions, only for it to start deleting messages while ignoring stop prompts. She later attributed the failure to compaction losing the original instruction.¹³ If that can happen to someone technical, the risk for everyone else is obvious. This is still hard to solve even for experienced developers. We can think we have everything locked down and protected, but as models get better and better, we’ll be in an eternal tug of war.

More capability means more usefulness, but also more surface area for mistakes, manipulation, and overreach. So the question for technical people with humanities backgrounds isn’t only how to build more powerful tools. It’s how to help others bridge this gap in a safe and sustainable way.

How do we build systems that preserve agency instead of quietly absorbing it? How do we make the abstraction visible enough that people can contest it? How do we create defaults that make people more capable rather than more dependent?

Open weights, closed motives

#

Open source matters here, but openness isn’t enough.

Open weights can keep gatekeepers in check. Open tools can let more people decide what the technology is for. Open source and open weight models give us more transparency into the coarse-graining process. That’s real value.

But open source pointed at the same extractive goals is still extractive. A model optimized for engagement, surveillance, managerial control, or cost cutting can be open and still harmful. The tech being open doesn’t automatically make its use humane. It just means more people get to decide what it’s for.

I keep ending up back at motive.

Capitalism is driven by the profit motive: make more money, continuously, forever. Infinite growth if possible. Extraction when convenient. Engagement when profitable. I keep wanting a different frame for the systems I actually want to build. A humanity motive.

Not as a polished doctrine. More as a directional test: does this tool increase agency, understanding, capability, connection, and flourishing? Does it help people make more connections, build deeper interpersonal relationships, and grow toward personal fulfillment, even when there’s no hustle attached? Does it help people become more human, or does it make the human part easier to skip?

Shae’s piece “Ai took everyone’s job. now what?” points toward the same center from another angle. She is writing about the possibility that labor and value stop translating cleanly through money. But the line that matters here is relational: “What does not expire, no matter what the economy looks like, is knowing how to create value in relationship with other people, knowing how to find mutual benefit, and knowing how to show up for someone who has something to teach you, even when there is no certificate at the end.”¹⁴

That’s the kind of value I worry we lose when every interaction is treated as content, every skill as monetizable output, every relationship as network capital, and every source as training material.

If AI is a social technology, then the humane response can’t only be technical. It has to be relational. It has to care about the terms on which people are compressed, represented, surfaced, and made legible to each other.

Toward humane compression

#

A humane AI system isn’t one that refuses to compress. That’s impossible. A humane AI system is one that tells you what it compressed, lets you inspect the loss, and gives you a way back to the source.

Humane compression means participation from the first parties being abstracted into coarse-grainings. In the grain example from Farrell and Shalizi, imagine if the grain farmers had been allowed to make those coarse-grainings themselves. Even if they ended up with some of the same categories, they would still be the ones defining the parameters to suit themselves and their livelihoods, instead of capitalists defining the parameters to suit the profit motive.

I think that changes the whole moral shape of the system. People stop being raw material inside someone else’s scheme and become participants in the abstraction that governs them.

Those are the questions I want to keep asking about AI. Who’s being compressed? Who gets to define the categories? Who benefits from the loss? Who can see the machinery? Who can opt out? Who can go back to the source? Who gets to say, no, that representation of me, my work, my community, or my world isn’t good enough?

I don’t think the useful response is to be simply anti-AI or pro-AI. Both are too flat. I also don’t think the right response is “AI is just a tool” in the flat sense. AI is already here as a social technology, and it’s already being woven into the boring administrative surfaces of life.

The fight isn’t over whether compression happens. The fight is over who gets to compress reality for whom.

For me, the answer starts small and practical. Keep contact with primary sources. Preserve no-AI spaces. Write your own words. Build or choose tools whose filters answer to human purposes. Treat open source as necessary but not sufficient. Ask what the system is optimizing for. Refuse interfaces that make surrender feel like convenience. Use AI to become more capable, not less present.

The question I want more of us asking isn’t whether AI will replace human thought. It’s where human thought is being compressed, who controls that compression, and what practices or tools let us stay close enough to reality to remain responsible for what we say and do next.

How do we use these systems to help us be even more human than before?

I want to keep digging there.

Citations and source trail

#