The Work We No Longer Have to Do

The Work We No Longer Have to Do

A recent study, AI Assistance Reduces Persistence and Hurts Independent Performance, asks an uncomfortable question about our increasingly helpful machines. Researchers recruited 1,222 participants for experiments involving fraction arithmetic and reading comprehension. Some participants received assistance from ChatGPT, while others worked independently. When the researchers withdrew AI assistance, the previously assisted participants performed worse and were more likely to give up on difficult problems.

The effects appeared after as little as ten minutes of AI use.

The findings invite an alarming interpretation: we're becoming less capable because machines are doing our thinking for us. But the study's limitations matter. The researchers measured performance immediately after removing assistance. They don't know whether these effects persist for hours, days, or years, or whether prolonged AI use might produce compensating skills. One particularly interesting finding was that participants who asked for hints and clarification fared better than those who requested direct answers, although that comparison wasn't itself randomized.

Rather than concluding that AI is making us stupid, I'd like to begin with a different question:

What happens to human competence when the activities through which we traditionally acquired it are no longer necessary to produce competent results?

When Stack Overflow Went Down

There's an old software developer joke: "Stack Overflow is down. I'm taking a long lunch."

It was funny because, well before generative AI, developers had accepted external information as part of their working environment. Nobody was expected to memorize every programming interface, configuration parameter, or obscure error message. You looked things up, found someone who had encountered your problem, and applied whatever you learned.

Of course, there were different ways to use Stack Overflow. You could read several explanations, understand the problem, and adapt a solution. Or you could copy the highest-voted answer, paste it into your code, and hope for the best. The latter occasionally worked, too.

Now consider an updated version of the joke:
"Hey, I ran out of Codex tokens."
"Well, you can still do your job, yeah?"
"[Sigh] Yeah, but I'm going to have to burn more tokens bringing the agents up to speed afterward."

I work in DevOps and site reliability engineering, where agentic coding tools have become a regular part of my workflow. The joke reflects an actual inconvenience. I can continue working without an agent, but the work I perform independently may subsequently need to be explained to it. Otherwise, its understanding of the project falls behind.

This isn't necessarily cognitive incapacity. It's workflow dependence. But the distinction becomes less reassuring when we ask how much independent practice we're getting.

Eight Sources and a Report

Consider a recent task: asking Codex to examine five Confluence documents and three code repositories, then produce reports describing an API service's functionality and operationalization.

Traditionally, this would involve hours of reading, searching, comparing implementations against documentation, tracing dependencies, and writing up findings. Much of that effort could now be delegated to an agent.

What, exactly, has been lost?

Possibly very little. If I already understand the architecture, the agent might simply accelerate information gathering and documentation. Or its findings might reveal connections I hadn't noticed, improving my understanding.

But there's a third possibility: the agent produces a convincing report without my developing an equally convincing mental model of the service.

All three scenarios could result in the same deliverable.

There's another problem I've come to think of as premature coherence. A language model can reconcile conflicting documents into a beautifully readable explanation, silently smoothing over inconsistencies that a human investigator might have found revealing. An undocumented dependency or contradictory configuration can disappear into an authoritative-sounding paragraph.

Consequently, I've developed the habit of asking agents to identify what I've missed, overstated, or minimized. It's an invitation to criticize rather than merely assist, although the criticism itself requires scrutiny. Models can manufacture objections as readily as they manufacture agreement.

The work hasn't disappeared. Some of it has relocated from producing an explanation to evaluating whether that explanation deserves to exist.

Crate Digging and Stem Browsing

A friend recently sent me a musical collage assembled using Logic Pro's stem-separation capabilities. It drew upon recordings by Nina Simone, Kendrick Lamar, Donny Hathaway, Roberta Flack, Sly Stone, Ornette Coleman, and several others. The result reminded me of the greatest DJ Shadow recording that had somehow never been made.

Our friend Adam observed that there's a considerable difference between crate digging and stem browsing.

For sample-based musicians such as DJ Shadow and John Oswald, working with existing recordings traditionally involved considerable friction. Finding an interesting fragment might require listening to hundreds of records. Extracting a useful passage meant working around everything else captured in the original mix.

Stem separation changes the available materials. A vocal, bass line, or drum performance can be isolated from a completed recording and placed in an entirely new musical context.

The technical obstacle has diminished, but the artistic decision hasn't disappeared. Our friend's experience as a professional musician and producer informed his selection and arrangement of those otherwise improbable collaborators.

Someone else could have access to exactly the same recordings and software without producing anything remotely as compelling.

Still, Adam's distinction raises another question. What happens to the discoveries that once occurred during the hunt? Crate digging wasn't merely an inefficient retrieval process. Unexpected encounters were part of the education.

Making a task easier can eliminate frustrations and opportunities simultaneously.

What the Modular Synthesizer Left Out

I've encountered something adjacent in my own musical collaborations. A friend in the UK periodically sends me short modular synthesizer recordings, which I turn into instrumental compositions.

Two recent snippets suggested a groove once I processed them through a Ringshifter and delay. A funk-pop drummer in Logic Pro helped establish the rhythm. Later, while walking the dog, I imagined a keyboard figure inspired by the harpsichord-like lead in Broadcast's "The Book Lovers."

The resulting 124 BPM instrumental acquired a Vintage Clav lead and Mustang bass line. In its second half, a high-register Bass VI, processed through a rapid rotary effect and a Tera Echo, takes over a variation of the keyboard pattern. A Matrix-12 emulation supplies the underlying pad. The modular fragments reappear in an interlude and at the end.

What's important is that my collaborator's contributions created negative space for the other elements to occupy. Neither of us necessarily knew what the eventual composition would become.

AI didn't compose that music, although software assisted several parts of the process. The work consisted largely of listening for possibilities and making choices. The tools enabled decisions that might once have required considerably more equipment, time, or expertise.

Compare that with prompting a generative music model for a completed instrumental. The result might be perfectly enjoyable, but the activities involved in producing it would be substantially different.

And that difference returns us to the original study.

What We Practice

The researchers investigated skills for which independent practice is itself valuable. Learning fractions requires working through fractions. Reading comprehension develops through reading and interpreting passages. If an assistant supplies the answers, successful task completion may conceal the absence of learning.

Professional and creative work are less straightforward. An engineer doesn't necessarily need to memorize every API declaration. A producer doesn't need to master obsolete audio-extraction techniques to make interesting music. Technologies routinely make certain competencies less necessary while creating demand for others.

What concerns me isn't cognitive offloading in itself, but the possibility that we'll stop practicing forms of judgment because the tools make their exercise optional.

An agent may generate correct Terraform without teaching its operator why the configuration works. It may identify an operational vulnerability without the engineer learning how to recognize the next one. It may produce persuasive criticism that the recipient accepts simply because it sounds informed.

Conversely, an engineer might use those same tools to investigate more systems, test more hypotheses, and encounter more architectural possibilities than previously feasible. A musician might explore arrangements that were once technically inaccessible.

The study establishes a short-term cost of certain kinds of AI assistance. It doesn't establish which of these longer-term trajectories will prevail.

I'd be interested in an experiment comparing engineers who work independently, engineers who delegate extensively to agents, and engineers whose agents are instructed to question their assumptions. Measure their productivity, certainly, but also their understanding, error detection, ability to recover from failures, and effectiveness when the agents are unavailable.

We might discover not simply greater or lesser competence, but different kinds of competence.

And perhaps that's the difficulty facing us. The artifacts we produce increasingly tell us less about what we personally know how to do. A beautiful musical collage, a comprehensive operational report, or a successfully deployed service may represent deep engagement, sophisticated delegation, or something between the two.

The output alone cannot tell us which.

The question isn't whether we should preserve every tedious activity technology makes unnecessary. We never have, and there are good reasons for that. It's whether we can recognize which apparently unnecessary activities were also teaching us something worth retaining.

I don't particularly miss spending hours hunting through documentation for an obscure configuration. But I would rather not discover, during a production incident, that the investigation was where I learned to understand the system.

The machine can do more of the work. What remains to be determined is what that leaves us knowing.

Subscribe to The Grey Ledger Society

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe