AI is the new medium shift: Why data confidence will define the future enterprise

Years ago, as I was graduating and moving on from my studies at the George Church Lab at Harvard Medical School, my mentor, Eswar Iyer, handed me a book called As the Future Catches You by Juan Enriquez.

Written in 2001, right at the peak of the internet boom, the book explored how a single wave of technology was going to fundamentally reshape industries, economies, and what it meant to work in science. When I first read it in 2017, the ideas felt provocative - but distant. Looking back now, most of what Enriquez described has come true. And it happened faster than almost anyone predicted.

Lately, I’ve been thinking more about his argument that genetics is a language.

DNA encodes information using four letters: A, T, C, and G. Enriquez drew a connection between the digital revolution and our growing ability to read and alter biological code. Computing would change how we understood living things and, eventually, what we could build with them.

I came to Transcend from genetics, healthcare, and machine learning, so that idea had a particular hold on me. It also raised questions that helped shape what I wanted to work on.

At the time, I thought about those questions in terms of personal data. But a recent conversation within our engineering team about our code review process recently brought me back to these questions from another direction.

The proliferation of AI generated code has broken the code review process that existed at Transcend for nearly a decade. Understanding every line of code is no longer plausible, and the types of errors that need to be caught in review have shifted from logical errors to high level intent.

Several engineers raised a concern I take seriously: reviewing code is how you maintain a mental model of the system you’re responsible for. If you stop following the changes, what happens when you’re the person debugging it at three in the morning? There is something real to lose when less of the work passes through your hands.

Increasingly, our confidence in code reviews comes from several kinds of evidence. Written statements of intent give reviewers something to evaluate the implementation against. Coverage gates require tests before a change can proceed. Agents review changes before a human sees them. Monitor and post-deploy validation checks proactively query the datastore for corruption instead of waiting for a customer to report a problem.

Each of those checks has limits, but I find it incredibly fascinating to see the art of software engineering shift and elevate as we get further and further away from reading every last line of code. We are raising the stakes of every decision an engineer needs to make, work that’s increasingly demanding as the consequences of being wrong grow with it.

Something about this felt familiar.

Genetics has always required people to work beyond what they can inspect and comprehend directly. A human genome contains roughly three billion base pairs. We can read its sequence, but understanding what that sequence means is a different undertaking. You cannot work through those three billion letters one by one and emerge with an understanding of the organism. Instead, researchers work at a higher level of abstraction. They form hypotheses. They identify associations. They design experiments that test whether those relationships hold under particular conditions.

AI is pushing software across a similar threshold, and I think it’s worth naming plainly what kind of threshold this is.

And one of those rules is how we establish confidence in the software we build. For most of the history of software engineering, we have treated comprehension as an important source of confidence. Someone wrote the code. Someone else read it. Collectively, the team maintained a reasonably detailed mental model of what the system was doing. AI is weakening that assumption.

As the amount of software we can produce grows faster than the amount any person can inspect, trust has to move up a layer, especially for systems processing sensitive personal data.

Consider a customer who changes a privacy preference. The preference is recorded successfully, but the change never reaches an advertising system downstream. That system continues doing exactly what it was built to do. Its uptime can be perfect. Its tests can pass. Its dashboards can look healthy. And it can still be doing the wrong thing.

Or consider an AI model trained on data it was never authorized to use. It may produce excellent answers. Model quality tells you almost nothing about whether the data should have been there in the first place.

These are different kinds of failures. They are intent and permission failures instead of execution failures. As software becomes more autonomous, these are the types of failures that can easily slip through the cracks, but are the kinds of failures that a human writing code would naturally catch in the process of development.

An engineer might once have made a few consequential decisions about data in the course of building a feature. An agent can make thousands of decisions about what data to retrieve, combine, infer from, or act on. We cannot realistically put a person in front of every one of those decisions.

Over time, that conviction has increasingly condensed into one deceptively simple question: Can I use this data?

Answering this question is very difficult. You need to know where the data is stored, what someone (or something) is trying to do with it, what the customer has permitted, what the organization has promised, and what rules apply in that context.

And those answers are not static. Permission for one purpose does not automatically imply permission for another. A decision made when data was collected may not settle whether an AI agent can use it six months later for something nobody had imagined at the time.

In software, genetics, and increasingly the broader data economy, the underlying language is becoming too large and dynamic for any one person to fully comprehend. This doesn’t mean we should give up on trying to understand what is happening, but for us to scale our impact, we need to get much better at defining the conditions that matter, gathering evidence that those conditions hold, and deciding where human judgment still belongs.

And I think we are only beginning to understand what those conditions should be.

A healthcare company deploying AI will encounter questions we do not see inside an infrastructure company. A financial institution, consumer brand, research lab, or media company will draw different boundaries. The technical problems overlap, but the consequences, incentives, and definitions of trust can be completely different.

Those are conversations I want to have more of.

Introducing The New Data Economy

Over the past year, I’ve found myself wanting a place to explore these questions with the people actually making these decisions.

So I'm excited to share that we are launching a new podcast: The New Data Economy.

Every episode, I'll sit down with the operators, innovators, and executives who are living at this intersection - people who are making real decisions about how to build with AI responsibly, how to earn and keep customer trust, and what it means to compete in an economy where data is the primary resource and permission is the primary constraint.

We'll cover the hard questions. What does a trustworthy AI initiative actually look like inside a Fortune 500? How are data leaders rethinking governance when agents make thousands of data decisions a second? What does the next five years look like for enterprises that are still trying to unify consent signals across fragmented systems?

These are the questions I'm thinking about every day - and they're the ones I want to explore in conversation with the people doing the work.

If you're building in this space, leading through it, or just trying to make sense of it - I'd love for you to join me.

The New Data Economy Podcast is coming soon! Follow along on LinkedIn and wherever you listen to podcasts.

Mike Farrell is the CTO and Co-Founder of Transcend, the autonomous "can I use this data" platform trusted by Fortune 500 companies and emerging category leaders. Transcend is headquartered in the San Francisco Bay Area and has been recognized as a Leader in IDC's MarketScape for Worldwide Data Privacy Compliance Software and twice named to the Deloitte Technology Fast 500™.