← All notes

FIELD NOTE ·

I knew the destination. The AI had the compass.

What ten months building with AI taught me about vision, judgement, and knowing when to overrule a confident answer.

7 min read
TRAVIS KEIR

TL;DR: After ~10 months building a Solana Seeker mobile app with AI agents — including a stretch where I quit because the machine kept remixing my vision into something that wasn’t mine — I’ve landed on a working relationship. Not “AI writes my app.” Not “AI is a fancy autocomplete.” Something more like navigation: I hold the map, it holds the compass, and neither of us gets there alone. Here’s what that actually looks like, including the part where it confidently handed me the wrong answer and I had to overrule it.


I gave up on this project once.

Not because the tech didn’t work — because it worked too eagerly. Early on, every time I handed an AI agent a piece of my idea, it would come back with something adjacent. Reasonable. Competent, even. And subtly not what I’d pictured. Do that enough times and you look up one day to find the app has drifted somewhere you never intended to go. The agent hadn’t built my vision; it had built a remix of my vision, and every remix moved a step further from the original. So I stopped.

Ten months is a long time in this field. The tools now are not the tools then — the improvement has been close to exponential, and more importantly I learned how to work with them instead of at them. This post is about that: the division of labour that finally clicked.

The compass and the map

Here’s the metaphor that fits, and I mean it fairly literally.

I envisioned the entire app. I mapped it out in my head — the whole thing, end to end, before a line of code existed. I knew the destination. I knew what it should feel like to use, where the edges were, what “done” meant.

What I didn’t always have was the bearing at any given moment — the exact next technical step through unfamiliar terrain. Solana account models, permanent storage economics, the fiddly internals of getting a signing library to run inside a serverless runtime. That’s where the AI came in. It had the compass. I had the map. Together we navigated.

That framing matters because it puts the responsibility in the right place. A compass doesn’t know where you’re going. If you don’t hold the map — if you don’t already know the destination — a compass just helps you get lost more efficiently. That was my mistake in the quit-the-project phase: I was letting the compass choose the destination. It’s very good at pointing in a direction. It’s not supposed to pick the direction. That’s my job.

The part where it was confidently wrong

I want to be honest about this because the honest version is more useful than the brochure version.

At one point I had a nasty bug — a mint that succeeded on-chain but never finished its final step. I handed the investigation to the agent. It came back with a clear, well-reasoned root cause and a clean fix. Genuinely good work.

It was also wrong. Not wrong about the code it found — that was a real flaw — but wrong about this bug. Its explanation didn’t match what I’d actually observed: the fix addressed a rejected mint, and my mint had succeeded. Those are different animals.

So I overruled it. Not with a better theory — I didn’t have one yet — but with the evidence: “your explanation doesn’t fit what I saw. The mint succeeded and took the fee. A rejection wouldn’t do that. Look again.” And it did, and the second pass found the actual cause.

This happened more than once. The pattern held: the AI was superb at investigation, code-tracing, and generating a plausible, well-structured explanation — and I was the one who had to notice when the plausible explanation didn’t fit the ground truth. That’s not a knock on the tool. It’s the division of labour. It produces the compass bearings; I check them against the map. When a bearing says “this way” and the map says that leads off a cliff, the map wins.

The failure mode to avoid is the seductive one: a confident, coherent answer is easy to accept, especially when you’re tired and want the bug gone. The discipline is holding it against what you actually observed and being willing to say “no, that’s not it” even when you can’t yet say what is.

So what’s it actually good at, and what isn’t?

Since I’m building toward an AI and automation consultancy, people ask me the real version of this question: where does this stuff help, and where doesn’t it?

It’s extraordinary at: tireless investigation (it will trace a code path or diff two merges at 2am without complaint), writing specs and plans, spotting the assumption you didn’t question, generating options, and — underrated — producing a clear, structured artifact of a messy conversation so future-you knows what past-you decided.

It’s not a substitute for: knowing where you’re going, judgement about what’s actually true versus what’s merely plausible, and the standards that decide what “good enough” means. It’ll happily meet whatever bar you set. If you set no bar, it’ll meet no bar, cheerfully.

And here’s a nuance I didn’t expect: sometimes the answer is that you’re better off with the tools you already have — and the AI’s best contribution is helping you get more out of those, rather than replacing them. It’s a guide to your existing toolkit as much as a new tool itself. Some of the highest-value moments in this build weren’t “the AI wrote X.” They were “the AI helped me understand why the boring, cheap, already-in-hand option was the right one, and talked me out of the shinier thing.”

Lesson learnt

Plan, research, then plan again — before any code is written. That double-plan is the whole game. The first plan is you getting the destination clear in your own head. The research is the compass work — what’s actually possible, what it costs, what the terrain looks like. The second plan is you and the agent getting aligned on the route before a single line is committed.

Every time I skipped that and just started building, I got drift — the remix problem, the agent wandering off toward its own reasonable-but-wrong destination. Every time I front-loaded the alignment, the build went clean.

You hold the map. Let it hold the compass. But never, ever let the compass tell you where you’re going.