Part 5: Scope Creep During the AI Age

This is Part 5 of the Modern Software for Small Businesses and Nonprofits series. This one is a personal reflection, not research.

AI agents made building software cheaper. They did not make keeping track of it cheaper. I ran into scope creep twice recently, in two very different projects, and the pattern was the same.

The short version

In both projects the list of open items grew faster than my understanding of them, and I want to be closer to the details than I am. I don’t have a fix yet. I do think naming the problem is the first step.

Scenario 1: what happened when my agent kept asking the client questions?

I was working with a client, and we made our first release. Along with the release notes, I had 22 questions that my Claude Code wanted to ask the client. I had inherited the codebase from another team, so there was a lot the agent could not know on its own.

The client replied to all 22 questions, along with more comments and answers on the release. We both acknowledged that we use AI tools to draft our emails. To be honest, we were both proud of our AI usage.

By release 2, Claude Code had 13 more questions, on top of the follow-up to the first 22. By then the agent was dominating my email.

Now we are on release 4 and there are around 100 things going on. I am not at the level of understanding of all those 100 things that I want to be. I am almost like the “non-technical” people manager for my project. The only difference is that now I don’t have any people to manage.

What would 25 open items have meant in my earlier career?

In my previous careers, if we wanted to track 25 such things, I am sure we would have had a dedicated PM, in some cases more than one. We would have had 3-4 teams, and each team would have been working on and following up on 5-6 items in detail.

Now it is me, an inbox and an agent.

Scenario 2: what happened when I built an SEO tool with agents?

I had this wonderful idea to build an SEO tool, the one I described in Part 4. It creates business profiles in a semi-automated fashion, to help small businesses have affordable SEO.

As usual, I used my AI agents. I started with a $20 Codex subscription, which, the way I was using it, hit its limit about every 30 minutes, and ended up on my Claude $200 subscription. The agent built something, asked me questions with recommendations, and of course I went with them. Finally, we had an app running on my laptop.

Then I had a problem. First, I didn’t know how to use it. When I clicked a button, I didn’t know whether the button was broken or was not supposed to work yet. Agents also sometimes won’t build, even when you say “build it”. They may need another “go”.

And since I was busy building many things, I had no time to test, because I was leveraging agents to test as well. I am not testing and I am not “actually” building, so I am very far from my own project. I know very little about it, because the scope is just too big to handle next to many other projects.

Is this an isolated incident?

I don’t believe so. I think it is happening almost on a daily basis to all of us. We are sending emails and building apps that we don’t fully understand.

How do we solve this? I don’t really have an answer. But as with many problems, the first step in solving it is identifying it as a problem, and that part is done. :)

What do I take from the two scenarios?

Scenario 1: we all knew this would happen. At some point our AI agents will talk to other people’s AI agents, and we should watch out for the traps. Take an online store as an example: its agent will be optimized to sell. The customer’s agent is optimized to find cheap deals. Who wins? Let the future unfold.

Scenario 2: maybe we should do fewer projects, so we can stay in the details of all of them. Maybe this is a learning process. We do things, we make mistakes, next time we avoid them, fix them and adapt. Slowly we get things right with the new tools.