That depends on what you mean by "small startup". In a VC-backed Silicon Valley startup where you're hiring fast and aiming for massive growth in as short a time as possible? You're probably right. Small startup that's self-funded or close to it with a tiny team, like what happens in most of the rest of the world? Yeah, you're probably rolling up your sleeves and getting your hands dirty for quite a while to come, because the code that you're selling has to come from somewhere.
Different startups have different constraints. It sounds like you're used to the SV-style startup where the constraint is time, rather than money. A lot of businesses have their most senior technical staff write code because they don't have the cash to hire more developers but their product still needs to be built.
So, if you're the CTO in your second year of business and still writing most of the code, you're probably a small business.
That said, in my experience, if you're a CTO and not able to at least understand the source code, you're a liability. I've worked for CTOs like that, and they were unable to manage risk or estimate beyond random guesses. And, it's unlikely that such a person can forecast or expand business safely without at least some trusted advisors.
I think Paul Graham uses the term "scalable". But I have seen talking points from lots of big VCs and other figures in business who seem to think it's more about age, profitability, and management structure that define a startup.
To focus on the scalability point, I tend to agree that many only think of scalable businesses as startups, but I think the predicted numbers don't have to be in the billions to be considered a scalable growth business.
If you are a one man shop building an innovative product and you raised seed money from friends and family and hire 2 employees and expect to start making $1m a year as a 3 man shop in a few years, I think most people would call that a startup.
On the other hand, if you raise funds and build just another restaurant on the corner but with no innovation in either product or scale, then I think you are a small business.
Using Google NGram viewer, I looked up "startup" and quickly noticed that the word (in modern usage) seemed to be short for "startup costs". Interestingly, "startup costs" seemed to only come into use after the 60s and 70s and was often connected to financial instruments. So, I also plotted the word "options" and was surprised that the shape of that curve was the same as the curve for "startup". They just had different magnitudes.
I also learned that stock options became tradable instruments in the 70s. And, also, startups began being formed in the 70s. That is, they seemed to go hand in hand. In particular "startup" was used to describe speculative research and development ventures with high capital requirements.
More recently though, the word "startup" became being associated with small teams, minimum viable products, and micro seed funding. So, that could be what you mean.
In a small business, consultancy, or more bootstrapped world you're completely right. Theres plenty of technology businesses that I've worked with where the CTO is one of the co-owners or the engineer who's been there the longest and they'll come on site to fix equipment or get on the phone with me to walkthrough unexpected behavior/etc.
EDIT: yep, just as I suspected :D https://www.quora.com/What-will-John-Carmacks-role-be-at-Ocu...
The analogy I use is that a CTO doing coding is like a CFO doing invoice processing or a COO dealing with routine office management matters.
What this seems to be calling CTO is more akin to a most senior engineer/fellow/hacker. I've seen it called Chief Engineer before. That's the person the the CTO should be able to hold their own in a conversation with but being that person would seem be unlikely for an exec team member as the business grows.
*Titles are more to less meaningless unless there is internal conflict or you're interacting with someone external, ignore that bit and think in terms of roles