Don't do it

Now I'll also do it out of spite! :)

Don't say I didn't warn you

Naw, you still don't want to, at least not in a context in which you're billing enough to be a real workout, because the non-engineering aspects will still get you. Like this: https://news.ycombinator.com/item?id=37557989 which, in short, refers to the fact that nobody in the company considers billing a system to be designed in the first place, so it gets approached with a truncated Waterfall that doesn't even have a requirements analysis phase, so you'll have no support whatsoever for your attempt to bring higher quality to billing.

Also, with all due respect to those working on billing I have seen that it is generally handed to engineers who are, shall we say, adequate for the task but no more, and unless you're 100% greenfielding you're going to be dealing with their stuff no matter what.

From another cynical point of view, it is a situation where you can fail miserably and bring great pain to the company that is traced directly to you, but there is no success; a functioning billing system is just what is expected. It's all downside and no upside for you personally.

I have a billing system (a component of the larger whole) and I'm pleased to be able to hand it off next month. I've not enjoyed my stay there. Though it was miles better than when I brushed the data lake/data analysis world. If I had to maintain this system for another few years I could and I wouldn't scream if I had to do it again someday. Dealing with the data analysis world beyond just throwing them some data is a "never again".

> ...miles better than when I brushed the data lake/data analysis world

The data pond, puddle, and lakeside cottage methodology would have spared you that pain.

More seriously, my experience with that world reinforced my view that it's all monkey motion (much effort, negative value).

Otherwise, spot on summary of IT dev work. Which now applies to "web services" too.

I've long been befuddled by the rise of "agile". It's not meant for product dev, right? Well stupid me took forever to realize that most work is actually IT (aka data processing) and not product dev.

> I've long been befuddled by the rise of "agile". It's not meant for product dev, right?

It...is?

Is "agile" meant for product dev? IMHO, no.

Very loosely, "product dev" projects have concrete deliverables, deadlines, and high cost of change (updates). So therefore are ameniable to PMI-ish critical path like methodologies. (Avoiding the evergreen unresolvable sophistry wrt "waterfall".)

The original motivation for "agile" was to protect the dev team from terrible customers (clients). Those who cannot or will not commit to requirements, actively resist any kind of formalism, have no clue what "done" even means.

Meaning most IT (data processing) efforts. And now most web stuff.

"Agile" is totally rational in those unwinnable situations. Because the dev team would always have something to show for their suffering. Lessening the likelihood of being the scapegoat come the inevitable failure.

Unfortunately, "agile" became a fad, a belief system, an identity. Curtailing any kind of introspection. Only in the last few years have we been able to have calm, frank discussions about "agile". Mostly because some of its progenators have broken ranks and done mea culpas.

> Very loosely, "product dev" projects have concrete deliverables, deadlines, and high cost of change (updates).

Why? A web based product can iterate, for example. We see the most experimentation in places like Amazon, where they're constantly deploying updates and experiments. Why should such a product have a high cost of change?

For APIs, backward compatibility, mostly.

Back when we burned CD-ROMs, distributing updates like bug fixes involved non-trivial effort.

Today, libs/packages and apps are somewhere between "agile" and PMI.

FWIW, for web stuff, I'm very interested in "test into prod" methodology. Neither agile or critical path; its something new. First popularized by the gilt.com fella (IIRC). But haven't yet found any teams (to join) that bold.

> For APIs, backward compatibility, mostly.

But you can still evolve APIs; breaking changes can occur much less than normal changes. And even then you can version them and run old and new in parallel if you want. I don't see why that would be slow.

I also don't see why making things in a way that's hard to change suddenly makes something amenable to agile.

I think maybe our wires got crossed somewhere.

I have never advocated for "agile", for any purpose. Ironically, IMHO "agile" is silly make work (ceremony) and too heavy weight. I was just trying to steelman the original justification for "agile" methods.

The PMI based processes my prior teams used had minimal communication and project mgmt overhead and very little drama. I see no reason why IT / web projects shouldn't be similarly managed. It's just that PMI is now like a lost art.

I only mentioned "test into prod" because I'm very curious about its methodology innovations. I think it would have better modeled and explained some of my prior experiences.

For instance, one of my teams would live code and then commit to prod changes while we were talking with our customers (for immediate verification and so forth). Almost zero overhead or ceremony.

We just came up with that system intuitively and couldn't really explain ourselves to others. (We had one senior VP in particular who reacted very negatively to our processes.) If we had the "test into prod" narrative, maybe things would have gone better.

> but there is no success; a functioning billing system is just what is expected. It's all downside and no upside for you personally.

I just wanted to say thank you for giving a robust voice to the frustrations I've felt after building multiple billing systems. This and your linked comment were a pleasure to read, and simply spot on.

Coincidentally, I am also pursuing a career transition toward data analysis. If I understand correctly, you are lamenting "data engineering" in contrast to "ad hoc data analysis deliverables"?

The problem is there are a lot of detail that is different just for the sake of being different and those will suck your soul. There are interesting problems in billing. However most of the issues are another government has a law that is essentially the same as every other government, but some small detail is trivially different so you need to figure out which subset of customers that government covers and then apply those details correctly to them. Nothing hard about this, just the same old tax but the details are annoyingly different.

Note that you need to get the above details 100% correct in all cases. They can put you in prison for mistakes (almost always they will just fine the company a lot of money, but they have the right to apply prison and you never know when they will want to make an example of someone)

Despite what jerf said, which I whole heartedly agree with, there are plenty of interesting billing problems for a good engineer to sink their teeth into.

But there is also a long, long tail of boring problems that you won’t have time to do properly.

To put it a different way: even after you’ve solved the “mathematical” problems of billing, you still need to solve the operational stuff. Like payments integration, GL integration, invoice formatting, tax, reports, etc etc.

I think most engineers underestimate both the difficulty of the “mathematical” problems, and the length of the long, relatively boring tail.

I’ve spent the last 12 years doing payments-and-billing-focused engineering for various companies. Most engineers, in fact, do not want to do this work. It’s like specializing in plumbing for septic tanks, most people have an innate sense of revulsion of self-protection that keeps them from doing it. You screw up the money pipes, and you have a Real Problem.

However! The problems are interesting, the challenges are endless, and it’s possible in some orgs to get recognized as the person who is keeping the money flowing.

I do have a sense of frustration that every company seems to start off with a bad design for billing, so I’m always cleaning up after negligence… but it pays the bills.

Sometimes I think I want to build an open-source billing engine. To show off what it looks like if you do it right. But would that be killing the golden goose? Currently there’s endless demand from people who don’t know.

My vision just now was to create a sand box with a DSL (that doesn't use floats) and have others write the rules into it.

   VAT.include(country, type, amount){
     if(type == "high" && country == "NL"){
       return percentageOf(price, VAT.NL.high) + amount;
     }
     throw new Error("Error in VAT.include: VAT not found."); 
   }