News for hackers. Stuff that matters.
Re:Technical debt as an opportunity to accumulate technical ... by kristianc on Monday August 15, 2016 @01:45PM (#12292039)
Here's a question: if you have a team and an internal culture that managed to create a legacy code mess in as little as six months in their existing app, how is giving them a microservices architecture and Docker to play with going to help?

If you give that team that structure, it seems to me you're highly unlikely to end up with beautifully bounded context and highly likely to end up with even more of a mess than you started with.

Re:Technical debt as an opportunity to accumulate technical ... by hinkley on Monday August 15, 2016 @01:58PM (#12292150)
That's a very excellent question, to which I have most of an answer, if not 'the' answer.

Daylight is the best antiseptic.

It's not giving them a microservices architecture and Docker that solves problems. It's expecting them to use it that causes problems to be solved, or at least formally identified.

I do a lot of root cause analysis whether my boss asks for it or not. Human factors are almost always in play. A significant source of bugs that make it into late QA or even production are caused by wishful thinking about the scope and impact of code changes. Things that should have been red flags are brushed off because you can't prove that their code caused the problem, and as long as there is reasonable doubt about the source, some people won't look at code they already declared done.

When the code is developed and tested in an isolated system then the only source of state changes on that system were caused by the new code. It takes a real set of brass ones or an extremely dense skull to deflect concerns about problems seen on a system that only contains your code changes. People either shape up or get labeled as untrustworthy. The former result is preferable, but at least with the latter you get predictability out of the system.

Re:Technical debt as an opportunity to accumulate technical ... by 0xfaded on Monday August 15, 2016 @03:11PM (#12292680)
I would argue it would be simpler to introduce test coverage tools that automatically call out bad test coverage during code review.

Microservices remind me of a particular C++ protocol implementation I wrote as a novice programmer. Since the protocol was structured, I had high level classes broken down into simpler classes. e.g.

  virtual class Reader {
    void read(Buffer *buf);
  };

  class TCharacter: public Reader {
    TString *name;
    TList<TStat> *stats;
    ...
  };

  class TStat: public Reader {
    TWord32 *remaining;
    TWord8 *percent;
  };

  class TWord32: public Reader {
    uint32_t v;
  };

  class TWord8: public Reader {
    uint8_t v;
  };
And on, and on, and on, and on ...

I thought it was cute that down to the simplest PODT, every type satisfied a well defined `Reader` interface. I hand wrote many constructors, deconstructors, read and several other interface definitions for each type.

Elegant, arguably. Inefficient, yes. Overly complex, undoubtedly.

Every argument for microservices has in some way reminded me of this code, with it's well defined interfaces and overly verbose implementation.

My current strategy is to utilise the crap out of language features to safely achieve what I want in the minimal amount of code possible. In C++ this would be done by initialising the protocol structure classes straight out of the memory buffer, using #pragma pack as required. No error prone constructors / deconstructors / read() required.

Haskell and Scala are two languages I use a lot, and each have powerful type systems which I can use to prevent myself from doing something stupid. Programmable macros are great for removing boilerplate. Want less technical debt? Write less code.

In my own controversial opinion, if somebody can't work out how to modify my code because they can't work out how to correctly update the type definitions, that person has no business modifying my code. Problem solved! Interfaces are well defined and modifiable only by people who truly understand the code.

Re:Technical debt as an opportunity to accumulate technical ... by ScottBurson on Monday August 15, 2016 @03:50PM (#12292918)
> Want less technical debt? Write less code.

I love this Dijkstra quote: If we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger.

Re:Technical debt as an opportunity to accumulate technical ... by bluejekyll on Monday August 15, 2016 @04:21PM (#12293113)
When I was in College, I really only appreciated Dijkstra as a genius who invented neat multi-processing techniques.

As a formal Software Engineer, I now consider him to be possibly the most enlightened engineer that's ever existed in the field.

Re:Technical debt as an opportunity to accumulate technical ... by duaneb on Monday August 15, 2016 @05:21PM (#12293552)
> As a formal Software Engineer, I now consider him to be possibly the most enlightened engineer that's ever existed in the field.

On one hand, his quotes have the ring of wisdom. On the other, the man was famously not an engineer and never brought a software project to fruition, to my knowledge.

These aren't necessarily contradictory, but you should keep it in mind. Knuth is much more of an engineer in the sense it's used on HN.

Re:Technical debt as an opportunity to accumulate technical ... by bluejekyll on Monday August 15, 2016 @07:33PM (#12294378)
Knuth is brilliant and wonderfully pragmatic, no doubt. I think the greatest thing Knuth really got into me is that code is meant for people, not computers.

As to Dijkstra; enlightenment doesn't necessarily mean productive. Some of the least successful people I've ever met, can have absolutely sage like advice. Dijkstra was able to visualize things others could not, and explain novel solutions to those. On top of that he had a deep understanding of the life of a programmer and the complexities faced day in and day out.

Re:Technical debt as an opportunity to accumulate technical ... by geon on Tuesday August 16, 2016 @08:19AM (#12296930)
> Some of the least successful people I've ever met, can have absolutely sage like advice.

You learn from mistakes. Unsuccessful = many mistakes?

Re:Technical debt as an opportunity to accumulate technical ... by jhayward on Tuesday August 16, 2016 @09:07AM (#12297186)
The kind of creativity and acceptance of uncertainty that leads to insight is often at odds with the narrow focus get-it-done tenacity associated with success. Some people have more of the former without the ability to switch to the latter.
Re:Technical debt as an opportunity to accumulate technical ... by ht_th on Tuesday August 16, 2016 @07:22AM (#12296674)
Dijkstra worked as a programmer at the Mathematisch Centrum (mathematics center) in the 1950s. He was responsible for most of the systems programming of three subsequent machines that were build and used there. His PhD dissertation (1959) was on the operating system he wrote for the Electrologica X1 computer that was being built by the first Dutch computer industry Electrologica.
Re:Technical debt as an opportunity to accumulate technical ... by computerex on Monday August 15, 2016 @04:14PM (#12293064)
The primary offering of microservices is decoupling and simplicity. They are language agnostic, communicating via HTTP /messaging/pattern matching. It's the opposite of verbose and traditional monolithic architectures that relied on a rich domain model. Microservices should be small enough to be owned by a single developer. With pattern matching, you can extend functionality by creating new microservices instead of modifying existing ones. It boils down to change management. For complex systems, the benefits microservices bring to the table make the overhead cost of maintaining these services worth it.
Re:Technical debt as an opportunity to accumulate technical ... by dasil003 on Monday August 15, 2016 @10:01PM (#12295022)
The problem which I don't see discussed nearly enough when people start drinking the microservices koolaid is what does the interface between the microservices look like? If you can define a nice stable interface that changes much less frequently then the constituent services, then it's mostly a question of operational overhead. However, if the microservices comprise many cross-cutting business concerns, then the churn on the interfaces is a massive source of pain compared to (for example) running one process that loads multiple modules where you can leverage all the power of that languages to do basic wiring / type checking ensuring the whole things fits neatly together.

IMO, microservices or SOA or whatever you want to call it, is primarily a method of organizing large teams so they don't get hamstrung by ops coordination. In that case you absolutely need it, but for smaller teams the overhead will usually far outweigh the benefit. I'm glad the space is being explored so that we get better tooling and techniques to lower that overhead, but when the dust settles I expect many small teams will discover that we collectively overestimated the better-understood pain of monolithic architectures and underestimated the less-understood pain of microservices.

Re:Technical debt as an opportunity to accumulate technical ... by 0xfaded on Monday August 15, 2016 @04:37PM (#12293225)
> Microservices should be small enough to be owned by a single developer

This seems bad for code review, and disastrous with turnover. Whoops, now we need to find a scheme engineer whose eccentricities are roughly equivalent to this last guy.

There are also ways to achieve microservice-like architectures without requiring multiple code bases. See Actors (scala akka). Streams are an extension which give automatic load balancing. And if you need it to run over the network, akka-remote has you covered.

Re:Technical debt as an opportunity to accumulate technical ... by hinkley on Monday August 15, 2016 @05:16PM (#12293523)
I've not been happy with this kind code ownership in the past. Those who want to misbehave have a nice comfortable place to hide out while appearing relevant because they have a monopoly on an idea that they don't have the skills to be responsible for.

So then you have your coworker who wrote a wrapper around the code to sanitize all of the inputs and outputs and it's bigger than the actual code and once in a while it guesses wrong about ambiguous data.

Re:Technical debt as an opportunity to accumulate technical ... by duaneb on Monday August 15, 2016 @05:22PM (#12293562)
This somewhat sidesteps an advantage of microservices that they are language agnostic.
Re:Technical debt as an opportunity to accumulate technical ... by 0xfaded on Monday August 15, 2016 @05:34PM (#12293653)
I've never been convinced this is an advantage.

Ryanair (the discount carrier) has a fleet of 354 boeing-737s.

This means they cannot fly some routes the bigger jets can, and other routes they burn more fuel than the smaller jets would.

BUUUT ... maintenance and logistics are drastically cheaper and efficient.

By analogy, I would argue that costs sunk either in training or hiring for specific skills will pay off in the long run.

Re:Technical debt as an opportunity to accumulate technical ... by twunde on Monday August 15, 2016 @07:05PM (#12294190)
I will say that it can be an advantage, but it's rare. It becomes an advantage when your primary language lacks the tools to do something well. One of my previous companies was a PHP shop and they used Python to chop up audio files, which would have been very cumbersome to do in PHP (few if any existing libraries). My current company implemented complex data pipelines in PHP, which really should have been done in a different language (python or scala would have been natural choices).
Re:Technical debt as an opportunity to accumulate technical ... by gedrap on Wednesday August 17, 2016 @10:07AM (#12304766)
I am in a very similar situation right now and I totally agree with you on this one.
Re:Technical debt as an opportunity to accumulate technical ... by crdoconnor on Tuesday August 16, 2016 @06:17AM (#12296480)
>The primary offering of microservices is decoupling and simplicity.

Microservice architecture is very much orthogonal to loose coupling. I've worked on several microservice architectures with tight coupling. The microservice aspect actually exacerbated the pain caused by the tight coupling because it added the risk overhead of serialization/deserialization and network failures, which simply don't exist in a 'monolith' context.

Microservices when implemented are usually either a reflection of conway's law (ID team in building B has its own service; as does the CMS team in Europe), or fashion-driven development (one team working on 17 different services... which can be nasty).

Re:Technical debt as an opportunity to accumulate technical ... by crdoconnor on Tuesday August 16, 2016 @06:14AM (#12296471)
>Want less technical debt? Write less code.

Also: write less powerful code.

* Replace 'clever' metacode with straightforward imperative code where possible.

* Replace imperative code with declarative code where possible.

* Replace declarative code with configuration where possible.

Re:Technical debt as an opportunity to accumulate technical ... by mattmanser on Monday August 15, 2016 @04:06PM (#12293007)
What does this even mean?

The previous poster is basically saying "they screwed it up last time in 6 months, they'll screw it up again".

Microservices don't help. If they couldn't properly design their code without micro-services, throwing docker and micro-services in the mix won't make it magically better. It'll add massive amounts of complexity right off the bat. They'll still be bad coders and bad architects. They still won't know how to lay out their code. Having to split up their code will probably make a bad situation even worse.

They'll put the wrong methods on the wrong microservices, and share some 'key' code that shouldn't be shared as the method's on the wrong microservice, or worse still, cut and paste code and have it decay at different rates. They'll create new services that should actually be on an existing service and then gradually they code will duplicate, but with random subtle bugs.

The code debt won't disappear, it'll accelerate until they have a bunch of services they daren't touch and one of those services will become "the monolith" and eventually no-one's allowed to deploy to it or the whole thing will come falling down.

And all you're saying is "daylight".

What does "daylight" mean?

The very idea that micro-services will remain 'isolated' in a bad coding team is utterly deluded.

Re:Technical debt as an opportunity to accumulate technical ... by calinet6 on Monday August 15, 2016 @04:16PM (#12293076)
It's not microservices that help.

It's changing the culture around how things are developed and understood, and the parent poster is saying that exposing the team to daylight using something such as an architecture like microservices (or another "good practice") can help push the team toward a better understanding of what a good codebase and stable state should look like.

Stop thinking about the specific buzzword hyped up trendy term you're stuck on here; start thinking about the psychological factors that caused the situation in the first place, the environmental pressures that resulted in poor decisions, and the ways you can teach the team—by doing—how to create better applications.

Also:

> They'll still be bad coders

Most often that's not the root cause. Most often the root cause is poor management and poor leadership causing otherwise decent programmers with decent instincts to make decisions against the best interest of themselves and the company. The most common classic debt generation situation is trading off long-term quality for short-term speed and functionality. That is not "bad coders" at work; it's bad managers.

Re:Technical debt as an opportunity to accumulate technical ... by chupy on Tuesday August 16, 2016 @07:16AM (#12296663)
I am not a manager and though I agree that sometimes there can be issues with managers/leadership I am really curious why you are defending coders."Coders" that actually do not try to improve themselves and read up on good practices and need managers to actually impose those practices are not really decent coders.
Re:Technical debt as an opportunity to accumulate technical ... by calinet6 on Thursday August 18, 2016 @10:29PM (#12317609)
I'm not defending coders, I'm rejecting the individual mentality—that people are just good or bad by nature, and aren't in some (large) sense a product of their environment as well.

Attribution bias tells us that we tend to see behavior as more a product of individual traits than the system that produced it. The system is more responsible than we interpret in almost every case.

Re:Technical debt as an opportunity to accumulate technical ... by hinkley on Monday August 15, 2016 @04:40PM (#12293247)
The 'daylight' line is me butchering a quote from Louis Brandeis about transparency, 'sunlight is said to be the best of disinfectants' commonly rendered as 'sunlight is the best disinfectant'. That's what I get for posting from mobile.

I'm neutral on microservices, pro Docker.

What I'm properly alergic to is arcane setups where it's easier for everyone to share a server than it is for anybody to set up a copy of the system that is theirs and theirs alone. In a big enough shop, running all of the microservices you need on your dev box might be possible while running the whole system isn't, because you run out of memory before everything loads.

The bad coders often continue to be bad coders because nobody can 'prove' that it's their fault and so they keep dazzling the managers with bullshit and implying that you are the one with the problem, not them. Isolated, repeatable systems means you have to look at how crazy your architecture is instead of ignoring it, and regarding this conversation, there's a paper trail backing up your version of the story.

When people don't know which solution is better, they tend to back the side that has more trustworthy people on it, where trustworthy is "doesn't make messes, or helps clean them up when they do".

If you make it clear who's the problem and the project management still doesn't intervene, take your skills elsewhere. You're quitting with cause and many managers will value your commitment to sanity.

Re:Technical debt as an opportunity to accumulate technical ... by pavel_lishin on Monday August 15, 2016 @04:10PM (#12293036)
Imagine a single monolithic jar file, that contains all of the code that lives on any given Docker instance, that decides which services and API endpoints to expose based on poking and prodding environmental variables.
Re:Technical debt as an opportunity to accumulate technical ... by andrewmcwatters on Monday August 15, 2016 @04:24PM (#12293129)
I almost never see comments like this on HN, because everyone prefers these cryptic, pretentious responses. Surprised this wasn't downvoted.
Re:Technical debt as an opportunity to accumulate technical ... by karmelapple on Monday August 15, 2016 @04:46PM (#12293293)
I really didn't feel it was cryptic or pretentious. However, if you do, can you help describe what made it difficult for you to understand?

The daylight word worked great for me, since the idea is to have everyone be able to see what's happening when they introduce changes.

If there's just one staging / integration environment where people are building things and patching over one another, and your local environment maybe has deviated a lot from either the staging, production, or even a clean local env, then there's all kinds of unexpected problems that might happen.

Getting more things, including your environment, under version control and requiring people to deploy things via files that live in version control rather than a smattering of deployment or env setup commands that not all of the team has visibility into, helps the entire development process go more smoothly.

Re:Technical debt as an opportunity to accumulate technical ... by hinkley on Monday August 15, 2016 @04:26PM (#12293147)
| cryptic, pretentious response

:(

Re:Technical debt as an opportunity to accumulate technical ... by gvb on Monday August 15, 2016 @02:50PM (#12292552)
"Corgibytes often introduces new clients to Docker, making it much easier and faster to set up new developer environments." (emphasis mine)

She isn't advocating using Docker for the target environment, she is advocating using Docker to set up a standard development environment that allows the developers to be productive[1] in 10 minutes rather than 10 hours.

I've been in the situation where the development environment is a poorly documented, massively multi-step steaming pile. The result is I spent a lot of time chasing down those responsible for the steaming pile (the "rockstar" in the department) asking them how to get the development environment working. Typically they pull the keyboard away from me, type a bunch, and then say "there, it works." Which it does. For a while.

Rinse and repeat for every developer. Regularly.

[1] Well, at least building the system rather than staring at mysterious very broken error messages.

Re:Technical debt as an opportunity to accumulate technical ... by pavel_lishin on Monday August 15, 2016 @04:13PM (#12293055)
> Typically they pull the keyboard away from me, type a bunch, and then say "there, it works." Which it does. For a while.

This would happen if they distributed Docker environments, too. Your setup process would be to download a Dockerfile, crank up everything you need, discover it doesn't work, call the Rockstar over, have him or her explain that there's a few manual steps that they haven't yet added to the codebase (or to the wiki), and then you rinse and repeat that process, instead.

Re:Technical debt as an opportunity to accumulate technical ... by fapjacks on Tuesday August 16, 2016 @01:12AM (#12295666)
No, because once your Dockerfile works, it works fucking everywhere.
Re:Technical debt as an opportunity to accumulate technical ... by qznc on Tuesday August 16, 2016 @04:44AM (#12296236)
Oh wait, you need to configure your networking bridge differently ... oh wait, the DNS needs this little tweak ... oh wait, you still use this old version from your distros repo? ... oh wait, you must give it this docker volume ...
Re:Technical debt as an opportunity to accumulate technical ... by fapjacks on Wednesday August 17, 2016 @12:08AM (#12302451)
I can tell you're frustrated from too much yak shaving. But once you become more experienced with systems in general, you'll feel much more comfortable around Docker.
Re:Technical debt as an opportunity to accumulate technical ... by pavel_lishin on Tuesday August 16, 2016 @10:22AM (#12297615)
Which is great if you're the person in charge of distributing it, but if the person in charge keeps giving out the "slightly broken" version, everyone will keep fixing it.

Or, and I'm not sure if this is worse, a few people will fix it in slightly different ways, and start distributing those, leading to a weird ecosystem of competing Dockerfiles none of which are quite correct.

Re:Technical debt as an opportunity to accumulate technical ... by fapjacks on Wednesday August 17, 2016 @12:12AM (#12302462)
Well, what I mean is that once you get your Dockerfile to the point where it works, it will work literally anywhere you've got Docker installed. That is one of Docker's core value propositions. It's very hard to get something "halfway working" in Docker, because you've either got a process running containerized, or you don't have anything at all. "Slightly broken" in the context of Docker isn't really something you run into very often.
Re:Technical debt as an opportunity to accumulate technical ... by collyw on Tuesday August 16, 2016 @09:05AM (#12297180)
You have to set up docker first.
Re:Technical debt as an opportunity to accumulate technical ... by hinkley on Monday August 15, 2016 @04:48PM (#12293304)
You know what else happens in that situation?

Nobody will volunteer to help you fix an obscure bug that only happens reliably in your version. Because it's too painful for them to shelve their local configuration and grab yours. And once people have that hesitation, 'all hands on deck' emergency fixes become window dressing, because only a few people can actually participate in tracking down and triaging the bug. Everyone else is sitting around waiting to be needed for the two minutes that they can contribute.

Re:Technical debt as an opportunity to accumulate technical ... by calinet6 on Monday August 15, 2016 @04:17PM (#12293084)
And really, the fact that we're caught up on this at all is concerning. It was one small suggestion in a sea of rather intelligent discourse mostly on team environments, structures, and management. Not technology.
Re:Technical debt as an opportunity to accumulate technical ... by rogerdpack on Monday August 15, 2016 @05:01PM (#12293411)
Isn't "setting up your environment" pretty much a one time cost, though? In my head at least, losing a couple days up front doesn't matter as much, it's the time after that the needs to be optimized. But maybe that's just my work place where people tend to stay on the same team for years...
Re:Technical debt as an opportunity to accumulate technical ... by marcosdumay on Monday August 15, 2016 @05:28PM (#12293609)
Shouldn't be.

When you automate "setting up your environment", you are able to switch between several of them, create new ones for forks or extra tests, and roll back to debug production issues.

Re:Technical debt as an opportunity to accumulate technical ... by infinii on Monday August 15, 2016 @11:08PM (#12295273)
Ignoring the one time aspect. It's the standardization that is important. Docker environments enforce uniformity that are beneficial for testing. No more of the "it works on my machine" excuses. Although the article mentioned development environment specifically, there is recurring benefit in being able to power up a Docker VM for testing purposes.
Re:Technical debt as an opportunity to accumulate technical ... by collyw on Tuesday August 16, 2016 @09:05AM (#12297177)
While I agree this sort of thing can be frustrating when starting a new company, I find that it is useful. If i can't get the system working on my dev machine, how can I be expected to debug the production site when there are problems?
Re:Technical debt as an opportunity to accumulate technical ... by marcosdumay on Monday August 15, 2016 @02:56PM (#12292587)
Hum...

You are trading an opaque irreproducible environment for another opaque environment that you reproduce by copying verbatim.

Technically, it's progress. But I can't imagine how people get satisfied by this.

Re:Technical debt as an opportunity to accumulate technical ... by Arcsech on Monday August 15, 2016 @03:05PM (#12292642)
I'm not too experienced with Docker, but aren't Docker images built using Dockerfiles, which are essentially glorified shell scripts?

In that case, wouldn't the Dockerfile be a relatively transparent way to see what's needed to reproduce an environment?

Re:Technical debt as an opportunity to accumulate technical ... by lallysingh on Monday August 15, 2016 @04:04PM (#12292993)
Yes, yes they are.

And they're written down where you can read them yourself. And maybe even version controlled!

Re:Technical debt as an opportunity to accumulate technical ... by GrinningFool on Monday August 15, 2016 @05:25PM (#12293587)
On the other hand, tracing back a dockerfile that pulls in an obscure one-off base image, then repeating that all the way up the chain, you can find yourself left with a dozen Dockerfiles that you need to read in the right order to know what you have.

That's if you can even find the original Dockerfiles - I recently ran into a situation where a container image was in the registry, but the original Dockerfile was completely lost.

Docker/containers in general move the complexity to a different plane, but they only move it - they don't eliminate it. Care needs to be taken to avoid situations just like the one above - or any of a number of other pitfalls - or you end up with an environment that's far more complex to understand than a VM housing a monolithic dev environment.

Re:Technical debt as an opportunity to accumulate technical ... by biocomputation on Monday August 15, 2016 @01:52PM (#12292102)
Depending on what you're doing, the right decisions might only be visible in hindsight.

To this point, you might scaffold out a system to get something running and then incrementally / organically improve the system. I regard this as 'getting the system to the point where it can be dogfooded and/or criticized in a helpful way'.

At this point, despite what you might have running, it may be exceedingly clear that you should have used a different strategy. Maybe you like the overall results, but you need things to be implemented in a different way.

For example, I'm a C++ programmer. It's not always obvious at first that you need a specific set of virtual functions to make something work really nicely.

It's impossible to always know these sorts of things in advance.

Re:Technical debt as an opportunity to accumulate technical ... by Ericson2314 on Monday August 15, 2016 @02:03PM (#12292188)
I assume you are saying this is why refactoring is so important? If so I absolutely agree.

Designing good design/abstractions like anything else takes practice. Armchair critiques won't cut it. If one's unwilling to fix bad design once it's identified, they're not practicing good design, they're practicing cobbling together something on a rickety foundation. How's that going to help next time one rights something from scratch?

Re:Technical debt as an opportunity to accumulate technical ... by biocomputation on Monday August 15, 2016 @02:06PM (#12292211)
Yes, to me, 'software development' is 'refactoring', just like some authors say that 'writing' is actually 'rewriting'.
Re:Technical debt as an opportunity to accumulate technical ... by ben_jones on Monday August 15, 2016 @01:58PM (#12292155)
What if microservices are poorly architected such that they are actually coupled? It's worth thinking about such an outcome especially if microservices are being seen as a way of making dysfunctional teams functional.
Re:Technical debt as an opportunity to accumulate technical ... by biocomputation on Monday August 15, 2016 @02:05PM (#12292201)
You're right, I'm just saying that it's not always possible to see these problems in advance, and that it can be very difficult to try and get something done if you apply too many rules to a nascent system ( too many rules == design by committee ).
Re:Technical debt as an opportunity to accumulate technical ... by maxxxxx on Monday August 15, 2016 @02:11PM (#12292253)
Yeah, that's the 'shiny new toy' fallacy. In every project there are points when it's clear that something wasn't designed right. Some people decide to get deeper into their mess and some people decide to refactor. It's a judgement call and not an easy one.

I don't think I'd want to work with microservices and Docker containers designed by the same people who messed up another codebase.

Re:Technical debt as an opportunity to accumulate technical ... by sbov on Monday August 15, 2016 @02:35PM (#12292447)
Considering management has a very large role in most messed up codebases I've seen, even if they fire 100% of the engineering team who coded it you probably are working with (for) the same people who messed up a codebase.
Re:Technical debt as an opportunity to accumulate technical ... by mseebach on Monday August 15, 2016 @02:52PM (#12292569)
That's a pretty dangerous mindset, and one that seems to be pretty popular around here, that all failure is automatically the fault of management. Yes, management plays a role, that's almost tautological, but it takes two to tango, and at the end of the day, the dev team wrote the messed up code. Grandly messing up in just six months, even if encouraged by management, takes willing participation from the developers.

Also, I've seen plenty of developers who were perfectly capable of screwing up codebases without being induced to do so by management (I've also seen lots of inept management, don't get me wrong. But I wrote the bad code, they didn't.)

Re:Technical debt as an opportunity to accumulate technical ... by sbov on Monday August 15, 2016 @03:27PM (#12292784)
Yes, it takes two to tango, which is the correct answer. Any other answer tends to oversimplify the problem.

A lot of engineers also seem to evaluate other engineers by their worst work. This is a toxic attitude. You can even take an engineer performing terribly in a good environment, and put them in a different good environment, and they can perform exceptionally. I've seen it happen more than once.

Re:Technical debt as an opportunity to accumulate technical ... by e28eta on Monday August 15, 2016 @08:23PM (#12294617)
As I understand it, it's a very common human failing called the Fundamental Attribution Error. I only have a passing familiarity with it, but its very seductive.

https://en.m.wikipedia.org/wiki/Fundamental_attribution_erro...

Re:Technical debt as an opportunity to accumulate technical ... by bluGill on Monday August 15, 2016 @04:26PM (#12293144)
It is always the managers fault: they hired bad people, and didn't train them. Or maybe they setup bad expectations (doesn't mater if the people are good or bad). Their job is to know how to hire the right people and get the best out of them.

Note, the engineers have an ethical responsibility to not be bad. However bad is sometimes creating perfect code when something else is called for.

Re:Technical debt as an opportunity to accumulate technical ... by GrinningFool on Monday August 15, 2016 @05:34PM (#12293655)
We're using 'management' as a standin for 'business requirements' here, at least in the most cases I've seen that lead to accumulation of 'debt'.

These things generally start with, "We need feature X for customer Y in Z days.". Engineers dive in, and then comments start popping up - usually variations on a theme: "This is a mess, but it works for now - we'll come back later and clean it up."

The unhealthy process that leads to that is a different discussion. But as engineers, our responsibility is to be aware that it's a reality, and work within those constraints.

Yes, there will be time/resource/budget limitations. No, we're not going to fix that overnight. But knowing it going in, we can take a closer look at some of those 'good enough for now' decisions, and do our best to not produce dog shit.

Sometimes dog shit can't be avoided, but if the pattern above is the normal way of doing business in a company, engineers do have the ability (and responsibility) to do the best they can within those constraints.

Often that means pausing and saying, "I know this is going to come back and bite us, what can we do now to make it better?"

Re:Technical debt as an opportunity to accumulate technical ... by mseebach on Monday August 15, 2016 @05:13PM (#12293507)
Telling yourself that failures are someone else's fault (while, presumably, still taking credit for good work) is a great way to never learn from your mistakes.
Re:Technical debt as an opportunity to accumulate technical ... by tormeh on Monday August 15, 2016 @05:13PM (#12293506)
What about when management (and financial realities) pushes relentlessly for new options, customizations and deployments? When all time not spent shipping new variants is spent developing features? What then? When is the code going to be refactored? In the developers' spare time?
Re:Technical debt as an opportunity to accumulate technical ... by mseebach on Monday August 15, 2016 @06:03PM (#12293828)
What about it? Yes, that environment will obviously be less conducive to good code than otherwise. But bad coders will do (a lot!) worse than good coders.

Don't assume that all good code bases out there were written by teams with reasonable, well-defined and stable requirements, plenty of time and money and perfectly enlightened management. Very few projects are like that. Generally, I think you'll find they were written by good developers who kept their heads cool in the face of a range of challenges.

Absolutely, they will also largely not have had bosses that death marched them or changed requirements three times a day - as I said, management do obviously play a role.

Re:Technical debt as an opportunity to accumulate technical ... by maxxxxx on Monday August 15, 2016 @05:58PM (#12293795)
Certainly not in spare time. It has to be part of the daily work. I compare it to running a restaurant. Cleaning up the kitchen every night takes time but it has to be done no matter the circumstances. You can't skip it or the health department will come after you. Unfortunately in programming you can get away with taking shortcuts for a long time.
Re:Technical debt as an opportunity to accumulate technical ... by mseebach on Monday August 15, 2016 @06:10PM (#12293873)
Incidentally, good cooks are extremely tidy, they clean obsessively as they go - and they are under insane pressure, and don't wait for management to allocate time for cleaning.

They clean continously, not primarily to make the end-of-shift cleaning easier, but because it allows them to execute faster, better and more consistently (which in that environment is a necessary condition for executing at all).

Re:Technical debt as an opportunity to accumulate technical ... by taeric on Monday August 15, 2016 @10:07PM (#12295048)
This is... Not necessarily true. I've known some really great cooks that have to have a cleaning staff.
Re:Technical debt as an opportunity to accumulate technical ... by tboyd47 on Monday August 15, 2016 @03:11PM (#12292683)
But it's only manager's fault when they try and do their employee's jobs for them, and this is a lot more common than you think.
Re:Technical debt as an opportunity to accumulate technical ... by maxxxxx on Monday August 15, 2016 @02:40PM (#12292485)
Management is a part of the team that messed up the codebase, not only the engineers. That's why I am always wary of managers bringing in new methods (microservices, scrum, whatever). They had a big part in previous failures and if they don't admit their own failures nothing will change.
Re:Technical debt as an opportunity to accumulate technical ... by collyw on Tuesday August 16, 2016 @09:20AM (#12297270)
The majority of managers don't give a shit about code quality. A fairly significant percentage of developers don't give a shit either.
Re:Technical debt as an opportunity to accumulate technical ... by mikeryan on Monday August 15, 2016 @03:04PM (#12292640)
Considering management has a very large role in most messed up codebases I've seen, even if they fire 100% of the engineering team who coded it you probably are working with (for) the same people who messed up a codebase.

It seems to me you should stop working for managers who are writing code.

Re:Technical debt as an opportunity to accumulate technical ... by maxxxxx on Monday August 15, 2016 @03:15PM (#12292704)
Even if they don't write code they can set unrealistic deadlines and actively not allow refactoring. My solution is usually not to tell anybody who doesn't need to know and just refactor but that can be a dangerous path too.
Re:Technical debt as an opportunity to accumulate technical ... by mikeryan on Monday August 15, 2016 @03:27PM (#12292786)
Even if they don't write code they can set unrealistic deadlines and actively not allow refactoring.

I know it might sound glib but if thats the case then you're allowing yourself as a programmer to be set up to fail and be left holding the bag when things go to shit.

Either manage up in those situations or find a new job.

I've definitely cut corners for deadlines in my life but everyone always understands what debt we're accruing and when we'd get stuck paying that debt. If your status quo is "Do shitty work" you really need to find a new gig.

Re:Technical debt as an opportunity to accumulate technical ... by maxxxxx on Monday August 15, 2016 @04:59PM (#12293395)
Unfortunately that's the situation most devs find themselves in.
Re:Technical debt as an opportunity to accumulate technical ... by jasonlotito on Monday August 15, 2016 @03:40PM (#12292858)
> I don't think I'd want to work with microservices and Docker containers designed by the same people who messed up another codebase.

How often are the people who are currently maintaining the messed up codebase the ones who also originally designed it?

Re:Technical debt as an opportunity to accumulate technical ... by maxxxxx on Monday August 15, 2016 @04:18PM (#12293088)
They are the ones who didn't refactor it though.
Re:Technical debt as an opportunity to accumulate technical ... by beat on Monday August 15, 2016 @02:37PM (#12292465)
Doesn't need to be an "and". Just getting the development/ops environments into a virtualized architecture like Docker can make a tremendous difference in a monolithic app. Quite often, the technical debt is due to configuration differences in manually managed environments - development, QA, and production can differ in small or large ways. I don't know how many times I've seen things like developing on Windows and deploying on Unix, or testing on a single server but deploying into a cluster. Docker (or Vagrant, or others) can fully automate provisioning and configuration, simplifying and enforcing consistency between environments.

Likewise, microservices from monoliths can be done without containers. Just pick a chunk of the monolith that can be pried loose, and start with that. This can solve all sorts of problems.

Re:Technical debt as an opportunity to accumulate technical ... by lostInTheWoods3 on Monday August 15, 2016 @01:48PM (#12292061)
Agreed. Cleaning and correcting a code base is only part of the solution. Putting in place the policies and culture to make sure the code stays clean is the other part.
Re:Technical debt as an opportunity to accumulate technical ... by mscottford on Monday August 15, 2016 @06:04PM (#12293832)
You're right to say that a microservices architecture and Docker won't solve the problem. That's not an accurate view of what Corgibytes advocates for. Microservices and Docker are tactical tools that can help many teams, but they don't work everywhere.

What does work is assisting that team to slowly clean up the mess that they've built[1] by focusing on incremental improvements.

[1]: And we find that almost all messes were created for very understandable reasons.

Re:Technical debt as an opportunity to accumulate technical ... by benologist on Monday August 15, 2016 @02:51PM (#12292561)
It's not even "technical debt" until someone thinks "technically ... we could have done this more easily in X".

Now it's a question of developer comfort and huge investment will be made to make sure everyone building the product are having the best experience doing so.

All the old code and knowledge is now debt because it can't come with us on this wonderful journey.

Re:Technical debt as an opportunity to accumulate technical ... by codingmyway on Tuesday August 16, 2016 @10:18AM (#12297590)
It's only debt if you are paying for it going forward.

The thought should be "things would be easier going forward if we'd done X".

If it does the job and doesn't create issues then you just purchased it without taking on debt. This is easier if it is isolated/loosely coupled.

There's no point going back to improve it unless it saves you time/money in the future. If you make a better one to get the 'best experience' you are throwing away time and money for a slightly better mousetrap.

Re:Technical debt as an opportunity to accumulate technical ... by benologist on Tuesday August 16, 2016 @03:05PM (#12299771)
It's ironic because we're trying to build software others will depend on, meanwhile the software that we depend on tries to avoid the kind of disruptive change that accompanies rewritten-with-X... that's why we can depend on it.
Re:Technical debt as an opportunity to accumulate technical ... by jdavis703 on Monday August 15, 2016 @01:57PM (#12292146)
It's easier to just start over again with one small part. I've worked at two places with massive monolithic tech debt ridden codebases. It was nearly impossible to rewrite anything because everything touched everything. Now the place I currently work has micro services. Some are legacy tech debt that no one wants to touch. But if needed they could probably be rewritten in a couple of months. It's the difference between selling AAA bonds and junk bonds. You're probably far more likely to pay off those low interest AAA ones than the junk bonds.
Re:Technical debt as an opportunity to accumulate technical ... by hinkley on Monday August 15, 2016 @02:29PM (#12292396)
The tradeoff though is that you can't get a handle on how expensive a particular call is because the domino effect of knock-on calls is very difficult to see. You can get a system that 'works' on test data but doesn't scale to production (from a user's perspective, this code doesn't work, even though the dev team will insist that it does)
Re:Technical debt as an opportunity to accumulate technical ... by computerex on Monday August 15, 2016 @04:28PM (#12293161)
Engineering is about trade offs. Microservices are a fairly recent innovation, and there are many kinks that need to be worked out. Another issue is the overhead cost of maintaining a large number of decoupled, independent services. It's plausible to have hundreds or thousands of microservices, and the tools to visualize/manage these systems haven't yet been created.
Re:Technical debt as an opportunity to accumulate technical ... by hinkley on Monday August 15, 2016 @05:12PM (#12293495)
I'm often in the minority opinion so I'm used to it, but to me microservices aren't fundamentally different than the message passing systems which we've tried many times. But when something doesn't work we try the opposite of it instead of just less or more. Monolithic server got you down? MICROSERVICES. Microservices got you down? Put everything into one process space!

Uh, how about we put all of the related stuff together so we have 8 services instead of 800? Could we maybe try that? How about something really crazy, why don't we give Conway's Law a try and arrange our code around organizational boundaries or vice versa?

Re:Technical debt as an opportunity to accumulate technical ... by erichocean on Monday August 15, 2016 @05:04PM (#12293432)
> Microservices are a fairly recent innovation

Yeah, no. Micro-services are neither recent, nor innovative.

Fun fact: a friend of my consults for large pensions funds in The Netherlands. One of the projects he was brought in on earlier this year to get an outside perspective on was this gigantic monument to micro-services that had been built up over a two year period by ~200 developers, at a cost of around 50 million euros.

After two months, every developer was fired, the project cancelled, and the entire 50 million euro cost written off.

But it had great micro-services, hundreds and hundreds of them!

Re:Technical debt as an opportunity to accumulate technical ... by chadcmulligan on Monday August 15, 2016 @08:33PM (#12294668)
they are to the kids who make web apps - most of this stuff they talk about on the internet is by kids who make web pages for start ups. I used to rant and say similar things, I now see the value of peer reviewed material. Basically 90% of developers advocating things on the internet are not peers, I look for how many projects they've developed successfully in the new gee whiz tech before reading too much. Really most of this stuff was originally thought of in the 70's and 80's, we just keep going in circles.
Re:Technical debt as an opportunity to accumulate technical ... by austingunter on Monday August 15, 2016 @05:13PM (#12293503)
Correct. Microservices are a fundamental tenant of how Unix pipelines. They go back (at least) to the 70s. Source: https://articles.microservices.com/the-origins-of-microservi...
Re:Technical debt as an opportunity to accumulate technical ... by tormeh on Monday August 15, 2016 @05:20PM (#12293547)
Tell me about it. The problem is moving parts. Whether you spread it out over several programs or not does not matter. If you manage to really decouple programs so they can do useful work independently of one another then that's great. Congratulations to you. But microservices can just as easily turn into one huge program separated into many small ones, none of which make any sense on its own, and now all you've done is make things harder for devops.
Re:Technical debt as an opportunity to accumulate technical ... by hinkley on Monday August 15, 2016 @11:29PM (#12295352)
I heard a phrase a couple months ago that I believe sums this up. What you want is local reasoning. You need to be able to estimate the scope of consequences of a change so you can act accordingly. You also need to be able to look at a problem and estimate the scope of code that could be causing the problem.

We have a bunch of rules and guidelines that work toward that goal but if you don't know the destination you may never arrive.

Re:Technical debt as an opportunity to accumulate technical ... by matwood on Monday August 15, 2016 @06:05PM (#12293843)
One of the main causes of technical debt and bad code overall is breaking out of the conventions of the code base. Good code reviews can help this, and solid leads of course, but things can still slip through.

Like or hate microservices, one thing they do is maintain a clear, hard boundary on the interfaces. There is no let me just reach into this class here, or call that there. It does work to keep people honest when the pressure is on.

Will it turn bad programmers into good ones? No, but I think the base assumption is that you're not necessarily dealing with bad programmers.

Re:Technical debt as an opportunity to accumulate technical ... by collyw on Tuesday August 16, 2016 @09:34AM (#12297357)
Personally I would say that crappy abstractions (or no abstractions) are what makes code shitty.