Correct, writing code does not make money. However this change is a little bit like starting a blog... You don't see the immediate benefits straight out but you'll realise them over time. You can put it off and put it off but the longer you do the harder it gets, especially in this scenario.

This architecture doesn't make sense for everyone and we could have continued with Django. But making these changes now mean that we will be able to fix all the above a lot quicker going forward.

Don't take this the wrong way, but you have 3 people writing code right now according to your Team page on the website.

Isn't it way more confusing to have a bunch of different code repos that only a single person owns? Sounds to me like you'll be locking up too much code in a single individual's head. Not to mention if they're using different frameworks and libraries you're only going to add to the headache.

For smaller teams, it usually is best to stick with one or two codebases/languages for your backend. Sure there are often "better" tools for specific applications but at the end of the day there's usually a way to handle them with your existing codebase that will let you scale for now.

As an example, we needed a robust messaging framework to support our chatrooms and private messaging features. We originally built it out in Tornado, then switched to NodeJS with a rabbitMQ backend, and finally realized it was easier for us to just use PubNub and fire off the pushes from our main Django app. It'll scale with us for probably 10x our current size and by that time we'd have a team of 4-5 people dedicated just to this feature anyways.

Finding architecture that makes sense to everyone is a tall order.

And of course, ignoring the problem of duct-taping all the services together.

"Oh we'll just RabbitMQ everything together", sure, great, until you discovered certain services needed a smaller latency, more bandwidth, etc

And RabbitMQ libraries are a pain and are not smart

actually this is a question that I have - how do you render the final output ?

I mean lets say I request a web page which depends on 5 microservices. What happens ? does rails or whatver block till all five have returned the output - or do you use something like ESI ... how are the microservices composed finally ?

Adopting a microservice architecture does nothing to define how you "render your output"- the devil is still in the details, but that is why a microservice is so interesting and flexible.

The way you will tackle this will largely depend on many questions like what services are safe to be exposed publicly, what services are dependant on other services, do you have a public API?

For example, you could have a public facing API on each service, or endpoints that render HTML templates. The services may or may not talk amongst themselves, either via the same public API, or a private one. If you make a request to one of the services, and that service needs to make a request to another service to generate the response, you've got to make a call on whether you want a blocking or non-blocking approach. Consider:

1) Blocking until the other services have responded

2) Pinging the server to check when it's done

3) Using web sockets

4) Whatever the hell else you can think of

Alternatively, you might want not want to expose your services publicly, and choose to have a single service that talks to the services on the clients behalf, a middleman. Your client speaks to this service, and the service sends of various requests to the other services. But the problem still stands. To block, or not to block?

AFAIK some big names like Amazon, Spotify, (Twitter?) have multiple public services. You may sometimes notice individual components on their [web] applications failing without bringing down the whole site, since the failures are isolated to the different services. Neat, huh?

One approach that makes sense in an enterprisy way is to use an ESB. BPEL can help here. Your ruby app calls the ESB on a service. Behind the scenes the ESB composes the calls efficiently into a final product for you. The enterprise system bus will aggregate the parts into a useable whole defined by your domain.

A benefit of this is that people get a unified view of the system. You can pull parts as needed. You can survive partial outages better than a monolithic app.

Of course you can still call each of the services independently. The ESB is just another tool in your box.

Upvoted.

Tried RabbitMQ, had huge issues that standard library barely functioning. Running direct code samples from support failed, finally supporting dev confessed, "I don't know java and the java guy is busy."

I was forced to move on.

This is not as terrifying as you might think. You can use this well for prototyping. You can make a simple endpoint fairly quickly in Go, Erlang, Clojure, Ruby. It could do some small task and pop out an answer. You can deploy this behind the wall of an named service.

If the service proves useful, you can migrate it to a more appropriate language. By appropriate, I mean a language that fits your companies culture. Java Shoppe? Java. C# Shoppe? F#.

You can also get a better match for the domain. For example Python has some great math libraries. They are easy to pickup, pleasant to use and reasonably performant. Say you need such features. Boom! Python service doing what it does best while the rest of your shop can use the tools that work well for the domain.

Micro-services force separation of concerns, theoretically making a system more maintainable. Micro-services do little for complexity though - they simply move complexity from a monolithic app to somewhere else and spread it out.

Yeah, from my experiences with my current company, standard language/framework == good, one-off apps that now need to be maintained in production == bad.

what happens when those devs who chose some new language leave? Who gets stuck with all that code?

That's a good reason to have shared responsibility of code. There is no one owner of a codebase, an entire team should own the code. An entire team leaving at once seems highly unlikely.

Yeah, but that's harder when everyone knows Erlang, and one guy goes off and writes something with, say, Django for fun.

But if someone goes "lone wolf" and writes an application in a language he or she chose without informing other members of the team/organisation something else might be going wrong in the chain.

I get the idea you're trying to convey here but in the case of a well-organised team/organisation that won't really fly as everyone will be involved at some point and choices are being made together.

Pair programming comes in handy here.