Re:How We Use Microservices and Event Sourcing to Instantly ...
by cjsaylor on Friday May 22, 2015 @01:17PM
(#9589692)
I'm currently working on a side project that has this sort of microservice layout with RabbitMQ and have found similar findings as the article.
One thing that has been a big win (which isn't mentioned in the article) is the concept of "duplication of data". Our microservices are currently listening to everything and only taking action on things they care about. As such, they are also storing data about things they care about. This makes it so that no single datastore is a bottle neck in a distributed architecture. This doesn't mean we don't have a centralize DB, but in the heat of operation it doesn't rely on a centralized DB. Using event sourcing makes it very easy to take this distributed data and store it in a consumable way for reporting, etc.
Another thing being it's a much different metric you have to worry about than traditional apps. Instead of worrying about load on a server, your main concern is messages processed per time frame. This can vary depending on what the service is doing, but in general you can simply monitor RabbitMQ's queue processing rate to determine if you need to scale a particular service or not.
It's a bit hard to debug some concurrency problems, but is normally solved by asking: "Who's responsibility is this?" and then adjusting who emits what.
They mention planning out events; we go a bit further and use json schema to describe exactly what is expected in that event. We actually have a dedicated development microservice that simply listens to all events and validates them against the json schema and records/notifies what events (and the source) is not being compliant.
In the beginning of development, our throughput was dreadful, but now that we have all the tools in place, adding a new service or handler of an event in a service is incredibly fast.
Re:How We Use Microservices and Event Sourcing to Instantly ...
by lukeroberts1990 on Friday May 22, 2015 @01:22PM
(#9589728)
Hey author here!
One thing that has been a big win (which isn't mentioned in the article) is the concept of "duplication of data". Our microservices are currently listening to everything and only taking action on things they care about. As such, they are also storing data about things they care about.
I couldn't agree more with this statement. I wish I'd gone into this, I was trying to cover a lot here about our new architecture in general.
In the beginning of development, our throughput was dreadful, but now that we have all the tools in place, adding a new service or handler of an event in a service is incredibly fast.
Yeah, we're adding a lot of new functionality and handlers really fast now. The great thing is when we need to split a service out into a new service we simply grab all the handlers for that event and the new service is pretty much already in a proof of concept state!
Re:How We Use Microservices and Event Sourcing to Instantly ...
by cjsaylor on Friday May 22, 2015 @02:14PM
(#9590004)
Absolutely.
On top of that, it also makes it easy for front end interfaces to be designed with a mock microservice (or handler) that simply emits it for their testing with a backend developer going through and filling it. As long as the API isn't overly complex for emitting events, almost anyone on the team can do this.
Re:How We Use Microservices and Event Sourcing to Instantly ...
by mamcx on Friday May 22, 2015 @05:14PM
(#9590922)
Any resource in how do Event Sourcing from the start? I have read a bit now, but is mostly high-level talk. Wonder if exist something like the django start tutorial...
Re:How We Use Microservices and Event Sourcing to Instantly ...
by awsmsrc on Saturday May 23, 2015 @05:07AM
(#9592396)
It would be really tricky to provide a resource on starting event sourcing. Its usually only necessary when you have a pretty complicated and unique problem.
It's more of an approach to higher level architecture than a technique. (think MVC as opposed to Rails). There is no starting MVC tutorial.
Re:How We Use Microservices and Event Sourcing to Instantly ...
by awsmsrc on Saturday May 23, 2015 @04:41AM
(#9592354)
Kafka in the long run is probably a better choice, we went with Rabbit as our core developers were pretty familiar with it and we had several libraries that were ready to go. That said Rabbit is providing more then enough throughput right now and once we're bigger enough for rabbit to be our bottleneck/problem area that will be a lovely problem to have :)
One thing that has been a big win (which isn't mentioned in the article) is the concept of "duplication of data". Our microservices are currently listening to everything and only taking action on things they care about. As such, they are also storing data about things they care about. This makes it so that no single datastore is a bottle neck in a distributed architecture. This doesn't mean we don't have a centralize DB, but in the heat of operation it doesn't rely on a centralized DB. Using event sourcing makes it very easy to take this distributed data and store it in a consumable way for reporting, etc.
Another thing being it's a much different metric you have to worry about than traditional apps. Instead of worrying about load on a server, your main concern is messages processed per time frame. This can vary depending on what the service is doing, but in general you can simply monitor RabbitMQ's queue processing rate to determine if you need to scale a particular service or not.
It's a bit hard to debug some concurrency problems, but is normally solved by asking: "Who's responsibility is this?" and then adjusting who emits what.
They mention planning out events; we go a bit further and use json schema to describe exactly what is expected in that event. We actually have a dedicated development microservice that simply listens to all events and validates them against the json schema and records/notifies what events (and the source) is not being compliant.
In the beginning of development, our throughput was dreadful, but now that we have all the tools in place, adding a new service or handler of an event in a service is incredibly fast.
One thing that has been a big win (which isn't mentioned in the article) is the concept of "duplication of data". Our microservices are currently listening to everything and only taking action on things they care about. As such, they are also storing data about things they care about.
I couldn't agree more with this statement. I wish I'd gone into this, I was trying to cover a lot here about our new architecture in general.
In the beginning of development, our throughput was dreadful, but now that we have all the tools in place, adding a new service or handler of an event in a service is incredibly fast.
Yeah, we're adding a lot of new functionality and handlers really fast now. The great thing is when we need to split a service out into a new service we simply grab all the handlers for that event and the new service is pretty much already in a proof of concept state!
On top of that, it also makes it easy for front end interfaces to be designed with a mock microservice (or handler) that simply emits it for their testing with a backend developer going through and filling it. As long as the API isn't overly complex for emitting events, almost anyone on the team can do this.
It's more of an approach to higher level architecture than a technique. (think MVC as opposed to Rails). There is no starting MVC tutorial.
Check out https://geteventstore.com/ for a standardized bit of tech with usage docs