> Now I think that this internal email is what has actually stuck with me the most. Bezos realized that he had to change the internal communication infrastructure [..].
> He understood that a radical organizational change was required to arrange the internal dynamics in a way that would allow the creation of something like AWS.
This is quite a strong and opinionated statement. I'd agree that Jeff made this change to improve communication infrastructure for Amazon.com. I wouldn't agree that he made it to enable AWS - for two reasons.
First, Amazon started thinking about EC2 and S3 way way later than this. Second, Kindle and SimpleDB (now Dynamo) were similar independent bets. These projects were happening at the time of moving to SOA, but they weren't tied to it.
The one thing common across all of these products is - making informed bets on market fundamentals and enabling teams to deliver on them.
Now, I know of a few fairly senior Amazonians who read those forums. They have more context than I do. So, if they chime in and say that Jeff sent this email to enable AWS, I'll gladly take it. :)
Until then, I wouldn't extrapolate Steve Yegge's post to mean something about Jeff B.'s intentions, and build an overarching argument over it.
ozgune is correct as far as I remember.
The beginnings of AWS, though, were for the retail site. We exposed search and browse and item metadata via an API first for a deal with AOL to provide them with product search and then later we opened it up to everyone. People built stuff like Simple Amazon on top of it, which I thought was pretty nice. I don't think this part of AWS exists anymore (please correct me if I'm wrong).
The first service exposed as what we'd today consider AWS was SQS and at the time it was kind of a head scratcher. Only later did we understand that it was glue for other services.
S3 and EC2 were quite a bit later I think.
Minor nitpick, these are two separate services. SDB is still independently accessible and probably will be for a while. That being said they definitely don't encourage any new use of SDB and push Dynamo instead.
Kind of a shame because a modern NoSQL store with SQL support would be great for rapid prototyping before a concrete data schema is established.
sdb is build on the same principles outlined in the dynamo paper: https://www.allthingsdistributed.com/files/amazon-dynamo-sos...
Between 2000/2003 Amazon started to learn how to properly make something that resembled AWS (launched in 2006), it was a necessity targeted towards scalability. What Yegge letter meant is that Amazon was doing SOA to all its depts, making these be useful both internally and externally
>"5) All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world."
So you see, it wasn't to enable AWS as the digital service as we know it, it was to build a scalable company that could use everything it had both to serve its needs, and also to sell it as a product (then, all services we all know EC2 etc). Bezos applied SOA as a company, not only to its digital philosophy. And this corporate move, with no precedent, enabled not only AWS, but the whole Amazon as it is today.
Given the recent 20,000 startup ideas post, this is the perfect algorithm to cut through the noise
There's also:
- Hire an intern / "Customer Service Representative" / "Technical Account Specialist" to manually copy data from one service into another
- Dump some file in a directory and hope something is treating that directory like a queue
- Read/write from the same database (/ same table)
Or the classic Unix trajectory of increasingly bad service communication:
- Read/write from the same local socket and (hopefully) same raw memory layouts (i.e. C structs) (because you've just taken your existing serialized process and begun fork()ing workers)
- that, but with some mmap'd region (because the next team of developers doesn't know how to select())
- that, but with a local file (because the next team of developers doesn't know how to mmap())
- that, but with some NFS file (for scaling!)
- that, but with some hadoop fs file (for big data!)
Obviously all of these are at some level an 'application programming interface'. But then, technically so is rowhammering the data you want into the next job.
"Think of the acronym CSV. Don't look up the definition of the format, just meditate on the idea of the format for a bit. Then write your data in the format you have just imagined is CSV, making whatever choices you feel personally best or most elegant regarding character escapes. Pass this file on to your downstream readers, assuring them it is a CSV file, without elaborating on how you have redefined that."
My personal experience comes from ingesting product feeds from online stores. Misapplication of \ from other encodings was the most common sin, but I'm pretty sure I saw about three dozen others, from double-comma to null-terminated strings to re-encoding offending characters as hex escapes. (And, of course, TSV files called CSV files, with the same suite of problems.)
On-hand work experience.
This wouldn't quite fit the "obfuscated C" contest, but I feel like there should be a prize for a system that does useful work this way.
Or it's unencrypted files on an FTP server containing literally the lifeblood of the American economy: https://engineering.gusto.com/how-ach-works-a-developer-pers... -_-
This gets abused even within one service. If I could get my coworkers to FFS stop using rows in a database as a degenerate kind of communications channel between components (with "recipients" slow-polling for rows that indicate something for them to do), I'd be a lot happier.
> rowhammering the data you want into the next job.
I hope the aforementioned coworkers don't read HN.
Screen scrape the other service and do data exchange via a Selenium script.
Directly interact with the other service's database.
CSV files and nightly batch jobs.
Apparently it was so hard to deal with the developers that instead of exposing an API they would automate clicking around on Internet Explorer browser windows.
Thankfully I haven't heard much about it lately.
- The page has automatic history & merge conflict resolution - There's built-in role-based security to control both visibility and actions (read-only vs. read-write) - You can work on a draft and then "deploy" by publishing your changes - You can respond to hooks/notifications when the page is updated
Considering that the alternative is either editing a text file on disk or teaching business users to use git, a Confluence page is not so bad.
The problem is that these kinds of things have to be built to the lowest common denominator, which is usually the customer anyway. The customer in enterprise software is usually not a tech company they typically have outdated IT policies and less skilled developers than a pure tech company would have. Even if the developers are capable of doing something like interacting with a queue they also need to be supported by a technology organization which can deal with that type of interaction.
Some times you get lucky and someone in the past has pushed for that kind of modernization. Or your project really won't work without a more advanced interaction model and you have someone in the organization willing to go to bat for tech enhancement.
But otherwise the default is "Control-M job to consume/produce a CSV file from/onto an SFTP"
- Building one large application (monolith). Parts of the application communicate with each other via function calls. Everything runs in one large process. You can go quite a long way with this approach, especially for parts of the application that are stateless. (You can also build components of the application using a service/client metaphor within the process as well.)
- Multiple separate applications might communicate with the same database, file system, or some other data store. Before we had distinct distributed systems components taking on the role of queues, event buses, and things like that, it was common to represent queues using folders or database tables. These approaches are still seen today, though they're uncommon in new applications.
https://www.ibm.com/products/datapower-gateway
It's a middleware appliance that allows any API to talk to any other API and handles just about any data format. You can script it with javascript or XSLT. It can handle ad-hoc things like ftp polling.
It has the added benefit that you can add security for outside facing clients.
Disclaimer: I helped develop this appliance (but I no longer work for IBM)
So basically it will be expensive, require an army of IBM consultants and become yet another integration point instead of actually solving anything.
In showed an evolution from a monolithic spaghetti codebase, to an SOA, to realizing there are now spaghetti connections between services in an SOA, to a very clean looking ESB architecture, to showing how all the chaos is still there, just inside the ESB where it’s even harder to reason about or change.
You integrate a "service" by creating and linking a library that implements the entire service, including data access. Now try tracking down everything accessing the "service" database, or rolling out an upgrade to the "service".
But enough about the web...
I think the core of the Amazon invention was to mandate that the API be a strict boundary around a service. No cheating by making assumptions about the internals of the implementation. Force teams to work off the published API documentation, and make teams the guardians of each service. Microservices probably aren't developed exactly like that in many companies.
Considering 'microservices' wasn't a well-known term for another decade [1], maybe better to ask: isn't 'microservices' just this ?
[1] https://trends.google.com/trends/explore?date=all&q=microser...
Yeah it is (though not so "micro-": it's really one service per team, which is the scale that actually makes sense IMO). This is kind of a "seinfeld is unfunny" case: this memo was in 2002, and predated (arguably helped create) most of the modern microservice approach.
> Also isn't this terrible for latency and debugging ?
When two codebases are owned by different teams (and potentially in different languages), it's far better to have them separated by a network connection and a well-defined HTTP API than living in the same address space where they can corrupt each other. Since the interfaces are well defined, you can isolate any problem down to a request that's not getting the correct answer from the service it's calling, and then hand that over to the other team to investigate. Since the request and response are plain text, it's easy to see what's happening.
If responding to a single request from the end customer requires chaining through several independent codebases maintained by different teams, you have bigger problems than request latency. Each team should own a piece of end-to-end functionality, not a horizontal layer.
>3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
API first... except if you want to be a number of certain new services that somehow managed to get away with not presenting an API, even though an API would make every service team's life easier.
> 5) All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
Except, similar to above, where teams apparently decided they didn't want to think that way at all and management just let them.
> 6) Anyone who doesn’t do this will be fired.
Unless your exception is perceived providing value to the company. Then you'll get lauded, and everyone is told they'll need to use your js laden, web only interface, and to hell with any automation.
Mostly those exceptions just codified further in my mind about just how right the Bezos email Yegge paraphrased actually was.