mtdowling

You don’t have to use Gradle with Smithy. You can also use the self-contained CLI which doesn’t use Gradle: https://smithy.io/2.0/guides/smithy-cli/index.html

VPlanet is another great glimpse in time at QBasic and the game dev community it had in the late 90s early 2000s: http://vplanet.petesqbsite.com/

Pretty much, but Smithy can be used for anything and isn’t specific to AWS. The AWS modeling support in Smithy is all through extensions that aren’t part of the core.

It’s also protocol agnostic and can be used in a lot of applications (HTTP, MQTT, and we are even experimenting using Smithy to generate C ABI bindings for non client server stuff).

Smithy has something called event streams that send async datagrams: https://awslabs.github.io/smithy/1.0/spec/core/stream-traits....

This is currently used in Amazon S3, Kinesis, Transcribe, and other services.

Smithy doesn’t have a service registry today. However, models can be vended and shared via Maven. Client codegen was designed explicitly to not require coordinated releases of clients and servers (that’s impossible for AWS SDKs).

Yup. Smithy is from a lineage of internal tools that have been in use at Amazon since the early 2000s. From my dive into software archaeology at Amazon (I work there), there was a bit of SOAP in use in the 2000s, and some other internal model formats that are now obsolete claimed to be SOAP like. As the years went on, other internal formats came out to replace other formats, until the internal model format became something distantly inspired by SOAP, but very practical and tuned for cross language code generation so it could power AWS (that’s my take at least). That was in use for well over a decade before we built Smithy to improve on and open source the internal format.

We expect to have Smithy code generation for every language we support as official AWS SDKs.

It's not really a fully finished project yet, so not much. We shipped the AWS SDK for JS v3 with Smithy, the AWS SDK for Go v2 with Smithy, and just launched an alpha of the AWS SDK for Rust using Smithy. More are in the works. We're currently iterating on their code generators to make them easier to use outside the AWS SDKs. AWS SDKs are being built in a layered approach where there's a generic code generator that's really extensible, and then the AWS SDKs extend it to add AWS-specific stuff like regions and credential handling.

We're working to get projects like these to GA: https://github.com/awslabs/smithy-typescript, https://github.com/aws/smithy-go, and https://github.com/awslabs/smithy-rs. And we're also working on service code generation.

Smithy is very focused on codegen, so the model is highly normalized. So for example, defining a list of something needs to be done using a `list` shape. This kind of list is something you'd see directly serialized and sent over the wire. For example:

list Messages { member: Messages }

Then you can reference Messages from other places in the model, like from a structure:

structure Something { messages: Messages }

In contrast, the `[City]` syntax is used in other places in the IDL to define a relationship to a shape. This isn't something that gets sent over the wire, it's just used to form essentially a relationship in the service graph from a service to resources, a resource to operations, an operation to errors, etc. For example:

service Weather { resources: [City, Sensors] }

Smithy today doesn't support any serialization formats that require fixed ordering. We do recommend that any additional members added to structures are added at the end to help with C++ and Rust codegen though.

That said, traits can be used in Smithy to enforce constraints on structures, so if you ever needed explicit indexing like that, it could be done via traits and protocols (Smithy's nomenclature for describing how clients and servers communicate). In fact, protocols are defined by traits, and traits can enforce requirements on the rest of the model using a DSL called selectors... Probably way too much info other than -- it's possible and easy to support this in Smithy if it's ever needed.

As for required vs optional -- today it's treated as server-side validation only and not used in client codegen. This allows service teams to remove the required trait from members if something ends up needing to be optional in future without breaking clients. We're working on some ideas too to see if we can generate even better code for SDKs in languages like Rust where optionality is very explicit, but without sacrificing the ability of being able to remove the required trait.

And, in general, backward compatibility issues are caught with Smithy diff, which also supports custom rules: https://github.com/awslabs/smithy/tree/main/smithy-diff

No Python codegen yet. Our plan is basically that as we migrate AWS SDKs to Smithy, we'll also offer generic client generators too. The Python migration hasn't started yet.

We haven't built a WebSockets based protocol yet. And yeah, without actual server-side support, it is a little meta right now. We're working on it and hope to roll out a few languages this year.

Protobuf and gRPC are great, and AWS will continue to make sure developers can be successful using them with AWS. I’ll try to explain how we ended up at Smithy instead of using other existing tools.

We started working on Smithy around 2018 because we wanted to improve the scale of our API program and the AWS SDK team to deal with the growing number of services (over 250 now!) and languages we want to support in official AWS SDKs (like the newly released Rust SDK). We had a ton of existing services that we needed to be compatible with, but we also wanted to add new features to improve new services going forward too.

We needed a very flexible meta-model that allows us to continue to evolve the model to account for things like integrating with other systems and to model service-specific customizations that each AWS SDK team can implement independently. Smithy's meta-model is based on traits, a self-describing way to add more information to models. Lots of validation can be built in to custom traits, which helps to ensure that service teams are using traits properly and adhere to their specifications. Smithy's resource modeling helps us here too because it allows AWS service teams, as they adopt Smithy, to essentially automatically support CloudFormation resource schemas. Resources also help us to point service teams in the right direction to make their services work well over HTTP (which methods to use, URIs, safety, idempotency, etc).

We needed an integrated model validation, linting, and diff tool to keep services consistent and detect breaking changes, and it needed to support company-wide standards as well as service-specific standards. We use Smithy’s validation system to automatically enforce API standards, and service teams often create their own service-specific rules to keep their own internal consistency.

We needed built-in input validation constraints so that they're standard across services and clients (e.g., length, range, pattern, etc). We didn't want to rely on third-party extensions to provide this feature since validating inputs is important. AWS uses internal service frameworks that enforce these constraints and are compatible with Smithy models. We're working to create open source service frameworks for Smithy as well.

We also wanted to support various serialization formats so that clients work with all of our existing services spread across JSON, XML, query strings, RPC, and HTTP APIs, but we also wanted to be able to evolve our serialization formats in the future as new technology comes along. That's why Smithy is protocol agnostic (like gRPC actually). The serialization format is an implementation detail. Smithy has some support for MQTT as well.

And finally, we need our code generators to be really flexible to support service customizations. There's quite a few customizations across AWS services, and we needed a way to inject custom code generation logic in various parts of our generators.

Smithy is still in heavy development, and we're working on building out more of the tooling so it can be used easily outside of AWS SDKs too, including client and server code generation.

Kind of. The AWS CLI uses an Amazon internal modeling format used to define services that's based on another Amazon internal modeling format that has been in use for about 15 years (and it's based on another internal model etc..). Smithy is basically the open source v2 of both, but with a public spec and tooling. Eventually all the AWS SDKs and the AWS CLI will adopt Smithy. (I work on the AWS SDKs and created Smithy)

Awesome project! I got Guzzle set up on Travis within about an hour. They even had node.js pre-installed, so I could use node.js as a test web server. http://travis-ci.org/#!/guzzle/guzzle