Or expressed a bit differently: what are the infra + 2 data stores doing that's a major improvement over processing raw AWS Config + CUR data which should easily fit in laptop memory. We're talking megabytes of data for a point in time snapshot, right?
I'm assuming like most things AWS, this is not targeted at your small-sized startup running 2 VPCs, a handful of subnets, and 10-20 EC2 instances. This is likely built specifically for major enterprises running a hundred thousand instances, with hundreds of VPCs, thousands of subnets, hundreds of thousands of security groups, etc. In such environments, a static SVG isn't feasible and you might want something that allows more analytics instead, which I think is the entire point of using Neptune.
That's a good question.
Whilst Perspective sources a lot of data from AWS Config, it also supplements and enriches the data from other sources where resources are not supported or where we wanted to show more detail. We'll be adding in other data sources in future, too.
Additionally, the AWS Config query language is quite helpful but doesn't expose a graph of the resources. Our approach is to pull the individual relationship data out and represent those in a graph DB (Amazon Neptune). Backing on to a graph database gives us the ability to let users navigate through the relationships of interest on the fly, rather than dumping what would inevitably be a complicated mess on the screen if we just showed everything.
Using this approach is also one of the ways we've tried to solve the UI/UX challenges presented in this space. Rather than say "show me all the things" that would lead to a complex mess of interconnections (and probably explode browsers) we instead encourage a workflow that starts with a resource or workload and explores outwards from there. It also allows us to implement some interesting features in future -- we intend to keep working on the project.
The ElasticSearch cluster is used by the discovery process to locate resources based on the metadata we collect; it's part of the way we enrich the data and relationships beyond what we get from Config.
No, the diagrams of Perspective were not created with Perspective -- we had to stick to existing style guidelines.
If there are more you'd like to see added, please raise an issue: https://github.com/awslabs/aws-perspective
An eventing system that lets you hook node/edge events world be awesome as well.
Will take a look tonight, really interested in this.
I'm from AWS and my team are responsible for development of this project (AWS Perspective).
It's important to note that Perspective is not a traditional diagramming tool like draw.io or Visio. Perspective is intended to display resources that already exist rather than draw new ones.
To do this, Perspective inventories resources in your accounts (Lambdas, DBs, EC2 instances, ENIs, EBS volumes etc.) and can automatically create a diagrammatic representations of that data. Perspective also discovers the relationships between resources and will automatically expose them on the diagram.
The underlying data sits in a graph database (Amazon Neptune). This allows you to explore the relationships in the graph (diagram) on the fly visually to discover how your resources interact with one another.
We also have the early stages of a feature to show the estimated cost of the resources in the diagrams you create.
The project is open source and our roadmap is public. I encourage you to check it out and add any commentary for features you'd like to see (or bugs you find!).
https://github.com/awslabs/aws-perspective
I'm also happy to answer any questions.