Cartography: Graph view of infrastructure assets and relationships between them

Re:Cartography: Graph view of infrastructure assets and rela... by alpha_squared on Thursday March 28, 2019 @09:31PM (#19518387)

Literally working on an extremely similar project, albeit to power security; seems inevitable that an open-source solution would've popped up. Glad to see this. Any exploration in doing this cross-cloud?

[ Parent | on HN ]

Re:Cartography: Graph view of infrastructure assets and rela... by alexchantavy on Friday March 29, 2019 @04:37PM (#19525438)

We're definitely open for collaboration to allow multi-cloud :-). This could simply be another [intel module](https://github.com/lyft/cartography/tree/master/cartography/...).

[ Parent | on HN ]

Re:Cartography: Graph view of infrastructure assets and rela... by duked on Friday March 29, 2019 @08:06AM (#19520875)

I'm working on something related to security too. I would love to connect but no email on your profile.

[ Parent | on HN ]

Re:Cartography: Graph view of infrastructure assets and rela... by alpha_squared on Friday March 29, 2019 @01:17PM (#19523606)

Sure, let's connect; you can shoot me a message here: andrew(a)armaneous.com

[ Parent | on HN ]

Re:Cartography: Graph view of infrastructure assets and rela... by chris408 on Thursday March 28, 2019 @09:40PM (#19518425)

Not sure, but for now AWS is the focus.

[ Parent | on HN ]

Re:Cartography: Graph view of infrastructure assets and rela... by alpha_squared on Thursday March 28, 2019 @09:43PM (#19518444)

Not sure if I'm missing it in the documentation, but how does it keep in sync with changes in the account?

Edit: OP changed response to remove certain references, presumably because something sensitive was revealed. Changing my response out of respect.

[ Parent | on HN ]

Re:Cartography: Graph view of infrastructure assets and rela... by alexchantavy on Friday March 29, 2019 @04:34PM (#19525397)

Hi, I'm one of the devs on the project.

Good question. As is, this does not keep anything in sync.

To keep the graph in sync with changes in the account, simply set up a cronjob to run `cartography` whenever you would need a refresh. Each sync run should guarantee that you have the most up-to-date data.

Here's how a sync works: when the sync starts, set a variable called `update_tag` to the current time. Then, pull all the data from your AWS account(s) and create Neo4j nodes and their relationships, making sure to set their `lastupdated` fields to `update_tag`.

Finally, delete the left over nodes and relationships (i.e. those that do not have up-to-date `lastupdated` fields). This way the data stays fresh, and you can see this in the [cleanup jobs](https://github.com/lyft/cartography/tree/master/cartography/...).

[ Parent | on HN ]

Re:Cartography: Graph view of infrastructure assets and rela... by alpha_squared on Friday March 29, 2019 @08:34PM (#19526782)

Makes sense, thanks for the additional details!

Our approach requires us to stay as real-time as possible, so we're actually using CloudWatch events to keep in sync -- the deletes become a little hard after that.

I look forward to the progress of Cartography, though!

[ Parent | on HN ]