We are using steampipe to connect 30 AWS accounts via SQL. In a single call we can query all our services. Using metabase to visualise it all. I highly recommend you give it a go
How are you making the metabase<-->steampipe connection? Are you able to connect metabase directly to a running steampipe/ postgres server instance? or are you doing some ETL into metabase. Thanks in advance (metabase is new to me, but steampipe has been really handy)
Re:Cloud Infrastructure as SQL
by nathanwallace on Friday September 17, 2021 @01:01AM
(#28561397)
Use `steampipe service start` to run Steampipe in the background [1]. It will print out a standard Postgres connection string running on port 9193 of your local machine. You can then simply point Metabase (or any other DB visualization tool) at it using the standard Postgres setup.
Re:Cloud Infrastructure as SQL
by nathanwallace on Thursday September 16, 2021 @10:59PM
(#28560733)
Awesome! Would love to hear more about your use case and any suggestions for improvements if you want to DM me.
Re:Cloud Infrastructure as SQL
by heinrichhartman on Thursday September 16, 2021 @02:18PM
(#28555194)
Neat. Does that use steampipe services/API under the hood, or does it consume the AWS/GigHub APIs/... directly?
Re:Cloud Infrastructure as SQL
by nathanwallace on Thursday September 16, 2021 @02:20PM
(#28555218)
It talks directly to the cloud provider APIs. Plugins are written in go-lang (similar to Terraform providers).
OSS plugins - https://github.com/topics/steampipe-plugin
Re:Cloud Infrastructure as SQL
by heinrichhartman on Thursday September 16, 2021 @02:21PM
(#28555224)
WOOT! That's awesome.
Re:Cloud Infrastructure as SQL
by Aaronstotle on Thursday September 16, 2021 @04:37PM
(#28557223)
As someone who has to do lots of compliance activities, this is a fantastic tool, thanks!
Re:Cloud Infrastructure as SQL
by nathanwallace on Thursday September 16, 2021 @04:41PM
(#28557269)
Great to hear :-) Please check out our open source mods (e.g. AWS Compliance, GitHub Sherlock, DigitalOcean Thrifty) - we'd love your feedback & help! https://hub.steampipe.io/mods
Ooh this looks really powefull, any plans to add hetzner and backblaze?
Re:Cloud Infrastructure as SQL
by nathanwallace on Thursday September 16, 2021 @11:03PM
(#28560747)
Both seem to have reasonable Go SDK libraries, so it should be easy to create new plugins for them. I've added them to our internal backlog, but all our plugins are open source if you'd like to try building it yourself! The Linode plugin is a simple, similar example to use as a starting point [1]. We also have docs for writing plugins [2].
1 - https://github.com/turbot/steampipe-plugin-linode 2 - https://steampipe.io/docs/develop/writing-plugins
Re:Cloud Infrastructure as SQL
by Aaronstotle on Thursday September 16, 2021 @05:34PM
(#28557972)
Will do! Thanks again
We are a Postgres FDW under the hood with Go-based plugins, so write would be possible, but we've chosen to focus on read only so far. Definitely interested to see how you approach create, update and delete in the SQL model!
Notes: Not related to iasql. I'm a lead on the Steampipe project.