I am trying out Cloudmarker. It claims to be extensible.
> As a result of this plugin-based architecture, Cloudmarker can also be used as a framework to develop your own plugins that extend its capabilities by adding support for new types of clouds or data sources, storage or indexing engines, event generation, and alerting destinations.
We have many VMs in Digital ocean. It can be nice if I can make Cloud Custodian or Cloudmarker connect to DigitalOcean. The plug-in framework of Cloudmarker looks neat. I am writing digitalocean.py similar to https://github.com/cloudmarker/cloudmarker/blob/master/cloud.... I can write read() function that returns the JSON from DO. The rest of the framework takes care of running it and putting it in the selected DB (Elasticsearch for me).
If the Digital ocean plugin runs fine I will write a GitHub plugin after it.
We've considered digital ocean support, and I use DO as well, but the primary use cases custodian serves around governance/compliance, security, cost optimization haven't seemed as needed with DO across our user population (ie hasn't been requested), so its not been as high a priority on the roadmap atm, that said its a community project so contributions welcome.
All that said, if your just looking for a periodic dump to a datastore, then perhaps cloud marker is going to be a better fit.
Can you give me Custodian doc or tutorial that will help me do this in Custodian? I saw the Kubernetes link. It has many files to read. Can you give me simpler starting place?
For folks interested in this domain, its worth checking out cloud custodian, https://cloudcustodian.io
Its open source and supports AWS, GCP, Azure and provides more functionality (imo) and supports more resources. Its designed for compliance as code gitops style workflows with policies in yaml that query/filter/take action on resources, and provides for reporting, notification, and remediation use cases. It integrates with all the clouds serverless runtimes to provide real time response/inspection of api calls, provisioned directly from its yaml policies, as well as integrating with the cloud provider's specific security/compliance offerings (ConfigRules, GuardDuty, SecurityHub in AWS, CloudSecurityCommandCenter in GCP, etc). Its got developers from AWS and Azure working directly on it, and a community of hundreds of contributors.