msnut

Thanks. I posted a comment update with simple Cloudmarker plugin I tried to get GitHub dump - https://news.ycombinator.com/item?id=19900677

Can you give me Custodian doc or tutorial that will help me do this in Custodian? I saw the Kubernetes link. It has many files to read. Can you give me simpler starting place?

I am trying this tool. First impression: The plugin framework works like charm. Creating a small plugin to get GitHub org repo info. Working with only two files. Plugin is one file and configuration yaml is other file.

I am not Python developer. I develop in Powershell. So don't judge my code. :)

I put this in ghcloud.py.

  import urllib.request
  import json

  class GhCloud:
      def __init__(self, org):
          self.org = org

      def read(self):
          url = 'https://api.github.com/orgs/%s/repos' % self.org
          data = json.loads(urllib.request.urlopen(url).read().decode('utf-8'))
          for d in data: yield d

      def done(self):
          pass

I put this in cloudmarker.yaml.

  plugins:
    ghplug:
      plugin: ghcloud.GhCloud
      params:
        org: python

  audits:
    ghmon:
      clouds:
      - ghplug
      stores:
      - esstore
      - filestore

  run:
  - ghmon

I run tool.

  PYTHONPATH=. cloudmarker --now

Cloudmarker runs my ghcloud.py and puts data into localhost:9200 and /tmp automatically. I can do it without cloning Cloudmarker code. I only hack my code and Cloudmarker runs it.

Is Cloud Custodian extensible? Can we make it connect to other clouds?

I am trying out Cloudmarker. It claims to be extensible.

> As a result of this plugin-based architecture, Cloudmarker can also be used as a framework to develop your own plugins that extend its capabilities by adding support for new types of clouds or data sources, storage or indexing engines, event generation, and alerting destinations.

We have many VMs in Digital ocean. It can be nice if I can make Cloud Custodian or Cloudmarker connect to DigitalOcean. The plug-in framework of Cloudmarker looks neat. I am writing digitalocean.py similar to https://github.com/cloudmarker/cloudmarker/blob/master/cloud.... I can write read() function that returns the JSON from DO. The rest of the framework takes care of running it and putting it in the selected DB (Elasticsearch for me).

If the Digital ocean plugin runs fine I will write a GitHub plugin after it.

Python is the go to language for multicloud automation. For Azure only Powershell is another go to language for it.

I'm doing some checks like this using Azure powershell. Microsoft has well documented powershell commands for Azure. My powershell scripts have less features and they can't run in unattended mode like this Cloudmarker tool. But I work on windows so powershell is always there. Has anyone run powershell as daemon on Linux? If that is present I will setup a Linux system with my powershell scripts for devops.