News for hackers. Stuff that matters.
Re:Show HN: My Single-File Python Script I Used to Replace S... by rollcat on Thursday September 21, 2023 @01:19PM (#37600818)
"This simple tool solves X at my org" is probably the most underrated type of project. There's not enough room to overcomplicate something that isn't a core part of the business, it must be practical to maintain, simple&stupid enough so that onboarding is not a hurdle, etc.

I encourage everyone to share your "splunk in 1kloc of Python" projects! Some of my own:

- https://github.com/rollcat/judo is Ansible without Python or YAML

- https://github.com/rollcat/zfs-autosnap manages rolling ZFS snapshots

Re:Show HN: My Single-File Python Script I Used to Replace S... by olafalo on Thursday September 21, 2023 @01:34PM (#37601059)
For me, it's configinator[0]. Write a spec file for a config like [1], get a Go file that loads a config from environment variables like [2]. Code-gen only, no reflection, fairly type-safe, supports enums, string, bool, and int64. I made it because it was gross to add new config vars in a project at work, and it's come in handy a lot!

[0] https://github.com/olafal0/configinator

[1] https://github.com/olafal0/configinator/blob/0576a53970bcb4d...

[2] https://github.com/olafal0/configinator/blob/0576a53970bcb4d...

Re:Show HN: My Single-File Python Script I Used to Replace S... by eigenvalue on Thursday September 21, 2023 @01:27PM (#37600944)
Thanks, based on the dismissive replies to my original comment in the Splunk acquisition discussion, I thought this would get a lot of hostile takes saying that it was dumb, that I reinvented the wheel because I didn't want to spend 2 weeks trying to figure out opentelemetry nonsense and tools X, Y, and Z, that it was trivial, that it wouldn't scale, etc.

But people are actually being surprisingly nice and friendly! I guess people just really hate Splunk!

Re:Show HN: My Single-File Python Script I Used to Replace S... by egwor on Thursday September 21, 2023 @01:52PM (#37601340)
I suggest you sell it to Oracle, get some popcorn and watch the Cisco vs Oracle log war begin!
Re:Show HN: My Single-File Python Script I Used to Replace S... by RexM on Thursday September 21, 2023 @02:46PM (#37602148)
mired in antitrust lawsuits
Re:Show HN: My Single-File Python Script I Used to Replace S... by _aavaa_ on Thursday September 21, 2023 @03:50PM (#37603250)
> reinvented the wheel

I hate this meme. It's as if cars, trains, and airplanes all use the same wheels. Or that wheels under my stove, my tiny filing dresser, and my shopping cart are all the same.

Oh yeah, re-inventing the wheel, what a stupid idea and something we obviously don't frequently do and for good reasons.

This meme is almost as bad as the horrible misquoted "premature optimisation is the root of all evil".

Re:Show HN: My Single-File Python Script I Used to Replace S... by facorreia on Thursday September 21, 2023 @05:01PM (#37604271)
Your project sounds like something that definitely could come in handy. I forked it as a "bookmark". I particularly like the idea of storing the data in a local SQLite database. Not everything needs to be "web scale".
Re:Show HN: My Single-File Python Script I Used to Replace S... by eigenvalue on Thursday September 21, 2023 @05:30PM (#37604654)
The real power is that with datasette you can instantly turn that SQLite DB into a full fledged responsive web app with arbitrarily complex filtering and sorting, which all gets serialized to the URL so you can share it and bookmark it for future use.
Re:Show HN: My Single-File Python Script I Used to Replace S... by silasb on Thursday September 21, 2023 @06:37PM (#37605411)
Not only that, but with https://litestream.io/ things becomes even more interesting.

I'm currently using this for a small application to easily backup databases in docker containers.

Re:Show HN: My Single-File Python Script I Used to Replace S... by rollcat on Thursday September 21, 2023 @07:48PM (#37606019)
> But people are actually being surprisingly nice and friendly! I guess people just really hate Splunk!

Best things in life come through love and passion. Frustration can be a good motivator but don't let it guide you.

> I thought this would get a lot of hostile takes [...]

To be entirely honest with you, recognizing and praising the good parts is a lot easier than giving proper feedback on what needs to be improved;)

Re:Show HN: My Single-File Python Script I Used to Replace S... by aaviator42 on Thursday September 21, 2023 @01:45PM (#37601225)
My org's apps heavily use this simple key-value interface built on sqlite: https://github.com/aaviator42/StorX

There's also a bunch of other purpose-built tiny utilities on that GitHub account: https://github.com/aaviator42?tab=repositories

Re:Show HN: My Single-File Python Script I Used to Replace S... by emodendroket on Thursday September 21, 2023 @01:54PM (#37601374)
> There's not enough room to overcomplicate something that isn't a core part of the business, it must be practical to maintain, simple&stupid enough so that onboarding is not a hurdle, etc.

You would think. But no, there is lots of room to make it over complicated without the corresponding efforts to manage the complexity.

Re:Show HN: My Single-File Python Script I Used to Replace S... by dynisor on Thursday September 21, 2023 @02:14PM (#37601639)
Don’t worry, that’s just tech debt and we will deeefinitely come back to it next sprint.
Re:Show HN: My Single-File Python Script I Used to Replace S... by xorcist on Thursday September 21, 2023 @02:20PM (#37601714)
Your software is cool but the description is a bit unfair to Ansible. Ansible works by solving for desired state. This software runs scripts, it replaces "for host in; do ssh $host < script.sh; done".
Re:Show HN: My Single-File Python Script I Used to Replace S... by matrss on Thursday September 21, 2023 @05:19PM (#37604512)
Last time I checked, ansible playbooks were also essentially just sequential steps to execute via ssh, albeit in yaml format. There was certainly no way to describe a desired state nor did ansible accomplish consistently bringing a system into some desired state. Two executions of the same playbook could result in a very different system state for example, depending on what happened in between.

The only systems I am aware of which are reliably capable of "solving for desired state" are nix and guix.

Re:Show HN: My Single-File Python Script I Used to Replace S... by eigenvalue on Thursday September 21, 2023 @05:33PM (#37604683)
Ansible is definitely all about solving from a specified target state and ensuring that it is followed. It’s even at the level of syntax for ansible, which is how it can be used totally declaratively. And if you stick to the native idiomatic ansible way of doing everything (as opposed to doing hacky stuff with ad hoc shell commmands), you get automatic idempotence and other nice stuff “for free”.
Re:Show HN: My Single-File Python Script I Used to Replace S... by matrss on Friday September 22, 2023 @04:05AM (#37609248)
So, the last time I used ansible, which was quite a while ago to be fair, there was a builtin way to install packages with apt on the target system. You could add packages to a list to be installed and that would work. But removing them from that "declared state" would not remove them on the target system on the next playbook run. You would have to add an explicit uninstall command. And that is where ansible failed to be declarative. Did that change in the meantime?

Ansible might provide idempotence for the builtin things (although I would argue it doesn't, at least not on a bit-by-bit level, since you can't pin down specific versions of package repositories and stuff like that), but to be declarative it would need to provide a 1-to-1 mapping from declared state to running system state. And if what I described above is still the case, then it simply does not do that.

In my experience, ansible tries to build a declarative interface to an imperative mode of system management, which works to some extent, but breaks down in more complex cases because building this declarative interface can only be a leaky abstraction without the right foundation.

Re:Show HN: My Single-File Python Script I Used to Replace S... by xorcist on Thursday September 21, 2023 @06:37PM (#37605414)
You can run sequential commands with ansible, but then you're just using it as a replacement for "ssh $host < script.sh". You would be missing out on most of the usefulness of the tool. It's meant to be used declaratively. The manual describes it quite well. In the same type of tools are puppet and indeed nix, but with one important difference for the latter: nix is also a package manager, which allows for more a fine grained state specification.
Re:Show HN: My Single-File Python Script I Used to Replace S... by matrss on Friday September 22, 2023 @04:31AM (#37609466)
True, nix is also a package manager, and that is the crucial step necessary to actually provide a declarative interface to system state. Without it you can only get the leaky abstraction that is ansible.

To illustrate my point, imagine this playbook:

  ---
  - name: Bring system into some state
    hosts: localhost
  
    tasks:
      - name: Install hello
        ansible.builtin.apt:
          name: hello
        become: true
Apply it and you get GNU hello installed. Now remove the package installation step, which really is just a glorified "ssh $host < script.sh":

  ---
  - name: Bring system into some state
    hosts: localhost
  
    tasks:
Apply that and you will still have hello installed, even though it was removed from the "declared state". This just pretends to be declarative but really isn't, the tasks are still imperative steps.

Not to blame ansible for this, it is just that ansible is build on a foundation that inherently makes declarative system management impossible to begin with and ansible is more or less the best thing you could do given the constraints.

Re:Show HN: My Single-File Python Script I Used to Replace S... by rollcat on Thursday September 21, 2023 @07:25PM (#37605836)
This is fair criticism.

When I first created Judo, I envisioned some sort of a standard library as a sibling project, sort of an "executable rosetta stone" for Unix, where you could declaratively say things like "ensure this user exists", "ensure this package is installed".

In practice I found out it's fairly easy to just write your scripts to be idempotent. It was the secret "2. and do not overcomplicate things" step that most initially simple software seems to gradually forget about.

Re:Show HN: My Single-File Python Script I Used to Replace S... by adamrezich on Thursday September 21, 2023 @03:03PM (#37602438)
I work for a school district. we use (an old version of) Skyward, a popular school management system. it's very common for various people in the district to want "reports" from Skyward's database. the way this works in Skyward is, you build the report file in a separate piece of software called Crystal Reports. you write an SQL query, and use Crystal Reports to format the report output. then, you go into Skyward, and create a new "Custom Report", and upload the Crystal Reports file. then you have to run it once (for some reason)—once it's been run, you can then schedule the Custom Report to be run periodically, and emailed out to whatever users you specify.

Crystal Reports is a robust piece of software that lets you do whatever you want when it comes to designing a layout. for example, I used it to design the new high school report cards for the district. but it also lets you output to CSV instead of PDF, which allows other secretarial staff to use it in Excel, and do whatever they want with it, like filter for values and such. when designing a to-be-output-to-CSV report, you don't do anything design-wise but drag the columns from the SQL query in that you want to export.

I found this process to be very tedious, plus time-consuming to maintain—if you want to update a query or adjust which email addresses receive the report, you have to make a bajillion clicks. so, I made a simple PowerShell script on one of our servers, which looks in a Rules folder, which contains subfolders of arbitrary depth, and, at the end, has a SQL file and a config file. the config file contains the schedule (daily, weekly, monthly), and the list of email addresses who should receive the report. the script takes a parameter for which schedule it's going to run, and Task Scheduler tasks are set up to run each different parameterized version of the script daily, weekly, and monthly. the script looks at all the Rules, determines which ones it should evaluate, evaluates them, saves the results to a CSV file, and emails them to the relevant staff. updating a "rule" is as easy as modifying either the SQL file or the config file. adding a new rules is as easy as making a new subfolder, and making a SQL and config file. the script has a parameter to do a "dry run", outputting the results in table form to the console, without sending an email. (though, for now, I still have to test the queries in SQL Server Management Studio first, because I can't figure out how to get Invoke-Sqlcmd to output linked server errors.)

this has been a huge boon for productivity and maintainability of CSV reports, allowing myself and my coworkers to easily create and manage these "data validation rules", such that our data analysts and HR people can get periodic updates on all sorts of various data validation issues, without having to do things the complicated Skyward way—and all it took was about a day of work setting up and testing the PowerShell script.

this is just one of many such PowerShell scripts I've written to automate things at work—PowerShell is great. I didn't know how to use it before taking this job, and none of my coworkers did, either. but for simple stuff like this, it's fantastic.