News for hackers. Stuff that matters.
Re:The OpenTF Manifesto by fishnchips on Tuesday August 15, 2023 @11:51AM (#37135452)
Marcin here, co-founder of Spacelift.

Even though I strongly believe the OpenTF fork could open up incredible possibilities for the community (I could go on and on about it), it is an equivalent of a civil war. It doesn't serve the community and our only interest is in the continued strength of the community that we continue to build for.

Based on my immense respect for what's been built under Hashi's umbrella I'd rather see a change of mind, and an opportunity to honor our pledge of resources (5 FTEs for 5 years) to the common rather than partisan cause.

Re:The OpenTF Manifesto by busterarm on Tuesday August 15, 2023 @12:40PM (#37136092)
Hashicorp isn't going to budge here. The same argument that you've made about Terraform being the underpinnings and needing to be open-sourced can be applied to their other important products like Vault, Consul and Nomad as well. The ecosystem of those three is plainly a direct competitor to Kubernetes which is open-source.

There's really no move for them to make here. It's unfortunate.

Re:The OpenTF Manifesto by fishpen0 on Tuesday August 15, 2023 @02:29PM (#37137521)
Tons of organizations run vault and consul as part of their k8s ecosystem so they don't directly compete. The vault CSI driver might be the single most installed CSI driver across all the orgs I've worked for
Re:The OpenTF Manifesto by busterarm on Tuesday August 15, 2023 @02:54PM (#37137784)
You're completely missing it.

If you are running Nomad as your orchestrator, because of the tight integrations you are almost certainly running vault for secrets and consul for service discovery/service mesh. The ecosystem of the three is the competitor to K8s.

s/ someone who runs both ecosystems at scale.

Re:The OpenTF Manifesto by twunde on Tuesday August 15, 2023 @03:16PM (#37138061)
While the Nomad stack is a direct competitor to k8s, Consul and Vault are both heavily used alongside k8s. In fact, Consul had features that were only for k8s the last time I checked
Re:The OpenTF Manifesto by busterarm on Tuesday August 15, 2023 @03:17PM (#37138074)
While these facts are true, that's totally not the fucking point and has nothing to do with the argument. I say again.

You can have software that both supports and competes with the k8s ecosystem. That's even the same type of problem all of these companies have with Hashicorp software now under the BSL.

Gruntwork builds tools that everyone using Terraform uses but they also offer services that Hashicorp would prefer that you pay them for instead.

You telling me "but people running K8s also use vault/consul" is like telling me that Gruntwork makes terragrunt which terraform users use. It doesn't mean that Hashicorp doesn't view them as a threat.

Re:The OpenTF Manifesto by hdaz0017 on Tuesday August 15, 2023 @04:08PM (#37138724)
There is a big difference though Terraform is the out and out winner in its market.

All their other products are at best small x% share of a crowded market or dominated by another product.

Re:The OpenTF Manifesto by ghshephard on Tuesday August 15, 2023 @05:32PM (#37139538)
Genuinely curious - other than Vault - what other product is there for secret management in the cloud infrastructure space. I get that CyberArk Conjur is big in the enterprise space, but I thought cloud users, even with k8s, mostly went with vault.
Re:The OpenTF Manifesto by candiddevmike on Tuesday August 15, 2023 @06:17PM (#37140000)
Most companies are probably using the secret manager provided by their cloud platform instead of Vault, especially after they tried to buy Vault.
Re:The OpenTF Manifesto by Alupis on Tuesday August 15, 2023 @07:23PM (#37140655)
Which is more than likely Vault + new carpets and paint.
Re:The OpenTF Manifesto by xyzzy_plugh on Tuesday August 15, 2023 @10:50PM (#37142325)
No, it's astronomically unlikely to be Vault at all.
Re:The OpenTF Manifesto by Alupis on Wednesday August 16, 2023 @03:52PM (#37152737)
It's far more likely to be Vault as a base, actually. The MPL would allow someone like Amazon to use Vault's source as a base, and so long as the core source wasn't modified Amazon would be under no obligation to make their modifications public.

The MPL is a lot more "business" friendly than the GPL.

The very fact that HashiCorp is changing their license and restricting use clearly indicates they see this threat as reality. Amazon/Microsoft/Google/Whoever using HashiCorp's work/effort but keeping all the money to themselves.

Re:The OpenTF Manifesto by candiddevmike on Wednesday August 16, 2023 @07:29PM (#37155253)
I don't know what you've heard but none of the cloud providers use HashiCorp Vault as the foundation for their secrets manager.
Re:The OpenTF Manifesto by xyzzy_plugh on Wednesday August 16, 2023 @10:10PM (#37156312)
Yes, they could, but no major cloud provider does. They're all bespoke. Read their white papers and it'll make more sense.
Re:The OpenTF Manifesto by oneplane on Tuesday August 15, 2023 @06:27PM (#37140097)
You generally only run Vault if you don't run on a public cloud.
Re:The OpenTF Manifesto by Alupis on Tuesday August 15, 2023 @08:06PM (#37141025)
There's a lot more reasons to run Vault than those.

Having a standardized way to "do secrets" for any team, any service, any app within the organization is very nice. Becoming cloud-agnostic for your secrets (connecting your local Vault with the cloud provider's vault) is another great benefit. Automatic secret rotation is also another great benefit. Secret versioning and auditing... etc.

It's not just "can't have this secret in VCS or viewable via kubectl".

Re:The OpenTF Manifesto by oneplane on Wednesday August 16, 2023 @08:46AM (#37146238)
> It's not just "can't have this secret in VCS or viewable via kubectl".

That is exactly what it is.

You seem to misunderstand (and thus downvote?) the statement I made. I'm not saying "haha vault bad", I'm answering "what other product" (from ghshephard) with the reality of today.

This has nothing to do with what Vault is or isn't, but just with the concept of storing secrets in a uniform way in clouds for use with cloud workloads what is being used right now.

Re:The OpenTF Manifesto by Alupis on Wednesday August 16, 2023 @03:20PM (#37152241)
I did not downvote you, no. Downvoting because we disagree isn't how that's supposed to work, even though some use it that way.

Regardless, the use of Vault is not exclusive to cloud environments.

All of the listed features of Vault have benefits within larger organizations even if they don't use the "cloud" and deploy monolithic applications.

Most frameworks have built in ways to fetch secrets/config from Vault, making it an easy standardized way to do things across all of your applications/teams.

It doesn't mean you need to use it, of course, but it has a lot of perks for many different situations.

Re:The OpenTF Manifesto by oneplane on Wednesday August 16, 2023 @03:56PM (#37152805)
I totally agree that Vault is more than a glorified password manager (because that's what most Clouds have in their implementation of a secrets store), but the thing is, everywhere I go, I don't see people use Vault, I see them use whatever AWS/Google/Azure happens to have (and often badly).

I'm not sure if that's even what ghshephard meant when he was curious for 'products', since technically all those cloud-integrated services aren't really stand-alone products for that matter.

In AWS for example, with or without EKS (and then something like External Secrets Operator in the EKS case), it's all just AWS Secrets Manager and sometimes Parameter Store. In a few cases people do manual encryption (using KMS), but in no case was HashiCorp Vault used.

Often, it's even worse: no secrets management at all. Stuff just gets pumped into environment variables (more often than not they get committed as .env files to Git), and there's just no drive to change that, even when a business policy is in place. Some even 'work around' this by storing secrets in password managers like 1Password and LastPass so they can check the compliance box without actually protecting the secrets (since they also live in plain text in VCS and at runtime in the environment).

In terms of 'products', I'd say Vault and the cloud ones don't really compare, but reality is depressing and secrets are often not as secret as the name implies. From a developer perspective, they might compare them because they desire the secrets to be injected into the environment either way, and as such the source doesn't matter much. I'm not sure if we should see that as a feature or a bug.

Re:The OpenTF Manifesto by ghshephard on Thursday August 17, 2023 @03:54PM (#37167067)
We use vault as a framework that associates authentication with secret engines via a policy framework. The Secret Engines could be AWS, PostGres Database, PKI, SSH Certificate Signer, Key value stores, etc... and the Authentication Framework might be LDAP, OKTA, or plain tokens. The Policy framework is pretty dynamic and has many thousands of possible policies mapping various authenticated entities to various authority (read, list, write, etc...) to various secret engines. Combine that with the syntactic niceties of template-rendering integration with the chosen secret-store, and maybe some clever stuff around single-use token wrapping - and I think of all of those features as belonging to a single product.

I'm relatively new to this field - and see tons of Vault at colleagues companies, and have friends who run/support Conjur (Enterprise more than cloud). Those are the only two secret-management framework/products I'd hear of - so was interesting in knowing what else had mindshare.

Re:The OpenTF Manifesto by oneplane on Saturday August 19, 2023 @03:42PM (#37192350)
I wonder if some of this also depends on how the secrets are consumed (and created), I'd imagine that if you store things like an API key and secret for a third party API, someone needs to 'enter' that data at some point in time and then set an ACL to allow a person or system to then consume it.

But if you have two programs what exchange secrets between multiple instances of each other, (one can do CRUD, the other only Read), you'd have much more interaction. Same as with a system creating secrets and a human reading it.

As for where it would make no sense at all: automated workload identities where you get time-limited temporary credentials that represent a role; most public clouds have some sort of link-local API, an injection method or mount method to provide ever-rotating secrets which gets picked up by the client SDK automatically. If you are using something like AWS, you'd be able to consume hundreds of services without ever persisting a secret anywhere.

This is also where my 'cloud' (and K8s) remarks are based on; when your workload and your resources speak the same authn/authz with a centrally coordinated policy system, there really isn't much value in adding something in the middle of that, and as such you don't see a lot of Vault and Vault-like implementations.

That said, as soon as you add something disconnected like local virtual machines, on-prem stuff etc. where authentication has historically been extremely bad and unless you brought a proper Kerberos setup you're screwed beyond mitigation. That's where Vault (when it came out) delivered a lot of value. It's probably also why we see AWS, IBM, GCP, Azure, in the same list with Vault and CyberArk. I'm surprised VMware doesn't have anything yet, but perhaps they recognise they lost this one already.

Re:The OpenTF Manifesto by hdaz0017 on Wednesday August 16, 2023 @02:11PM (#37151218)
HashiCorp Vault - is big in Secrets management but its not the Terraform of Secrets management it's just too easy to make another product.

AWS Secrets Manager

AWS Systems Manager Parameter Store

AWS KMS

Google Cloud KMS - Cloud Key Management System

Azure Key Vault

confidant - https://lyft.github.io/confidant

keywhiz - https://github.com/square/keywhiz

knox - https://github.com/pinterest/knox

strongboxsafe - https://strongboxsafe.com

conjur - https://www.conjur.org

+ many more

Ansible Vault - probably not competing directly

I am pretty sure companies like netflix, facebook, uber, tesla and others are probably using their own in house creations.

Re:The OpenTF Manifesto by tedivm on Tuesday August 15, 2023 @11:58AM (#37135553)
I really appreciate that, and I do think it's right of you to at least make the attempt.

That being said, I don't expect this attempt to work and I fully believe that a fork is going to be inevitable. I also think a fork is an amazing opportunity to standardize the language and prioritize the features developers want.

It isn't just about the license, but the way that Hashicorp has maintained the Terraform project. The github insights show that they don't have nearly as many people working on it as I would expect, and most of them are split into also working on Terraform Cloud. At the same time they don't work with the community that well- there are open issues and pull requests that just get ignored as Hashicorp clearly doesn't see value in open source contributors. This isn't just a Terraform issue either- my company had to move off of nomad due to the lack of development and support (as well as broken features).

I have strong concerns about the future of these projects in general beyond just the licensing. An open foundation that had multiple companies involved would by definition need to find a way for those people to collaborate together, and once they do that it makes it easier for them to invite community collaboration. So while I do appreciate that it is a drastic step, I think it's one that would also be far better for the ecosystem and project as a whole.

That said, maybe this is the wake up call hashicorp needs to fix these problems. If you provide five FTEs that basically doubles the size of their Terraform development team (they have more people working on it than five, but those people are split into other projects), and once they start working with other groups maybe they'll work with the community more as well. I'm not holding my breath though.

Re:The OpenTF Manifesto by lamontcg on Tuesday August 15, 2023 @12:58PM (#37136295)
Smells like the end of Chef. Management doesn't understand how much it takes to maintain the open source project and is just pouring resources into sales and marketing and products that they can charge for, and don't see how that erodes goodwill and the technological foundation of the company.
Re:The OpenTF Manifesto by miah_ on Tuesday August 15, 2023 @02:25PM (#37137467)
I also saw that parallel with Chef. I think its the story of all VC funded software that attempts to be "Open Source". For them, Open Source means "You can read the source code, and potentially fix a bug", for us, it means community, transparency, and fixing bugs beyond those your paying customer has.

I looked at github /chef/chef and github /inspec/inspec and its the same as it was shortly after I left. The only changes are from the one person who carried over after the sale to Progress, and the contracting team out of India, with dozens are unanswered queries and pull requests from the community.

What really ruffles my feathers was when they had us define oss-practices (https://github.com/chef/chef-oss-practices), clearly nobody outside our small team read (or understood) those words and goals. It feels like it was work to make us look better in OSS in order to bolster the company sale.

Re:The OpenTF Manifesto by lamontcg on Tuesday August 15, 2023 @03:14PM (#37138031)
There was a whole lot of community window dressing going on. I still wonder if they weren't trying to ship maintenance of the open source code off onto the community thinking that if all that worked appeared (or thinking that it was actually going on--believing their own bullshit about how involved the community was) that they could just leach that work.

There's probably some manager at Hashi right now trying to argue that they should offload TF maintenance entirely onto the community and they should pivot to hosting services and consulting and making money off of all that free work.

Re:The OpenTF Manifesto by hdaz0017 on Tuesday August 15, 2023 @03:55PM (#37138547)
chef said, did and tried some really dumb stuff and lots of it failed for obvious reasons, it's like docker took a chunk of their playbook and their business went the same way.

It's all good and "fun" on the IPO up.

(in x months we will rewrite the whole universe).

Re:The OpenTF Manifesto by FrenchTouch42 on Tuesday August 15, 2023 @04:31PM (#37138977)
Thank you Miah for all your contributions (beyond InSpec)
Re:The OpenTF Manifesto by rattus on Monday August 21, 2023 @05:05PM (#37215359)
Cygnus made it work. Long way from those origins to the redhat or hashicorp of today though.
Re:The OpenTF Manifesto by PeterZaitsev on Tuesday August 15, 2023 @01:37PM (#37136897)
Hi Marcin,

I recognize you're in interesting position in Spacelift. Per your recent analyses you may not be impacted and in this case you probably do not want to pissoff Hashicorp folks :)

In reality though force better be responded with force and showing Hashcorp what what was Terraform will be successful as Open Source project with or without them is best way to get them to reconsider.

Re:The OpenTF Manifesto by RcouF1uZ4gsC on Tuesday August 15, 2023 @04:46PM (#37139089)
> it is an equivalent of a civil war.

Why. It is open source. A fork should be no big deal, and definitely not a “civil war”. I think the community should be quicker to fork open source projects that are not serving the needs of the community.

The corporations are trying to have the benefits of open source without the responsibility. Forking is a normal, acceptable part of open source and we should normalize it.

Re:The OpenTF Manifesto by enneff on Tuesday August 15, 2023 @05:39PM (#37139606)
What would it mean to “normalise” forking? The costs of maintaining a fork are significant, and if one group of programmers are being funded to work on the project then it can be very difficult to fork a project in any meaningful way without significant resources behind it.

Also IIUC most of the parties in this conversation are corporations. They’re all trying to enjoy the benefits of open source development for a variety of reasons.

Re:The OpenTF Manifesto by lmm on Tuesday August 15, 2023 @08:41PM (#37141302)
> The costs of maintaining a fork are significant

Why? If we have tooling or workflows that assume forks don't happen, maybe we can fix those. Forking should be cheap, easy and frequent.

Re:The OpenTF Manifesto by waynesonfire on Wednesday August 16, 2023 @12:57AM (#37143019)
Right below your comment was another comment that started with,

"The truth is that a fork hurts everyone."

So, which is it?

Re:The OpenTF Manifesto by lmm on Wednesday August 16, 2023 @02:13AM (#37143419)
Currently forks are painful, because they aren't normalized i.e. our tools and workflows don't expect them. I'm saying rather than discouraging forks we should adapt our tools and workflows to expect them.
Re:The OpenTF Manifesto by ericb on Wednesday August 16, 2023 @05:19AM (#37144567)
Sure, tooling could be better.

But the real work is all the hard work that goes into a fork. I've watched open forks die all the time--all it takes is no one to step up and do/pay for the work, which is basically the default, because it is in everyone's interest if someone else is the one to do that.

I think that's really the crux of the problem--there are plenty of folks willing to maintain software for money, and a whole lot of people who'd rather it not cost money and if it does, not their money.

If the tooling is better, who is going to maintain this?

Re:The OpenTF Manifesto by throwaway2037 on Wednesday August 16, 2023 @05:27AM (#37144631)
I assume OP meant cost == community. That is the hard part of a fork.
Re:The OpenTF Manifesto by kragen on Tuesday August 15, 2023 @05:45PM (#37139670)
it's not war

hashicorp has decided they don't want to contribute to open source any more

they're totally within their rights to do so, and it doesn't harm anybody; it's not the equivalent of going around blowing up buildings, raping women, and napalming children. at most we can wish they had continued doing the beneficial things they were previously doing

maybe they'll change their minds, as you say, but that's no reason for the community to sit around twiddling its thumbs hoping for such a change. what's important now is that the people who are still willing to cooperate can do so successfully, and that's what opentf is about

that's even more obviously not the equivalent of going around blowing up buildings, raping women, and napalming children. it's very much the opposite, in fact

it's unclear to me which of the parties you intend to accuse of doing the moral equivalent of burning innocent people alive en masse, but either way, maybe you should think about walking back that rhetoric a bit

Re:The OpenTF Manifesto by fishnchips on Tuesday August 15, 2023 @06:10PM (#37139929)
I apologize if I appeared to downplay the horror of actual war. I'm lucky to never have experienced it first-hand.

I mainly wanted to focus on the "civil" than "war" aspect of the possible schism.

Re:The OpenTF Manifesto by anyoneamous on Tuesday August 15, 2023 @06:31PM (#37140129)
FWIW there are still people (dozens of us!) who don't feel the need to be artificially offended by a bit of hyperbole on an internet forum.
Re:The OpenTF Manifesto by bifrost on Tuesday August 15, 2023 @08:01PM (#37140996)
I concurr.
Re:The OpenTF Manifesto by kragen on Wednesday August 16, 2023 @12:28AM (#37142889)
my objection is not that mass graves, piles of mangled bodies, your close friends unexpectedly disappearing into pink mist, and terrible stenches are too sacred to be used as a metaphor for something else

my objection is that warfare involves people intentionally harming each other, and that doesn't seem to be what's going on here. it's not that war is a more extreme version of the situation; it's that it's directionally different

rather, hashicorp is struggling to not go bankrupt, so they've decided to switch to making a proprietary product instead of an open-source products; and terraform users, naturally enough, are reluctant to make their infrastructure vulnerable to a proprietary software license. hashicorp is not intentionally harming terraform users, and terraform users are not intentionally harming hashicorp

they're just not continuing their previous cooperation

Re:The OpenTF Manifesto by 0ct4via on Wednesday August 16, 2023 @12:22PM (#37149419)
Not sacred enough for you to invoke the Holocaust in a discussion about data protection legislation and the "right to be forgotten" within mere hours of this statement, however...

Such hypocrisy, attacking others for "rhetoric" that needs to be "dialled back a bit" when you're every bit as guilty of the exact. same. thing.

Re:The OpenTF Manifesto by kragen on Wednesday August 16, 2023 @01:12PM (#37150201)
please note that the comment you are replying to says the opposite of what you are implying it does

the comment you are referring to, for anyone who is interested, concerns the question of whether or not there is a higher standard of morality to which legislation can be held, or whether legislation itself is the ultimate moral authority, or whether there is in fact no objective standard of morality at all. anyone who is interested in that kind of thing can read it at https://news.ycombinator.com/item?id=37147305

Re:The OpenTF Manifesto by kragen on Tuesday August 15, 2023 @06:30PM (#37140123)
it's not a possible schism. hashicorp has clearly and unmistakably abandoned the open source community. conceivably they'll change their minds, but their communication doesn't have any ambiguity in it

i have no idea what you could possibly mean by 'focus on the "civil"'. it's good that people are being civil to one another, isn't it?

Re:The OpenTF Manifesto by tedivm on Tuesday August 15, 2023 @07:11PM (#37140515)
You're missing the point- the OpenTF group wants to mend the schism if possible, by getting Hashicorp to change their licensing back. If they immediately fork then that's not likely to happen, so they're attempting this first.

I don't think it will work, but I think it's good of them to try.

Re:The OpenTF Manifesto by kragen on Tuesday August 15, 2023 @08:33PM (#37141244)
mending the schism would be great, but we probably can't do that by pretending it doesn't exist, analogizing it to blowing children's extremities off, or analogizing acknowledging its existence to blowing children's extremities off; I'm still not sure which of the latter two was the intent of the comment I was responding to
Re:The OpenTF Manifesto by 0ct4via on Wednesday August 16, 2023 @12:22PM (#37149424)
Don't you think you should check yourself before attacking others?

> "it's unclear to me which of the parties you intend to accuse of doing the moral equivalent of burning innocent people alive en masse, but either way, maybe you should think about walking back that rhetoric a bit"

... but only earlier today were you equating a data protection law with the Holocaust... seems you could do with a bit less projection and rhetoric yourself, Kraggy.

Re:The OpenTF Manifesto by kragen on Wednesday August 16, 2023 @01:18PM (#37150291)
this is careless reasoning; rather than equating a so-called 'data protection law' with the holocaust, i said the justification others were using for that law was incorrect, because if it were correct, it would also justify the holocaust

this is not an extremely advanced form of logic, but i understand that it is not within everyone's grasp

the comment in question is https://news.ycombinator.com/item?id=37147305

octavia's other, rather astounding comment in https://news.ycombinator.com/item?id=37149528 is probably also useful context

my objection to the war rhetoric in this case is that it casts people as opponents who are not, in fact, opponents, just different parties pursuing largely independent interests. in that context playing 'let's you and him fight' seems unlikely to improve the situation

Re:The OpenTF Manifesto by 0ct4via on Monday August 21, 2023 @04:15AM (#37206712)
Why use so-called? It's literally a law concerning data protection. Unless you feel that the empirical truth somehow shouldn't be used to describe the GDPR and colloquial "right to be forgotten" aspects it entails, you're just trolling for trolling's sake -- either that, or you don't even know that you're using "so-called" improperly.

Once again you jump to mansplaining and condescension, followed by failing to even get a username correct when it's literally on your screen.

Your repeated attempts to tell people what they think, what they do or do not know, and where they live, show that every observation of you being arrogant, condescending, and disingenuous, is patently correct.

Unsure if you need a shovel to get out of that hole you've put yourself in, but you're sure backpedaling quickly, yikes.

Re:The OpenTF Manifesto by jeffchao on Tuesday August 15, 2023 @05:56PM (#37139797)
As a user and collaborator of TACOS, agreed that it could open up opportunities. Though I echo trade-offs (as in other replies) that it could start a civil war that makes it difficult for end users and collaborators -- reminds me of Python2 -> 3, Presto/Trino, and many other stories. Pledging resources is a great approach.