What programming language stack do you (and a lot of other developers) use that’s closed source with no realistic open source alternative? It’s not C, Python, Java, Ruby, JavaScript, Lisp, Fortran, C++, PHP, Perl, Go, SQL, Rust, Objective-C, or Swift? (All of these, of course, not only have industrial strength FOSS implementations, but the first or second most popular implementation is the FOSS one.)

>What programming language stack do you use that’s closed source?

Verilog/System Verilog (generally HDL/FPGA/ASIC design/verification).

Yes, it's a niche (some may even say it's not software, it's hardware, but it is indeed a large code base that requires a software engineering environment/toolchain/practices/etc...). Again, just answering your question for the benefit of the discussion. It would be interesting to hear of others such closed source areas.

[0] https://en.m.wikipedia.org/wiki/Q_(programming_language_from...

[1] https://en.m.wikipedia.org/wiki/Kdb%2B

Wikipedia has an article on Time series databases and they contain a short list of popular TSDBs [0]. On that list 11 are libre-software to some degree and 4 are commercial. Relational TSDBs are new to the open source space but they're still being built despite there being the existing kdb+. This shows that people will spend considerably more funds to engineer a replacement (and give that replacement away for free) to avoid using a closed source product for this use case.

[0] - https://en.wikipedia.org/wiki/Time_series_database#List_of_t...

Market adoption could be better, but the license costs around $100,000/year (probably the most expensive software per kilobyte). Fintech can afford it, other industries can’t.

They have a an SQLish interface, that allows non-specialists to get work done, but anything serious needs to move beyond that.

I haven't used it in a few years, but queries had to be carefully optimized -- swapping the order in a where clause could cause order of magnitude differences in performance. Also it is append-optimized. If you need to update or insert data, it is a nontrivial exercise.

Single machine, because it can't handle parallel queries, at all, and there are no options to scale it.

Every financial institution I've worked at is busily unwinding their investment in q/kdb. It's legacy tech.

I even met a programmer who hacked something together in Java using the same columnar kdb layout but, you know, multithreaded, so different basket optimization jobs could run simultaneously on the same store.

This seems pretty much next generation k.

We can see how things goes.

[0] https://www.reddit.com/r/FPGA/comments/a5pzs5/prediction_ope...

1) If there are some widely used open hardware devices or open standards. The fact that FPGAs are proprietary designs and can't be copied is a significant barrier. If there was an analogue of ARM in the FPGA world that could open up open source dev tools too.

2) DIY FPGA. This one is almost certainly much further down the road than any of the other options. But if fab and design tools of the chips themselves become a commodity then there will be open source dev tools. It used to be pretty difficult to make a custom PCB, but now with PCB as a service open tools are seeing more use. I think the reason this is so far down the line is because fpga offers cutting edge performance for specialized applications. If you wait several generations or are ok with reduced performance then you can stick with generic CPU/GPU. With Moore's law running down there may be more and more things that aren't possible without fpga. And I don't think it would be until then that open source dev tools follow more diy fpga.

https://symbiflow.github.io/

There is an open FPGA hardware implementation:

https://github.com/haojunliu/OpenFPGA https://www.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-4...

More resources:

https://wiki.debian.org/FPGA

For my comment I was thinking about HDL simulators, which are very expensive/proprietary tools (vendors like Synopsys, Cadence, Mentor Graphics), with a lot of vendor lock-in, but yet are based on standard/open languages: Verilog/System Verilog. I know there are some open-source simulators that have been around for a while, but they aren't widely used, and the big 3 EDA vendors still seem to have a strangle hold on the HDL simulator market.

Simulation is generally more important for ASIC hardware than FPGA hardware, since you can quickly iterate on an FPGA design with logic/design fixes, whereas with ASICs design iteration can be prohibitively expensive and time consuming.

https://riscv.org/ https://openrisc.io/ https://www.mips.com/mipsopen/ https://en.wikipedia.org/wiki/SPARC#Open_source_implementati...

But the IC/hardware toolchain is a separate issue from the design itself. It's largely proprietary, probably due to the niche market and large R&D costs involved in some of the tools.

http://iverilog.icarus.com/ https://www.veripool.org/wiki/verilator http://www.clifford.at/yosys/ https://symbiflow.github.io/ https://github.com/Qucs/ADMS http://www.myhdl.org/ https://chisel.eecs.berkeley.edu/ https://wiki.debian.org/FPGA

I’m aware of chisel used to verify some RISC-V cores that were fabricated, but that’s a research POC, not a volume/production ASIC.

We have chips shipping in volume.

I think Python definitely wins as a scripting language lingua franca though. It is free as in beer, open-source, available on any OS I care about, and most of the software I use has a built-in Python API.

Mathematica is big in academic math and physics programs, but I haven't seen it in industry as much as Matlab. Believe it or not, my company had a major accounting application written in Mathematica, so it does pop up in weird places.

It seems like using the SDK in your language of choice would be a reasonable way to leverage the engine where it makes sense.

IME even getting binaries from an antenna vendor can be like pulling teeth, are there some good open source solutions out there?

Matlab?

MATLAB

To give some more pushback on this assertion... I've been developing software for 21 years now, and the only positions I had where there was negligible open-source software were in the first couple of years (in the automotive firmware space [OnStar], and a brief stint in some defense contracting). For the last 18-ish years--firmware, systems, enterprise, virtualization, demos, security--everything I've worked on has largely been based on open-source software, or the product was open-source itself. What I've never done professionally: Web app development.

Honestly, I think it would be hard to overstate the prevalence and impact of open-source software in the field of programming (and really, the entire tech field).

I suspect it was phrased strongly because the GP made the same claim in the opposite direction: "Most software is grounded in an open source development and deployment tool chain".

Are you serious? I don't do web apps. But almost 100% of our stack is based on open source technologies. Python, Twisted, Go, etc. Every single thing we use is open source.

It is ironic that you are accusing someone else of having zero concept of what programming looks like outside of web apps when one could easily think the same thing about you.

And even most Go software is proprietary, hidden behind Google's bar.

GCC was so revolutionary because it brought a pro-level (although not at the start) compiler toolchain for free and allowed developers who were so inclined to improve and modify it.

And lots of Python code is for things like machine learning, Go runs lots of DevOps/neo-infrastructure stuff, and I can’t comment on Twisted.

Admittedly I go out of my way to avoid including closed source components in my work but that's not just an ideological decision. As the parent points out, and especially when you're a developer, there's a massive amount of freedom and power to open source tools. Don't like something? You can tweak it any way you like. You don't have to worry about the IP owner pulling the rug from under you without notice. I want to use that code on my MIPS32 target? I can try to port it, I don't have to beg the vendor to release a version just for me.

If anything in my experience the archetypal "modern web dev" crowd (the one that got into programming through Javascript, basically) is the one that benefits the least from this because often they don't dare to patch their window manager or their shell to customize them. These days C is often considered a weird arcane language, something like the Latin of CS (including in this very forum) so people don't feel empowered to modify these applications. The fact that modern DEs like to treat their users like slow children instead of exposing the tools that could help them become "power users" probably doesn't help.

Then of course for people who don't actually try to read or modify the source of the programs they use "open source" is more of a political statement than anything else. There's no real benefit for them.

I'm making generalities of course, I'm not saying that all web devs fit this description.

I hope this trend will change as newer, more "friendly" languages like Go and Rust make forays in the system programming space. Hopefully that'll motivate the new generation to get over their fear of system programming and start embracing and customizing the tools their use.

Not at the GUI level. Xcode and Instruments make use of open source toolchains underneath, but aren't open source themselves, nor are the Apple frameworks they're built on.

Except for perhaps optimizers (which some of the best are closed source, e.g. Mosek), the entire rest of the stack is open source.

Neither is the hardware, nor is Intel’s or the Motherboard’s, etc. Would it be ideal? Sure. Is it possible to have every level be completely open source? Maybe, but you’d struggle.

I still believe the point is clear, though.

In my lab, we attempt to use as much open source software and parts as is possible without literally spending years waiting for RISC-V to become mainstream along with fully open source GPUs and computing cores, open motherboards, RAMs, etc... this doesn't even include fabrication or other useful software which runs our machines (do you want to have a several-million-dollar photolithography machine running on poorly maintained OSS?). Yes, should that machine be open in a perfect world? Of course, that would be incredible and in the spirit of verifiability. But right now it is not and there is little we can do about it.

It's clearly impossible (or completely and utterly impractical) to pretend that somehow everything can be done in a fully open source way, and it's easy to go down the rabbit hole pretending to be on the high horse of "our stuff is more open." My point is that what can be made with OSS, we almost always attempt to make with OSS, not everything we make is perfectly open and transparent, because it literally cannot be without so much effort as to render the entire thing completely useless.

So, sure we use Mathematica in some things when the OSS alternative is mostly unworkable or unusable, but by and large we attempt to be as open as possible such that people can verify our results without needing to spend thousands of dollars on software whose internals we don't generally know. Do we rely on Intel software and hardware which could have another floating point bug? Sure. But it's hard to be at all productive if you don't at least assume a bare minimum, even if that standard is not completely open.

Proprietary closed source has a built in mechanism for funding. That's a major plus.

Like Intellij?

https://github.com/hughperkins/coriander

It's Excel for engineers.

The Matlab IDE is not bad though and is a lot like Python's Spyder IDE. Also, a lot of Matlab commands have fully browsable source provided I've been told.

I'm not talking about the dollars, that's relatively minor. It's all the things around it, especially with licenses becoming more complex, including core counts, including all these weird provisions related to when and how and what features can be used in dev and staging and production, what versions of other software they can work with, etc...

The software that we can buy and just use without thinking about it is great, but it's really few and far between these days.

We can spend weeks or months analyzing the things surrounding the commercial licensing for a product, and that is spending one of our most valuable resources: attention.

(No, being able to use a sandbox is not sufficient to properly learn any language because that's not how you use languages to build stuff)

And before anyone says how important Wolfram Engine is and that this is the going rate for specialized software: nobody cares. Because you are probably wrong if history is any guide. Lots of software used to be niche software and isn't today. Things change. If it's expensive and only available at a steep price, then only those who need it will buy it. The curious will get their kicks elsewhere.

There used to be a time when database professionals stomped their feet and yelled if you suggested Oracle might consider a Linux port of Oracle. "Impossible! People who use Linux aren't into paying for stuff!". And then it got released and then Linux became its most important platform.

Things change.

In order to get people to include it as an infrastructural component you probably have to ensure that it is free for non-commercial use. I'm not sure if you have to open source it. I use lots of stuff that is free for non commercial use - and then I use it commercially and pay for a license. Occasionally really expensive licenses.

I've looked at Wolfram Engine a few times, but in the end we never used it because it just didn't have a great enough following among programmers. I couldn't just tap into a talent pool when recruiting. It was hard to find experience.

I tend to choose tools based on the _people_ I can find to use them. In fact, usually the people I find choose the tools. That goes for how I choose languages, software, cameras, cars, motorcycles...you name it. No vibrant community means I don't buy in. And sure, there are lots and lots of cameras that are better than mine in that price range. But the cameras are not all that important. The people are. Who I can collaborate with, who I can learn from, who can help me solve problems.

Also, it tended to be a bit on the expensive side for what I wanted to use it for. Sure, I could have contacted Wolfram and said "look, we want to use it for this, but the licensing model doesn't quite fit", but how many developers do this?

(I remember thinking about buying Mathematica several times to tinker with it, get to know it and then perhaps see if I could use it for something. But it was expensive, and I'm not spending that amount of money for something I'll maybe going to play a bit with and then perhaps forget for a year. This too makes Wolfram Engine a bit of "interesting, but I'm not going to invest time in it".

And I can't say I've ever blamed Stephen Wolfram for it. He has a product that makes a bunch of users happy and him rich. Who am I to tell him that he should do things differently? So he exists over there and I exist over here and everyone is happy.

I'm excited about this change. I wish it had happened sooner so it could have had more of an impact. It certainly put Wolfram Engine back on my radar.