Imagine saying this with a straight face. Maybe in web development, but the vast majority of programmers are using closed source tools to produce closed source software.
You're either thinking of a tiny picture vs the whole picture (web devs vs programming in general) or you honestly have zero concept of what programming looks like outside of web apps.
What programming language stack do you (and a lot of other developers) use that’s closed source with no realistic open source alternative? It’s not C, Python, Java, Ruby, JavaScript, Lisp, Fortran, C++, PHP, Perl, Go, SQL, Rust, Objective-C, or Swift? (All of these, of course, not only have industrial strength FOSS implementations, but the first or second most popular implementation is the FOSS one.)
>What programming language stack do you use that’s closed source?
Verilog/System Verilog (generally HDL/FPGA/ASIC design/verification).
Yes, it's a niche (some may even say it's not software, it's hardware, but it is indeed a large code base that requires a software engineering environment/toolchain/practices/etc...). Again, just answering your question for the benefit of the discussion. It would be interesting to hear of others such closed source areas.
[0] https://en.m.wikipedia.org/wiki/Q_(programming_language_from...
Wikipedia has an article on Time series databases and they contain a short list of popular TSDBs [0]. On that list 11 are libre-software to some degree and 4 are commercial. Relational TSDBs are new to the open source space but they're still being built despite there being the existing kdb+. This shows that people will spend considerably more funds to engineer a replacement (and give that replacement away for free) to avoid using a closed source product for this use case.
[0] - https://en.wikipedia.org/wiki/Time_series_database#List_of_t...
Market adoption could be better, but the license costs around $100,000/year (probably the most expensive software per kilobyte). Fintech can afford it, other industries can’t.
They have a an SQLish interface, that allows non-specialists to get work done, but anything serious needs to move beyond that.
I haven't used it in a few years, but queries had to be carefully optimized -- swapping the order in a where clause could cause order of magnitude differences in performance. Also it is append-optimized. If you need to update or insert data, it is a nontrivial exercise.
Single machine, because it can't handle parallel queries, at all, and there are no options to scale it.
Every financial institution I've worked at is busily unwinding their investment in q/kdb. It's legacy tech.
I even met a programmer who hacked something together in Java using the same columnar kdb layout but, you know, multithreaded, so different basket optimization jobs could run simultaneously on the same store.
This seems pretty much next generation k.
We can see how things goes.
[0] https://www.reddit.com/r/FPGA/comments/a5pzs5/prediction_ope...
1) If there are some widely used open hardware devices or open standards. The fact that FPGAs are proprietary designs and can't be copied is a significant barrier. If there was an analogue of ARM in the FPGA world that could open up open source dev tools too.
2) DIY FPGA. This one is almost certainly much further down the road than any of the other options. But if fab and design tools of the chips themselves become a commodity then there will be open source dev tools. It used to be pretty difficult to make a custom PCB, but now with PCB as a service open tools are seeing more use. I think the reason this is so far down the line is because fpga offers cutting edge performance for specialized applications. If you wait several generations or are ok with reduced performance then you can stick with generic CPU/GPU. With Moore's law running down there may be more and more things that aren't possible without fpga. And I don't think it would be until then that open source dev tools follow more diy fpga.
There is an open FPGA hardware implementation:
https://github.com/haojunliu/OpenFPGA https://www.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-4...
More resources:
For my comment I was thinking about HDL simulators, which are very expensive/proprietary tools (vendors like Synopsys, Cadence, Mentor Graphics), with a lot of vendor lock-in, but yet are based on standard/open languages: Verilog/System Verilog. I know there are some open-source simulators that have been around for a while, but they aren't widely used, and the big 3 EDA vendors still seem to have a strangle hold on the HDL simulator market.
Simulation is generally more important for ASIC hardware than FPGA hardware, since you can quickly iterate on an FPGA design with logic/design fixes, whereas with ASICs design iteration can be prohibitively expensive and time consuming.
https://riscv.org/ https://openrisc.io/ https://www.mips.com/mipsopen/ https://en.wikipedia.org/wiki/SPARC#Open_source_implementati...
But the IC/hardware toolchain is a separate issue from the design itself. It's largely proprietary, probably due to the niche market and large R&D costs involved in some of the tools.
http://iverilog.icarus.com/ https://www.veripool.org/wiki/verilator http://www.clifford.at/yosys/ https://symbiflow.github.io/ https://github.com/Qucs/ADMS http://www.myhdl.org/ https://chisel.eecs.berkeley.edu/ https://wiki.debian.org/FPGA
I’m aware of chisel used to verify some RISC-V cores that were fabricated, but that’s a research POC, not a volume/production ASIC.
We have chips shipping in volume.
I think Python definitely wins as a scripting language lingua franca though. It is free as in beer, open-source, available on any OS I care about, and most of the software I use has a built-in Python API.
Mathematica is big in academic math and physics programs, but I haven't seen it in industry as much as Matlab. Believe it or not, my company had a major accounting application written in Mathematica, so it does pop up in weird places.
It seems like using the SDK in your language of choice would be a reasonable way to leverage the engine where it makes sense.
IME even getting binaries from an antenna vendor can be like pulling teeth, are there some good open source solutions out there?
Matlab?
MATLAB
To give some more pushback on this assertion... I've been developing software for 21 years now, and the only positions I had where there was negligible open-source software were in the first couple of years (in the automotive firmware space [OnStar], and a brief stint in some defense contracting). For the last 18-ish years--firmware, systems, enterprise, virtualization, demos, security--everything I've worked on has largely been based on open-source software, or the product was open-source itself. What I've never done professionally: Web app development.
Honestly, I think it would be hard to overstate the prevalence and impact of open-source software in the field of programming (and really, the entire tech field).
I suspect it was phrased strongly because the GP made the same claim in the opposite direction: "Most software is grounded in an open source development and deployment tool chain".
Are you serious? I don't do web apps. But almost 100% of our stack is based on open source technologies. Python, Twisted, Go, etc. Every single thing we use is open source.
It is ironic that you are accusing someone else of having zero concept of what programming looks like outside of web apps when one could easily think the same thing about you.
And even most Go software is proprietary, hidden behind Google's bar.
GCC was so revolutionary because it brought a pro-level (although not at the start) compiler toolchain for free and allowed developers who were so inclined to improve and modify it.
And lots of Python code is for things like machine learning, Go runs lots of DevOps/neo-infrastructure stuff, and I can’t comment on Twisted.
Admittedly I go out of my way to avoid including closed source components in my work but that's not just an ideological decision. As the parent points out, and especially when you're a developer, there's a massive amount of freedom and power to open source tools. Don't like something? You can tweak it any way you like. You don't have to worry about the IP owner pulling the rug from under you without notice. I want to use that code on my MIPS32 target? I can try to port it, I don't have to beg the vendor to release a version just for me.
If anything in my experience the archetypal "modern web dev" crowd (the one that got into programming through Javascript, basically) is the one that benefits the least from this because often they don't dare to patch their window manager or their shell to customize them. These days C is often considered a weird arcane language, something like the Latin of CS (including in this very forum) so people don't feel empowered to modify these applications. The fact that modern DEs like to treat their users like slow children instead of exposing the tools that could help them become "power users" probably doesn't help.
Then of course for people who don't actually try to read or modify the source of the programs they use "open source" is more of a political statement than anything else. There's no real benefit for them.
I'm making generalities of course, I'm not saying that all web devs fit this description.
I hope this trend will change as newer, more "friendly" languages like Go and Rust make forays in the system programming space. Hopefully that'll motivate the new generation to get over their fear of system programming and start embracing and customizing the tools their use.
Not at the GUI level. Xcode and Instruments make use of open source toolchains underneath, but aren't open source themselves, nor are the Apple frameworks they're built on.
Except for perhaps optimizers (which some of the best are closed source, e.g. Mosek), the entire rest of the stack is open source.
Neither is the hardware, nor is Intel’s or the Motherboard’s, etc. Would it be ideal? Sure. Is it possible to have every level be completely open source? Maybe, but you’d struggle.
I still believe the point is clear, though.
In my lab, we attempt to use as much open source software and parts as is possible without literally spending years waiting for RISC-V to become mainstream along with fully open source GPUs and computing cores, open motherboards, RAMs, etc... this doesn't even include fabrication or other useful software which runs our machines (do you want to have a several-million-dollar photolithography machine running on poorly maintained OSS?). Yes, should that machine be open in a perfect world? Of course, that would be incredible and in the spirit of verifiability. But right now it is not and there is little we can do about it.
It's clearly impossible (or completely and utterly impractical) to pretend that somehow everything can be done in a fully open source way, and it's easy to go down the rabbit hole pretending to be on the high horse of "our stuff is more open." My point is that what can be made with OSS, we almost always attempt to make with OSS, not everything we make is perfectly open and transparent, because it literally cannot be without so much effort as to render the entire thing completely useless.
So, sure we use Mathematica in some things when the OSS alternative is mostly unworkable or unusable, but by and large we attempt to be as open as possible such that people can verify our results without needing to spend thousands of dollars on software whose internals we don't generally know. Do we rely on Intel software and hardware which could have another floating point bug? Sure. But it's hard to be at all productive if you don't at least assume a bare minimum, even if that standard is not completely open.
Proprietary closed source has a built in mechanism for funding. That's a major plus.
Like Intellij?
It's Excel for engineers.
The Matlab IDE is not bad though and is a lot like Python's Spyder IDE. Also, a lot of Matlab commands have fully browsable source provided I've been told.
I'm not talking about the dollars, that's relatively minor. It's all the things around it, especially with licenses becoming more complex, including core counts, including all these weird provisions related to when and how and what features can be used in dev and staging and production, what versions of other software they can work with, etc...
The software that we can buy and just use without thinking about it is great, but it's really few and far between these days.
We can spend weeks or months analyzing the things surrounding the commercial licensing for a product, and that is spending one of our most valuable resources: attention.
(No, being able to use a sandbox is not sufficient to properly learn any language because that's not how you use languages to build stuff)
And before anyone says how important Wolfram Engine is and that this is the going rate for specialized software: nobody cares. Because you are probably wrong if history is any guide. Lots of software used to be niche software and isn't today. Things change. If it's expensive and only available at a steep price, then only those who need it will buy it. The curious will get their kicks elsewhere.
There used to be a time when database professionals stomped their feet and yelled if you suggested Oracle might consider a Linux port of Oracle. "Impossible! People who use Linux aren't into paying for stuff!". And then it got released and then Linux became its most important platform.
Things change.
In order to get people to include it as an infrastructural component you probably have to ensure that it is free for non-commercial use. I'm not sure if you have to open source it. I use lots of stuff that is free for non commercial use - and then I use it commercially and pay for a license. Occasionally really expensive licenses.
I've looked at Wolfram Engine a few times, but in the end we never used it because it just didn't have a great enough following among programmers. I couldn't just tap into a talent pool when recruiting. It was hard to find experience.
I tend to choose tools based on the _people_ I can find to use them. In fact, usually the people I find choose the tools. That goes for how I choose languages, software, cameras, cars, motorcycles...you name it. No vibrant community means I don't buy in. And sure, there are lots and lots of cameras that are better than mine in that price range. But the cameras are not all that important. The people are. Who I can collaborate with, who I can learn from, who can help me solve problems.
Also, it tended to be a bit on the expensive side for what I wanted to use it for. Sure, I could have contacted Wolfram and said "look, we want to use it for this, but the licensing model doesn't quite fit", but how many developers do this?
(I remember thinking about buying Mathematica several times to tinker with it, get to know it and then perhaps see if I could use it for something. But it was expensive, and I'm not spending that amount of money for something I'll maybe going to play a bit with and then perhaps forget for a year. This too makes Wolfram Engine a bit of "interesting, but I'm not going to invest time in it".
And I can't say I've ever blamed Stephen Wolfram for it. He has a product that makes a bunch of users happy and him rich. Who am I to tell him that he should do things differently? So he exists over there and I exist over here and everyone is happy.
I'm excited about this change. I wish it had happened sooner so it could have had more of an impact. It certainly put Wolfram Engine back on my radar.
The reason for its dismissal goes a bit deeper than the simple fact that it’s not open source. We pay for other software (like E&M solvers), why not this? Because its use is not compartmentalized. It’s a programming language with the world’s most expansive standard library. Just as is the case with—say—Python, folks using this have no end to what they would build, including company IP. The code would absolutely not be portable, and your tech stack would be vendor-tied. That’s a danger zone for any serious enterprise.
I learned this lesson (albeit from a distance) when Microsoft killed off Visual Basic 6 and replaced it with Visual Basic .Net.
A closed source solution, where all you have is documentation (potentially flawed), including manually authored change information, just feels archaic at this point.
Companies should derive that trust factor from other factors based on their unique circumstances. If you are Amazon's competitor for instance it is probably not wise to use AWS for anything, but if you are a startup it could be very wise to take a bit of lock-in to avoid rolling your own infrastructure. If you are an established company maybe use AWS but in a way that you could move to another provider without too much pain, but balance that against some of the goodies you get by staying locked in.
I guess the same nuanced considerations have to be made to decide to use Wolfram Engine or not. There is no clear answer that suits all end users.
I've been reading as much as I can on RMS's site recently because I want to expand my mind on these sorts of topics. I really want to understand how "free" software can work at a practical level, where you say "here is a piece of software, it's $1000 from me, and you are free to give it to your friends, modify it, and resell it, even to people who would have paid me $1000, but now they wont need to" and how that ties in with making a sustainable business. I think it takes some luck and good imagination to make such a business work by selling some by-product such as consulting services or whatever. But for a lot of companies non-free is required in my opinion to exist as a business. Happy to have my mind changed.
With AWS, you can pack up and go to a different cloud provider, and get the same Linux and the same gcc and the same TCP/IP stack. There's probably some other code you'd have to change, but the essential core is non-proprietary and nobody can take it away from you. Abstracting further, if Linux decided to go in a weird direction, you can get a POSIX-conformant Unix-like OS from other groups, like FreeBSD or Illumos. The Linux codebase can be forked as well, providing an even stronger insurance relative to what happens when some company's or business unit's incentives no longer align with yours.
And that part about aligning incentives is powerful. In the closed-source world, it's also fragile: One company being bought out can shift things tectonically for all of the software that company made. It becomes a treadmill, or a movie routine of constantly jumping from platform to platform as each of them dies off or is pulled out from under you as "corporate synergy" realigns strategic chakras feng shuis your old codebase into worthlessness.
If you are "seriously" using AWS: A lot. Value of AWS is not that it offers compute nodes. But a full catalogue of further services and tooling APIs and migrating of that can be a big project.
maxima has existed for ages, maple has very similar fatures
1. It is arguably the most sophisticated CAS available. Open source tooling has crept up in performance and completeness in recent years, but Mathematica still dwarfs every open source system in performance and feature availability. Competing proprietary systems like Maple are capable of beating Mathematica in certain specific domains (like PDEs), but that leads me to my second point.
2. Mathematica isn't just a CAS. It also supports sophisticated data analysis and ingestion, visualization, (some) machine learning, natural language processing, speech recognition, signal processing, climatology, meteorology, geography, financial analysis, and limited forms of convex optimization.
Maxima is just a computer algebra system- one small part of what Wolfram does.
The parent said it doesn't compete "in the breadth or depth of Mathematica" and you responded with "it does compete because it's free!"
Your response does not refute the parent's argument.
With that said, I don’t speak for everyone! If folks find this tool useful, love writing Wolfram code, have no expectations about the portability or maintainability of the code, trust Wolfram and the contract to which they’re agreeing, then by all means! More value for people is better!
I'm a bit puzzled about how strongly you seem to be feeling about this topic. Given your comments, you seem to be quite far from the target audience of Mathematica. It's not for software engineers. Most productive work done with Mathematica is not "building software" ...
But given all this hostility, have you downloaded it, and have you tried it? Or is all this just philosophical (i.e. the least interesting) criticism? There seems to be no experience backing your comments.
I hope folks aren’t finding my comments hostile. I have used the Mathematica product before in their GUI and at the command line, and some of my colleagues use it for various calculation tasks. I’ve opted to stick to open source alternatives like Sage, wxMaxima, GAP, and Axiom/Fricas.
The engine doesn’t give you a license to Mathematica, so I don’t presume the target audience is the collection of folks who want to do differential equations with plots. The article emphasizes lots of software things (languages, integrations, etc.), so actually I do believe they’re targeting software engineers.
You should note that I have not criticized their features, but rather restrictions on the use of the product in the first place . Maybe that would be interesting to read, but instead I have commented on the leading question in Wolfram’s article. I have not and will not download their Wolfram engine in its current state though for the reasons I’ve outlined. Do recall that using it means I’m bound to a legal contract, and that’s not one I’m willing to engage with.
[0] https://arxiv.org/abs/1904.10541
[1] https://github.com/rigetti/quilc/blob/master/src/compilers/a...
As an old Lisp programmer, it makes me feel good seeing CL applications.
I broadly agree with your points in this conversation but I am downloading the Engine right now. I also did a ten day free trial of Mathematica recently. I have a little time to spare to kick the tires.
BTW, in the same way that Keras has functionality to download some standard data sets and cache them locally, I would like to see more CL projects do the same. Julia has a few such packages. Having easy to access data is a selling point for Wolfram Language.
As such, this comment:
>But given all this hostility, have you downloaded it, and have you tried it? Or is all this just philosophical (i.e. the least interesting) criticism? There seems to be no experience backing your comments.
does not seem to apply
and using an AGPL licence, there would be no danger for them!
Please explain how/why FOSS is human birthright?
Making good software requires significant effort (both up-front and on-going) and being compensated for that effort is reasonable
The problem of big cloud providers ripping off OSS works is a new one, but it is explicitly allowed by OSS licenses - really not sure what a "good" solution is here, but personally I like the route TimescaleDB has gone down.
TimescaleDB has a good blog post about why they went this route[0]
I do think that perhaps we need to stop being so snooty about OSS vs open source, and recognise that companies with open source products are allowed to make a profit while preventing the big cloud providers from scalping them.
[0] https://blog.timescale.com/how-we-are-building-an-open-sourc...
Azure and AWS already have competing stacks to Wolfram, it is just various and sometimes incompatible hodgepodges of R, Python, Jupyter, and so on and so forth.
If Wolfram open source their stuff and people use it, it won't necessarily be a vindication. At the moment they have people who are willing to part with cash to get what they make. That's a definite vindication for anyone who makes things.
Really, with the kind of license proliferation that was going on due to every Tom and Harry that wanted to publish their own license, it was difficult to understand which license was really safe to use and which wasn't. FSF and OSI stepped in to bring some sanity to this situation by categorizing all the free and open source licenses by their permissiveness and restrictions. The majority of the open source development community adopted these licenses.
Any license that restricts making money would neither be considered a free license by FSF nor an open source license by OSI. The license could discourage making money without contributing back to the community and the GPL family of licenses do achieve that to some extent but they cannot outright restrict making money.
The OSD has its roots in Debian Free Software Guidelines which for a long time has been the community standard for what guarantees free software must provide. FSF's four freedoms of free software are also similar. OSD adopted Debian's guidelines to create the open source definition.
If a license does not allow us to run a software how we wish we lose freedom 0 to run the software how we wish. We lose the right to use the software for any field of endeavor. You might want to consider it "open source" because you can still see the source but widely accepted terms like "open source" do not get redefined so easily based on how some people feel about the term. The term's meaning still remains intact due to the meaning it holds for the vast majority of open source software developers.
There are other terms for source which is available but proprietary, such as "shared source".
His response belittled the question, calling it naive. His smugness was astoundingly tone deaf.
Until he figures out how to answer questions like this without using shame to cow developers into line, I’m staying away.
Not to mention the fact that probably he doesn't own all rights for doing that. Software with such a history often contains third party licensed modules.
Although, perhaps this step is kind of a result of stagnating growth, IDK.
that is simply not true and a rather insular view. i build large complex systems, and all of it is done with proprietary software. in fact, it is practically impossible to do my job exclusively with open source software. most open source software is completely unreliable and ill-fitted for many domains. and just try getting good support for use cases only found in industrial use from most open source developers. i have tried, and it never works out. the response is usually "you figure it out and let us know".
furthermore, a lot of proprietary software is fairly open source in that you get access to a fair amount of source code. and it isn't dwindling at all.
“Source [partially] available” and “open source” are generally considered distinct concepts these days. See for instance [0] or even look up what Google’s dictionary says about the term.
[0] https://en.m.wikipedia.org/wiki/The_Open_Source_Definition
This alone makes your "uncommon" claim void, and there are other examples outside of the Windows ecosystem equally well-rooted in proprietory tools.
Now if MS had their own language (they did...) that was closed source (it was...), the argument would be more akin to this Wolfram debate. But MS learned that wouldn’t fly, and adopted an OSS strategy.
>> Visual studio
> I don't think that's a strong argument
Lovely talking to you.
I would say Windows development in Visual Studio is by far uncommon when you compare with technologies like Python, C (with GCC or Clang), Java (OpenJDK), JavaScript, etc.
Doing it in one big jump would probably impact revenue so much that it would take too long for them to grow out of it. Hopefully this change will increase their growth rate without much change to revenue, making it faster to grow out of, and prepare the way for further changes.
1 Fear of losing control (over their vision and centralized design of computation and representation of the Wolfram language).
This is just false, as they would be in control over their project even if open-sourced.
Forks would be possible but would not influence the original source (if not wanted).
I cannot see why this would be true.
I see how that could be a problem. One solution would be funding (s. point below)
to pay devs for the boring stuff.
Their main point (and probably a big issue of a lot of other companies
thinking about open-sourcing their software) and a very valid one indeed.
@Wolfram: The most important assets of Wolfram are the devs. working there, not the software itself. Given this huge knowledge base around the topics covered by the Wolfram language, the support-business-modell could very well work! Also from a philosophical point of view: Wolfram claims to unify disparate fields of all of science which kind of is a scientific achievement on its own. Scientific results should be available for all people of our society.
That depends on the software development segment. If you are creating 3D games you would expect to build on top of Unity or Unreal because unless you are a major studio building the infrastructure yourself is expensive, slow and not as good. The question is "is computation" such a segment?
My question is: do you use Google?
No one asked about programming languages or standard library (except you when you misunderstood my question).
Therefore you implied that a good reason not to use a technology is that it's not open source. Literally.
Then why are you using Google?
WL is essentially just a bunch of useful functions collected into a common platform, and I think we already have the open source equivalent of that - it's called npm, and it's great, but it is very different, and it has some pretty big downsides which are the direct result of it being an open source project rather than a commercial product. I don't know if WL is as elegant or high-quality or deeply functional as Wolfram claims, but if it is, isn't that likely a result of it being built the way it's built?
It might be that there's a middle option - a way to open some core part of the language, while still keeping enough closed to make up a service layer that they can charge for, as you described - but today's announcement is pretty strong evidence that they considered that and rejected it. It seems pretty presumptuous to assume they got it wrong.
It’s not open source.
Even if the engine is “free”, I don’t want to build an open source product with it and hope to be granted a “free production license.” If I build something on my own time I don’t want to ask my employer to purchase a “production license” as soon as it becomes useful.
Wolfram believes that mathematics software (or “computational knowledge” or whatever he calls his entire enterprise now) must be proprietary and paid-for in order to exist. Maybe it’s true; all of the successful and wildly popular computer algebra systems are closed source. (wxMaxima is rough to use, Axiom has 3 or 4 different forks, each with fewer than 10 developers, SymPy just isn’t there, Sage is absolutely wonderful but not polished or easy to deploy, ...) But that’s completely at odds with how most software engineers work these days. Most software is grounded in an open source development and deployment tool chain. There is a market for proprietary developer tools, but it had been dwindling since its prime-time in the 90s.
I think Wolfram needs to think quite hard about how he wants to get his technology in the hands of developers while maintaining a business. Not that my opinion matters, but if he can manage to do it by open sourcing Wolfram Language, Wolfram Engine, or something like that, while keeping his business intact, I might again consider him to be the genius he was lauded to be in his 20s.