News for hackers. Stuff that matters.
Re:Malleable Systems: Software must be as easy to change as ... by saagarjha on Monday April 13, 2020 @02:04PM (#22858254)
I don't think this is a very popular opinion, but for those shipping software, consider having at least one programmatic way to "monkey patch" it. These are things like a plugin API, dynamically linking it, or using a language with an exposed runtime–things that allow programmatic injection of code, so that users who really care about tweaking the tool can do so. It doesn't even take much effort, either: my text editor has no plugin interface, but one of the reasons I stick with it is because I can hook its calls to libc to customize some of the things I don't like. My preferred mail client happens to lack a certain entitlement that most people don't even know about and that enables an entire cottage industry of software for it. Browsers let you specify some JavaScript to run on each page and suddenly the balance of power on the web tilts significantly towards the user. Even (or especially) if your tool is open source, consider supporting such an interface: it's much nicer to add on to piece of software than fork it.
Re:Malleable Systems: Software must be as easy to change as ... by bcrosby95 on Monday April 13, 2020 @03:21PM (#22859090)
The problem is support. Even if you put big caution tape around it, if you break it or change it in a way that isn't backwards compatible users will complain loudly about it. And if you have users paying lots of $$$ relying on it, it might force your hand.

Building a plug-able system in a way that doesn't blow up in your face can be hard. Firefox got a lot of bad press for redesigning their plugin system.

Re:Malleable Systems: Software must be as easy to change as ... by arghwhat on Monday April 13, 2020 @08:14PM (#22861588)
Arguably, it's not just about support: It's often just not a good idea, complicating the existing product, still only gives you pre-defined hooks.

Jenkins is one of my favorite examples for why plugin systems are evil. As users accumulate plugins, it all becomes a giant, unmaintanable monolith that ends up unable to perform even the most basic tasks due to some unrelated/uninteresting/forgotten plugin somewhere doing something weird.

Instead of stuffing everything into a monolith with plugins, split it out into individual composeable units with minimal features.

Re:Malleable Systems: Software must be as easy to change as ... by egypturnash on Monday April 13, 2020 @08:50PM (#22861771)
Here is a thought experiment for your preference of “individual units with minimal features”: How do you take an artist’s toolbox app (Photoshop, Illustrator, Blender, Painter, Procreate, Clip Studio, etc) and decompose it into individual units that can still be interacted with via a stylus as the primary UI?
Re:Malleable Systems: Software must be as easy to change as ... by arghwhat on Tuesday April 14, 2020 @12:07PM (#22867656)
While I was primarily referring to things with a backend, trying to redesign these "everything but the kitchen sink" apps is actually an interesting task.

First of all, Photoshop isn't a toolbox. It's a tool, but with plugins. The real-world analogy would be a KitchenAid kitchen machine, where you can attach awkward tumors like a spaghetti machine or entire food processor to their power take off. The kind of accessories that, after purchase, make you look long and hard in the mirror, trying to figure out just when it happened that your life fell apart.

A toolbox, on the other hand, is just a vehicle for making tools easily available. That, and being a blunt weapon in emergencies.

If I were to imagine a universal image toolbox, I'd imagine it as system that can visualize a stack of buffers. Then, it would be able to launch other applications, providing access an API to access and manipulate the buffer stack. That's all.

These applications can provide your usual floating toolbox UI's just fine, and nothing should stop it from having the same UX as Photoshop, but plugin-less. Depending on things, these tools may use usable standalone, for bulk use, command-line use, from different toolboxes, whatever. You'd be able to swap out features that are otherwise Photoshop built-ins, as they're just another application here.

This distinction between plugins (i.e. extendable tool) vs. toolbox (isolated tools) is important: Instead of awkward tools designed entirely around their ability to attach awkwardly to a KitchenAid, you get a tool designed around doing its job right.

(That KitchenAid accessory metaphor works a lot better than I initially anticipated—the PTO is the annoyingly limited hooks you have to work around, and the results are as awkward as any plugin system.)

Re:Malleable Systems: Software must be as easy to change as ... by tcgv on Monday April 13, 2020 @08:48PM (#22861760)
> As users accumulate plugins, it all becomes a giant, unmaintanable monolith that ends up unable to perform even the most basic tasks

I agree that most of the time it's best to give users only one way of doing things, so that they don't get creative and go down a rabbit hole.

Nevertheless, if a product targets a specific type of user, which is expected to be knowledged about and follow certain best practices while using it, making the product more customizable can be a competitive advantage.

We can't always blame the product for being "too complex" if it's purposely designed that way so that well-read users can take the most out of it.

Re:Malleable Systems: Software must be as easy to change as ... by pfranz on Tuesday April 14, 2020 @03:21AM (#22863472)
Sure, I think that's a market to target. But it sounds like it's a slog to develop for and often a slog to use (or at least set up).

Often the opinions made when designing the software is part of what you're selling. Otherwise, I just ship you a box of 1s and 0s and you can put it together yourself =)

Re:Malleable Systems: Software must be as easy to change as ... by arghwhat on Monday April 13, 2020 @09:20PM (#22861904)
Complex ≠ useful. A TV and a set top of your choice box is simple useful. A smart TV does the same thing, but is complex and inflexible.
Re:Malleable Systems: Software must be as easy to change as ... by munchbunny on Tuesday April 14, 2020 @12:43AM (#22862821)
I just attached a desktop PC to mine... so now it’s more complicated and does everything I want. =P

Doesn’t require much complexity from the TV though.

Jokes aside, sometimes simplicity is judged on different criteria. Everyone’s favorite example is Craigslist (ugly, but actually quite simple once you grok it), but I can name a few others where the actual criteria for simplicity can be hard to define.

Bash, for example. It takes a while before you really realize what the scripting can actually do and what Unix pipes can do. Once you see it, it makes a ton of sense, but until you grok the abstractions, it's just a pile of syntactic mush you can't remember. It's simple for what it does, but the problem itself has a lot of complexity.

I believe the same thing about Vim. It requires training because while its modes become simple after you start to get it, it's an irritatingly arduous road to get there. So from the perspective of a lay user... not simple at all. From the perspective of an expert, it's effective to the point of religion.

Re:Malleable Systems: Software must be as easy to change as ... by arghwhat on Tuesday April 14, 2020 @12:12PM (#22867723)
True. SmartTV's may appear simple, but I am fairly certain that their simplicity crumbles for most users quickly.

Say, once a user just wants to play games on a console and has to wait for a TV to boot and prompt you to install updates, when a user wants to use AppleTV instead of Android TV, or when they want to upgrade the smarts.

I think the man reason people think this is simpler is that manufacturers force it down consumers' throats as this setup sells more units due to planned obsolescence.

Re:Malleable Systems: Software must be as easy to change as ... by saagarjha on Monday April 13, 2020 @10:45PM (#22862276)
And I can often run my own software on the set-top box, too, if I’m really interested in customizing it. With TVs this is usually harder.
Re:Malleable Systems: Software must be as easy to change as ... by ethbro on Monday April 13, 2020 @08:28PM (#22861673)
> Instead of stuffing everything into a monolith with plugins, split it out into individual composeable units with minimal features.

This feels like microservices-for-everything as a product.

Then you've designed a platform, with a reference combination. Which carries its own set of challenges and over-complications.

There's something to be said for a clean, batteries-included product... with a way to hack the hell out of it.

Re:Malleable Systems: Software must be as easy to change as ... by arghwhat on Tuesday April 14, 2020 @12:26PM (#22867898)
Microservices is a polar extreme to monoliths.

It's not about making each macro-service into 1000 micro-services. It's about making your macro-services smaller, instead of all-consuming monoliths.

The problem is that monoliths will never be hackable. A LEGO-kit is hackable. A monolith just have hooks and connectors, but the product itself will always be static.

You can get things shipped as pre-assembled LEGO kits (both metaphoric or not)?

Re:Malleable Systems: Software must be as easy to change as ... by detaro on Tuesday April 14, 2020 @03:45PM (#22870575)
An interesting comparison to Jenkins in the same space is Buildbot. Which embraces being written in a scripting language, and is "configured" by writing code that describes the builds, using existing primitives and extending or replacing them where necessary.
Re:Malleable Systems: Software must be as easy to change as ... by enos_feedler on Monday April 13, 2020 @03:36PM (#22859219)
This is a great point and something to consider when designing runtime programmable tools. It should mean moving slow, cautious with minimal surface area and opinions about the future is probably a way to go. Having strong isolation is probably good too. This is something that made me uneasy about highly programmable/hackable platforms. I like when the core maintainers understand _how_ a system wants to be extended and shaped and provides precise hooks for doing so.
Re:Malleable Systems: Software must be as easy to change as ... by Cthulhu_ on Tuesday April 14, 2020 @03:55AM (#22863630)
Interesting you mention Firefox because, when you think about it, a web browser (just like an operating system) is a massive pluggable system. They changed the plugin system, but they haven't (as far as I know) broken web standards, so that 20 year old websites still work today.

In that regard, browsers aren't that bad.

Re:Malleable Systems: Software must be as easy to change as ... by funkymike on Monday April 13, 2020 @06:49PM (#22861034)
It seems to me the key then would be to have that programmable interface be used to actually implement the features shipped as part of the project. It will be a lot easier to get the API right if the developers are using it themselves, plus that adds a lot more incentive to maintain backwards compatibility as well.
Re:Malleable Systems: Software must be as easy to change as ... by nine_k on Monday April 13, 2020 @07:16PM (#22861232)
Yes, the fabled Emacs way.
Re:Malleable Systems: Software must be as easy to change as ... by denster on Monday April 13, 2020 @03:35PM (#22859214)
¯\_(ツ)_/¯

Card carrying member of the disagree-club here (malleable clan), but I think it's a tall order to ask users to write procedural code.

Tools that are:

(1) declarative by nature (HTML, DSLs)

(2) based on direct manipulation (PSA: we're lucky to have Bret Viktor on this planet)

is what will win out in my opinion.

Or will we be writing procedural code 30, 50 years out? :)

--

[1] YMMV, founder of https://mintdata.com here, where we recently opened public access & are seeing the above come true.

Re:Malleable Systems: Software must be as easy to change as ... by nijave on Wednesday April 15, 2020 @08:28PM (#22884812)
Who actually likes DSLs and why? You introduce the complexity of a lexer and parser in the form of an abstraction on top of an API. What does that buy you?

In my experience, it just makes workflows more difficult since you now need to create custom editors, linters, formatters, testing frameworks, etc. You need a whole new ecosystem before you can truly become productive on top of learning an entirely new syntax.

Re:Malleable Systems: Software must be as easy to change as ... by slifin on Monday April 13, 2020 @03:59PM (#22859404)
Mintdata looks cool, what does the imperative layer that drives the declarative layer look like? It'd be nice if this was a platform where engineers could sell the imperative layers
Re:Malleable Systems: Software must be as easy to change as ... by denster on Monday April 13, 2020 @05:29PM (#22860260)
Thank you! Genuinely warms my heart to hear it :)

We've been using our own "Visual SDK" to build large parts of MintData for the past few years.

It's 100% written in TypeScript/React, using decorators (cough: annotations!) and allows us to:

1) add new spreadsheet functions in TypeScript

2) wrap any React component

3) we get the auto-magic property panel UI "for free" & it instantly talks to/from the spreadsheet

For our cloud/backend imperative layer:

4) we use our own real-time stream processor (think: Apache Storm but minus the Clojure stacktraces)

5) the SDK lets us quickly write flow blocks in Java using annotations & connect to all sorts of enterprise-y systems (SQL/NoSQL stores, Queues, File Systems, etc)

@slifin -- in terms of engineers writing & selling those layers, we're open to it -- ping me if you have ideas & we can give you access to the docs & SDK.

--

[1] I'd put a link here, but the flamethrowers have come out on this thread :)

Re:Malleable Systems: Software must be as easy to change as ... by chrisweekly on Monday April 13, 2020 @06:32PM (#22860920)
Looks pretty cool; IMHO it's a good idea. (I once participated as a consultant / dev on a project to convert a Byzantine set of Excel spreadsheets into a deployable application built on web technologies, and this might have been a saner approach.)
Re:Malleable Systems: Software must be as easy to change as ... by denster on Monday April 13, 2020 @07:08PM (#22861178)
Thank you!

Before the others come with torches, I’ll point out we turned on public/free access recently, would be happy to hear your thoughts/feedback on the MintData approach.

Re:Malleable Systems: Software must be as easy to change as ... by jmercouris on Monday April 13, 2020 @03:30PM (#22859169)
I agree :-)

The user should have the power. Also the interface should be as powerful as possible. One such browser that embraces this philosophy is Next https://github.com/atlas-engineer/next. Source: biased author

Re:Malleable Systems: Software must be as easy to change as ... by denster on Monday April 13, 2020 @03:48PM (#22859313)
Very cool! I'll show this to our team [1]; we have a few keyboard ninjas & I think they'll absolutely love it.

--

[1] team of https://mintdata.com, where we've been known to appreciate a fine keyboard shortcut or 12.

[2] partly an inside joke to our team who reads HN, since we recently shipped keyboard shortcuts for our top menu (Google Sheets, we borrowed what we could to ease the pain :D )

Re:Malleable Systems: Software must be as easy to change as ... by bem94 on Monday April 13, 2020 @02:25PM (#22858467)
I agree that people would and do find such interfaces very useful in many of their programs and tools.

Those with nefarious purposes find them extremely useful too. It's a lot of work to secure those interfaces against abuse.

Re:Malleable Systems: Software must be as easy to change as ... by saagarjha on Monday April 13, 2020 @02:44PM (#22858666)
This is why I believe it to be an "unpopular opinion" these days as most computing platforms move towards being more locked down and less extensible; for security or complexity reasons these kinds of things seem to get little love or are even axed completely. Usually, the hard part is not security per se, but identifying user intent. You want to come up with a process that ensures that the user who enables potentially dangerous interfaces knows what they are doing. Preventing the user from doing this entirely is an easy but fairly disappointing "solution" to this problem.
Re:Malleable Systems: Software must be as easy to change as ... by bem94 on Monday April 13, 2020 @02:50PM (#22858747)
> Preventing the user from doing this entirely is an easy but fairly disappointing "solution" to this problem.

For sure. And if individuals are happy with trading some security for some functionality/flexibility in their software, I'll support their right to make that choice.

It gets more complex when the people affected by that choice aren't the people making the choice. Which is more likely in the case of network programs.

> Usually, the hard part is not security per se, but identifying user intent. You want to come up with a process that ensures that the user who enables potentially dangerous interfaces knows what they are doing.

Indeed. I'd add identifying developer intent to that as well. What functionality am I trying to enable v.s. what else am I actually enabling? Coming up with that process is the herculean task there.

Re:Malleable Systems: Software must be as easy to change as ... by saagarjha on Monday April 13, 2020 @07:45PM (#22861420)
Usually in the case of network programs you want users to have a way to control their view of the shared backend/database/whatever, which means providing things like REST APIs and hackable clients.
Re:Malleable Systems: Software must be as easy to change as ... by fergie on Tuesday April 14, 2020 @03:12AM (#22863434)
"Those with nefarious purposes find them extremely useful too"

Whilst I don't disagree with this point, it has all to often been used as an argument against open software, and ultimately leads to more closed and less secure software.

Re:Malleable Systems: Software must be as easy to change as ... by conradev on Monday April 13, 2020 @03:29PM (#22859158)
Re:Malleable Systems: Software must be as easy to change as ... by hinkley on Tuesday April 14, 2020 @11:10AM (#22866919)
It’s challenging to do, but there is something to be said for having features you don’t advertise.

You don’t have to make them work for everyone. They don’t have to be safe in all possible scenarios. But they are there for Support to offer to users who are caught in a bind and need a solution.

The about: box is a bit like this. It’s there, if you really need it, but they aren’t going to flood their menus for all the 2% cases.

Re:Malleable Systems: Software must be as easy to change as ... by noizejoy on Monday April 13, 2020 @07:49PM (#22861443)
It exists for software that considers itself a platform in the form plugins, themes and such. And it’s very different, much harder, software play for support and reputation management.