News for hackers. Stuff that matters.
Re:Systems ideas that sound good but almost never work by travisjungroth on Tuesday December 31, 2024 @02:25PM (#42561096)
I've written two DSLs (one with a team) and I'd consider them both successful. They solved the problem and no one cursed them out. I think the most important factor is they were both small.

They were very similar. I even reused the code. One was writing rules to validate giant forms, the other was writing rules to for decisions based on form responses.

Ok, just ranting on DSLs. Good DSLs take someone from can't to can. A DSL that's meant to save time is way less likely to be useful because it's very likely to not save you time.

In both of my DSLs, it's that we needed to get complex domain behavior into the program. So you either need to teach a programmer the domain, partner a programmer with a domain expert, or teach a domain expert how to program.

Putting the power in the hands of the domain expert is attractive when there's a lot of work to be done. It frees up programmers to do other things and tightens the feedback loop. If it's a deep domain, it's not like you want to send your programmer to school to learn how to do this. If it's shallow, you can probably have someone cheaper do it.

A DSL comes with a lot of cognitive overhead. If the other option is learning a full programming language, this becomes more reasonable.

A time saving DSL is where someone already knows how to write code, they just want to write less of it. This is generally not as good because the savings are marginal. Then when some programmer wants to change something, they have to learn/remember this whole DSL instead of more straightforward code.

Actually, this makes a simpler rule of thumb. A DSL for programmers is less likely to be a good idea than a DSL for non-programmers.

Re:Systems ideas that sound good but almost never work by Majromax on Tuesday December 31, 2024 @04:06PM (#42561835)
I think the theory of domain-specific languages is very valuable, even if there's rarely a need for a full implementation of one.

As I see it, a DSL is just the end-state of a programmer creating abstractions and reusable components to ultimately solve the real problem. The nouns and verbs granted by a programming interface constrain how one thinks, so a flexible and intuitive vocabulary and grammar can make the "real program" powerful and easy to maintain. Conversely, a rigid and irregular interface makes the "real program" a brittle maintenance nightmare.

Re:Systems ideas that sound good but almost never work by travisjungroth on Tuesday December 31, 2024 @09:42PM (#42563551)
I agree. The line between a DSL and regular old programming abstractions is fuzzy. Learning some language design is very helpful because you’ll see every abstraction is a little piece of language.
Re:Systems ideas that sound good but almost never work by f1shy on Tuesday December 31, 2024 @02:33PM (#42561157)
DSL are just great, have never failed for me. I have done it several times, the last one just the past year, for programming complex ASICs. I‘ve seen uncountable times working like a charm.

I cannot understand why seems it was bad for him…

Re:Systems ideas that sound good but almost never work by travisjungroth on Tuesday December 31, 2024 @03:21PM (#42561538)
I think there are two big reasons DSLs often underdeliver. The first is not understanding the cognitive overhead budget. If it's something where it's being used infrequently or by a lot of new people, that's a lot of overhead to be spent each time. Sometimes people think of writing DSLs for tests because they have to write a bunch, but it's really easy for this to suck. I have a test turn red and now I have to learn a DSL to deal with it? Ew.

The second is fuzzier. It's putting a DSL over something complex and hoping this will fix things. Writing SQL queries for this system takes a bunch of time and is error prone? Just put a DSL over it! Except all those details and errors are probably going to leak right through your DSL.

You want to master the domain before you put a DSL over it.

Re:Systems ideas that sound good but almost never work by f1shy on Tuesday December 31, 2024 @03:51PM (#42561737)
>> The first is not understanding the cognitive overhead budget. If it's something where it's being used infrequently or by a lot of new people, that's a lot of overhead to be spent each time. Sometimes people think of writing DSLs for tests because they have to write a bunch, but it's really easy for this to suck. I have a test turn red and now I have to learn a DSL to deal with it?

What is the alternative to the DSL with lower cognitive load? I do not follow. Every single DSL I’ve seen REDUCES the cognitive load, by allowing to express the concept in the mere language of the problem at hand, for which the SME should be more than familiar with.

About the second point: I see many critics in this thread based on DSls above SQL. Whatever somebody is doing above SQL and selling as a DSL, it is not. Period. I cannot think in any possible way of doing a DSL above a query language. No doubt people hate the idea. Is a BAD one.

Re:Systems ideas that sound good but almost never work by travisjungroth on Tuesday December 31, 2024 @09:39PM (#42563538)
> What is the alternative to the DSL with lower cognitive load?

In the test example, writing it directly in the programming language. This will usually lead to code that is more verbose and repetitive, but understanding the first example will be faster.

I think of cognitive load like a line. X is the number of cases you’re working with, Y is cognitive load [0]. For someone who already knows a programming language, the DSL is going to have a higher Y intercept since you have to learn something new before you understand the first case. Hopefully, it’s a shallower slope so as you deal with more cases the upfront cost gets paid back. If you have lots of people dealing with one case or doing it infrequently enough they have to relearn each time, this payoff never happens.

This model extends past DSLs to all abstractions. It’s why people often end up happier with test code that’s less abstract/DRY. The access pattern supports it.

Looking at it this way also explains why a DSL for a non-programmer is more likely to be useful. The intercept can be lower than an actual programming language, so you’re ahead from the start.

[0] It’s really more of a curve, but the line model works conceptually.

Re:Systems ideas that sound good but almost never work by epgui on Tuesday December 31, 2024 @03:51PM (#42561740)
These examples sound like things that aren’t really DSLs… Or in other words it sounds like someone is trying to make something “simpler than it actually is”.

DSLs are supposed to be for making it easier to perform computation in a specific context. Software tests have about as many degrees of freedom as the programming language they are written in, so I’m not sure they are an ideal use case for a DSL— not without a lot of discipline at least.

For a DSL to make sense, IMHO, you need to be able to write down a complete and correct specification for it. I doubt that is even possible in the given examples :shrug:

Re:Systems ideas that sound good but almost never work by bjourne on Tuesday December 31, 2024 @08:45PM (#42563312)
There are hundreds of DSLs for ASIC design but not a single one of them has ever been used for actual tapeout. It's 100% unheard of. Hence, I doubt your DSL saved any time over using an RTL language directly. Sorry for sounding harsh, but if you work in the area you understand my skepticism about ASIC design DSLs.
Re:Systems ideas that sound good but almost never work by layer8 on Tuesday December 31, 2024 @09:08PM (#42563430)
Are your DSLs used by other people and do they share your opinion? In my experience DSLs are nice to work with for the creator, but it’s much more work in documentation, training, intelligible error handling, and so on, to make a DSL that’s easy for others to learn and use.

I do like DSLs, but the value proposition is often difficult, IMO.

Re:Systems ideas that sound good but almost never work by throw16180339 on Tuesday December 31, 2024 @09:56PM (#42563609)
> I cannot understand why seems it was bad for him…

There are many poorly designed libraries, and DSL design is no easier. While I haven’t personally encountered any, I’m sure there are numerous half-baked DSLs out there.

Re:Systems ideas that sound good but almost never work by crabbone on Wednesday January 01, 2025 @06:29AM (#42565409)
There's a difference between "works once for my very specific problem" and "works most of the time for a wide range of problems".

DSLs, in my experience, usually fail in the later definition. It's very hard to make a small language that precisely captures its domain of application, will produce easy to manage programs no matter the size, would be easy to analyze in terms of performance and side-effects.

Re:Systems ideas that sound good but almost never work by d0mine on Wednesday January 01, 2025 @12:57AM (#42564241)
Here's a few DSL (for programmers) examples that might make sense (the context is embedding DSLs into Python literal strings): https://news.ycombinator.com/item?id=41214365

For example, bash, SQL DSLs may be immediately useful by protecting against shell,sql injection: shutil.run(sh"command {arg}") may translate to subprocess.run(["command", os.fspath(arg)])

No shell--no shell injection. The assumption is that it enables sh"a | b > {c}" syntax (otherwise just call subprocess.run directly). Implementing it in pure Python by hand would be more verbose, less readable, more error-prone).

Re:Systems ideas that sound good but almost never work by travisjungroth on Wednesday January 01, 2025 @02:52AM (#42564602)
Yes, there are definitely counterexamples. It’s not black and white at all.
Re:Systems ideas that sound good but almost never work by lifeisstillgood on Wednesday January 01, 2025 @06:26AM (#42565398)
>>> A DSL for programmers is less likely to be a good idea than a DSL for non-programmers.

Nail on the head time - somewhere else in the thread is jooq which is (yet another) SQL DSL where you end up with from(table).where(student=bob)

This is a perfect example of why the programmer should (just?) learn SQL instead of the DSL - and your comment nails it

Re:Systems ideas that sound good but almost never work by PaulHoule on Wednesday January 01, 2025 @12:08PM (#42567351)
(1) JooQ shines for query generation. For instance at work we have a search engine that can search maybe 50 fields that are mostly materialized but involve the occasional subquery. Also you can easily write a template method like

   transitiveQuery(selectClause,relationshipGenerator,where1,...)
and have type inference do the right thing in the compiler and IDE and all of that. You get "hygenic macros", you can write an abstract Java class which is parameterized by a table and then subclasses can implement additional functionality on top of a table.

(2) Circa 2005 coding PHP I came to the conclusion that the ORM I needed was

   insert(table,{col1: val1, col2:val2, ...})
because writing simple inserts and updates against SQL is for birds, let freshers do it and they will forget to escape something. Such a "framework" can be so simple that you can bend it to the needs of your application. JooQ gives you something like that but backed by the Java type system.
Re:Systems ideas that sound good but almost never work by lifeisstillgood on Wednesday January 01, 2025 @03:52PM (#42569194)
I do get it (I think!) - but there is a world of difference between “I have 20 years SQL experience and do not want to spend hours maintaining 200 SQL templates, and believe the overhead of this DSL is worth the trade off” vs “use the DSL and you won’t have to teach junior Devs SQL!”

My comment is more aimed at the second part. SQL Is tied to the implementation and demands coders understand it all. A DSL can allow domain experts to express their understanding without having to worry about software trade offs.

The most successful “DSL” I know of like this is fitnesse tests - just a large number of simple tests where domain experts can spreadsheet style throw in the “gotchas”.

Something like that but more spool is fixated is a holy grail - Behaviour driven tests like cucumber come close but there is that weird intermediate translation from English phrase to random function - now you have to understand the function to use the phrase and suddenly you are reading real Code to be able to use the fake code and it never feels clean

One day I will be clever enough to be able to write a really good test DSL

It’s just whenever I think of “Given used is logged in, visit “textbox” and enter “word” .. it just looks like BDD test not a DSL. Like I said, one day I will be clever enough

Re:Systems ideas that sound good but almost never work by PaulHoule on Thursday January 02, 2025 @04:24PM (#42579099)
Sure. Note though that there's a long tradition of systems for embedding SQL in conventional programming languages such as

http://infolab.stanford.edu/~junyang/cs145/or-proc.html

which for whatever reasons never caught on in the open source world. (I'd blame limitations of current compiler technologies and the values of people who make compilers... If we had composable parsers you could just say "here's a spot for a SQL query in a Java method" in 10 lines of code) JooQ approaches that without requiring any change in the compiler. In the past it was awkward to embed SQL in Java because there were no multi-line strings. In Python you could write

do_query(" ... a really crazy complicated queries with lots of joins and subqueries that is carefully indented to fit in with the rest of the program ... ",{"arg1": val1, "arg2": val2})

but without real map literals, multi-line strings and such this was terribly awkward. (If you think List.of(), Map.of() and such are cool I was writing a computer chess program last month that used List.of(A,B) to create a list that was used in an inner loop and it was terrifying how slow it was compared to using an ArrayList)

Re:Systems ideas that sound good but almost never work by ramchip on Wednesday January 01, 2025 @02:10PM (#42568366)
I'm not familiar with jooq, but I've used Ecto a ton, and the point was never to avoid learning SQL. It's about making queries composable and mapping to domain objects automatically so eg. there aren't dozens of queries to update when you add a field to a domain object.