Fantastic DSLs and where to find them

http://serce.me/posts/29-06-2017-fantastic-dsls/

by SerCe • 9 years ago

180 140 9 years ago

Anyone else dislike the immutability fad?

Mutability has its place, and simply hiding it behind abstractions tacked on to the language (vars, refs, agents, and atoms in Clojure's case) isn't a productive way to deal with it. It has a lot of benefits, but the downsides are significant. Sometimes I just want to create a map and mutate its structure, and the language saying "no" is constraining me in an unpleasant way.

It's the old debate of whether the programmer should be constrained by the language, or the language should serve the programmer. Maybe it's a matter of opinion.

Also, typed function parameters are painful. Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? What if you want to change it? The cost goes up significantly, and I'm not sure the purported ability to sidestep a few bugs is worth it. If safety is an issue, good test coverage can be a sufficient substitute for types.

(The article is excellent, by the way.)

striking• 9 years ago

RE: Typed functions:

Kotlin lives on top of a statically typed architecture, so I think you do have to make that function typed. Some languages get around this requirement by inferring types, but type inference itself can then become Turing complete...

So it's not like there's a "healthy option". It's more that you're choosing the poison you prefer.

adrianN• 9 years ago

If you have sufficiently good test coverage to replace a type system you have to change tests whenever you'd have to change the type. No costs saved. Thinking more before coding is beneficial in my experience.

norswap2• 9 years ago

Agreed that immutability seems overrated right now. The pendulum will swing back in full force, eventually.

But of course, the art is in putting mutability and immutability in the right places. There are no fast rules, but in general mutability of local variables is harmless, and you probably need some mutability at the top level as well (as per: "functional core, imperative shell"). Mutability in other places can still be useful though.

I don't agree or really follow your comment on function parameters. It seems an argument against typing in general.

panic• 9 years ago

I think type systems derived from linear/affine/uniqueness typing will help a lot here. Mutability is fine -- it's sharing that's the problem.

sillysaurus35• 9 years ago

Re: function parameters, here's a concrete example.

Clojure's `reduce` takes a function `f` as an argument: https://clojuredocs.org/clojure.core/reduce

When you pass an empty list, `f` is called with no arguments. That way, `(reduce + [])` can return 0, whereas `(reduce * [])` can return 1.

It's very easy to write such a function in Clojure, thanks to a lack of types. What would that look like in Kotlin? A function that can either take zero arguments or takes two arguments of types specified by the input list?

nicktelford2• 9 years ago

Immutability is useful as a default, but languages should absolutely provide mutable alternatives for situations the programmer deems appropriate.

The reason immutability has become popular so recently, and the reason why I say it's the best default assumption is the rise in multi-threaded programming.

There are a growing number of strategies for dealing with mutability in a multi-threaded environment in a way that is both safe, and intuitive to reason about, the two that springs to mind are the rediscovery of actors (e.g. Akka), and go's goroutines.

Mutability certainly has its place, but so does immutability. Previously, immutability has been largely ignored in favour of the convenience of mutablity. I wouldn't say immutability is being heavily favoured now; more that it's back where it belongs, as part of a binary choice.

tdb7893• 9 years ago

Even in single threaded programs immutability can make things easier to reason about. I've had to look at some old Java projects and the easiest time I've had reasoning about them was a project that used mostly immutable objects. Ever since then I've used immutable as the default for my Java objects and it's been better for me. Your mileage may vary obviously

lostcolony7• 9 years ago

But even actors benefit from immutability. Imagine if the message you received and matched on was not the message you processed (because it was mutated in between). You either want immutability at least for message handling, or messages to be full copies, and there are tradeoffs to either choice.

zeptomu• 9 years ago

If your data is immutable or not is significant for program analysis and correctness, so I would say the distinction makes absolutely sense.

Sometimes I just want to create a map and mutate its structure, and the language saying "no" is constraining me in an unpleasant way.

Languages that provide immutable data structures have often 2 variants, an immutable and mutable. Why can't you just take the mutable in that case?

I think it's more about making immutability or mutability explicit.

pmontra3• 9 years ago

You can have both mutability and immutability in the same language. Example from Ruby:

    $ irb
    2.4.1 :001 > a = [1, 2, 3]
     => [1, 2, 3] 
    2.4.1 :002 > a.map {|x| x + 1}
     => [2, 3, 4] 
    2.4.1 :003 > a
     => [1, 2, 3] 
    2.4.1 :004 > a.map! {|x| x + 1}
     => [2, 3, 4] 
    2.4.1 :005 > a
     => [2, 3, 4] 
    2.4.1 :006 > a.freeze # make a immutable
     => [2, 3, 4] 
    2.4.1 :007 > a.map! {|x| x + 1}
    RuntimeError: can't modify frozen Array

Ruby is fundamentally a language based on mutability, but it shows that it should be possible to have both ways. Questions:

1) Is really there a language which is fully agnostic about mutability and immutability?

2) If not, it's maybe because designers have strong opinions about this matter? Actually IMHO designing a fully agnostic language would demonstrate strong opinions too.

moosingin3space4• 9 years ago

I'd argue most languages are agnostic about (im)mutability. It's libraries that aren't.

typon• 9 years ago

It's the old debate of whether the programmer should be constrained by the language, or the language should serve the programmer. Maybe it's a matter of opinion.

I like to think of it as the language saving the programmer from herself

sporkenfang• 9 years ago

Also, typed function parameters are painful. Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? What if you want to change it?

This is actually my favorite thing about languages with strong type systems where type inference is less emphasized -- if you're looking at a function definition, even if you're not terribly familiar with the language, you know what the thing is supposed to do. Sure, it's nice to let the compiler infer your types at compile or runtime, but it's really nice to just be able to read the code and be able to see what a function is at least meant to do, based on what it returns.

mpweiher4• 9 years ago

...and in fact this documentation effect is (so far) the only positive effect of static typing that has had at least some scientific validation.

reitanqild2• 9 years ago

Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know?

(Java programmer here.) Then return a sufficiently "broad" type up to, if necessary, Object.

That said I almost never fall that far back, I'd see that as a sign that I should stop and think.

Oh, and the advantage of a typed languages like Java: you can have extremely good tool support. Netbeans (or IntelliJ or Eclipse) will happily help you to change types if it later turns out you were wrong.

colanderman• 9 years ago

What if you want to change it?

If you need to change the argument/return types of a function, you have a LOT more work to do than just changing a one-line signature.

lomnakkus• 9 years ago

What if you want to change it?

Then you change it and let the compiler/IDE help you find all the other bits of code you need to change. I don't see the problem.

zintinio5• 9 years ago

Immutability is absolutely fantastic as a default (especially for typical business logic). I'm not sure I want to go back to mutability except in constrained environments or time-sensitive software. Mutability should be harder to reach for because it forces you to think hard about the implications to other parts of your program. I can think of a number of times we've had to parallelize a routine or share resources between workers, and it has been almost trivial _because_ of immutability. In contrast to that, I inherited a hairball of C code using pthreads, mutexes, global variables, etc which had to be completely gutted and rewritten as it was impossible to understand the dependencies in the software (we were witnessing segfaults, deadlock, etc).

kqr4• 9 years ago

Anyone else dislike the immutability fad? [...] Sometimes I just want to create a map and mutate its structure, and the language saying "no" is constraining me in an unpleasant way.

Immutability does not prevent you from incrementally creating new versions of a map! At this point, you are conflating no less than three different and orthogonal concepts:

A) A convenient way to write code that incrementally creates new versions of a value. This is trivial in most languages using immutable values, but it often requires unfamiliar idioms, giving people the false impression it cannot be done.

B) A performance optimisation that performs such local incremental updates in place. This is a real concern. Any immutable-by-default language should support a safe (!!) way for the programmer to ensure updates happen in place.

C) A mechanism whereby new versions of values can be distributed to other parts of a program. This one is tricky as all hell to do safely, and most good solutions work just as well with immutable values.

"But kqr, why did you take something so simple and made it so complicated?"

It was complicated from the start. Traditional languages have just ignored this complexity and solved it in an "undefined behaviour" sort of way: let's just do wha the metal happens to do, however unsafe!" You'll note tha such languages have no way to do safe in place updates, and value propagation is never sychronised or otherwise guaranteed not to cause partial views. It's a low-level hack that's far outstayed its welcome in the age of huge systems with all sorts of interaction effects.

mpweiher4• 9 years ago

It was complicated from the start.

No it wasn't. It's only "complicated" from a certain very specific and constrained POV, which you seem to have adopted wholesale and now assume as the only valid reality.

Mutability is not just how "the metal" works, it's also how the rest of the world works. If I eat a banana, the contents of my stomach (and by extension I) change. You don't get a new me that's no longer hungry + the old me that hasn't changed. If I put gas in my car's tank, the car changes. I don't get a new car. Etc.

You are also conflating immutability with atomic updates, as far as I can tell.

emidln• 9 years ago

Mutability has its place, and simply hiding it behind abstractions tacked on to the language (vars, refs, agents, and atoms in Clojure's case) isn't a productive way to deal with it.

In Clojure if you want mutability, you don't use an PersistentHashMap i.e. `{}` inside an atom `(atom {})`, you import and instantiate a mutable java class `java.util.concurrent.ConcurrentHashMap` or `java.util.HashMap` `(let [m (doto (java.util.HashMap.) (.put "foo" "bar") (.put "spam "eggs"))] .... )` and bang on it just like you would in Java. Clojure doesn't attempt to solve mutability because the host platform has already done a good job at that. Clojure provides semantics around state managment of persistent data if you need that sort of thing, but that's not necessarily a good replacement for mutability if you actually need a mutable thing.

codecurve2• 9 years ago

Few points on immutability and Clojure.

The abstraction that hides mutable data structures is transient[1], not vars, refs, agents and atoms.

Those abstractions are all to do with mutable references, not mutable data. They're there to make sure that you have safe ways to coordinate state in concurrent environments.

I agree that immutability can be annoying in languages that weren't designed with it in mind, but the languages that were almost always make it easier, safer and performant to create copies rather than mutate data.

If you're writing Clojure or Haskell and you think you're going to save yourself time by "just creating a map and mutating its structure" then you're misunderstanding the purpose of the language. These constraints are what enable the guarantees and contracts those languages make.

[1] https://clojure.org/reference/transients

sillysaurus35• 9 years ago

But it does save time. JS is proof of that. There are no immutable data structures and the world rolls on. People will hate on JS, but it's effective.

You can learn to get into the "immutability mindset" if you train yourself to, but are you certain it's worth the time investment? It seems like there's at least a chance that it's not.

borplk• 9 years ago

You pay for not specifying those parameters and so on in other ways later on.

By leaving those things out you are broadening the problem model and denying the compiler information that it needs from you as the programmer so that it can do a good job of solving the problem.

You have to remember that the computer can't synthesise knowledge about the problem domain that you deny it in the first place.

So sure tightening things down by specifying them may be tedious but it is the reality of the problem that you are solving.

Convenience of leaving it out will translate to instability such as run-time exceptions not to mention a massively reduced ability in tooling support (like IDE auto-complete).

dtech• 9 years ago

Also, typed function parameters are painful. Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know?

Why are you writing the code if you don't know yet what your input will be? A function transforms input into output, how can it do that correctly when you don't know what its input is?

If you don't know the input type yet, you should first be writing the things that will deliver the input, or leave the function as an unimplemented placeholder and define only the output type.

What if you want to change it

This is one of the best reasons to use types. Directly after the change you'll be told by the compiler which other code needs changes. Compare this to un-typed languages where you'll need a lot of tests to ensure the same or risk runtime errors.

norswap2• 9 years ago

The crispness and terseness is definitely a big part of Kotlin's appeal.

What I find suprising is that these ideas don't exactly seem breakthrough. Why aren't more language implementing them or taking them further?

pmontra3• 9 years ago

Because they're not new. Check this XML DSL for Ruby, March 2008 http://www.codecommit.com/blog/ruby/xmlbuilder-a-ruby-dsl-ca...

What's happened here is that it's being done in a statically typed language which is harder than in a dynamically typed on. So, good job done.

tormeh• 9 years ago

It's not new in statically typed languages either. There are already too many DSLs in Scala...

athenot2• 9 years ago

(For those who were wondering, the article is about DSLs in Kotlin).

DSLs can be very elegant ways to improve code readability (as long as the assumptions and language they use is meaningful to the team).

This is an area that I wish node.js had invested in. Perl6 has "slangs" which are extremely powerful, see [1] for some examples that just flow elegantly. Combine that with custom operators to write code like "4.7kΩ ± 5%" that would make sense to electronicians [2] and you can get some really user-friendly syntaxes.

(Again, it bears repeating that a DSL improves readability by sacrificing generality, so a DSL is for audience that's usually a subset of that language's users. That electronics code would be useless to the average web developer.)

[1] https://github.com/tony-o/perl6-slang-sql

[2] https://perl6advent.wordpress.com/tag/dsl/

kqr4• 9 years ago

a DSL improves readability by sacrificing generality, so a DSL is for audience that's usually a subset of that language's users.

You'd think that'd be self-evident from the "domain specific" in the name, but I guess a lot of things are obvious once they're pointed out to you...

wahern• 9 years ago

Wow. The Perl6 slang feature is awesome. I can't believe I missed it.

I haven't had a chance to use Perl 6 but I would really like an excuse. I love Lua's LPeg module. Perl 6 has deeply embedded PEGs (i.e. Grammars, aka parsing combinators) into it's design. Knowing first-hand how powerful PEGs can be if the engine and language integration is implemented well, I've never doubted the awesomeness and elegance of Perl 6. It's unclear to me if Perl 6 Grammars' Action feature is as powerful as LPeg's capturing and inline transformation primitives[1], but the fact that you can plug a grammar into Perl 6's parser is crazy awesome.

I'm not surprised Slang exists--I've known it was possible; just surprised that it's a module and idiomatic pattern and I hadn't read about it before.

[1] See http://www.inf.puc-rio.br/~roberto/lpeg/#captures Most PEG engines just return your raw parse tree as a complete data structure and require you to fix it up manually. LPeg has really well thought-out extensions that allow you to express capture and tree transformations much more naturally; specifically, inline (both syntactically and runtime execution) with the pattern definition(s).

sametmax4• 9 years ago

4.7kΩ ± 5%

And now ask somebody to type those unicode chars on the top of his/her head...

Plus how do you look what it does in google ? In a a documentation ?

How do you generate documentation for those ?

Cause you have no method name, no class name, nothing.

And then what's the exact implementation behind it ? Does it do something funny ? So I have to mix the dozen of funny details of all the code from the guys who though the 20 times they do this super-important-operation they need to recreate a whole new syntax ?

TeMPOraL7• 9 years ago

And now ask somebody to type those unicode chars on the top of his/her head...

That's about the only problematic thing with this, though your editor should let you do it relatively easily; if it doesn't, find a one that doesn't suck ;).

Plus how do you look what it does in google ? In a a documentation ? How do you generate documentation for those ?

Like with any other piece of API code. There's nothing special here.

And then what's the exact implementation behind it ? Does it do something funny ? So I have to mix the dozen of funny details of all the code from the guys who though the 20 times they do this super-important-operation they need to recreate a whole new syntax ?

Check the associated documentation, or source code if available.

I get the impression programmers nowadays are afraid of reading the source of stuff they use.

evincarofautumn3• 9 years ago

Programming languages characteristically haven’t adopted Unicode syntax because Unicode input methods aren’t “there yet”. But if we start accepting Unicode in programming languages, at least as an option with ASCII as a fallback, then input methods, searching, and so on will be forced to improve.

I currently use Emacs’ TeX input method, so I can type “\forall\alpha. \alpha \to \alpha” and get “∀α. α → α” or ”4.7k\Omega \pm 5%” for “4.7kΩ ± 5%”, which isn’t too bad. And in Haskell at least there’s Hoogle, which lets you search for unfamiliar operators, even Unicode ones. For instance, searching for “∈” gives me base-unicode-symbols:Data.Foldable.Unicode.∈ :: (Foldable t, Eq a) => a -> t a -> Bool and tells me it’s equal to the “elem” function which tests for membership in a container.

Using ASCII art instead of standard symbols introduces some cognitive load for beginners as well. When I was tutoring computer science in college, I had students get confused by many little things, like “->” instead of “→” (“Minus greater than? What could that possibly mean?”) or writing “=>” instead of “>=” because “≤” is written “<=”.

LunaSea• 9 years ago

Having read a lot of Python code that uses scientific libraries that override operators and key indexing I'll have to disagree.

The result is easier to read if you're familiar with it but exponentially more complicated if you're not and still learning. It creates marginally shorter code but I feel like the sacrifice of the explicitness / verbosity of a normal method / function call isn't worth it.

goatlover6• 9 years ago

Those libraries also make it easier to write that code. The majority of people using scientific libraries will be used to mathematical and scientific notation, so the closer the programming is to that, the better for them.

posterboy• 9 years ago

key indexing

sure, that should come with a huge warning sign in the documentation. Edit: and as code should be self documenting, that warning sign could be an extra function, sure.

agentgt2• 9 years ago

I used to be fervent embedded DSL fan many years ago (particularly in college with Scheme/Racket) but since then I have grown to hate them particularly Scala, Groovy and Ruby DSLs.

External DSLs (that is a DSL independent of the host language)... I love them. An external DSL is like a protocol or the ultimate formal API (an example would be HTTP).

Why do I dislike embedded DSLs: I guess the crux could be that you are essentially mixing two languages when the reality it is just one so you are doing non canonical things in the host to make the guest language look pretty (eg going to town with implicits, extension functions, and operator overloading (scala)). Basically a whole bunch of confusing stuff to normal developers just to make the DSL look pretty.

With Scheme/Lisp/Clojure it is OK because embedded DSLs using sexp look like lisp. In Haskell it is sort of OK because Monads and various other things Haskell provides (laziness).

With Scala its the absolute worse with operator overloading, implicits, and various other gotchas.

In my experience it also gets really confusing trying to figure what is going on when a embedded DSL breakk as well as extremely difficult to secure an embedded DSL. `ie System.exit(1)` is just one call away.

There are exceptions of course. DSLs that mimic external DSLs one-to-one. One example being jOOQ as it aims to be one to one with an external DSL ... SQL.

pluma• 9 years ago

Embedded DSLs often seem like the typical "bike shed coding styles" taken to the extreme. In my opinion they often value form over function, creating an entirely new (subset of the) language just because.

I can obsess about code formatting as much as anybody else but creating my own subset of the language just for a specific domain just seems excessive. It puts additional burden on anyone else trying to work with it because they now not only have to worry about the "host" language but also the idiosyncrasies of the embedded DSL.

swsieber• 9 years ago

Counter argument for statically enforced dsl (e.g. Kotlin):

I have an idea for a DSL - parse the templates for a site to generate type information about what buttons, inputs, interpolated values, etc can be found on each page. Combine that with parsed route information to allow one to generate dsl binding for a all the pages / routes.

If we're doing that with Kotlin, then afterwards we can get a really nice, statically enforced testing api (for Selenium, etc), with IDE completion. And you don't even have to run them to make sure that the element selectors work (barring of course any bugs in your parsing setup).

I had this idea several months ago but haven't had the opportunity to use it. I'm itching to because it seems like a feasible, if not slightly insane, way to making writing automated end to end tests easier to write and maintain.

tl;dr if you're going to do some code gen for code generation, why not make the interface a DSL?

codemac• 9 years ago

Would you say this is the battle between syntax & semantics?

If you are inventing new syntax to support semantics you don't fully understand along the way, it usually ends up as some franken language.

Lisp makes this a feature by limiting what most people end up doing into one syntax, and with things like macroexpand-1, you can see what "real code" is being generated.

agentgt2• 9 years ago

I failed to explain why with Lisp and Haskell it feels ok but I think you sort of nailed it with the "expansion" part. With Haskell and Lisp you think in expressions being expanded.

The DSLs in Ruby, Groovy, Scala (often but not always) are not macro expansion. Often times these DSLs are very mutable and are like a state machine (and in fact have a state machine underneath which causes a whole bunch of concurrency issues).

sametmax4• 9 years ago

+1. And you have to learn the all damn dsl wich is always badly documented and with zero tooling or support instead of just using the regular language API for witch you are equiped for.

westoncb• 9 years ago

I have a theory on why people maybe overvalue embedded DLSs: many problems become trivial when the terms of a language are a natural fit for the problem domain, where 'terms' are just any abstractions (new types, instantiations of old types).

So, maybe people are assuming that this benefit derives from having a full language which is a natural fit for a problem domain? (My contention being that the terms are what's really significant and the rest can be nice, but diminishing returns + tradeoffs.)

edit: fixed phrasing. I'll also add: I think certain sub-problems in an application can be specialized enough to need something really different (e.g. SQL, Prolog), but it seems like a relatively uncommon thing.

lmm4• 9 years ago

I find them very useful in Scala, because they make for a way to do "config" that you can refactor safely under the normal rules of the language. E.g. in akka-http/spray if there's a bunch of directives that I keep repeating in my HTTP routing config then I can just pull that out into a variable that's just a plain old Scala variable, and use that variable like a normal variable. Whereas if I want to pull out a repeated stanza from a routing config file I have to find out the rules for that config file and maybe it's not even possible.

The stuff Scala does can be confusing to some people, but I find a lot of it actually simplifies things. Certainly I never want to go back to a language where operators are treated as a special case differently from normal functions (at the same time there are certain Scala libraries I avoid because those libraries give their functions stupid names - but that's a problem that exists in any lanugage). Implicits make for a nice middle-ground if you use them the right way, to represent things that would have to be either painfully verbose or completely invisible in another language but really want to be almost-but-not-quite invisible. If you use them for something that should be more explicit, or for something that's not worth tracking at all, then that can become a problem (though again a problem you'd have in any language).

A lot of the stuff certain libraries do with "embedded DSLs" in Scala because it just shouldn't be a DSL at all - IMO there's no reason unit test code shouldn't just be plain old code. But the cases where you really do need a DSL more than make up for it. The wonderful thing I've found in Scala is that you never need an external config file or magic annotation or anything like that - everything is just Scala, and once you understand Scala (which, granted, probably takes a little longer than other languages) you never need to worry about understanding what some framework or external DSL is doing.

timv2• 9 years ago

I find them very useful in Scala, because they make for a way to do "config" that you can refactor safely under the normal rules of the language

If you see the choices as either DSL or external config file, then I can understand why DSLs are attractive.

But I found that Spray suffers because it tries to hard to be a DSL rather than just being Scala. When our team tried to do something that wasn't 100% obvious, or didn't fit with the documentation we ended up spending a lot of time trying to understand how the DSL worked under the covers so that we could work out what we needed to do, and then work out how to turn that back into the DSL.

It's been a while since I worked with it so (a) it might have changed, (b) I'm probably remembering details incorrectly, but I recall several conversations with team-mates who were just trying to do something simple like extracted a value from a request in a slightly non-standard way, where I would have to explain "no, this part is installing a spray directive, it runs at class initialisation, this is the part that runs at request time". Once you understand how spray is implemented, you can work those bits out, but the DSL doesn't make it clear at all, and if getting something to work requires that you understand the spray design, and you need to know Scala, then the DSL is just one more thing to learn, when a idiomatic standard scala API would be clearer all round.

My experience was that Spray (and Slick) fell right into the trap of: you need to know how the tool implemented to be productive in it, but once you know that the DSL is no longer adding enough value to justify its awkwardness.

moosingin3space4• 9 years ago

I used to like embedded DSLs a lot too, but then I realized they (mostly) were just the builder pattern + a lot of "extra" syntax (never written anything large enough in a Lisp or Haskell). The builder pattern feels more readable and has fewer context switches when writing code.

marcosdumay• 9 years ago

A "nice syntax to the builder pattern" describes every single applicative based DSL I've seen on Haskell. (I think this generalizes for all theoretically possible ones, but I need more coffee for being sure.)

But you get the advantage that applicative is a standard syntax, what you don't get on most builders.

goatlover6• 9 years ago

Pandas & Numpy are extremely useful DSLs in Python. Being able to add vectorization to the language and use structures like DataFrames in an R-like way is very powerful. There are some good use case scenarios.

Language features like operator overloading do exist for a reason, and that's because the language designer was aware of situations where you do want to use those features.

makmanalp2• 9 years ago

Do pandas and numpy count as DSLs? This always confuses me. I think it's just a library, with the same language and the same semantics, but different data structures.

syngrog66• 9 years ago

they are not DSLs. they are literally Python and they are APIs.

lasermike026• 9 years ago

I like ruby's rspec and Rake. Very powerful.

lostcolony7• 9 years ago

Out of curiosity (because this is what I was hoping from from the article), has anyone found a DSL in the wild that actually achieved its objectives?

That is, it either simplified the code necessary to write a program that there was a net savings for the project compared with developers just writing it in a language they were familiar with, or it simplified the business logic aspect to such a degree that even non-devs were able to be productive with it?

I admit this is partly determined by what we decide a DSL is, since it is, ultimately, at some level just an abstraction. But almost every instance I'd squarely say "that was an attempt at a DSL", it became just as complex as the underlying language, with additional gotchas, if it even 'worked' at all (i.e., allowed you to solve the domain specific problems it was intended to solve).

baddox• 9 years ago

I don't know if it would be considered a DSL, but the Rails router API is pretty great. Then there's Rspec, which I find atrocious.

pitaj• 9 years ago

JSX is similar, and I'd say it's a huge positive

agency• 9 years ago

This may partly be Stockholm syndrome, but I'm actually somewhat fond of Gradle (a Groovy DSL for configuring builds/managing dependencies for Java projects). It's certainly not perfect and I think it would be overkill for simple builds, but I find the DSL well suited to writing custom build tasks.

As an example, I wrote a Gradle plugin for a project I work on that lets you write:

  apply plugin: 'mycompany.service'
  
  mainClassName = 'mycompany.Main'
  service {
    id = 'serviceid'
    port = 1234
  }

This results in a `buildImage` task being added which builds a Docker image that runs `mycompany.Main` and exposes the specified port (and does some Docker tagging with the service id).

Implementing this plugin was straightforward, less than 50 lines of Groovy, most of which is invoking another DSL[1] for creating a Dockerfile.

[1] https://github.com/bmuschko/gradle-docker-plugin#creating-a-...

vorg2• 9 years ago

Gradle (a Groovy DSL

Gradle is not a Groovy DSL. Gradle enables you to write its build files in either Kotlin or Apache Groovy, and perhaps there'll be more choices later on.

Implementing this plugin was straightforward, less than 50 lines of Groovy

Did you know Gradleware now recommend using Kotlin for writing plugins for Gradle versions 3.0 and later?

TeMPOraL7• 9 years ago

Plenty of such project-specific microDSLs can be seen in Lisp world. They do a great job at reducing the boilerplate one would otherwise have to write, and they're relatively easy to inspect, if you need to know what's being done underneath.

lostcolony7• 9 years ago

Right, but that hinges on the definition of a DSL. It's not a complete language for solving a domain specific need; it's just a set of abstractions intended to reduce boilerplate. To me that's not that different than, say, a library in any programming language; it's a set of nouns and verbs to make certain problems easier, but it still requires the full knowledge of the underlying language to make useful.

zserge• 9 years ago

I can give Anvil as an example (https://github.com/zserge/anvil). It's a DSL for writing Android view layouts in Java and Kotlin. The DSL looks nicer than linear code of creating views and binding their properties, its look follows the hierarchy of the views, and the way it works results in a very fast rendering mechanism.

cle2• 9 years ago

One in particular that my team and I have used extensively is Spock, a unit testing Groovy DSL (see http://spockframework.org/spock/docs/1.0/index.html). In particular, we make heavy use of data tables and data pipes to generate sophisticated test data that would be a huge pain otherwise. And the accessible mocking syntax makes mocking so easy that devs end up writing more tests and testing more edge cases. The systems we have that use Spock the most have incredible test coverage and are very robust, because Spock makes it very easy and quick to test all your edge cases.

brabel2• 9 years ago

I second this. Spock is the best testing framework I've ever used. Same results on my team: much better test coverage just because writing tests is so much easier and fun.

zem3• 9 years ago

regular expressions work so well people have stopped thinking of them as a dsl and just see them as a language feature.

evincarofautumn3• 9 years ago

Character escapes and string formatting (printf, String.Format in .NET) fall into this category as well. They’re weird little domain-specific sublanguages that are entirely unlike their host languages, which people don’t even think of as DSLs because they’re so pervasive and (somewhat) consistent across languages.

lostcolony7• 9 years ago

Good example! And maybe that's what happens; extremely successful DSLs stop being thought of as DSLs, but as languages or features in their own right.

zem3• 9 years ago

racket's `match` is a very successful dsl in terms of simplifying code: https://docs.racket-lang.org/reference/match.html

mnarayan01• 9 years ago

it simplified the business logic aspect to such a degree that even non-devs were able to be productive with it

* bash (or any shell) -- Might seem odd to call it a DSL, but it certainly embodies the abbreviation.

* Matlab/R/etc.

* Excel (and even VB depending on usage) -- This even plays into "business logic"

You're probably more talking about something along the lines of Gherkin (Cucumber) though. While I've certainly seen it make devs unproductive, I don't know that I've even heard of it achieving the converse unless you loosen the criteria to meaninglessness.

lostcolony7• 9 years ago

Yeah, I actually was thinking Excel, and then was like "but that kind of depends on your definition of DSL", hence in part the question. A shell is an interesting thought of one. I don't know that I'd consider Matlab/R/etc given they're full on programming languages, just ones with a math-y slant.

tavert• 9 years ago

JuMP, in Julia: https://github.com/JuliaOpt/JuMP.jl/blob/master/examples/bas...

It's rapidly displacing proprietary DSL's like AMPL and GAMS in operations research. Neither of those happen to be embedded in a host language, they're special-purpose. Though there are comparable tools out there embedded in Python or Matlab, they don't perform as well as JuMP or the special-purpose tools.

cptn_brittish2• 9 years ago

Loop in Common Lisp is a good one.

flukus• 9 years ago

I wrote one once, a bastardized xpath that was used in a web crawler. I think it worked because it was extremely limited in what it could do and because normal code was a fallback in several places, it was never intended for non-developers. With it we could condense several hundred lines into < 10, with much lower error rates. Come to think of it, I'm not sure if the term "DSL" had been coined at the time, or I was unaware of it in any case, "creating a language" was not something I was attempting to do.

Since then I've encountered several in the wild that have been maintenance nightmares. Some were too simple that they were useless. Others were too complicated, so we ended up having to maintain a language and the code in that language.

There's one lying around here that was intended for support people to program in via an expression tree editor with a clunky UI. That was a disaster. We've got another one which is a terribly bastardized regex that was built because "support people don't understand regex". The don't understand this one any better, they would have been better off learning a life skill like regex.

lmm4• 9 years ago

Spray is the best approach to HTTP routing I've ever used, though I suppose you could argue it's just a normal library and you just write normal Scala. I'm not sure how you'd really draw a line between that and a "DSL" - to my mind any good library lets you write what you're doing in a simple, declarative, domain-appropriate way.

draegtun• 9 years ago

Rebol/Red's VID (Visual Interface Dialect) is a DSL which did achieve it's objective - http://rebol.com/docs/easy-vid.html

And another success is the Parse dialect which I use daily and comes baked-in with Rebol / Red - http://www.rebol.com/docs/core23/rebolcore-15.html

Rebol / Red is an excellent language for creating DSL's (Dialects) and there are plenty of examples in the wild (I've seen Rebol dialects for creating PDF's, Excel spreadsheets, Scheduling tasks, etc)

didibus2• 9 years ago

In CS, there are no good and bad things, there are only trade offs. DSLs are only good when the effort to learn to be proficient in the DSL repays the time it took you to learn it. Otherwise, the cognitive load of learning the semantics and the challenges often with debugging are not worth it.

An example of a great DSL is linq in C#. List comprehensions in Clojure. JQuery in JavaScript. Etc.

Osmium• 9 years ago

Unlike many other languages, it allows developers to create another language inside it.

Is this true? e.g. Swift has extensions on generic types too, doesn't it, and could also be used to write DSLs?

(Sincere question, asking because I genuinely don't know how rare this kind of functionality is)

robotpony• 9 years ago

DSLs can be great abstractions and interfaces, in that they can provide a way to state program knowledge in a form that non-programmers may be able to work with more easily.

One example that stuck with me early was the Quake (and Doom) shader files, as well as the other game resource constructs from those old WAD-based games. The shader syntax wasn't much more than a rough wrapper on the program's `struct`s, but it allowed the graphic artists to twiddle with the resources manually (and later made a great program interface for the level editors).

Quake map files were [pretty neat too](https://quakewiki.org/wiki/Quake_Map_Format), compiling down to playable levels as BSPs (binary space partitioning files).

XML (specific to document interchange), HTML (specific to hypertext interchange), and CSS (specific to DOM settings) were all DSLs that came about over the course of browser evolution. They offered ways to define content and configuration in a way that was both easy to understand and somewhat isolated from the underlying source code.

I've developed several DSLs in various systems I've worked in. Mostly you're pushing stuff out of the source code that doesn't belong there: configuration, repeated definitions, and things that may change from installation to installation. One of the abstractions we built was a series of custom state machine scripting languages that mapped to serial and TCP/IP protocols in a way that reduced boilerplate, and made the guts of implementing specific communications scenarios easier. The amount of time spent in developing a DSL was generally many times smaller than the gain in flexibility, transparency, and exposure to who could interact with customizing the system.

ruleabidinguser2• 9 years ago

The immutable data structures example was revolting

Fantastic DSLs and where to find them

Related Stories