Language Oriented Programming: The Next Programming Paradigm

http://www.onboard.jetbrains.com/is1/articles/04/10/lop/index.html

by Hexstream • 17 years ago

17 23 17 years ago

onboard.jetbrains.com

Hexstream5• 17 years ago

Here's an article from 2003 that I think is still very relevant inspiration for what's to come. I think we're "getting there" and I plan to be part of this revolution ;)

I especially liked page 3. It's hitting nail after nail right on the head.

edit: On second thought I'm not sure we're getting there because I'm not aware of any IDE that comes close to approaching what I have in mind (which is pretty close to what the article is describing). I guess I just have those concepts in mind so much that I think everyone else sees what I'm seeing. Anyway.

Hexstream5• 17 years ago

Erm, I mean from 2004. October 2004.

yters6• 17 years ago

I thought pg was thinking this way with arc. I remember reading something where he was planning to add some kind of syntax definition capability to arc, but haven't been able to find the article since.

yters6• 17 years ago

I'm trying to think of an easy way this idea can be hacked together with existing tools, instead of writing a bunch of new stuff.

Could this be handled by macro programming on a lisp machine? Lisp is essentially the parse tree, which all languages have, so it provides the syntax agnostic representation. Macros allow you to easily specify DSLs, plus the macros themselves are lists and thus manipulable by macros. The lisp machine is essentially emacs that really is an OS, so the entire programming environment is programmable. But, emacs on top of some shell isn't so bad either.

The only thing missing is the syntax layer. Maybe regexps could suffice? Also something like LaTeX can provide a rich variety of symbols and symbolic structures for syntax.

GeoJawDguJin2• 17 years ago

> Could this be handled by macro programming on a lisp machine?

Sure!

Now all we need is a lisp machine that anyone can run on modern hardware. Any ideas?

Hexstream5• 17 years ago

I'm not sure if you're joking. It has always been the case that you could run Lisp on stock hardware. Lisp machines were just an optimization for back in the day when hardware was slow and Lisp compilation techniques weren't well-known.

yters6• 17 years ago

This guy's trying to write one in Movitz:

http://www.emmett.ca/~sabetts/

Hexstream5• 17 years ago

I'm pretty knowledgeable about Lisp so if these ideas could be implemented just by using some existing tools together I'd probably know. This article goes much further than that.

As high-level as Lisp is, it's still a general-purpose, turing-complete language and as such there's always going to be a DSL that's better suited than it to solve a particular domain-specific problem.

While it's true that you can embed DSL's in Lisp easily with macros, that doesn't address the editing issue at all. Current IDE's are not so easily extensible to support the high-level semantics of new DSL's, so even if your DSL is 20x more expressive than vanilla Lisp in your domain, you lose a lot of that productivity to the lack of tool support for it which we've grown accustomed to having with general-purpose languages (syntax highlighting, dependencies, reverse-dependencies, parameter hinting, refactoring, etc).

yters6• 17 years ago

Do you know if anyone has tried abstracting out the elements of a good IDE? Your list looks like it could be implemented with a table structure. All languages have similar dependency structures (as far as I know), so the IDE would just need to know the right syntax elements. I'm probably way over simplifying though. I'm not broadly experienced in many languages.

silentbicycle2• 17 years ago

Regexps alone are a really poor fit for dealing with recursive structures, such as finding matching pairs of braces. You can use them for a lexing step, but you're probably better off using something like yacc or LPEG for the parsing.

Forth, Factor, and other stack-based languages also have this same property of "syntax agnostic representation". While on the surface they're kind of a backwards Lisp (due to postfix notation, with intermediate results pushed on a stack), both are dealing with nested lists of symbols, and very adept at dealing with things expressed the same way.

yters6• 17 years ago

That's true about the nested structures. I was wondering if it is worthwhile to make lexing and parsing first class and changeable at runtime, and whether regexps would be a good way of doing this. The problem with relying on an external program is that making syntax conversion first class gets much more convoluted.

But, it isn't clear to me whether it is a good idea to make everything about LOP first class. I do like that feature of Lisp though.

mst• 17 years ago

Or, possibly, perl6 rules. One of the things I keep meaning to have a go at is taking Larry's gimme5 stuff (a slow but working implementation of much of the rules system in perl5 he's using to bootstrap the grammar) and bolting it back into perl5 so we can experiment with it more easily.

kragen2• 17 years ago

Forth is not adept at dealing with nested lists of anything. It doesn't have linked lists, garbage collection, or traditionally even heap allocation. It does have recursion, but it doesn't traditionally have local variables, and mutual recursion (really handy when you're doing anything related to ASTs) requires using DEFER or something. Do you have a lot of experience writing Forth?

dusklight• 17 years ago

this is "new" only if you don't know ruby or lisp.

Hexstream5• 17 years ago

Wrong. The lack of tool support for DSLs is a real issue. We've barely scratched the surface of what IDEs can do. We often have to deal with lackluster editing support (especially so in the case of DSLs) and I find this deplorable.

As an example, if a DSL's function C references a lisp function F, the IDE should integrate the two languages enough that when you ask who calls F the IDE will list C among the calling functions.

edit: Suddenly I have a feeling people will have to see the power of a "real" IDE for themselves before they see that all the seemingly small improvements are really worthwhile. It's a bit like explaining Lisp macros to a beginner: the simple examples are easy enough to understand but aren't very compelling, while the complex, more worthwhile examples are too difficult to understand. So let's talk again in a few years when I have an implementation ;P

johngunderman• 17 years ago

It sounds like all he needs is a VM, such as the JVM.

For example, the JVM gives us:

Java, Clojure, Jython, JRuby, etc.

All of these languages are compatible with each other and can easily extend each other. Especially with Clojure, it would be trivial to implement another DSL on top of these, but the platform has always been there.

snprbob86• 17 years ago

This only solves part of the problem.

If you have ever used JetBrain's IntelliJ IDEA, you will understand why standard text editors are grossly inefficient. IntelliJ is a Java IDE, but imagine for a moment that it was an "all JVM powered languages" IDE. It's a very interesting vision, and I for one am confident that JetBrains can produce something compelling. Their products are top notch.

gfodor• 17 years ago

After several years of development, Jetbrains released a beta of the Meta-programming System for implementing this vision:

http://www.jetbrains.com/mps/index.html

jerf• 17 years ago

I think the problem with claiming this is the "next big paradigm" is that the average programmer, and frankly, even the merely above-average programmer, is not going to be competent to design a non-trivial special-purpose language of any value.

Once you step out of the domain of totally-trivial languages, language design immediately becomes tricky, subtle, and prone to exotic interactions and quirky tradeoffs that even our absolute best teams of language designers can only mitigate, not eliminate... and it's not all going to be "absolute best" teams, after all.

And... then you want multiple languages to be sitting there interacting, too? That's not even possible without somehow limiting those interactions ("thy languages shalt have lexical binding that works thusly"), and now you're just making another meta-language like Lisp, which, presumably, doesn't fit the bill. Either you cut off the diversity of the language or you get the sort of evil interactions the likes of which have never been seen in a real language. The only middle ground there is to do both at once.

If your paradigm requires a genius to use it, that's not a fair comparison against other paradigms; start a genius out on OO and they can get pretty far there, too.

Language Oriented Programming: The Next Programming Paradigm

Related Stories