Lispy - Code-as-data in Ruby, without the metaprogramming madness.

by chrislloyd • 15 years ago

90 38 15 years ago

bguthrie2• 15 years ago

I really, really like this idea. It's difficult enough to employ Ruby's metaprogramming constructs without making a mess, so the idea of instantly decomposing into nested lists is really appealing.

I realize the library is only, like, three days old, but questions/feedback:

- Do the DSL methods get turned into into proper Ruby methods eventually? I'd hate to see this whole thing built on the back of method_missing.

- Picking apart nested arrays can lead to its own kind of spaghetti code. When people build DSLs, I recommend they immediately turn their high-level method calls into nicely testable objects. Can we see an example of that here?

- The author specifically calls out libraries like ActiveRecord, which would suggest that the primary use case for this library is for integration as a DSL layer into your own libraries, but he doesn't provide any examples of that usage--just Lispy.new. I'd like a way to extend Lispy into my own class or module and get that same functionality.

ryan-allen4• 15 years ago

G'day,

Thanks for the feedback! I was hoping to do a little more work on it before it being posted to somewhere like HN, but oh well!

A lot of the work I've done with this kind of Ruby, it often only is run once, so I tend to use method_missing. The reason why you'd want it converted to methods is for performance, right?

I agree with your testing comments, there's a lot left out so far, specifically a guide on how you might use such a library, I'm working on that.

And yes, you're right about AR being not the best example, it's use is for what you've described. Integration with your own code would be an ideal use. Implementing it as a module would be trivial.

pi18n• 15 years ago

Is there a specific reason you chose to do arrays of arrays instead of hashes for subelements? I'd want the lists to come out as (maybe using a pluralizer or maybe not)

:items => [{:priority => :normal, :desc => '...'}, ...]

Then you could do more intuitive things like

todo_list.items.sort{|a,b| a[:priority] <=> b[:priority] }

which is somewhat stupid for symbols but not for numbers.

bguthrie2• 15 years ago

Performance and improved introspection. Once they're first-class they get included in that class's list of methods, which is helpful sometimes. Mostly I just like to avoid m_m when I can.

AR is actually an okay example - it's as good an illustration as any of a place that employs these kinds of techniques, and the source of a lot of complexity in Rails.

sleight42• 15 years ago

The man has a point. We should consider using sexps more often when developing internal DSLs.

That's not to say (stupid iPad) that we dont still need define_method and instance_eval; however, decoupling the language definition/API from the language impl is good sauce.

raganwald2• 15 years ago

I like it, and I speak as someone who is slowly recovering from a year in a sanatorium gibbering about how I killed Mozart by using a library to decompile Ruby into sexps, then rearranged the sexps before reconstructing Ruby code:

https://github.com/raganwald/rewrite_rails

I'd still be in a padded cell if I hadn't run into a doctor who eschewed the more complex forms of therapy and simply told me to stop doing that.

extension2• 15 years ago

I called this a Domain Agnostic Language:

http://code.extension.ws/post/169602795/dal-rb

I don't actually like the approach though. Ruby makes DSLs as easy as "normal" interfaces. Why not take advantage of that? Generating and parsing an AST is exactly the kind of extra complexity that Ruby lets you avoid.

raganwald2• 15 years ago

Here's a use case: Imagine that you want a form validation library, something like active record's domain-specific language, but it is not attached to models, just to forms being submitted. But anyhow, you write your DSL. Now you have two options:

1. Execute it directly in the controller of your Ruby server 2. Convert it to sexps, export those as JSON, and write a Javascript validator that validates forms on the client-side.

extension2• 15 years ago

That is indeed a nifty use case, but also kind of a special case in which a serialized AST is the required output of your program. For the general case where the DSL is just a sexy API, I wouldn't recommend building a parse tree.

EDIT: Actually, for your case I don't think I would use sexps. I would write some primitive validators in Ruby that can also generate equivalent JavaScript code, then use those to build higher order validators in Ruby alone. Compare that to maintaining complete validator engines in two different languages.

regularfry• 15 years ago

One advantage of this approach is that it lets you not care about the order of definitions if you don't want to. Doing things the usual way, ruby's execution model can have you tied in knots if you're not careful.

jamesbritt2• 15 years ago

How does this compare to the 'sexp' library?

http://www.artima.com/rubycs/articles/patterns_sexp_dslsP.ht...

JonnieCache• 15 years ago

Ruby metaprogramming hell (it is a real place).

But meta-sin is so much fun!

Nice lib with a lot of potential. It will definitely come in handy and may well prompt me to take some unfinished projects off the back burner and get them in the oven as it were.

Also, I enjoy your README writing style. It walks the fine line between amusing irreverence and technical specificity very well.

ryan-allen4• 15 years ago

I'm glad you enjoyed the README, I had fun writing it (aided by some Jameson).

jamesbritt2• 15 years ago

Also, I enjoy your README writing style. It walks the fine line between amusing irreverence and technical specificity very well.

I found the README distracting and hard to follow. I'm still unclear just what the library does or why one might need it, possibly because of an over-emphasis on jokes over technical detail and sufficient examples.

However, there seems to be an under-current of "Ruby is hard to use correctly, so use this."

Do people really find Ruby meta-programming so hard?

bitwize• 15 years ago

Ruby metaprogramming hell is a myth used to frighten school children into doing the dishes.

C++ metaprogramming hell -- ohhhh, that's real.

jcapote• 15 years ago

1000 internets to this man. This is excellent!

zwp2• 15 years ago

From the Examples:

    fart => [ :fart, [] ]
    fart 1 => [ :fart, 1 ]
    fart 1, 2 => [ :fart, [ 1, 2 ] ]

Why isn't the single argument call:

    fart 1 => [ :fart, [ 1 ] ]

ps. this is also fun: http://parsetree.rubyforge.org/ruby_parser/

dustmop• 15 years ago

That's funny, I was expecting the opposite behavior:

    fart => [ :fart ]
    fart 1 => [ :fart, 1 ]
    fart 1, 2 => [ :fart, 1, 2 ]

Seems more lispy to me, and from a quick glance I can't see where it would break.

zwp2• 15 years ago

Yes, I prefer this, definitely more lispy.

I blame seeing too much of clojure's "defn fart [ args ]" syntax ;-)

KirinDave• 15 years ago

I'm a bit confused what this has to do with lisp. Seems more like a classic Smalltalk building pattern. If it's a reference to "code as data" from the Lisp tradition (which is a friendlier colloquialism for "homoiconicity"), then they bear only a tenuous relationship.

Which is not to say that this library wouldn't be useful. But it is to say that it doesn't really bring any of the power of homoiconicity or consequential techniques thereof to Ruby.

moron4hire• 15 years ago

"Code-as-data in Ruby, without the metaprogramming madness. — Read more http://www.youtube.com/watch?v=aD4bn5pp32w

link is a rickroll. What was the point of that? I don't think we need to be super serious about everything, but at least don't be anti-productive. I was hoping to have a link to a talk or screen-cast demonstrating what Lispy was about. Instead, backwards rickroll.

ryan-allen4• 15 years ago

I'm sorry! I've updated the link to the RubyGem. I'm also working on a tutorial for intended usage, I'll post it when it's done.

bluedanieru• 15 years ago

It seems this hit the public consciousness a little earlier than the author expected is all. That might have just been a placeholder.

mardigan• 15 years ago

There was a recent ruby-talk announcement for live parse trees being available in 1.9 now. Presumably gives the functionality of ParseTree. http://quix.github.com/live_ast

helium• 15 years ago

This is very close to something I built earlier this year...just nicer.

https://github.com/michael-erasmus/Flippant

pavelludiq2• 15 years ago

OT: I'm very tempted to be an asshole smug lisp weenie, and make some Greenspun's rule joke, but instead I'm going to try to be helpful. Ruby hackers pride themselves for being early adopters of new and cool technology, learning new skills, and libraries. What I've failed to notice is any interest in the old technologies, from which ruby got most of its good ideas(lisp, smalltalk). There is tremendous value in learning from the past(a lot of really good ideas didn't make it to the "cool kids languages" of today). This isn't specific to ruby, but its a good example. So my attempt at useful advice is: do learn lisp, instead of reinventing the wheel badly.

Anyway, I've been thinking about this kind of stuff, and it seemed like a good opportunity to mention it.

erikpukinskis4• 15 years ago

learn lisp, instead of reinventing the wheel badly

When someone says something like this, my typical response is "That's true, I should dig more into Lisp. Oh, look! A bug in my software I need to fix. * forgets Lisp entirely *"

If you were to say something like "the difference between Lispy and Lisp is that in Lisp you can do X", then my likelihood of actually going out and learning about X, pulling out the Lisp interpreter and playing around goes up about 1000x.

pavelludiq2• 15 years ago

Yes, finding the time and motivation to educate ourselves is sometimes hard, especially when we have to deal with other crap, I've been meaning to look into smalltalk for ages now, myself.

If you want an example of something cool you can do with lisp, in principal the common lisp object system is built with macros on top of the non-object oriented lisp. Could i use Lispy to add a CLOS style object system on top of ruby? What about a prototype based one? This isn't a rhetorical question, I'm actually curious if ruby can do these kinds of things.

spacemanaki• 15 years ago

I was so impressed by pavelludiq's comment (I had the impulse to post a comment citing Greenspun but refrained, their comment was so much better than what I would have posted) and I tried to come up with a tiny example of a defmacro which would show where Lispy is reinventing the wheel, but really, why bother when much better writers and teachers have already. I think Peter Seibel's Practical Common Lisp is a great intro and this chapter is so early in the book it nearly stands on its own:

http://www.gigamonkeys.com/book/practical-a-simple-database....

There's a couple of simple macros in there and it's not that long a read.

Then there's Casting SPELs in Lisp which is another great concise intro to macros that shows them off:

http://www.lisperati.com/casting.html

I don't think I could do better than that. Both of those are short enough that you can read them in an evening. If you're not hooked by then, at least you haven't spent much time on it.

Confusion• 15 years ago

It would be helpful if you would point out how you feel that Lispy is 'reinventing a wheel badly'.

Also, if a fullblown lisp is what you want, then I wonder whether the correct alternative isn't 'embed a lisp interpreter'. If Lispy is not intended to be as powerful as a fullblown Lisp, then perhaps it just suffices?

samps• 15 years ago

This looks pretty cool, but as a non-Rubyist, I remain confused about the purpose of all these "DSL" libraries that lispy aims to simplify.

In the examples, at least, the language was only used to declaratively build hierarchical list/dictionary structures. The only constructs I can see used are blocks and method invocations; there aren't any loops or conditionals or anything else imperative. Can someone more experienced with Ruby DSLs explain what lispy accomplishes that something like YAML or a simple custom parser doesn't? (Other than a bunch of extra "do"s and ":"s.)

erikpukinskis4• 15 years ago

There aren't any loops or conditionals in the example, but since it's using blocks, there could be arbitrary code. But yeah, if you're not doing any of that you might as well just use YAML, or a hash.

kleiba• 15 years ago

Saying that, I've written about 10 lines of LISP in my life. Pete keeps telling me to go read SICP and I keep meaning to but then I end up at the local bar listening to some rock band and I'm like "crap, it's 3am".

Man, you're such a rebel!!

soamv• 15 years ago

I see that lispy is using instance_exec -- which means that the scope of code inside that code is the lispy instance, which might not be what the programmer expects. For example if he does this inside a method he'd find his instance variables missing. (Is that right, or am I misreading the code?)

You could use some sort of global or thread local variable to maintain the context, and then you wouldn't lose the instance scope, but then this code will not mix well with yield (and continuations and fibers etc.)

So then it looks like the only way to solve this problem really neatly is in a language that has macros.

(I made some attempts at this problem too, at https://github.com/soam/blox . It is slightly more complex than lispy; see for example https://github.com/soam/blox/tree/master/samples/webservice .)

Lispy - Code-as-data in Ruby, without the metaprogramming madness.

Related Stories