Goblins: A transactional, distributed actor model environment

by snicker7 • 5 years ago

134 30 5 years ago

nerdponx• 5 years ago

Cool! I hope one day Racket will take off as a serious language for app development.

I know it's not really the point of Racket, but it has all the right attributes^, all it needs is a bigger library ecosystem.

^: including a package manager & build toolchain, an IDE, nicely-formatted web hosting for docs, runtime is reasonably fast, has plenty of batteries included, has cool stuff like contracts, supports multiple programming paradigms.

tartoran3• 5 years ago

That would be great but even if doesn’t get there, Racket is still an amazing ecosystem for exploring new ideas and will continue to love it. But of course any improvements are welcome

mark_l_watson• 5 years ago

I agree. Racketfest Amateur Night happened last Friday and Saturday and many cool projects were show cased.

giancarlostoro• 5 years ago

Kind of weird hearing that when we're posting on a website running atop Racket.

yen2232• 5 years ago

Hacker News isn't Racket, it's Arc

TeMPOraL2• 5 years ago

This website also runs on a single server[0], stores user state in closures[1] and uses tables for layout[2], in violation of just about every single modern web development practice.

And it's all the better for it. I wish more sites would follow HN's example.

[0] https://news.ycombinator.com/item?id=16076041

[1] - Mostly, at least - https://news.ycombinator.com/item?id=23374471. It used to be almost for every action, the pg's OG hack, but got optimized away.

[2] - Press CTRL+U.

Jeff_Brown• 5 years ago

Looks fun. How novel is it? Can I do these things already in some other language? (Esp. Haskell or Python -- my favorite language and my work language, respectively.)

paroneayea3• 5 years ago

If the version of CapTP we have is ported to other languages, then you could do some of it, but probably not all. Haskell might be better up for some, but Python won't be able to do some of the more interesting parts (like the time-travel support) quite as easily since it's a much more mutation-oriented environment. Still, you could get systems talking to each other over the network if a compatible CapTP was implemented.

zenhack• 5 years ago

The networked actor-model bit & CapTP goes back to E originally[1]. The other contemporary real-world protocol based on this design is capnproto rpc[2], which has implementations in several languages including both Haskell (of which I am the author) and Python.

The ergonomics of the Haskell implementation could use some work IMO, and I've got some ongoing refactoring work on a branch. But it does work, and folks are using it.

[1] http://erights.org/

[2] https://capnproto.org/

paroneayea3• 5 years ago

Oh hey neat, some coverage. Actually I'm right on the verge of the next big release (v0.8) which should make doing the networked programming significantly easier.

But here's some more background: I'm co-author/co-editor of the ActivityPub specification, which might give you some idea that I have some experience with trying to build networked systems. Goblins is part of the Spritely project, or even more accurately, the foundation for it: https://spritelyproject.org/

Spritely has some pretty wild ambitions... I won't go into them in detail, the video near the top of the above site explains better. But achieving those in a feasible timeframe means we need some better tooling. Goblins is the foundation of that; it's implemented as a Racket library, but can/will be ported elsewhere. The docs linked above explain some of the core ideas, but honestly I think the video linked from this section of the site explains it better: https://spritelyproject.org/#goblins

In general, people tend to realize that something interesting is happening when they see a cool shiny demo. So here are two pages that might give you an idea with some cool shiny demos:

- Demonstrating "time travel" in a space shooter game (no special code added for this other than exposing it in the GUI): https://dustycloud.org/blog/goblins-time-travel-micropreview... - "goblin-chat", which doesn't look quite as shiny but is interesting more that the code you see above is "end to end encrypted", but is written in a mere 250 lines of code for the whole protocol http://dustycloud.org/blog/spritely-goblins-v0.7-released/

What? How can the last one be only 250 lines of code? (And a mere 300 more for the GUI!) Well, that's because we're implementing a peer-to-peer distributed object programming protocol called CapTP (which includes wild things like distributed garbage collection over the network (for acyclic references), "promise pipelining", and is object capability secure). The next release will be advancing that work significantly. The Agoric organization is also implementing CapTP, but their stuff is in Javascript instead of Racket; we plan to have our CapTPs to converge. Thus it shouldn't really matter whether you write your code in Javascript or Racket, your code should be able to talk to each other.

Anyway, new release coming out soon. Hope that answers some things since the docs page might not convey what's fully interesting about it (and indeed, neither does this text, but the videos mentioned above do a better job).

carterschonwald• 5 years ago

This looks very cool! And some of the ideas look very interesting!

mathgladiator• 5 years ago

That's neat; the ability to move state backwards is a powerful tool. I'm building my own language around reactivity and state management ( http://www.adama-lang.org/ ), and I'm getting to the point where I could demo replaying a board game backwards.

ssivark• 5 years ago

Thanks for setting me off on a wonderful trail tonight! After a couple of hours of reading, I finally appreciate how useful, simple and coherent the capabilities+actors part of the story is! I particularly enjoyed the analogy/application [1] with federated social networking -- Seeing how easily they are modeled by ocaps actually makes capabilities much easier to understand! :-)

[1] https://gitlab.com/spritely/ocappub/blob/master/README.org

[2] This talk also has a nice & simple technical perspective: https://www.youtube.com/watch?v=2H-Azm8tM24

tartoran3• 5 years ago

What are the best use cases for distributed actor model solutions? Im just slightly familiar with it and it’s on my learning list. Anybody here have any experience with it? How about with Goblins?

unoti• 5 years ago

The use case for actors is any time you want to distribute work across multiple nodes. In general, actors are a great concurrency model that simplifies a lot of the complexity of distributed programming.

For more info on "why", see:

https://doc.akka.io/docs/akka/current/typed/guide/actors-int...

https://www.microsoft.com/en-us/research/wp-content/uploads/...

tartoran3• 5 years ago

Thanks, the first link answers some of my questions.

Use of actors allows us to:
Enforce encapsulation without resorting to locks.
Use the model of cooperative entities reacting to signals, changing state, and sending signals to each other to drive the whole application forward.
Stop worrying about an executing mechanism which is a mismatch to our world view.

Did anyone here use Goblins for any project and would like to share their experience?

blandflakes• 5 years ago

I don't want to distract from this comment, because I think it's spot on, but I think another interesting use for actors is as one of the few sane concurrency models for mutation. Pony does this - objects are actors by default. Things can act concurrently without fear external to an object, but all mutations within a given object are serialized. Seems like a nice set of defaults.

mhoad• 5 years ago

I was recently trying to answer this question while playing around with Dapr (amazing project by the way).

So despite it being a solution that has existed for distributed systems since the 70s I was watching a talk from Microsoft on the topic where they would refer to actors as “cloud native objects” and the model itself as an ideal way to do OOP on distributed systems. The marketing feels a little cringe at first but the mental model works well.

I have an object which contains some state. Normally I would be responsible for managing the entire lifecycle of this object including resolving race conditions for DB writes and a million other problems that make distributed systems difficult. With many modern implementations of the actor model however (Microsoft have done a lot of interesting stuff here building Azure and Xbox games like Halo using Actors heavily) you don’t have to worry about any of this. You can just call the object as though you had unlimited RAM and it was in memory for you and race conditions were no longer a thing that you had to worry about.

This paper is a good intro to the topic I found https://www.microsoft.com/en-us/research/wp-content/uploads/...

reubenbond• 5 years ago

I was watching a talk from Microsoft on the topic where they would refer to actors as “cloud native objects” and the model itself as an ideal way to do OOP on distributed systems. The marketing feels a little cringe at first but the mental model works well.

Was that my Orleans Deep Dive video? https://youtu.be/R0ODfwU6MzQ

I started referring to Grains as Cloud Native Objects as a way to better convey them to people who aren't familiar already. My view is that Grains are different enough from Erlang/Akka style actors that lumping them under the Actor terminology can cause confusion. In the research publications, the term Virtual Actor is coined, which better describes them.

That video doesn't discuss it, but Orleans also supports distributed cross-actor ACID transactions, largely thanks to Phil Bernstein: https://www.microsoft.com/en-us/research/publication/transac... and more recently https://www.microsoft.com/en-us/research/publication/resurre....

sriram_malhar2• 5 years ago

Can someone familiar with the framework answer a few questions?

Do transactions span distributed actors? Perhaps I missed it, but I see no recovery log.

It implements distributed garbage collection. This is a non-starter for any environment where the network cannot be taken for granted. RMI/DCOM/CORBA had no shortage of problems due to distributed GC alone.

evgen• 5 years ago

If you want to understand the tech more you should start with the E language it is based on at erights.org. In the case of distributed GC the system understands that the network can fail and plans for this: http://www.erights.org/history/original-e/dgc/index.html (caveat: I know original E and have no direct knowledge of whether or not this bit has been implemented in Goblins)

sriram_malhar2• 5 years ago

I am quite familiar with E. Or used to be during my PhD. As far I recall, neither E's implementation nor spec cover distributed transactions, fault-tolerance or GC issues.

TeMPOraL2• 5 years ago

I've just discovered someone built "a completely peer to peer, end to end encrypted chat system" working over Tor[0][1] using Goblins. Not sure how significant this is, but I found it interesting.

[0] https://gitlab.com/spritely/goblin-chat

[1] https://octodon.social/@cwebber/105992747855007110

rapnie• 5 years ago

someone

It is from the researcher / developer of Goblins himself, Christopher Lemmer Webber. Very interesting indeed.

Goblins: A transactional, distributed actor model environment

Related Stories