Go channels, goroutines and GC available in Nim

by pwernersbach • 11 years ago

147 83 11 years ago

kodablah4• 11 years ago

I always wanted to see a Go/Nim interop project in a little different vein. Since Nim is a superset of Go (and Go is fairly simple), why not a Go implementation in Nim via cross-compilation? Then you can even build a Nim macro that does a "Go get" and Nim developers can use Golang libraries right in their code as imports. Not to mention we get a full, alternative Go impl.

It should be easy to bootstrap by first using Go's parser to dump some AST that Nim code can then translate, and then once that version is done, just reference the Go parser using this new impl. The only real struggle with the project from what I can see is deciding which standard library items to use the Go transpiled implementation (probably most of them) vs which need to have Nim backends (e.g. runtime package). Meh, just rambling thoughts in my head I was considering playing with given the time...

stefantalpalaru7• 11 years ago

why not a Go implementation in Nim via cross-compilation?

Nim's macros have one limitation that prevents an accurate implementation of another syntax: they can't fully modify the existing syntax.

See how I had to use "scase" inside "select" blocks because the existing "case" keyword insists on having "of" after it. So Nim's semantics put some limits on the amount of hijacking one can inflict on it through macros.

stcredzero3• 11 years ago

If you front compilation with Go's parser, you could generate bog-standard Nim code from the AST.

kodablah4• 11 years ago

I don't believe they should accept the full Golang syntax, just help with the import. E.g.

    macro goImport(path: string): stmt
      // TODO
      discard
    
    goImport("golang.org/x/crypto/nacl/box")

aikah• 11 years ago

Since Nim is a superset of Go

what ?

kodablah4• 11 years ago

I was under the impression this was true feature-set wise. I may be wrong though and am happy to be corrected.

duaneb2• 11 years ago

Since Nim is a superset of Go

I seriously doubt that Nim is out of the box binary compatible with another runtime's fundamental types.

kodablah4• 11 years ago

I am not looking for binary compatibility. I am looking for feature parity. Go interfaces can be handled with concepts [1] or just macros. I am talking about an alternative Go implementation (i.e. just sitting on top of Nim), not ABI compat between the two runtimes.

1 - http://nim-lang.org/docs/manual.html#generics-concepts

bmurphy1976• 11 years ago

I'm very curious to hear the pros and cons of this from somebody who has intimate knowledge of Nim internals. I really really like Go's CSP model and would love to see it properly supported in another lightweight non-jvm language (yes I know about Erlang, it doesn't fit the bill for me).

vbit• 11 years ago

Well you can already use Nim threads (http://nim-lang.org/docs/threads.html) and channels (http://nim-lang.org/docs/channels.html) - the model is similar although the implementation uses system threads rather than coroutines.

Nim also has lightweight coroutines using `async` and `await` (http://nim-lang.org/docs/asyncdispatch.html) - you can run a bunch of these within one thread.

Also, have a look at gevent for Python.

dom96• 11 years ago

In addition to the heavyweight threads (http://nim-lang.org/docs/threads.html) there is also `spawn` (http://nim-lang.org/docs/manual.html#parallel-spawn-spawn-st..., http://nim-lang.org/docs/threadpool.html)

rspeer• 11 years ago

Don't look at gevent for Python, look at a responsible async library: asyncio if you like Py3, Twisted if you like Py2 and want something that's very featureful but old-school, Trollius if you wish you could be using asyncio but you're stuck on Py2.

gevent is too much magic. It aims to give you async without changing any of your code. To accomplish this, it monkey-patches the entire Python standard library, in a way that is 99% compatible with Python, but the 1% will constantly surprise and infuriate you. Its compatibility shows no signs of increasing given how much its development has slowed down.

You can use gevent as a quick hack, but you will hate yourself if you have to maintain gevent code.

digitalzombie• 11 years ago

Ada have a CSP model too, just fyi.

It's a mature language. You can look into that?

Manishearth• 11 years ago

This is awesome! Go's channels and green threads were one of the features I really liked about Go. One more reason for me to go past basic prime number programs in Nim :)

I'm hoping to see Rust get green threads/tasks/goroutines too. I'm working on a GC myself, and hopefully someone is trying out a green thread scheduler.

stcredzero3• 11 years ago

This document describes how the GC works and how to tune it for (soft) realtime systems.

The basic algorithm is Deferred Reference Counting with cycle detection.

I'm sitting here feeling very impressed. Deferred reference counting has very good semantics for games. Even better, you can control the cycle detection part separately and run that part at an advantageous time. (Though it's probably better to just let GC do its thing, unless you really know what you're doing.)

I am currently writing a multiplayer game server in golang, by making sure almost everything is allocated on the stack, and heap sizes are small. This gives me an efficient, nearly pauseless server. However, something like Nim could give me even more flexibility.

pcwalton9• 11 years ago

I'm sitting here feeling very impressed. Deferred reference counting has very good semantics for games.

Not if you need to be thread-safe.

Even better, you can control the cycle detection part separately and run that part at an advantageous time.

How would that work with multithreading? (Assuming you had a thread-safe GC, which Nim's isn't.)

Pharohbot14• 11 years ago

A more general approach to shared memory via lockable heaps is a feature Nim seems will implement soon.

fjehcjriendje2• 11 years ago

So, I've noticed that over the past six months that every time something about Nim gets posted to HN, you make an effort to discredit the language.

Care to offer an explanation why?

vvanders2• 11 years ago

If you're crossing threads in games stuff you're doing it wrong. There's a good chance you're running into false-sharing and other pitfalls.

Most games use worker-queues(in which case you can use non-GC objects) to deal with architectures like the CELL and for better cache coherency. In that case Nim is a pretty good fir.

stcredzero3• 11 years ago

Not if you need to be thread-safe.

Everything that has to be thread-safe uses channels. I use channels to sanitize everything for a purely synchronous game loop. As an optimization, in cases where there are atomic operations available, there are places where concurrent code can mutate values visible to the game loop, but this is strictly an optimization technique, to be used judiciously. (So only values like "speed" can use this technique. Anything that's a reference is verboten.)

How would that work with multithreading? (Assuming you had a thread-safe GC, which Nim's isn't.)

I didn't realize Nim's GC wasn't thread safe. In my current architecture, you'd only have to worry about the part using channels to sanitize things for the synchronous game loop. If everything outside of the game loop was written such that most everything was allocated on the stack, the GC would never have to collect anything outside of the game loop. So maybe it could work as a port. I couldn't say for sure, though.

jeremyjh4• 11 years ago

I thought the Go runtime runs foreign code on M:M threads; e.g. when Go time calls foreign code, it dedicates a thread to it. This is so foreign libraries (which are unaware of yielding to the Go scheduler when they do I/O) don't block a thread with multiple goroutines scheduled on it. I do not think Nim code can run "in a goroutine".

stefantalpalaru7• 11 years ago

I do not think Nim code can run "in a goroutine".

It does. Don't forget that this is gccgo so it is possible to use plain C functions as goroutines. Nim is translated to C and with the help of a macro I convert Nim functions with an arbitrary number of arguments into ones with a single void* arg that gccgo wants for its 'go' keyword implementation:

    extern void* __go_go(void (*f)(void *), void *);

jeremyjh4• 11 years ago

Yes, but when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? If it doesn't, then nothing prevents foreign code from blocking other goroutines. How does the Nim code yield the thread for other goroutines, does it have to register a callback?

Go channels, goroutines and GC available in Nim

Related Stories