State of Loom

https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html

by mxschumacher • 6 years ago

202 132 6 years ago

cr.openjdk.java.net

A virtual thread is a Thread — in code, at runtime, in the debugger and in the profiler.

To explain: if you debug async code on Kotlin right now, you cannot single-step in a debugger, because the debugger attaches to a thread. But coroutines get scheduled on different threads all the time!

I wonder what this means for ThreadLocals and Locks? Will they work as before? If so, that is huge! Because locks don't work with async code either, for obvious reasons. The consequence is that the whole ecosystem is split in two parts! You cannot just use a Guava cache in Kotlin coroutine-based code.

If Loom manages to avoid all of this, this is fantastic news.

pron17• 6 years ago

I wonder what this means for ThreadLocals and Locks? Will they work as before?

Yes. You can try that today: http://jdk.java.net/loom

MrBuddyCasino3• 6 years ago

This is outstanding. They solved the „what color is your function“ problem! The Rust ecosystem has a completely separate std-lib just for async. I think there is one other language that managed to avoid that problem too by using compile-time magic [0].

[0] https://github.com/ziglang/zig/issues/1778

kasperni3• 6 years ago

The consequence is that the whole ecosystem is split in two parts!

Which it might continue to be. As Kotlin needs to support other backends than just the JVM.

sgt2• 6 years ago

Why does it absolutely need to? I thought the purpose of Kotlin was to primarily be a JVM language.

GordonS2• 6 years ago

Hmm, the most popular C# IDEs (Visual Studio and Rider) don't have this problem - you can single step when debugging async code.

Is there something in particular about the JVM that makes it more difficult, or is it just that the work hasn't been done yet for some reason?

yrio2• 6 years ago

I don't know about single step when debugging, but it seems intelligible stack trace for async code is available only recently, starting with .NET Core 2.1

https://stackoverflow.com/questions/15410661/is-it-possible-...

ceronman2• 6 years ago

This is my favorite JVM project and I think it's going to be huge!

This model of concurrency is so much better than the async/await model used by many other languages. No more colored functions [1], or worse Completable<Future>, promises et al. Nice stacktraces, debuggers that work plus no need for thread pools anymore. I can't wait for this to be ready for production.

Only drawback seems to be when calling native code. I guess it's the same problem that Golang has. Good thing is that the Java ecosystem is not that dependent on native stuff, so I think it's a fair tradeoff to make.

[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

anonymoushn• 6 years ago

Will virtual threads be copyable?

mrkeen2• 6 years ago

No more colored functions [1]

That's a property of the function, whether or not you want the type system to help you with it. I can call a function myRedFunction, but if it calls a blue function, it's blue.

or worse Completable<Future> What is this? Future<Result> f = e.submit(() -> { ... return result; }); // spawns a new virtual thread
This model of concurrency is so much better than the async/await mode

  ThreadFactory tf = Thread.builder().virtual().factory();
  ExecutorService e = Executors.newUnboundedExecutor(tf);
  Future<Result> f = e.submit(() -> { ... return result; }); // spawns a new virtual thread
  ...
  Result y = f.get(); // joins the virtual thread

I don't get it. This just looks like async/await (submit/get) with extra steps. Why not just write

  Future<Result> f = async ( ... result; )
  Result y = await (f);

pron17• 6 years ago

That's a property of the function, whether or not you want the type system to help you with it.

The problem is that in, say, C# or Kotlin, subroutines with the same semantics come in two different colours. The type system "helps" you distinguish between things that aren't meaningfully distinguishable.

I don't get it. This just looks like async/await (submit/get) with extra steps.

You only need to submit if you want to do stuff in parallel and then join (also, there are no extra steps). The analogue to:

    var a = await doA();
    var b = await doB();

is just:

    var a = doA();
    var b = doB();

Skinney5• 6 years ago

Well, the first two lines are usually written somewhere at the top level. You won't actually write that anytime you want something to be async. The last two lines are what you usually write, so most of the time, the number of lines of code are equivalent.

Second, Java doesn't have async/await today. If Java was to introduce it, it wouldn't be compatible with the code written today. The big benefit of project loom is that code written today will get all the benefits of async/await without changing any code. In fact, because it's not important to pool threads, it actually gets easier.

The big problem with async/await in languages like C# or even coroutines in Kotlin is that it doesn't mix with the "old" api. The get the benefits you need to do a huge refactor of your code, and you need to make sure that any library you pull in is compatible.

Java's approach seems much better.

eggsnbacon1• 6 years ago

I don't get it. This just looks like async/await (submit/get) with extra steps. Why not just write

because altering the underlying thread pool can convert all existing code to fibers without syntax changes. This is the crux of "colored functions", two different signatures for async/non-async.

The implications for improving the performance of existing code are enormous. Most Java API's give you at least indirect control of the thread pool used. Which means they can be converted to fibers without updating the library.

In Java, executors are basically thread pools. If you can update the executor to use fibers the rest of the code is using fibers now. Your second example only converts a single call site to fibers. A thread pool could be used in hundreds of places.

In practice, this is a huge difference. I can update my Streams to use a fiber pool, and all across my app the hundreds/thousands of streams are using fibers now. Same story with REST and DB access. There's usually a single thread pool for each. A few lines of code to update the pools and my REST and DB calls are all using fibers.

balfirevic6• 6 years ago

I still have't fully made peace with the fact that C# choose async/await route :(

It could have been so much better.

nlawalker2• 6 years ago

Why didn’t C# or JS go this way?

albertzeyer• 6 years ago

Can you maybe elaborate why this model is better than async/await? I have to admit, I don't really know Loom, but from a first glance, it looks like this is just (green) threads, and async/await is anyway an orthogonal concept to this, or not? But maybe I am confusing things here.

Can you maybe recommend a good overview over all the current concurrent approaches and concepts, like async/await, and alternatives, and their advantages and problems? I would love to get a more recent overview on this.

I had the impression that most recent languages follow on the async/await concept. At least JS and Python. Maybe also Rust? Go? Erlang? And there are probably libraries for C++ to do the same, or maybe it is already integrated? I have to admit that I did not fully follow all this development.

Skinney5• 6 years ago

Project Loom provides the same benefit of async/await, but without changing the syntax in any way. In C# you now have a split API: blocking api's which rely on threads for concurrency, and async api's which require you to use async/await syntax. These api's are not compatible, so to take advantage of async/await, you have to refactor your code.

When project loom is implemented in Java, all code which makes use of threads today automatically gains all the benefits of the async/await model, without any refactoring.

Go and Erlang doesn't do the async/await thing. They only have one notion of concurrency (goroutines in Go, processes in Erlang). They don't require you to use async and await constructs, but they work similarly to async/await under the hood.

GolDDranks2• 6 years ago

Async/await split up your functions at their "blocking points" to fragments that can be run fragment-by-fragment, in an interleaved way by an executor / event loop, thus enabling high level of concurrency within a single OS thread.

The problem is that your split-up async function ceases to be a normal function. It doesn't use the stack in a similar way than normal functions. Furthermore, the executor / event loop is required for running those async functions.

So you can't call async functions from normal functions, because they aren't. You need to create an event loop and then hand the task over for it to process it. Your world splits up into sync and async functions.

Go and Erlang are going with the similar "lightweight thread" approach as Java's project Loom. On the other hand, JS, Python and C# have gone with async. I'm not sure if these languages actually need async, going with something similar as project Loom would have also fit, and would have been simpler from the user perspective, perhaps? But hindsight is 20/20.

Rust has also gone async, and I'd argue it's the only one of the pack who really have async/await as a necessity - this is because of two design requirements that differ from Java, C#, Python and JS: 1) must have native-level performance when calling foreign code that expects a C-like stack. Something like Loom doesn't provide that. 2) must not have an implicit default runtime.

hn_throwaway_99• 6 years ago

Couple specific examples you may be familiar with:

Coming from Java to JavaScript, I really, really yearn for the equivalent of ThreadLocals. For example, most Java logging libraries let you set a "context" object that can be used for logging a request ID or user ID anywhere down the call stack. You can't do that in Node, so instead you have to explicitly pass that context object down in all your function calls, or make use of brittle, complicated, hacky, not-fully-supported features like "continuation local storage".

Debugging stack traces in an async/await world is a mess because you don't get a nice "from the start of the request down to this call" view of the stack.

Perhaps most importantly, with Node at least you need to worry about starving your main event loop because there is only a single top-level main thread (but understand that's not inherent to the async/await issue in general).

ceronman2• 6 years ago

As others have pointed out, the big problem of async/await is that it splits the world in two kinds of functions that can't call each other.

For example, in Python you have this http client library called `requests`. It's very easy to use, but it is synchronous and you can't use it in when doing async/await. Instead you have to use a similar library called aiohttp. And it's like that for every single library or function doing any sort of IO, you need two versions of everything. Same thing happens in Node.js. You have `spawn` which is asynchronous and `spawnSync` which is synchronous. And so on.

And it doesn't stop with functions, this division goes across the language it self. For example in Python you have a `for` loop and an `async for` loop. Same in JS, you have `for` and `for await`. C# also has it's own `await forech` construct. It's like splitting the world for no reason.

The great advantage of Loom is that you don't need a parallel language, you can use the same http clients, database clients, filesystem libraries that you were using in a synchronous world, but now you will be able to use concurrently in millions of virtual threads.

As the article mentions, the whole reason why async/await was invented, is because doing concurrency with kernel threads is very expensive. They are slow to create and they use a lot of memory. Using Promises and an event loop you can run multiple concurrent operations in a single OS thread. This gives you the chance to achieve much more concurrency than running one operation per OS thread. The problem is that this changes the way you program a lot. Your code is not sequential anymore, your error handling is different, even some conditional operations are different. Async/await is just some syntactic sugar created to make this looks like the regular programming model. It makes the function calls look sequentials (by adding an `await` prefix), it makes the errors look like exceptions, it makes the for loops look like regular for loops. But this is just appearance. It's something that looks similar, but it's different, and that's why you can mix it with regular code.

Loom model is closer to Go and Erlang.

balfirevic6• 6 years ago

Can you maybe elaborate why this model is better than async/await?

Others have given actual explanations, but I'll chime in with a bit of a tongue-in-cheek one: it's better than async/await because it doesn't have async/await (yet has all the benefits provided by them).

closeparen• 6 years ago

Code that runs in goroutines looks no different from code that runs outside goroutines. You do regular blocking IO. The application code of a performant RPC / HTTP server never needs to refer to any kind of concurrency construct, unless it wants to make outbound calls in parallel. (Your HTTP or RPC library takes care of dispatching inbound requests onto goroutines).

As much as I want a more expressive language, all the explicit async/await, futures, callbacks, etc. junk needed just for your API handler to cooperate with concurrent requests (not even do its own parallel processing) keeps on sending me back to Go. I look forward to being able to write Java in the same style.

eiopa• 6 years ago

This reminds me a lot of python’s gevent.

I often wish the language had adopted it instead of the C#-like await async, since it’s just more straightforward.

dehrmann• 6 years ago

gevent has some serious drawbacks. Stack traces are incomprehensible, it works by monkey patching existing code, and it falls apart if you have a blocking operation gevent can't make async. Loom is a bit like monkey patching, but at the VM level, so I expect it to be much more stable.

jpgvm• 6 years ago

It's worth mentioning those drawbacks are implementation problems, not soundness or ergonomic problems with the model itself.

I would consider myself a pretty harsh critic of Python but even I appreciate the elegance of the gevent approach to concurrency.

doctor_eval8• 6 years ago

I was reading about Go threads just the other day and the article mentioned that Go uses green threads because they are more efficient.

I thought this was weird because IIRC Java 1.0 used green threads in Linux and it was a big deal when they moved to OS threads.

I’ve long believed that the IT world moves in cycles but this is a very clear example of exactly that. Java has gone from green threads to Posix threads and now back to green threads.

I do think it’s awesome (I love goroutines) and Threads in Java have become a but of a nightmare, made a little easier with executors and CompletableFutures. So this further improvement is great news.

But still... call me when the builder pattern is dead.

dullgiulio2• 6 years ago

As the article explains (I know it's hard to comment after actually reading), Java moved away from using several green threads on top of a single system thread.

What Loom and Go do is to schedule green threads on a bunch of system threads and spawn more system threads when they get blocked doing synchronous system calls.

doctor_eval8• 6 years ago

No need for snark, I did read the article. I was just pointing out the irony that it was a big deal when the JVM moved away from green threads and now it’s a big deal when it moves back. It’s a comment on the hype cycle. And, as I said in my OP, it’s a good change and I’m happy to see it.

(And if you want to be pedantic, IIRC the green threads were originally mapped to the Java process, not a thread, because threads were either unavailable or immature in Linux when Java 1 came out; I can’t remember which)

chrisseaton3• 6 years ago

I’ve long believed that the IT world moves in cycles but this is a very clear example of exactly that. Java has gone from green threads to Posix threads and now back to green threads.

No it’s not the same thing - Java was previously M:1, then became M:M, and now Loom is M:N and also preserves M:M at the same time.

doctor_eval8• 6 years ago

Sure, but back when Java had green threads on Linux, almost nobody had multicore processors, so it’s not surprising that it started as M:1. What is perhaps surprising is that Java then moved to using OS (POSIX?) threads instead of continuing to manage them internally. I remember OS threads were touted at the time as being faster than green threads.

It’s a comment on the hype cycle. As a long time (since 1.0!) java developer, I like that Java has adopted a more efficient threading model. But they are certainly returning to a state where the JVM manages the threads rather than the OS, which certainly is the same as the original release, at least on Linux.

chewbacha• 6 years ago

Hmm I love having cake...

...and eating it too

Igelau• 6 years ago

Anyone else disappointed this wasn't about Lucasarts Loom (1990)?

davidjhall• 6 years ago

Have to admit - I was excited to think this was about a SCUMM update or maybe a sequel.

aidenn03• 6 years ago

You mean the latest masterpiece of fantasy storytelling from Lucasfilm's™ Brian Moriarty™?

e12e• 6 years ago

A server can handle upward of a million concurrent open sockets, yet the operating system cannot efficiently handle more than a few thousand active (non-idle) threads.

Does anyone have a reference to some up to date notes on the state of scalability of Linux and/or free-/dragonfly-bsd scalability at the process level?

I don't doubt processes are a bit heavy for scaling to a million active threads of execution - but it'd be nice to see what one could expect on a low end server (say 16 cores, 64gb ram) today?

zokier• 6 years ago

I'd also be curious about this. C10k was a problem at the turn of the century but world has changed a lot since, both in terms of hardware and kernel.

Best reference I could easily find was this experiment that had 100k connections to mysql which has thread per connection model. Seems to handle it well

https://www.percona.com/blog/2019/02/25/mysql-challenge-100k...

MattTse• 6 years ago

Anyone else read this, and read it as google's Project Loon, and wonder what java has to do with it?

jmartrican• 6 years ago

I been waiting for this project for a while. I saw an amazing demo of this on youtube video. In the demo a regular Jetty server running a simple endpoint that slept for 1 sec was DOSd with multiple concurrent requests. You can see as the number of concurrent requests increased, the execution time increased. The presenter then changed the code of the Jetty server to create a new Fiber vs a Thread. Then reproduced the test and this time there was no increase in execution time.

What I really like about this project is that we can keep Spring MVC applications in non-reactive form. Spring MVC came out with reactive framework for implementing APIs. But I'm not a fan of this style for a few reasons, plus there is millions of LOC doing it the non react implementation. By utilizing project Loom, we do not have to switch over to the Spring reactive way, and we can increase the performance of existing code.

EDIT: Here is the video: https://www.youtube.com/watch?v=Csc2JRs6470:

- he goes into Loom at 19:30.

- he goes into the demo at 24:00.

artemonster• 6 years ago

Can Loom be used to simulate RTL & verification language on top of that (similar to SV or e)? I mean practically (since theoretically the answer is yes)

elric• 6 years ago

In case anyone else is confused about some of the internal links appearing to be broken (e.g. the Scope Variables link), the content is on the second page.

pizlonator• 6 years ago

This is great. If I grok right this is basically the best of native threads and the best of green threads put together.

I think I proposed this at a crazy idea talk at VEE a long ass time ago (10 years or more). It’s still a good idea but I guess not crazy anymore.

jayd163• 6 years ago

Seems fine but I'm confused by the async/await hate. At least in C# it seems to me thee is already a superset of Loom through async/await.

    Thread.startVirtualThread(() -> {});

Starts a new virtual thread that will run a synchronous method. Ok cool. C# already has:

    Task.Run(()=>{})

This will run synchronously on the shared thread pool. Tasks are futures and can return objects. Same as the Loom proposal.

Optionally you can decide to opt into the await unboxing sugar by adding async to your method signature. There's arguably some double dipping with the async keyword on a method. It allows the await keyword to be used inside the method but also forces the caller into a different calling style. You can argue this shows API intent for cooperative threading. That said, you can still use async and sync interchangeably.

The syntax seems very similar to me.

Is there something else going on in Loom that I'm missing? Is it a matter of how the virtual threads are scheduled/preempted vs how other languages with async/await schedule their tasks?

balfirevic6• 6 years ago

The point is that you don't have to write code in a different way if you want to avoid blocking an OS thread on IO operation.

jsiepkes• 6 years ago

Maybe I don't understand correctly what your saying but "Task.Run" just schedules something on a normal (kernel) thread by using a common thread pool. The Java equivelant is probably "CompletableFuture.runAsync" which does the same thing.

Loom with "Thread.startVirtualThread" will run something on a userland / green thread (ie. not a kernel thread so no context switching and blocking it "costs" practically nothing). async-await is actually a subset of wat Loom does in the sense that Loom allows for far more then just async-await. Most of the "hate" for async-await is probably because it leads to the "what color is your function" [1] problem.

[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

dang2• 6 years ago

State of Loom

Related Stories