PR to Merge Multicore OCaml

https://github.com/ocaml/ocaml/pull/10831

by c-cube • 5 years ago

388 155 5 years ago

github.com

sadiq7• 5 years ago

As usual, happy to answer any questions.

pdimitar2• 5 years ago

Is the previous plan for adding effects in OCaml 5.1 still in place?

I am very excited not only for the native parallelism that's coming in 5.0 but also about the effect handlers! I am sure many others are looking to start creating very interesting things with them, e.g. Rust-alike async capabilities or Erlang's preemptive green threads / actors runtime.

sadiq7• 5 years ago

5.00 (the first release with multicore) will include effects!

More info: https://discuss.ocaml.org/t/multicore-ocaml-september-2021-e...

Zababa3• 5 years ago

I've searched a bit for this and haven't been able to find a good answer. Let's say that I have m tasks that I want to run concurrently on n cores, for example handling HTTP requests or searching for something in files. I'd also like to not have to manage manually how the tasks are going to be distributed. Basically, have the same experience as when writing Go code. Is there a way to do this in multicore OCaml? I've found the task pool in domainslib [1] but I'm not sure if it's what I'm looking for.

[1] https://github.com/ocaml-multicore/domainslib/blob/master/li...

vphantom• 5 years ago

I think libraries like Lwt will be the ones to offer the level of abstraction which you're describing. I too would like to see the simplicity of goroutines make it into OCaml 5 ASAP.

sadiq7• 5 years ago

You could use eio with an event loop per domain and the domain manager to distribute work to other domains. The restriction at the moment is that the tasks you spin off to other domains can't do asynchronous io.

There is work on-going at the moment to bridge or even unify eio (concurrency via effects) and domainslib (nested parallelism via domains and effects) but it's a few months out.

rwmj• 5 years ago

I'm slightly concerned if this means changing C extensions (of which we have rather a lot).

We only have a few places that use "naked" pointers, which are in any case deprecated, and we should be able to fix those easily enough.

Mostly the rest are fairly ordinary extensions that just call C functions and use the usual CAMLparam stuff.

We do have a few places that register global roots. And several packages that do callbacks from C back to OCaml. We also use @@noalloc a lot.

Is there anything else? Since there's so much code, what should I be grepping for to find code that might be of concern?

Edit: Some examples of simple stuff:

https://github.com/libguestfs/libguestfs-common/blob/master/... https://github.com/libguestfs/libguestfs-common/blob/master/... https://github.com/libguestfs/libguestfs-common/blob/master/...

More complex stuff:

https://gitlab.com/nbdkit/nbdkit/-/tree/master/plugins/ocaml http://oirase.annexia.org/tmp/ocaml/

avsm2• 5 years ago

You may find the “nnpchecker” configure option in 4.13.0 useful. This will print a detected use of naked pointers to stderr, which can hopefully be triggered by test suites.

Noalloc and registering roots should all be the same, as are callbacks (for sequential code, which all existing code will be)

sadiq7• 5 years ago

To echo what avsm said, our intention is to preserve the C API. With the exception of naked pointers, if you follow the rules around the existing C API then your extensions should continue to work in sequential code running on 5.0.

We have a scheduled build and test of every package in opam with multicore: http://check.ocamllabs.io:8082/ to try to shake out C API incompatibilities and that's proved fruitful.

If you do find things that don't work on 5.0, please let us know (and if you can, get it in to opam so we test it automatically!).

jpf0• 5 years ago

A bunch of questions: Is multi-threading happening at the level of user-defined functions? How are threads scheduled? What's the underlying method or library for enabling multi-threading (Cilk, openMP, some LWT library...)? To what extent has this level of granularity been tested against other levels of nested parallelism (e.g. SIMD or otherwise parallel operators)? Have you tested performance by OS, and if so, have you noted any necessary OS-level modifications related to thread management? Is this part of a broader roadmap for accelerator integration?

sadiq7• 5 years ago

1. Domains are the unit of parallelism. A domain is essentially an OS thread with a bunch of extra runtime book-keeping data. You can use Domain.spawn (https://github.com/ocaml-multicore/ocaml-multicore/blob/5.00...) to spawn off a new domain which will run the supplied function and terminate when it finishes. This is heavyweight though, domains are expected to be long-running.

2. Domainslib is the library developed alongside multicore to aid users in exploiting parallelism. It supports nested parallelism and is pretty highly optimised (https://github.com/ocaml-multicore/domainslib/pull/29 for some graphs/numbers). The domainslib repo has some good examples: https://github.com/ocaml-multicore/domainslib/tree/master/te...

3. We've not tested against other forms of parallelism. There isn't anything stopping you exploiting SIMD in addition to parallelism from domains.

4. No, we've not compared performance by OS.

5. No plans for the multicore team to look at accelerator integration at the moment.

jez• 5 years ago

This is only glancingly related to the topic of multicore but I figure I’ll venture the question anyways:

There were three big reasons pushing me towards other languages (C++, Rust) for a certain class of throughput-focused workflows:

1. Lack of multicore execution

2. Lack of control over how many bytes a datatype is (useful for many things, like ensuring something fits in a machine word or can avoid chasing a pointer in a loop)

3. Difficulty controlling where memory is allocated/copied vs moved (though some of this is less relevant when everything uses reference semantics and mutability is tracked with ref/mutable)

I’ll be glad to see (1) fading into history! Do you have any personal tips/anecdotes for memory optimizations in practice, or any suggestions for something to read/follow/search?

sadiq7• 5 years ago

For both 2 and 3 you may find https://github.com/ocaml/RFCs/blob/unboxed-types/rfcs/unboxe... very interesting. I'm hoping that progresses beyond the RFC stage.

eatonphil• 5 years ago

Congrats!

Kototama• 5 years ago

I'm excited. Haskell gets a lot of attention but I have the feeling that OCaml may be better for engineers.

ghostwriter6• 5 years ago

How can parallel untracked mutations of untracked state be better for engineers?

sidkshatriya5• 5 years ago

There are many paradigms in programming. Each have their strengths. A purely functional approach ala Haskell is not the only way. Based on your comment, it would seem no one should use C/C++. Yet many do. It depends on what you want to achieve, what is your abstraction budget, your performance requirement, legacy code...

OCaml offers a pragmatic functional approach to programming. And now you are going to be able to have your OCaml code run in a truly parallel fashion on your multicore CPU.

In the future there are plans to add typing to effects though. (There is support for effects currently but its untyped and experimental). When that happens you can track changes to state (which is a kind of "effect") if you want to...

hedora• 5 years ago

I assume they were referring to the strong OCaml ecosystem (and collection of C/C++ bindings), not the relative merits of the threading systems.

grumpyprole• 5 years ago

It may not remain untracked, hopefully types for algebraic effects will land in the future.

naasking• 5 years ago

How can parallel untracked mutations of untracked state be better for engineers?

It basically requires eager evaluation which, believe it or not, is a huge ergonomic win for people wanting to work on practical problems.

OCaml also preserves the concise syntax and great type inference you get in Haskell.

All in all, a more pragmatic set of tradeoffs I'd say, although it certainly has its own problems.

ernst_klim4• 5 years ago

How can parallel untracked mutations of untracked state be better for engineers?

What is untracked? Is

    unsafePerformIO (printLn ...)

tracked? What is tracked? Is

    foo :: IO ()

thread-safe? Maybe, maybe not. What meaningful information does this signature says to me? That it does some IO? That's an extremely useless information, especially if most of your code is IO something.

What granularity does IO have? Does

    foo :: IO ()

throws any exceptions? Maybe, maybe not.

The need for effect tracking for writing correct programs is way overstated by some Haskell programmers. It's usually much more prudent to write DSLs which hide effects like variable mutations and logging, than expose them.

For example you can start your program with a pure correct-by-construction core DSL, and then add logging, mutable variables where needed underneath the DSL's terms without breaking the semantics of the DSL. With effect tracking you are doomed to either reinvent custom effects to be able to switch interpreters painlessly, or you'll need to break the DSL by the addition of effects.

Neither is prudent in real world, neither gives more value than takes. What is really funny is that some Haskell programmed believe that logging should be tracked, but allocation apparently should not (yes, it's a side effect).

Where in fact all the "effects" are need to be tracked only when they are meaningful, i.e. when they are part of our DSL's semantics, and not some hidden part of the interpreter I don't need to know about.

Imagine a theorem prover function which creates a conjunction of two terms:

    val conj : term -> term -> term

It doesn't matter if it allocates or logs, at the precision that is interesting to us, it's just a function creating a conjunction of two terms.

dleslie• 5 years ago

OCaml benefits from not looking like operator stew.

sidkshatriya5• 5 years ago

Actually OCaml has more operator stew than Haskell. Haskell's use of typeclasses/HKT avoids a lot of operator/function-name noise in the code.

Where OCaml wins is that it does not hate mutation. This makes the code more impure but simpler. Sometimes just incrementing a mutable variable makes more sense than introducing a state monad!

throwaway8943452• 5 years ago

When I tried to dabble in OCaml, it definitely looked like operator stew to me, but I’m coming to it from the C, Python, JS, etc world so maybe Haskell is even worse in this regard?

DonaldPShimoda2• 5 years ago

Totally agree. OCaml is, in general, a better choice for engineering and real-world application. I think Haskell gets a lot of attention because it's interesting (for lack of a better word). I don't mean "OCaml is a boring language", but rather "Haskell was specifically designed as a playground for advanced ideas in type theory, so lots of weird things happen in that ecosystem." For doing personal projects for fun, I'd rather work with Haskell, but for getting things done probably not.

gmfawcett4• 5 years ago

Please let's just all be friends. :) It's fair game to discuss and differentiate the two languages on features, but "better for engineers" is going to take us right into the valley of flame-wars, by way of No-True-Scotsman town. :)

Haskell and OCaml are both brilliant languages, with excellent feature sets and amazing dev teams... and lots of "engineers" who like them.

yawaramin7• 5 years ago

Note the difference:

Kototama: ...I have the feeling that OCaml may be better for engineers.

You: let's not say that OCaml is better for engineers.

People are allowed to feel things.

But I'll give you a concrete reason why OCaml actually can be better for engineers: named and optional arguments. This single feature does wonders for code readability and maintainability.

Actually I'll throw in a bonus reason: no laziness by default i.e. small and simple, predictable runtime that behaves almost exactly like a C executable.

jamespwilliams• 5 years ago

Agreed. Being able to drop seamlessly into an imperative style is very useful.

I’m someone who isn’t an experienced functional programmer, or that deeply knowledgeable about type systems and category theory etc, but OCaml is still approachable to me, I feel I can get stuff done in it. I didn’t feel that way with Haskell - I tried, but found the number of ways to do things overwhelming, and a lot of it felt very theoretical. Of course, Haskell’s pureness is an impressive and interesting feature; but it isn’t for me.

OCaml also has much better systems programming abilities compared with Haskell, IMO. It feels like OCaml is the Rust of the functional programming world (and this is no accident, the Rust compiler was first implemented in OCaml in fact).

This is all completely subjective of course.

baby4• 5 years ago

I see Rust as an imperative language with functional support, and OCaml as a functional language with imperative support

sidkshatriya5• 5 years ago

Some questions:

- Is merging this PR going to be more or less a formality now? I'm assuming subsequent PRs will make further improvements/fix any further bugs discovered. When is the merge to trunk likely to happen?

- (Not that it really matters, but I'm curious). Will this PR be merged as-is with nearly 4000 commits or will it be squashed? The history exists in the multicore repo but will the history be brought into the ocaml repo?

sadiq7• 5 years ago

1. Before multicore got to this PR stage it went through two phases of detailed review by the core team. A summary of this is on November's Multicore Monthly: https://discuss.ocaml.org/t/multicore-ocaml-november-2021-wi... . The tasks coming out of that which are marked post-MVP will be follow-up PRs before the 5.00 release.

2. That is unclear at the moment. There's a lot of useful history in those commits (which link out to issues and PRs on ocaml-multicore's repo) but at the same time, it also includes a lot of experiments that were ultimately backed out.

mst2• 5 years ago

Sounds like the sort of situation where I prepare a fastforwadable PR and then force git to create a merge commit anyway.

2wrist• 5 years ago

Great stuff!!

michaelmcmillan• 5 years ago

Finally a PR worthy of Hacktoberfest.

gmfawcett4• 5 years ago

The multicore team has done an excellent job of communicating their plans and progress over the past few years. Innovative research, solid engineering, and a top-notch communications game. Whether or not you care about Ocaml, many other SE projects could learn a lot from studying this team.

eatYourFood• 5 years ago

could learn a lot from studying this team.

How? Where?

gmfawcett4• 5 years ago

Fair question! Here's a good place to start, they've been doing monthly progress reports here since January 2020:

https://discuss.ocaml.org/tag/multicore-monthly

There have also been presentations, videos, academic papers, etc., which should all be documented in the monthly reports.

lgrialn• 5 years ago

Hurray!

My wish for Ocaml is that it could somehow be popular enough to have more… casual users, let’s say. I enjoy playing with it more than any other language I’ve run into, but everyone else falls so deeply into it that I don’t generally follow their discussions very well. (Maybe I’ll fall in deep myself in time, but it remains to be seen.)

ModernMech2• 5 years ago

I don't know if you've seen this language yet and I haven't used it, but Grain seems to take a lot of inspiration from Reason. Seems like they are trying to target a more casual audience.

https://grain-lang.org

Ericson23147• 5 years ago

Someone needs to get OCaml running on the Go runtime, so it can be an F# for that ecosystem.

whimsicalism• 5 years ago

What's the motivation? OCaml is roughly as performant as Go.

yawaramin7• 5 years ago

With all the problems arising from leaky abstractions and trying to adapt to a foreign runtime? Imho there's no point. Especially now that OCaml Multicore is on the way. A much better effort, for anyone interested, would be to vastly simplify the OCaml build and package management story to be more Go-like.

ernst_klim4• 5 years ago

This is a very bad idea considering Go runtime is optimized for way less allocations due to 1) how good Go in stack allocation is and 2) how mutable everything is.

OCaml on the other hand has an extremely well optimized runtime for lots of small heap allocations (typical in FP), and produces good asm code for functional patterns.

amaranth• 5 years ago

With F# or TypeScript or similar things you can compile down to a common bytecode or to the base language and use the result. Unless you were writing an OCaml to Go transpiler you'd need to ship a copy of the Go source code alongside a new OCaml compiler and still have to build a new build system and ecosystem and such. You couldn't just bolt this on to an existing Go installation or something.

frej2• 5 years ago

Which feature is prevalent in the go runtime that you must have? Or is it the ecosystem that is larger in go?

noncoml• 5 years ago

I really didn’t think I’d see the day.

keewee7• 5 years ago

F# is a ML/OCaml with solid concurrent and parallel programming features and has been multiplatform since Microsoft released .Net Core 1.0 in 2016.

klibertp2• 5 years ago

F# is a ML/OCaml

No. F# is an ML dialect, like OCaml. But, you should be aware that the "O" in "OCaml" actually means something. OCaml comes with beautiful object system that's prototypal in nature and structurally typed integrated into the language. F# on the other hand has the object system inherited from .NET, and it's simplistic in comparison.

Also, modules and functors.

So, no, F# is not OCaml. If you absolutely need to use .NET, and know some ML, you can reach for F#, but its limitations will drive you nuts quickly. F# is for C# programmers - they have "full power of .NET" still accessible, plus a few nice things like HM type inference, immutability by default, and maybe computational expressions.

pbiggar• 5 years ago

But basically no one uses the object part of OCaml

pjmlp7• 5 years ago

F# is not really for C# programmers, C# programmers just get by the F# features that keep being copied into C#.

F# is an ML that is kind of allowed to play on .NET because in a given moment management accepted to integrate it into Visual Studio, and keeps looking for the golden spot that it will take it beyond the VB/C# shadow, always a second thought when the .NET team designs new architecture features only taking VB and C# into consideration.

F# twittersphere is its own bubble, not always with nice opinions about Microsoft and the .NET platform it depends on.

bmitc• 5 years ago

I completely disagree that F# is for C# developers. For one, C# developers are usually fine sticking with C# and its massive ecosystem. F# and .NET more seamlessly run everywhere than Ocaml, which last time I tried is a lot of trouble on Windows. F# has had multicore support from the beginning and hasn’t needed a multi-year development process to get there.

I’d like to learn Ocaml (more than just the parts I’ve learned from F# and SML), but F# is by no means a simplistic language in terms of what you can accomplish. It’s easily one of the best designed and most pragmatic languages out there. From the syntax I have seen, F# is cleaner than Ocaml.

F# is an ML dialect, like OCaml

That’s kind of splitting hairs. The object system is different for obvious reasons. F# clearly started as a port of a subset of Ocaml to .NET and was even valid Ocaml code for some time in the early days. That’s exactly a dialect.

sally1620• 5 years ago

Don Syme explains in history of F# that there were multiple efforts to port OCaml to .NET, (Nemerle comes to mind), but none of those felt natural to use.

For F#, they started with OCaml syntax and semantics; then modified it to make it a true .NET language. They added features that were not in OCaml like inheritance & interfaces. and removed features that were unnecessary in .NET like functors.

https://fsharp.org/history/hopl-final/hopl-fsharp.pdf Early F# compiler even had ml-compatibility mode which would make syntax closer to OCaml, it was probably used for bootstrapping.

gmfawcett4• 5 years ago

F# is a great language, and definitely an ML. But it falls a bit short of being an OCaml. :) The type-system features are quite different -- e.g. F# has no functors and no GADTs, at least not the last time I checked. I think that F# has a different feel and perhaps a different target audience.

pjmlp7• 5 years ago

Great news! Congratulations to everyone involved.

nicolashahn• 5 years ago

If OCaml gets good tooling and ecosystem I'm totally in. I've heard it's close to Rust but GCed and that sounds extremely enticing.

nestorD• 5 years ago

Tooling got me from Ocaml to F#, I would recommend it if that is a criteria for you.

c54• 5 years ago

I like OCaml a lot but I agree that it suffers from this tooling/ecosystem problem. Even un-fancy languages with good tooling are just going to win, at the end of the day. Go is the main example of this. Rust also has great tooling that's gotten better over time.

I think over time Jane Street will (continue to) open source more of its libraries, and hopefully soon will also migrate onto Dune (internally they use jenga[0]). This should mean that the ecosystem within Jane Street more closely matches the external environment and tooling should get better as they push patches upstream.

Hopefully.

[0] https://discuss.ocaml.org/t/does-jenga-have-more-features-th...

wk_end3• 5 years ago

I mean, maybe. But Core and Async have been open source for a long time, and there's still plenty of people out there using Batteries/Containers/Iter/BOS/Lwt.

jasone2• 5 years ago

I started using OCaml in 2017, and had a rough start. But since then Dune (https://dune.build/) and the now-excellent LSP implementation have resolved the biggest issues I had. I haven't had such a pleasant development environment since Turbo Pascal! (To be fair, IntelliJ was also pretty good other than the shortcomings of Java.)

sideeffffect2• 5 years ago

Since you mentioned using IntelliJ, I can't resist asking the question: Have you played with Scala too? If yes, how would you compare OCaml to it?

baby4• 5 years ago

If you start a project today I would really try to use esy (https://esy.sh/)

I actually don’t use it myself but it seems to bring the modern programming language experience to OCaml

devmunchies• 5 years ago

I definitely will use it as my primary server language on side projects after it has a standardized async lib (effect handlers and eio). I was also hoping that rust would be my primary lang but its a little too verbose with all the "<>" for it to be my guilty pleasure. Been dabbling in OCaml since 2018.

wk_end3• 5 years ago

It's similar to Rust in certain respects, but it feels very different in several others.

* There's no ownership tracking. This might be obvious - it doesn't need it, it has GC - but is actually kind of unfortunate; even in GC'd languages, linearity can be used to safely model state machines/typestate or for optimization.

* There's no traits/ad-hoc polymorphism at all.

* There's no impls, though a common pattern with a similar feel is to define types inside modules with the actual implementation of the type hidden, and provide an interface via functions to create or interact with instances of the type.

* It uses exceptions for error handling (when it doesn't use an option type) and even as a lightweight control flow mechanism. And now there's algebraic effects as well.

* It has an extraordinarily powerful module system, with functors (pass modules into other modules to produce new modules) and first-class modules (modules that exist as values at run-time that can be passed around to functions).

* It has some more advanced type machinery - GADTs, polymorphic variants, and a very interesting, powerful, almost completely unused object system.

* This bleeds a little into "ecosystem" and a little into "no ad-hoc polymorphism" and a little into the oddness of the syntax and a little into subjectivity, but in general doing really bog-standard imperative stuff in OCaml, while possible, tends to feel more awkward than it does in Rust.

edwintorok3• 5 years ago

* exceptions for error handling: you can also use the 'result' type (an OK of 'a|Error of 'b type in the stdlib) if you don't like exceptions. At least the function's signatures will give more details in how it can fail and force you to handle the failures (or pass them through to your caller). Unfortunately you lose backtraces that way (though you can use wrappers like reword_error, or logging to give more details on what goes wrong, but backtraces can be convenient in locating the source of an error sometimes). A compromise is to use a wrapper like trap_exn (from Bos) that puts the exception with its backtrace in the Error type and then you propagate that...

baby4• 5 years ago

* There’s no explicit typing also, everything is polymorphic by default

* Compiler errors won’t help you much

* The tooling is lacking compared to cargo. It’s pretty much makefiles in a lisp language (better love parens)

* The ordering of your code matters, shadowing functions and structs is a a thing

* use A::* is pretty much the default, and transitive dependencies are imported at the top level, so be ready to be confused about the origin of a function or module or variable

I really wish more people would write OCaml so that we’d have a better ecosystem.

If you’re looking to get into a new language, you should seriously consider it as there are SO MANY low hanging fruits and opportunities to have an impact on the language.

dang• 5 years ago

Past related threads:

Multicore OCaml: October 2021 - https://news.ycombinator.com/item?id=29238972 - Nov 2021 (12 comments)

Effective Concurrency with Algebraic Effects in Multicore OCaml - https://news.ycombinator.com/item?id=28838099 - Oct 2021 (59 comments)

Multicore OCaml: September 2021, effect handlers will be in OCaml 5.0 - https://news.ycombinator.com/item?id=28742033 - Oct 2021 (3 comments)

Multicore OCaml: September 2021 - Effect handlers will be in OCaml 5.0 - https://news.ycombinator.com/item?id=28719088 - Oct 2021 (3 comments)

Adapting the OCaml Ecosystem for Multicore OCaml - https://news.ycombinator.com/item?id=28440385 - Sept 2021 (1 comment)

Adapting the OCaml Ecosystem for Multicore OCaml - https://news.ycombinator.com/item?id=28373155 - Aug 2021 (21 comments)

Multicore OCaml: July 2021 - https://news.ycombinator.com/item?id=28039219 - Aug 2021 (14 comments)

Multicore OCaml: May 2021 - https://news.ycombinator.com/item?id=27480678 - June 2021 (27 comments)

Multicore OCaml: April 2021 - https://news.ycombinator.com/item?id=27140522 - May 2021 (89 comments)

Multicore OCaml: Feb 2021 with new preprint on Effect Handlers - https://news.ycombinator.com/item?id=26424785 - March 2021 (29 comments)

Multicore OCaml: October 2020 - https://news.ycombinator.com/item?id=25034538 - Nov 2020 (9 comments)

Multicore OCaml: September 2020 - https://news.ycombinator.com/item?id=24719124 - Oct 2020 (43 comments)

Parallel Programming in Multicore OCaml - https://news.ycombinator.com/item?id=23740869 - July 2020 (15 comments)

Multicore OCaml: May 2020 update - https://news.ycombinator.com/item?id=23380370 - June 2020 (17 comments)

Multicore OCaml: March 2020 update - https://news.ycombinator.com/item?id=22727975 - March 2020 (37 comments)

Multicore OCaml: Feb 2020 update - https://news.ycombinator.com/item?id=22443428 - Feb 2020 (80 comments)

State of Multicore OCaml [pdf] - https://news.ycombinator.com/item?id=17416797 - June 2018 (103 comments)

OCaml-multicore now at 4.04.2 - https://news.ycombinator.com/item?id=16646181 - March 2018 (4 comments)

A deep dive into Multicore OCaml garbage collector - https://news.ycombinator.com/item?id=14780159 - July 2017 (89 comments)

Lock-free programming for the masses - https://news.ycombinator.com/item?id=11907584 - June 2016 (29 comments)

Lock-free programming for the masses - https://news.ycombinator.com/item?id=11893911 - June 2016 (4 comments)

OCaml 4.03 will, “if all goes well”, support multicore - https://news.ycombinator.com/item?id=9582980 - May 2015 (113 comments)

Multicore OCaml - https://news.ycombinator.com/item?id=8003699 - July 2014 (1 comment)

laylomo2• 5 years ago

dang

rackjack4• 5 years ago

Whoa, earlier than I expected (didn't think we'd see the PR within the year). Exciting!!

ijustboughtit2• 5 years ago

Last time (ca. 1 year ago) I tried learning OCaml, I ended up reading the beta version of Real World OCaml 2nd Ed. IIRC and for some reason all I remember now is that I didn't feel confident regarding the learning material.

The RWO-website https://dev.realworldocaml.org claims that the 2nd. Ed. has been published in Q4 2021, but I can't find it anywhere. Can somebody with experience tell me if RWO 2nd Ed. is the way to go?

avsm2• 5 years ago

(Coauthor of RWO here)

We are just finishing edits of a few chapters (the tooling, testing and GADT ones), and then it’ll be off to the publishers early in the new year. The online one is therefore pretty up to date.

There’s also a thread on the OCaml forums on this topic with more suggestions: https://discuss.ocaml.org/t/how-do-you-stay-productive-in-oc...

ijustboughtit2• 5 years ago

Thanks a bunch!

baby4• 5 years ago

since you're here, what do you think about starting a new OCaml project with esy as best practice today? I was working on https://o1-labs.github.io/ocamlbyexample/ and I was thinking of starting with that approach as it seems to provide a more modern experience than just opam/dune. (I read your book as introduction to OCaml and still struggled a ton with all the edge-cases of dune and opam. Even today I'm iffy every time I start working on some OCaml code again because I know I'll have to deal with opam/dune issues.)

jabl• 5 years ago

Is there an ELI5 writeup somewhere about what's so nice about algebraic effects?

I skimmed through some preprint on arxiv, and I got the impression that one could implement async/await as well as something like green threads if you select a suitable runtime. But beyond implementing other concurrency abstractions, why should I care?

classified• 5 years ago

https://speakerdeck.com/kayceesrk/effect-handlers-in-multico...

simplify2• 5 years ago

The short answer is, algebraic effects allow you to build your own abstractions like async/await. That's extremely powerful.

yawaramin7• 5 years ago

Check out https://overreacted.io/algebraic-effects-for-the-rest-of-us/

kcsrk• 5 years ago

Effect handlers are a foundation of all non-local control flow abstractions. Anything that requires fancy control-flow can be implemented with effect handlers. This includes green threads, async/await, generators (or iterators as some languages call it), but also higher-level applications such as algorithmic differentiation, probabilistic programming, model checking and fuzzing of parallel programs, etc.

A few of these applications were discovered only very recently. The aim is that this language primitive will inspire further discoveries for expressing useful abstractions elegantly.

Some of these ideas are summarised in this talk: https://m.youtube.com/watch?v=VEhkhxoGJSk

VitalyAnkh• 5 years ago

Oh the PR is so huge. Would this be difficult to review?

brabel• 5 years ago

It's probably a final PR from multiple smaller, reviewed ones, I would guess.

sideeffffect2• 5 years ago

I don't know much about OCaml, only Scala and Cats Effect/ZIO/Monix, but would appreciate if I could get a gist of what Multicore OCaml brings to the table.

Is there somebody who's familiar with both worlds and could compare them and explain how Multicore OCaml (and possibly the new effect system) work?

Thanks in advance!

yawaramin7• 5 years ago

Multicore OCaml works using a thread-safe generational garbage collector. Each domain (i.e. thread) gets its own minor heap, and the major heap is shared. Each heap is GCd independently. The domain-level minor heaps are restricted so different domains can't write to each other's minor heap. And the shared major heap is protected with more bookkeeping. More details in https://kcsrk.info/multicore/gc/2017/07/06/multicore-ocaml-g...

Scala of course just uses the set of GCs that are available on the JVM. OCaml's GC has an advantage in that it's optimized for quickly creating and collecting many small objects–perfect for functional programming. But some advanced JVM GCs probably come close to that.

The new effect system is basically 'resumable exceptions'. An effect is declared somewhat similarly to an exception. When the effect is 'thrown', it is handled by the nearest effect handler up the callstack. The difference from an exception is that once (if) it is handled, it actually resumes at the exact position in the code where it left off, not in a continuation created by a closure which is effectively what Scala and every other userland effect system does.

roguas• 5 years ago

LGTM +1

PR to Merge Multicore OCaml

Related Stories