Julia 2.0 isn’t coming anytime soon, and why that is a good thing

https://logankilpatrick.medium.com/why-julia-2-0-isnt-coming-anytime-soon-and-why-that-is-a-good-thing-641ae3d2a177

45 28 4 years ago

logankilpatrick.medium.com

I am interested to hear if folks think a 2.0 release would make Julia more appealing.

There are of course many breaking changes that could improve the language itself, but in my opinion the attention of maintainers is best spent on improving the ecosystem--while a 2.0 release may be exciting, I've been very pleased to see the language steadily improve with 1.x releases.

bobbylarrybobby3• 4 years ago

Yes, I'd love to see a revamp of the type system with real interfaces instead of the single inheritance with abstract types. I also think packaging and imports need a lot of work — for god's sake, there is no "import adjacent file" mechanism other than a C-style `include`, which just copies and pastes code from another file into the current file.

contravariant3• 4 years ago

Not sure if interfaces work as you'd expect them to in Julia. Values don't have methods for instance, you just define functions for the types.

So what would an interface look like? Best I can figure out an interface is equivalent to a requirement that a couple of methods are implemented and that a bunch of fields exist, which sort of gets taken care of by the duck-typing, the only problem is that it's not documented in code.

One approach that I'd been thinking about would be defining a method IMyInterface that returns a tuple of fields and methods defined by the interface and saying a type implements that interface if there's an implementation. Technically this performs the same job, but unfortunately it doesn't seem like it'd be easy to use without hefty syntactic sugar.

What do you imagine interfaces would look like in Julia?

jarbus• 4 years ago

It's been very exciting to follow progress on Julia, it's quite a wonderful language. Thank you for all the fantastic work. While I understand 2.0 making sense in terms of breaking changes, I think a 2.0 release would need a major improvement in one area to be worthy of the title, and get more users interested. For example, if there was an update that made compilation significantly faster, or an update that empowered users to never have to restart the REPL with Revise, I think that would be a "defining" thing that would interest a lot of people, more so than breaking changes.

wnoise2• 4 years ago

Well, yes, because there are several breaking changes that should happen, but can't before then, because of the adherence to semver.

blindseer• 4 years ago

I'd like to see a way to specify interfaces / traits, in the base package itself.

I would also like to see a better test suite, with the ability to run a single test if I want to. Test Driven Development on a large project is a nightmare (although it's gotten a little better recently). I still can't believe how people contribute to Base Julia, because running the tests takes forever.

Lastly, better support for debugging in Neovim, Vim, Emacs etc. Better linting tools too. There's a push in other languages to write core language tooling in Rust or Zig, I think Julia would benefit from that approach (there's `juliaup` but I'd like to see more official tools similar to this).

Personally I'd like to see SetField.jl or Accessors.jl in Base. I'd also like to see improvements in the ergonomics of using immutable structs. In Julia, users should want to use immutable structs all the time, and ONLY if they need reference semantics they should want to reach for mutable structs. I think the naming and the sematics here is REALLLYY confusing when trying to get a team to write performant code. I can't tell you how many times I've had this kind of conversation with various members in my team.

Me: Create a struct for holding all that related data together

F: Okay, let me create a struct. - Refactors code base for a day - Realizes the data is being updated - Uses mutable structs

Me: Don't use mutable structs unless you want reference semantics

F: What even is reference semantics? - Tries to refactor by removing the `mutable` keyword, code breaks when doing `a.b += 1`, writes messy code like `a = A(B(a.b + 1, a.b.c, a.b.d, ...), a.c, a.d, ...)`.

Me: Ugh, why didn't you use `SetField` instead?

F: No unnecessary extra dependencies to reduce complexity.

Project Manager: Where are we with this task? Just use mutable struct and move on.

Me: Sigh. Fine.

On a non-language side of things,

1) it would be nice to have a bigger focus on teaching how to write performant Julia code. Currently, most if not all the material is geared toward someone who already knows how to write performance code (in C++ for example) and showing them the "Julian" way to do it.

2) I'd like to see a usability survey done from Julia users to see what features should be tackled next.

ChrisRackauckas6• 4 years ago

Personally I'd like to see SetField.jl or Accessors.jl in Base. I'd also like to see improvements in the ergonomics of using immutable structs. In Julia, users should want to use immutable structs all the time, and ONLY if they need reference semantics they should want to reach for mutable structs. I think the naming and the sematics here is REALLLYY confusing when trying to get a team to write performant code. I can't tell you how many times I've had this kind of conversation with various members in my team.

I agree that one of these should be standardized to be a Base functionality. But that won't need a v2.0 for that to happen: it can happen in a v1.x.

it would be nice to have a bigger focus on teaching how to write performant Julia code. Currently, most if not all the material is geared toward someone who already knows how to write performance code (in C++ for example) and showing them the "Julian" way to do it.

The issue is that many of these tricks are actively being addressed, for example with escape analysis to reduce allocations. So what really needs to be taught is how to profile and understand what the code is doing, and how to effect it to do what one wants for performance. That then increases the barrier to entry. I think we shouldn't be teaching people "how to write performant Julia code" much more than is already out there though: since most of the tricks are really just workarounds for missing optimizations, those compiler optimizations should just get added (and there's some work showing it is possible).

rovr138• 4 years ago

While I appreciate it as a user, I hope that when it happens they learn from Python and, surprisingly, PHP.

PHP took over 10 years between 5 and 7. A bit push to upgrade from 5.6 was around 2017/2018 when support would stop. It was a mess because most developers simply didn't care or didn't know enough to handle this.

Now they're doing yearly releases (7.2, 7.3, 7.4) but also PHP 8 was released already (along with 8.1 and 8.2).

waffletower4• 4 years ago

I'd like to use Julia for audio and image processing. The language and library support is well suited for my applications. However, the current state of the PackageManager is reason for Julia to be a non-starter for me. I looked at the linked release notes with hope that PackageManager improvements would be slated for 1.9. The startup time for standalone executables is unacceptably large and baffling given the computing efforts required to compile and package them. I reached for Julia to improve upon Python and the start up costs far exceed the available system time for gains. The mere existence of the acronym TTFP indicates that the Julia community has neither an appropriate frame of reference nor prioritization of developing Julia into a truly general programming language -- Julia remains far from production ready. Computing generality goes far beyond the habitual use of: input, analysis, plot. Swift and Rust ought to be benchmarks for executable startup. From my perspective, Julia runtimes ought to be able to be constructed and frozen in dynamic shared libraries such that startup can approach that of Swift and Rust. Julia developers perhaps can look to GraalVM for inspiration. A long wait for Julia 2.0, particularly if standalone executable compilation would be addressed, can not possibly help the adoption of Julia from my perspective.

ChrisRackauckas6• 4 years ago

A long wait for Julia 2.0, particularly if standalone executable compilation would be addressed, can not possibly help the adoption of Julia from my perspective.

It's the opposite. Julia 2.0 would not bring standalone executable compilation, but instead would require compiler work for language changes. The whole point of saying "we don't need to do a 2.0 right now" is that folks on the compiler team believe working on things like standalone compilation and improved caching of LLVM code (i.e. fixing TTFP) are a better use of time than changes to language semantics (i.e. a 2.0).

waffletower4• 4 years ago

If PackageManager changes can be addressed in 1.10, 1.11+ etc., I would be elated. Again, TTFP does not need to 'addressed' -- the acronym instead needs to be eliminated from usage -- a shift to use of 'startup-time' can help a change of mindset to open the Julia community up to a broader range of applications and adopters. Otherwise, Julia simply continues to feed its currently narrow niche of applications.

joshjob422• 4 years ago

Can I ask what about your use case would involve starting Julia and importing packages over and over?

At least for my workloads, I often write a small .jl script that imports everything I want and then runs some functions on a list of files, for instance, or I just do my work in a Jupyter notebook, and I pay the load cost once for hours or days of computation.. Annuitized over the entire time I might use the kernel the startup time is basically nothing.

I'm not saying the load time should be a lot lot faster! That would be awesome! But I see folks complain about this and I'm wondering if they really need to be booting a new kernel constantly or if they could adapt how they work a little bit and get their desired result. (For instance, by running a .jl script instead of a bash script that calls Julia or something.)

waffletower4• 4 years ago

Currently, the adaptation cost is too high. The ecosystem that I work in has many small, distributed, fast-startup applications. While I could create a fairly complex client server application written in Julia to work with its limitations, and write compact fast startup applications to communicate with it, the complexity and time investment is undesirable. Again, to use Julia here one must bend to the ecosystem's weaknesses. I think the community truly doesn't understand what a general purpose language is. If you compile with LLVM startup really ought to be a non-issue. Look at the startup of GraalVM, and they began with Java bytecode.

jakobnissen• 4 years ago

In my last job, I built a data pipeline with Snakemake. Snakemake is a workflow manager where each command (often hundreds) is called from the shell. That was excruciating to do with Julia, and I can think of no alternative.

2OEH8eoCRo0• 4 years ago

Can we get a 5 year freeze on languages? There is too much fragmentation in this industry.

Julia 2.0 isn’t coming anytime soon, and why that is a good thing

Related Stories