Julia 1.8

https://julialang.org/blog/2022/08/julia-1.8-highlights/

by shakow • 4 years ago

254 93 4 years ago

mxkopy• 4 years ago

Feels like yesterday when I saw the same post for 1.7. Very happy things are coming along!

wiz21c3• 4 years ago

> A new tool has been added to provide insight into the way that loading dependencies contribute to the load time of a package.

That doesn't sound like an improvement to the dependencies loading... Which for me happens every time I change a struct and must reload Julia (which happens a lot when I'm prototyping code, which is what I do 90% of the time)...

adammarples• 4 years ago

This is very annoying but they recommend working in the REPL

wiz21c3• 4 years ago

Maybe I'm wrong, but I do work in the REPL. Each time I update a struct, I need to restart the REPL, hence reload dependencies... Of course, I could make my own sysimage, but it's a bit painful...

Is Julia "doomed" on this issue or is it just that they prefer to work on other use cases first ?

krull10• 4 years ago

It is useful in determining which packages are dominating a given package’s load times, allowing for a better understanding of where to invest effort to reduce runtimes. This has already helped in directing efforts towards a number of common dependencies that increase load times…

jarbus• 4 years ago

I saw this beautiful hack to get around this:

```

struct MyClass1

...

end

MyClass = MyClass1

```

Then, whenever you need to change MyClass, just add 1 to the version number of MyClass in those two or three instances, and the repl will just compile MyClass2 as a new class to be used for all future calls in the repl.

freemint2• 4 years ago

Not good if you want to also know stuff about the algorithms performance as that assignment is now accessing a global vs accessing a const.

t6jvcereio• 4 years ago

Had it ever occurred to you - if this was a real issue how would anyone ever get any work done? The answer is modules.

module X struct etc... end end #module

Change without reloading.

bobbylarrybobby2• 4 years ago

But doesn't defining a new module lead to code recompilation because all the functions in the module are new? Or does Revise.jl have some way around this?

amval• 4 years ago

That is not dependency loading.

About your issue, things you can do: - Put the struct inside another module. - Use this package: https://github.com/BeastyBlacksmith/ProtoStructs.jl . It gives you a simple @proto macro that you can prepend to the struct definition and makes all changes immediate.

wiz21c3• 4 years ago

It seems to address my problem of constantly restarting REPL (and yes, it triggers dependencies reloads; although not an issue with dependencies directly).

If it was stackoverflow, I would have accepted this answer with a kind comment :-) Thanks !

sundarurfriend8• 4 years ago

That doesn't sound like an improvement to the dependencies loading...

This gives the package authors a tool to basically "profile" the loading time of their package, which will help them optimize the loading time. So there _will_ be downstream improvement to package loading for us users too.

Which for me happens every time I change a struct and must reload Julia (which happens a lot when I'm prototyping code, which is what I do 90% of the time)

The workaround involving `MyClass = MyClass1` that a sibling comment mentions comes from [this Revise.jl manual page](https://timholy.github.io/Revise.jl/stable/limitations/). Revise in general enables a few different ways to work around this, and it's useful to integrate into your workflow (as much as it feels weird at first to go from full REPL to half REPL-half editor).

socialdemocrat2• 4 years ago

You can use a macro for your structs to make them changeable while prototyping. I think it is called @prototype or soemthing. You need a package for it. Google it.

leephillips5• 4 years ago

There is RedefStructs: https://juliahub.com/ui/Packages/RedefStructs/nrk8Q/0.1.2

and ProtoStructs: https://github.com/BeastyBlacksmith/ProtoStructs.jl

I’ve never tried them, but they’re advertised as doing the job.

thetwentyone• 4 years ago

Right, but there is a separate new feature which caches more lowered code.

https://julialang.org/blog/2022/08/julia-1.8-highlights/#imp...

The core devs also said that they are aiming to cache native code in the next year or so.

ChrisRackauckas• 4 years ago

> This gives the package authors a tool to basically "profile" the loading time of their package, which will help them optimize the loading time. So there _will_ be downstream improvement to package loading for us users too.

It lead to https://github.com/SciML/RecursiveArrayTools.jl/pull/217 . 6228.5 ms to 292.7 ms isn't too shabby.

Pepe1vo• 4 years ago

A few months back this article[0] was posted here (Why I no longer recommend Julia). Does anyone following the Julia community know if the situation is getting better?

[0] https://yuri.is/not-julia/

jakobnissen5• 4 years ago

In my view, yes, it's getting better, but only slowly. There has not been a decisive "let's fix this issue" moment.

Recent releases have had longer cycles with more thorough testing. Julia's CI has expanded. Static analysis is improving. The linter is improving. That all adds together to a more stable Julia with fewer bugs.

However, there has been made little progress on actually improving Julia's tests, or documenting abstract types, or on formalizing interfaces, nor have there been a big dedicated "fix bugs" push.

So yes: It's getting better, but slowly and not yet by leaps and bounds.

jessfyi2• 4 years ago

I mean if you go through them yourself and you'll see most (if not all) of those are closed. And the majority of the others dealt with composability issues between custom packages/libraries and not base Julia. Honestly it's a tooling + linter problem, and the lack of progress there (plus the general adversarial/celebratory tenor of that thread) made me pretty glad I've just moved onto Rust + python again. It's just too demoralizing from a dev's perspective to see.

Don't get me wrong, I love multiple dispatch and Julia's composability + speed make it a no brainer for science/quant econ teams to adopt (and using it to train models on TPUs is a actually a sneaky way to save time + $), but they've lost the "culture" war and I can't see the entrenched masses here or elsewhere changing their minds on it. Not sure whether it's for job security sake or they got sick of all the "Julia is faster than/will replace X" evangelism or Plots loaded too slowly the one time they tried it years ago or a combination.

nsajko2• 4 years ago

So you've decided to abandon a technology stack because of an unfavorable discussion thread on HN?

nsajko2• 4 years ago

Does anyone following the Julia community know if the situation is getting better?

Which situation exactly? The linked article lacks a salient point, so there's really no way to know what you're referring to.

Someone• 4 years ago

IMO, it has a clear salient point:

“My conclusion after using Julia for many years is that there are too many correctness and composability bugs throughout the ecosystem to justify using it in just about any context where correctness matters.”

jakobnissen5• 4 years ago

I disagree with that completely, and I'm deep into the Julia community. The article is very, very clear that the author considers Julia's level of bugginess to be unacceptable.

moelf2• 4 years ago

https://huijzer.xyz/posts/recommend/

sundarurfriend8• 4 years ago

Excited to see the effect of the precompilation improvements, saving more of the pre-compiled code. I've heard there's even more improvements in this direction with v1.9, and it seems like a good way to make significant progress regarding latency issues.

`status --outdated` is something I've wished for many times, nice!

Typed globals will, at the least, make life easier for people new to Julia (and those teaching them).

sundarurfriend8• 4 years ago

By the way, the allocation profiler visualization is a cherry on top of all the improvements to VS Code for Julia [https://www.julia-vscode.org/] that make for a really good development environment now. (See the JuliaCon talk "Julia in VS Code - What's New" [https://www.youtube.com/watch?v=Okn_HKihWn8] for details on others.)

I never thought I'd move away from Vim, but the ease of use and polish here make it worth switching (with maybe Neovim as the backend at some point), rather than try to recreate it in Vim.

clircle3• 4 years ago

Every time I try Julia on my computer, there is so much lag between input and output, I just go back to R.

clircle3• 4 years ago

I cannot edit my note, but to explain a bit more, I do know that Julia is a compiled language, and is fast for some workflows. But for my data science and statistics workflow, it is slower than R. And I have a pretty standard workflow: clean data, plot data, maybe fit a linear regression model or something like that.

moelf2• 4 years ago

it's called JIT, comparing to R, it probably would speed up some moderately complex calculation by 100x to 1000x, but if you don't have the type of workload, feel free to use R -- people should use whatever suits them the best!

bobbylarrybobby2• 4 years ago

The "right" way to use Julia is to connect to a REPL. Once you pay the cost of compiling, it's insanely fast. As long as you keep your REPL open your compilations will remain valid and cached, and you won't have to repay the cost of compilation until you restart your REPL (which you should do very rarely).

There's a well-known concept in Julia, [time-to-first-plot](https://discourse.julialang.org/t/time-to-first-plot-clarifi...), which encapsulates the large lag the first time you call a function. But there is no such thing as time-to-second-plot — the second plot is always fast.

clircle3• 4 years ago

Yes, I have heard that. I'm used to frequently restarting R to ensure that codes are reproducible and can run in a clean environment. That's a best practice in technical research that seems at odds with the Julia workflow.

bluenose69• 4 years ago

Startup time is not the only issue with Julia. Its error messages are long-winded and cryptic, and its manpages can be quite sketchy.

Even so, Julia is a good tool for intensive computation, and I think it will continue to replace Fortran for some such work.

Balance is the key. I use R all day long for data analysis, and don't really see Julia as competitive for such work. However, when it comes to numerical modelling, Julia competes well with Fortran and C/C++.

I think Julia should be thought of as a Fortran replacement for overnight/overweek jobs, not an R replacement for interactive work.

krastanov6• 4 years ago

YMMV. For me, Julia is a wonderful replacement of Python/numpy/scipy/pandas/Jax/Tensorflow/cython/sympy that I use mainly for interactive work. I am a fairly competent Python developer and know how to do idiomatic high-performance Python. I needed to learn new idioms and workflows when switching to Julia, but I am drastically more productive in it, both in interactive and in batch work.

adgjlsfhk16• 4 years ago

some of the error messages are about to get much better (see https://github.com/JuliaLang/julia/pull/46372).

I find Julia to be pretty good for interactive work, but there definitely are some rough edges that still need smoothing.

semi-extrinsic• 4 years ago

I'm kind of on the fence with Julia, never used it in anger. But to me it seems much more of a Matlab or Python replacement than a Fortran replacement? Meaning that (in my perhaps limited view) it's main usefulness is in making algorithm development much faster and smoother.

However, once a new very fast algorithm has been discovered and tested and everyone agrees it is good, someone will probably implement it in Fortran or C or even assembly and optimize the shit out of it.

One thing that supports this view is that there are several Julia packages that are wrappers around existing C/Fortran/C++ libraries, and basically no examples (that I know) of people porting existing libraries to Julia.

sundarurfriend8• 4 years ago

That is one of the things this update specifically addresses (and the next 1.9 will do more of). There's a tangible difference in the REPL, things feel snappier. I'm sure there will still be lags in the odd place, and there's still a first time compilation, but that compiled result is cached a lot more, and so when you restart session as you mention below, that cache will be preserved.

tpoacher• 4 years ago

Very welcome changes. Well done to the team.

The 'typed non-const globals' are a welcome syntax update for the sake of consistency with the rest of the language ... but it's worth pointing out that const globals effectively had the same effect, since you actually were allowed to redefine a const-labelled variable, and you were only prohibited from changing their type. Earlier versions of Julia silently ignored such redefinitions, but a warning was added later on whenever a redefinition of a const variable was detected. So I guess the only difference between the two now is that one will issue a warning and the other will not? I don't know if the two also have some hidden performance differences that are not obvious from this post. Regardless, still a small but welcome change nonetheless.

adgjlsfhk16• 4 years ago

The warning on redefining const globals is important. It's there because when you change a const global, that change may not get picked up in functions that use the variable (since you declared it as const). Using the type annotation prevents the compiler from saving the value of the global in other code and makes it do a lookup at runtime.

kristofferc• 4 years ago

Functions can inline global const values and these will not be updated when you change it in a way that emits the warning. So it is not the same, the warning is not there for no reason.

adenozine• 4 years ago

I’m waiting for interfaces in Julia 2.0…

I’m impressed with all that they’ve done, but for finding new talent and dealing with such a strange tool and with multiple dispatch being such an awkward paradigm compared to traditional OOP, I still don’t think it’s worth the trouble yet.

Additionally, everybody I’ve met thus far for interviews who knew Julia, also knew either Python or R, so it’s sort of a “why bother?” scenario.

freemint2• 4 years ago

Interfaces/Traits can be done with macros today.

mattkrause• 4 years ago

I did some dynamical systems stuff in Julia that absolutely blew Python and Matlab out of the water, both in terms of speed and ergonomics, so that's what baited the hook for me.

Multiple dispatch is odd. It took me a while to get used to it, but it turns out to be pretty useful for a lot of math-y things where there are speed-ups for special cases (symmetric matrix, etc) or you need to wrangle data into a common structure first (one variance, list of variances, covariance matrix, etc).

ssivark5• 4 years ago

IMHO multiple dispatch feels more awkward compared to OOP only because we’re far more used to the latter. And if that’s what you want to stick to, I guess you could program in that style by making the “object” the first method argument, and wrapping the functions along with the struct definition in a module.

Additionally, everybody I’ve met thus far for interviews who knew Julia, also knew either Python or R, so it’s sort of a “why bother?” scenario.

Well, if it isn’t substantially improving on a pain point that makes you really want to switch, then why bother? :-)

Eg: For a problem I happened to be working on yesterday, I probably could have reduced ~1000 lines of Python code to ~100 lines of Julia code (all because of multiple dispatch… not just by calling some library) and cut through a lot of the not just tedious but also error-prone boilerplate (lots of little indexing details).

leephillips5• 4 years ago

An odd take by the GP, as the usual version of OOP (I assume “traditional OOP” doesn’t mean Smalltalk, but the class-based horror used in Python or Java) is an awkwardly constrained, special case of multiple dispatch where we can only dispatch on the first argument.

sidpatil• 4 years ago

And if that’s what you want to stick to, I guess you could program in that style by making the “object” the first method argument, and wrapping the functions along with the struct definition in a module.

You might not even need to do that. This package was released recently, implementing OOP: https://github.com/Suzhou-Tongyuan/ObjectOriented.jl

Buttons840• 4 years ago

OOP is nice to stumble through because what's possible for a given object is spelled out. Your IDE can suggest a list of possible methods and you pick the one that sounds good. Whereas in Julia there's an infinite number of potential functions that can operate on the value you have, and I haven't seen a good way to list them.

sharikous• 4 years ago

macOS on ARM is still tier 2 and not tier 1?

Not related to this but I think Julia is too ambitious too succeed, at least in a normal timeframe. Languages like Rust and even Python managed to improve a lot more compared to their goals than Julia compared to the goal "a modern language, a modern top edge research level scientific library rewritten from scratch".

I enjoy the community that is backing it, really, but there is still no solid core. A lot of research level cool functions and algorithms, yes. But you still suffer from lagginess, inexact calculations and language idiosyncrasies. In short too much dispersion and divergence in this project. It's almost like this language was designed by academics...

adgjlsfhk16• 4 years ago

MacOS on ARM is technically tier 2, but we expect to promote it to tier 1 within a month or so. Basically we just want to make sure that 1.8 is as stable on M1 as we think it is. Until you have thousands of users testing everything, it's hard to declare confidently that everything works perfectly.

jakobnissen5• 4 years ago

You might be right that Julia's sky high ambitions make it seem like a failure no matter what it does. I mean, it could have strived for just being a pleasant dynamic language with a fast amortized running time, and then it would have succeeded in 2018.

But is it not better to both achieve you mediocre goals and also have extremely ambitious goals that may still be achieved, than merely having mediocre goals?

I don't really agree that Julia right now doesn't have anything to show for its huge amibitions. In my opinion it's already the best language for science by quite a large margin. I understand it still lacks some tooling, reliability and deployment options to be attractive for many industrial clients (though Julia is already making inroads in industry - I have an old friend from college who uses Julia in production). It's also understandable that some people used it and found the latency unacceptable. The thing is - there is no reason to believe that tooling, reliability and deployment options won't come around. The stuff is being built. It's just happening "too" slowly right now.

leephillips5• 4 years ago

You may have a particular measure of success in mind, but it seems to me that Julia has already succeeded: it is a new language for scientific computing that met its goals of providing high level abstractions and an expressive syntax while not compromising on performance. Its public release was in 2012 and it’s already one of only four languages used in high-performance scientific computing. Of course, in many areas its ecosystem is immature; but in others, such as differential equation solving, it’s a leader.

Mikeb852• 4 years ago

This. I was hoping Julia would be a quicker and more pleasant R. Instead it's turning into a super ambitious language that wants to be as fast as C++ but that makes it slightly less pleasant and useful than R. And if I need C++ speed I can just write C++...

jakobnissen5• 4 years ago

And if I need C++ speed I can just write C++...

Maybe I'm being pedantic here, but surely you can't "just write C++". If you could, why would you ever use R, which is way slower, less backwards compatible and can't be shipped as a binary?

I'm guessing you use R because it's easier to use than C++, interactive, and quicker to get stuff done in. That sounds to me like a list of criteria where Julia should aspire to beating R - if we can.

throwawaymaths• 4 years ago

What the heck?

Before making a claim like this you better define what you mean by "succeed" and what you mean by "normal timeframe".

Julia has been around since 2010-ish. The equivalent timeframe for python was late-90s.

The federal reserve bank uses (used?) Julia to model economics... Jokes about whether that is a succeed or fail for society at large, aside, it's hard to argue that adoption by at least one very significant player doesn't constitute success for a PL.

martinsmit2• 4 years ago

I don't think such a defensive response is necessary. Julia's claims have been rather lofty since the beginning and the degree to which they have been achieved is different to everyone.

Without defining more precisely what succeeding and the timeframe means it is difficult to know what OP was after but the fed example stands well on its own.

andrewl-hn• 4 years ago

Rust is keeping macOS Arm as Tier 2 as well, mostly not because of lack of interest but because CI platforms can’t support all the demand for Arm builds. MacStadium, Azure, AWS and others keep adding extra capacity but the build queues are still too long.

So you either promote Arm Mac as tier 1, but now every PR takes extra hours to get merged, or you keep it at tier 2 for now and only build on Arm, say, once a day. Rust developers choose the later, because with their commit traffic they can’t afford such delays.

As the build machines capacity increases arm macOS will be promoted to Tier 1. Don’t read too much into tiers. Tier 2 is still pretty well supported from a language user perspective. Julia situation is pretty much the same.

adgjlsfhk16• 4 years ago

We actually have a stack of M1 mac minis on top of a refrigerator to run CI. It's working pretty well for how janky it is.

blindseer• 4 years ago

Great improvements! Congrats on the release.

I don't like the ergonomics of `mutable struct` vs (immutable) `struct`. Adding `const` seems like another step in the wrong direction. Right now, there's `struct` + `Ref{T}` fields for having a immutable struct with fields that can be mutated VS `mutable struct` + `const` fields for a mutable struct with fields that can't be mutated. And defining mutability at the `struct` level is kind of painful.

Sometimes I want to accept user input in a type checked manner (i.e. not storing them in dictionaries and using typed struct fields) and then pass that struct into a core algorithm where execution needs to be fast (i.e. aligned memory, immutable structs etc). Rust does this by making the mutability declared at an individual function level or during access to the struct value. In Julia, I have to decide ahead of time and that changes how users would use a library of mine.

I also dislike that syntax `a.b += 1` is not allowed for fields in immutable structs. I would love it if `@immutable_mutate a.b += 1` de-sugared to `a = A(b = a.b + 1, a...)` or the equivalent. If this existed I think I would just use immutable structs ALL the time. If a macro for that existed in Base Julia that would be amazing. (I don't want to use third party libraries for this.)

I'm (cautiously?) optimistic that Julia will be absolutely awesome in like 5 years. Currently, 90% of my work in Julia is painful because of poor developer workflow / tools. But when I need to do the remaining 10% (performance / optimization of code), Julia is amazing. Right now, it is probably easier to write code in Python (with type hints) and drop down to Rust or Go whenever performance is needed, than to start with Julia in the first place.

borodi2• 4 years ago

The syntax sugar you are looking for "mutating" the immutable stricts is basically what Accessors.jl does. I don't really see a reason to bring it into Base Julia if it doesn't need special things to work. The compiler can and will optimize out the stack allocation

sebow2• 4 years ago

Wish julia would get some decent packages on main distros at least for the latest stable release. The tooling is slowly but surely improving, but that's hard to do when you're always 1/2 versions behind the latest release.

vchuravy• 4 years ago

Take a look at `juliaup`. It's allows you to manage multiple Julia versions and makes you independent of the distribution.

sebow2• 4 years ago

Wow, thanks a lot. I'm kind of surprised I didn't knew this because I recall searching quite a bit for such a tool/script. In my defense it was around when 1.6 got released ... if anything this shows how fast the julia community grows(or how slow I keep up).

Julia 1.8

Related Stories