Sukera

Could you expand on how these wraparound bugs happen in Rust? As far as I know, integer overflow panics (i.e. aborts) your code when compiled in debug mode, which I think is often used for testing.

No, you can write the kernels directly in Julia using KernelAbstractions.jl [1].

[1] https://juliagpu.github.io/KernelAbstractions.jl/stable/

Because adding the top limbs of two encoded numbers would overflow too soon. If you set both to 2^63 for example, they overflow immediately. Might be fine for wraparound arithmetic, but not in general.

For aggregation-like things, the interesting properties are often about the properties of the accumulation function, and not the entire aggregation, which should then be correct by extension. So for your `sum` example, you'd use PBT to test that your `+` works first, and only then coming up with things that should hold on top of that when repeatedly applying your operation. For example, once you have a list of numbers and know it's sum, adding one additional number to the list and taking the sum again should lead to the same result as if you had added the number to the sum directly (barring non-associative shenanigans like with floating point - but that should have already been found in the addition step ;) ).

There's a bunch of these kinds of patterns (the above was inspired by [0]) that are useful in practice, but unfortunately rarely talked about. I suppose that's because most people end up replicating their TDD workflows and just throwing more randomness at it, instead of throwing more properties at their code.

[0] https://fsharpforfunandprofit.com/posts/property-based-testi...

Throughout the rest of the paper, we assume an adversary who at some point managed to obtain the private identity key IK, the phone number pn, the API username unA, and the password pwA.

This is basically fully owned access already, right? If you have everything that is required to authenticate to the Signal servers, of course you can register new devices.. In that scenario, there's not much you can do to protect against this, and even the proposed countermeasures are conceivable to work around as an attacker. Signal also seems to view the paper like that:

We disclosed our findings to the Signal organization on October 20, 2020, and received an answer on October 28, 2020. In summary, they state that they do not treat a compromise of long-term secrets as part of their adversarial model. Therefore, they do not currently plan to mitigate the described attack or implement one of the proposed countermeasures.

  Location: EU, Austria
  Remote:  Possible
  Willing to relocate: Yes, for some offers
  Technologies: C, C++, Julia, Python, SQL, Linux, Git/Github, Docker, TrueNAS, Wireguard, CoreDNS, AVR, ARM, RISC-V
  Résumé/CV: ~3 years DevOps (mostly Python/Proxmox/TrueNAS); Email for more extensive CV 
  Email: valentin (at) bogad (dot) at   please mention "HNMay2024" somewhere in the email
  LinkedIn: https://www.linkedin.com/in/vbogad
  Site: https://seelengrab.github.io
  GitHub: https://github.com/Seelengrab

It doesn't only cover pure hobby projects:

(10c) the mere fact that an open-source software product receives financial support by manufacturers or that manufacturers contribute to the development of such a product should not in itself determine that the activity is of commercial nature.

(10) Accepting donations without the intention of making a profit should not be considered to be a commercial activity.

(10c).. for the purpose of this Regulation, the development of products qualifying as free and open-source software by not-for-profit organisations should not be considered a commercial activity as long as the organisation is set up in a way that ensures that all earnings after cost are used to achieve not-for-profit objectives.

See https://berthub.eu/articles/posts/eu-cra-what-does-it-mean-f...

50% of the top 100 most starred, i.e. used, packages are not representative of the entire community. Not to mention that the vast majority of those SciML packages is developed by a relatively small group of people, compared to the rest of the ecosystem. If all a potential user cares about is SciML, good for them! I've repeatedly said that users not looking for SciML are left behind.

Yes, SciML is doing good. I'm not denying that, and never have. Still, the rest of the community/package ecosystem is not good at catching up - which is what I'm criticizing.

with the interface checking, implicit interface support was turned mostly into explicit interface support where packages and types opt-in via defining traits (this is still continuing to evolve but is mostly there)

What interface checking? Base doesn't provide any such facilities. Package extensions are still ad-hoc constructions on a package by package basis; there is little to no consistency.

Of course, not every person or every package has changed, but what I described here are major interface, style, and cultural changes to some of the most widely used packages since 2020.

And none of that has landed in Base, none of that is easy to find out about/discoverable for non-SciML users. SciML is not all of Julia, and certainly not Base or packages outside of SciML. Please don't frame this as if SciML was the only thing that mattered here.

But often the packages also talk through simple accesses of values like `X.someproperty`. I've seen situations where maintainers would add and remove these properties and break their own other libraries. Better "enforcement" of these types of things - however that would look like - would be a huge improvement for the time sink that is maintaining a Julia package.

I think this is a cultural issue to a large degree. I think even if there were better enforcement, it's just a question of coding of discipline to then actually not break something. If it were easier to anticipate what sort of breakage a change could entail (say, by making clear & documented _by the core language_ what is considered a breaking change, and then actually not even doing "technically breaking" changes and using deprecations instead), this COULD change.

That being said, that requires a very different development workflow than what is currently practiced in the main repos of the language, so there's quite a bit of an uphill battle to be had (though such issues happen there all the time, so solving that would actually help here the most, ironically).

There is no rebuttal because nothing much has really changed culture wise. Sure, the various @inbounds issues and concrete bugs that are mentioned in Yuris post have mostly been addressed, but the larger point (that is, "what can I actually expect/get guaranteed when calling a given function?") definitely hasn't been, at least not culturally (there are parts of the ecosystem that are better at this, of course). Documentation of pre- and postconditions are still lackluster, PRs trying to establish that for functions in Base stall for unclear reasons/don't get followups and when you try to talk about that on Slack retorts boil down to "we're tired of hearing you complain about this" instead of trying to find a systemic solution to that problem. Until that changes, I have large doubts about Yuris post losing relevance.

My own efforts (shameless plug, https://github.com/Seelengrab/PropCheck.jl for property based testing inspired by Hedgehog and https://github.com/Seelengrab/RequiredInterfaces.jl for somewhat formalizing "what methods are needed to subtype an abstract type") are unused in the wider community as far as I can tell, in spite of people speaking highly of them when coming across them. I also don't think Kenos InterfaceSpecs.jl is the way forward either - I think there's quite a lot of design space left in the typesystem the language could do without reaching for z3 and other SAT/SMT solvers. I personally attribute the lack of progress on that front to the lack of coherent direction of the project at large (and specifically not to the failings of individuals - folks are always very busy with their lives outside of Julia development/other priorities). In spite of the fact that making this single area better could be a big boon with more traditional software engineers, which are very underrepresented in the community.

Lots of languages can mmap, it's not exclusive to C++ or C. The JSON package the Julia code is using seems to mmap too, for example.

(2020)

It would be great if you could share such an example work load - such big examples are often few and far between, even though they are VERY good for debugging compilation performance.

Can you be more specific? As is, it's hard to follow what the issue was that you encountered.

Doesn't look like it https://developers.cloudflare.com/cache/how-to/enable-crawle...

There's a lot more than release notes going into a release - we've had 3 release candidates for a reason, and those regressions/bugs need to be fixed first.

For me personally, I just think it's really fun to write julia code. Granted, I'm neither machine learning nor physical science, but the fact that I can go through the whole stack and choose an abstraction that's right for the problem at hand (Metaprogramming? Regular struct-based abstractions? External program? LLVM optimization? Inline assembly?) and still being able to understand what's going on while getting good performance at the same time, is just magical to me. Maybe that's not for everyone, but to me the ratio of dev time to run time is just really, really good.

The authors have previously also shown off that this can do 3D visualization in real time: https://twitter.com/gabrielweymouth/status/16486827416201953...

There will also be a release blogpost, highlighting the new stuff. The release will likely come with that.

I think Chris was referring to the leaky abstractions python grew on https://www.youtube.com/watch?v=qCGofLIzX6g

Are you afraid that you're going to "inherit" the issues the python ecosystem has through your goal of full compatibility? Or is Mojo more like Numba, in that only parts of python will actually be supported for full acceleration? At least your docs[1] seem to say so..

--

[1] https://docs.modular.com/mojo/why-mojo.html#intentional-diff...

Also, previously discussed: https://news.ycombinator.com/item?id=16084575

That definitely sounds like cause for concern! You seem to have spent some time on this, can you share an example?

Ah, I see, you're referring to linting capabilities. Yes, that's unfortunately a current limitation of LanguageServer.jl right now. I don't think it interfaces extensively with the compiler for type inference in a way that it crossreferences callers in the current code base, though it certainly could. That capability just hasn't been implemented yet I think.

Regardless, that's orthogonal to being type safe.

That's surprising to me, as @inbounds should at worst be a noop - it shouldn't ever result in slower code. I sadly can't benchmark right now (no suitable device for the next few days), but I'll see if I can get back to this after that.

I have to annotate all structs and function input types.

Can you elaborate on that? Argument types in function definitions do not help the compiler infer types - you can leave them completely untyped. It's really only required in struct fields, so that the compiler can know the size of your struct and doesn't have to box everything.

Interesting comparison! The criticism of julia is certainly valid, especially in terms of documentation. However, I wouldn't say that core language documentation should necessarily be geared towards any specific field.

In the implementation of the julia code there is a slight performance bug though (not sure if the authors submitted the article - the username certainly suggests it):

    function likelihood(o, a, b, h, y2, N)
        local lik = 0.0
       for i in 2:N
        @inbounds h = o+a*y2[i-1]+b*h
         lik += log(h)+y2[i]/h
        end
        return(lik)
    end

The source code is linked at the very top, under the heading "Computation Speed": https://modelsandrisk.org/appendix/speed_2022

Yes, the size in memory is definitely an issue. Do keep in mind that right now, the julia executable more or less just has to load all of LLVM as a shared object into memory, since we might compile stuff at any time. That alone adds ~84M of overhead on my machine for the current libLLVM14 shared object. There's probably some other stuff I haven't thought of OTOH. The (non-minified) runtime itself is 244K at the moment, which can probably be slimmed down further. There are currently efforts underway to statically compile more stuff and hopefully introduce a more traditional static linking approach, but it takes time & lots of effort.

Still, a large chunk of the memory being spent is not in the code that's produced or in the allocations happening in the code, but libraries that are loaded but not required. It's one of the downsides of focusing on interactivity first. The benchmarks in benchmarksgame don't reflect that, which I guess is up to interpretation/what's required - if the total amount of memory is a concern it's an important figure, if you only care about your core algorithm not having inherent allocation/memory problems, you probably won't care about the compiler chain as much.