staticfloat

I love that this article includes a test program at the bottom to allow you to verify its claims.

This is a very old answer about the M1, but yes what you’re saying is possible: https://stackoverflow.com/a/67590869/230778

While you can cause the compiler to run longer to squeeze the binary size down, the compiler has a baseline number of compiler passes that it runs over the IR of the program being compiled. These compiler passes generally take time proportional to the input IR length, so a larger program takes longer to compile. Most compiler passes aren't throwing away huge amounts of instructions (dead code elimination being a notable exception, but the analysis to figure out which pieces of dead code can be eliminated still is operating on the input IR). So it's not a perfect proxy, but in general, if the output of your compiler is 2MB of code, it probably took longer to process all the input and spit out that 2MB than if the output of your compiler was 200KB.

Just a small piece of feedback, it looks like there's a small typo on the last paragraph of the page:

Interviews can be scheduled (and rescheduled) interviews around the clock, 7 days a week.

I believe the second `interviews` should be omitted.

Other posters have pointed out that this is incorrect, but I wanted to give a bit of intuition as to how signals can be received when they are below the noise floor.

First, as a definition, below the noise floor means that the power of my signal at any given time is smaller than the power of the ambient noise in my channel, and usually this implies that you're only interested in a particular segment of frequency spectrum (e.g. within the 10MHz band centered at 1.8GHz). If we were doing a simple frequency-shift keying or amplitude-modulated signal, once the noise power exceeds the signal power, there is basically no hope of recovering anything useful, as those are both demodulation schemes that rely upon obtaining instantaneous estimates of the frequency or amplitude of the signal of interest.

However, spread-spectrum methods make a time/frequency tradeoff, where the signal of interest is "spread" across multiple points in time and frequency. A very simple example of this is to say "if I want to transmit a 1, instead of transmitting one cycle of a sinusoid at 18.GHz, I will transmit 10 cycles". Then, at the decoder stage, you average across 10 cycles of your carrier in order to detect whether a signal was sent or not. By doing this averaging across time, you get a 10x gain versus the noise which is expected to cancel itself out as often as not.

True spread-spectrum techniques are more advanced than this, they actually use wave shapes that are more complicated than just a sinusoid to make it easier to detect when they start and stop (whereas with a sinusoid there's a fair amount of ambiguity if you shift one period to the left or right) but the fundamental idea of averaging across time is the same.

Through this mechanism we are able to rescue out signals from far below the noise floor, although it reduces your maximum transmission rate. When dealing with digital radio systems we can even rescue out signals from below our quantization floor, although not too much lower, as eventually you lose the ability to average out a signal that is fluctuating by significantly less than a single bit.

Whenever I talk about making tradeoffs in transmission speed to aid in reception, I am reminded of the ELF systems in submarines [0]. While they did not use spread-spectrum techniques, (they just jumped between two frequencies, 76Hz and 80Hz) they still correlated across time to boost up their effective SNR. [0] https://en.wikipedia.org/wiki/Communication_with_submarines#...

I ran into an issue like this in my first ever job! I accidentally filled up a cluster with junk files and the sysadmin started sending me emails saying I needed to fix it ASAP but rm wouldn’t work. He taught me that file truncation usually works when deletion doesn’t, so you can usually do “cat /dev/null > foo” when “rm foo” doesn’t work.

Yes, that’s what it means for me. I’ve never heard someone use the word arbitrary to mean anything other than “a random choice”, or even “a poorly thought-out choice”.

My professors in grad school explicitly discouraged use of that word anywhere in technical writing, as they felt it would immediately give the reader the impression that the actions taken in the research were not thought through. Example: “This new technique enables arbitrary manipulations of data” should instead be replaced by something like “this technique enables a wide range of manipulations of data”.

I really like the integration of lower-level memory control in a superset of Python. Trying to maintain compatibility with such a large and varied ecosystem is a daunting task, and has the opportunity to be very valuable for many people who don't want (or are unable) to move away from the Python ecosystem. Kudos to you all for tackling such a difficult problem! Good luck, I look forward to seeing how you guys help to increase efficiency across the board!

It is a little disappointing that they're setting the bar against vanilla Python in their comparisons. While I'm sure they have put massive engineering effort into their ML compiler, the demos they showed of matmul are not that impressive in an absolute sense; with the analogous Julia code, making use of [LoopVectorization.jl](https://github.com/JuliaSIMD/LoopVectorization.jl) to automatically choose good defaults for vectorization, etc...

    julia> using LoopVectorization, BenchmarkTools, Test
           function AmulB!(C,A,B)
               @turbo for n = indices((C,B),2), m = indices((C,A),1)
                   Cmn = zero(eltype(C))
                   for k = indices((A,B),(2,1))
                       Cmn += A[m,k]*B[k,n]
                   end
                   C[m,n]=Cmn
               end
           end
           M = K = N = 144; A = rand(Float32, M,K); B = rand(Float32, K,N); C0 = A*B; C1 = similar(C0);
           AmulB!(C1,A,B)
           @test C1 ≈ C0
           2e-9*M*K\*N/@belapsed(AmulB!($C1,$A,$B))
    96.12825754527164

Perhaps they have more extensive examples coming that showcase the capabilities further. I understand it's difficult to show all strengths of the entire system in a short demonstration video. :)

EDIT: As expected, there are significantly better benchmarks shown at https://www.modular.com/blog/the-worlds-fastest-unified-matr... so perhaps this whole discussion truly is just a matter of the demo not showcasing the true power of the system. Hopefully achieving those high performance numbers for sgemm is doable without too much ugly code.

The Julia correctness issues are things like basic math functions, called in simple normal ways, returning wrong numbers.

If you have an example of this, I'd be interested in tracking it down. I didn't see this in Yuri's blog post, nor am I aware of any egregious examples of this right now.

Small correction; it’s DisplayLink, not DisplayPort. You have to use DisplayLink because they do some part in software as a part of their driver (which you must download and install) whereas the native Apple stuff has some internal hardware limitation.

I run three 1080p monitors off of my M1Pro, and I don’t notice any heavy CPU load from whatever software load the driver may be adding.

We actually use this in our CI system to limit write access outside of the build environment’s build folder.

You can see some Julia code that generates the sandbox config rules here: https://github.com/JuliaCI/sandboxed-buildkite-agent/blob/ma...

This linear scaling per-core doesn't match my experience with using the AMX co-processor. In my experience, on the M1 and the M1 Pro, there are a limited number of AMX co-processors that is independent of the number of cores within the M1 chip. I wrote an SO answer exploring some of the performance implications of this [0], and since I wrote that another more knowledgable poster has added more information [1]. One of the key takeaways is that appears to be one AMX coprocessor per "complex", leading us to hypothesize that the M1 Pro contains 2 AMX co-processors.

This is supported by taking the code found in the gist [2] linked from my SO answer and running it on my M1 Pro. Compiling it, we get `dgesv_accelerate` which uses Accelerate to solve a medium-size linear algebra problem, that typically takes ~8s to finish on my M1 Pro. While running, `htop` reports that the process is pegging two cores (analogous to the result in my original SO answer on the M1 pegging one core; this supports the idea that the M1 Pro contains two AMX co-processors). If we run two `dgesv_accelerate` processes in parallel, we see that they take ~15 seconds to finish. So there is some small speedup, but it's very small. And if we run four processes in parallel, we see that they take ~32 seconds to finish.

All in all, the kind of linear scaling shown in the article doesn't map well to the limited number of AMX co-processors available in Apple hardware, as we would expect the M1 Max to contain maybe 8 co-processors at most. This means we should see parallelism step up in 8 steps, rather than 20 steps as was shown in the graph.

Everything I just said is true assuming that a single processor, running well-optimized code can completely saturate an AMX co-processor. That is consistent with the tests that I've run, and I'm assuming that the CFD solver he's running is well-written and making good use of the hardware (it does seem to be doing so from the shape of his graphs!). If this were not the case, one could argue that increasing the number of threads could allow multiple threads to more effectively share the underlying AMX coprocessor and we could get the kind of scaling seen in the article. However, in my experiments, I have found that Accelerate very nicely saturates the AMX resources and there is none left over for future sharing (as shown in the dgesv examples).

Finally, as a last note on performance, we have found that using OpenBLAS to run numerical workloads directly on the Performance cores (and not using the AMX instructions at all) is competitive on larger linear algebra workloads. So it's not too crazy to assume that these results are independent of the AMX's abilities!

[0] https://stackoverflow.com/a/67590869/230778 [1] https://stackoverflow.com/a/69459361/230778 [2] https://gist.github.com/staticfloat/2ca67593a92f77b1568c03ea...

My guess is that it’s due to git looking at its own base name to figure out which command it’s supposed to run as, kind of like busybox does.

To add to Keno's sibling comment, Julia, as a JIT compiler, essentially creates large chunks of standalone, "static" code, and runs those as much as it can, breaking out into the "dynamic" runtime when it has reached the limits of type inference or for some other reason needs to return to the runtime to perform dynamic dispatch etc... in these instances, we break out of the standalone code and start using the runtime to do things like determine where to jump next (or whether to compile another chunk of static code and jump to that). Note that these chunks of static code can be both smaller or larger than a function, it all depends on what Julia can compile in one go without needing to break out into the dynamic environment.

In the Julia world, we make redistributable binaries for all sorts of things; you can find lots of packages here [0], and for LLVM in particular (which Julia uses to do its codegen) you can find _just_ libLLVM.so (plus a few supporting files) here [1]. If you want a more fully-featured, batteries-included build of LLVM, check out this package [2].

When using these JLL packages from Julia, it will automatically download and load in dependencies, but if you're using it from some other system, you'll probably need to manually check out the `Project.toml` file and see what other JLL packages are listed as dependencies. As an example, `LLVM_full_jll` requires `Zlib_jll` [3], since we build with support for compressed ELF sections. As you may have guessed, you can get `Zlib_jll` from [4], and it thankfully does not have any transitive dependencies.

In the Julia world, we're typically concerned with dynamic linking, (we `dlopen()` and `dlsym()` our way into all our binary dependencies) so this may not meet all your needs, but I figured I'd give it a shout out as it is one of the easier ways to get some binaries; just `curl -L $url | tar -zxv` and you're done. Some larger packages like GTK need to have environment variables set to get them to work from strange locations like the user's home directory. We set those in Julia code when the package is loaded [5], so if you try to use a dependency like one of those, you're on your own to set whatever environment variables/configuration options are needed in order to make something work at an unusual location on disk. Luckily, LLVM (at least the way we use it, via `libLLVM.so`) doesn't require any such shenanigans.

[0] https://github.com/JuliaBinaryWrappers/ [1] https://github.com/JuliaBinaryWrappers/libLLVM_jll.jl/releas... [2] https://github.com/JuliaBinaryWrappers/LLVM_full_jll.jl/rele... [3] https://github.com/JuliaBinaryWrappers/LLVM_full_jll.jl/blob... [4] https://github.com/JuliaBinaryWrappers/Zlib_jll.jl/releases [5] https://github.com/JuliaGraphics/Gtk.jl/blob/0ff744723c32c3f...

Aha! A chance to plug one of my favorite CGP Grey videos that explores this very question: https://www.youtube.com/watch?v=JEYh5WACqEk

Imagine instead of getting a grid of pixels once every 30th of a second, you instead get one pixel’s value, alone with its location, along with the time stamp at which the pixel’s change was noticed. Event cameras can have very fine time stamp resolution (orders of magnitude better than 1/30th of a second) and so a bright moving pixel can be tracked very accurately.

Note that the restriction on external monitors can be worked around by using a DisplayLink compatible dongle/dock, since it uses a custom driver (I assume it does something in software that would otherwise be limited in hardware).

I use the Dell D6000 and I run three (1080p) external monitors in addition to the built in monitor.

In case there's anyone out there with an M1 MBP that desperately wants multiple monitors, if you use a DisplayLink compatible external dock, something about the alternate driver that such a dock uses allows for multiple monitors. I personally use the Dell D6000 to run three external displays in addition to the built-in display off of my M1 MBP and it works quite well. You can see a list of DisplayLink docking stations here [0].

[0] https://www.displaylink.com/products/universal-docking-stati...

Hah! I did the exact same thing when I discovered the “net send” command. Only me and my friend were playing around so we sent eachother messages like “I know where you live”..... the school tech was cool with me so I didn’t get punished, but quite a few admins were freaked out by these strange messages appearing on their computers.

We call these "system images" and you can generate them with PackageCompiler [0]. Unfortunately, it's still a little cumbersome to create them, but this is something that we're improving from release to release. One possible future is where an environment can be "baked", such that when you start Julia pointing to that environment (via `--project`) it loads all the packages more or less instantaneously.

The downside is that generating system images can be quite slow, so we're still working on ways to generate them incrementally. In any case, if you're inspired to work on this kind of stuff, it's definitely something the entire community is interested in!

[0] https://github.com/JuliaLang/PackageCompiler.jl

Yeah, we've managed to get Julia itself running pretty well on the M1, there are still a few outstanding issues such as backtraces not being as high-quality as on other platforms. You can see the overall tracking issue [0] for a more granular status on the platform support.

For the package ecosystem as a whole, we will be slowly increasing the number of third-party packages that are built for aarch64-darwin, but this is a major undertaking, so I don't expect it to be truly "finished" for 3-6 months. This is due to both technical issues (packages may not build cleanly on aarch64-darwin and may need some patching/updating especially since some of our compilers like gfortran are prerelease testing builds, building for aarch64-darwin means that the packages must be marked as compatible with Julia 1.6+ only--due to a limitation in Julia 1.5-, etc...) as well as practical (Our packaging team is primarily volunteers and they only have so much bandwidth to help fix compilation issues).

[0] https://github.com/JuliaLang/julia/issues/36617

Small nitpick; its Project.toml (or JuliaProject.toml, to avoid name clashes) not Projects.toml

They are talking about total operations, but since big O notation ignores constant factors, and they explicitly ignore the addition steps (because for large n multiplication dominates) it is, in effect, only looking at the multiplications.

To answer the GP, the minimum amount of work necessary is to write a number for each element in the array, which scales by n^2 since there are n^2 elements. It is of course only possible to beat this if you can make assumptions like “the diagonal of my matrix is nonzero and everything else is zero”, but that’s not general matrix multiplication anymore, that’s a specialized sparse matrix routine. For a dense, “normal” matrix, O(n^2) is the absolute best we could hope for.

You pay a price for the super-small frame sizes; much worse compression ratios. Frame sizes smaller than 10ms disable the LPC and hybrid coding modes, which are quite advantageous. [0]

In my experiments with transmitting realtime audio signals using Opus, 10ms frame sizes are acceptable for 1-way synchronicity (e.g. if you want the user to perform an action and hear the result as simultaneous with the action) but it's definitely the upper bound. From what I remember my signal processing professor say, the threshold to try and hit is 7ms total latency for truly undetectable processing, but I can't find a reference for that, so take it with a grain of salt.

[0] https://www.opus-codec.org/docs/html_api/group__opusencoder....

Here in Seattle, in the absence of traffic forcing slower speeds, most people are going ~10mph over the speed limit. On the freeway, although the speed limit is 60mph most people will be going about 70mph. If you use Tesla's autopilot it will match the speed of the car in front of you, up to the maximum specified by the "speed limit". So I usually set the speed limit to something like 75mph so that the car follows the flow of the traffic around it, instead of forcing other drivers to swerve around me because I am going 10mph slower than everyone else.

I have always found magnum to be a very nicely constructed C++ codebase: https://github.com/mosra/magnum

The abstractions that are built up within that codebase feel very clean, like they get out of your way so that you can be productive, without drowning you in syntax.

It’s not a jab at Julia

I didn't take it as such; there are legitimate shortcomings to any tool, I just wanted to provide pointers to other readers that the devs are aware of it, and that there is ongoing development to address it. :)

Come on, jeez.. Julia’s compiler is a Lisp-based LLVM driver: of course it can do these things.

As someone who, before Julia, was firmly entrenched in C/C++/Python land, I suppose I am discovering many of these "obvious" things for the first time. :)

Julia's SIMD programming model is still very much a work in progress; I think we have a way to go in providing the kind of flexibility and control that languages such as ISPC, Halide, TVM, etc... provide.

That being said, packages such as SIMD.jl [0], and LoopVectorization.jl [1] are making fantastic progress, to the point that LoopVectorization forms the basis of a legitimate BLAS contender, in pure Julia [2]. It's not totally there yet, but it's close enough that real work is being done in LV at OpenBLAS-like speeds.

As an aside, I find it incredible that these kinds of extensions can be built in packages thanks to the fact that Julia's compiler is extensible enough to allow for direct manipulation of the LLVM intrinsics being emitted by user code.

[0] https://github.com/eschnett/SIMD.jl [1] https://github.com/chriselrod/LoopVectorization.jl [2] https://github.com/MasonProtter/Gaius.jl