6d65

Keep up the good work and thank you for sharing.

I wasn't sold about the source level snapshot tests, they seemed clunky and not that universal. But, I am definitely giving snapit a try after this blog post, and the other two linked.

I might have to add a few things, most notably the support for multiline strings for more readable ascii art based prints, but I have a good feeling about this, but will see how it goes.

I have been tracking 5950x prices and anything over $300 seems unreasonable. And I think I'm just going to get a 7700x and a new MB and 32GB of RAM, it should be a $500-sh upgrade. It should have M3 like single threaded perf, multi threaded perf better than 5950x, slightly worse than M3 Max, better memory bandwidth than the 5950x.

You're right about just having a beefier PC, but maybe a baseline M2/M3 Air to SSH into a local machine + a beefier PC is a good combination. Luckly I switched to an all in terminal dev experience with Neovim/Helix/Tmux and the advantage is that one can seamlessly mix ssh/mosh shells in this workflow. Don't even need to set up any VNC/RDPs/Moonlights. Maybe even that is not needed if your languages supports ccache like tools.

I'm mostly interested in compilation speed, maybe running some programs faster on beefier hardware and I don't mind working on Linux.

So to recap, a baseline Air for portability(or some light ryzen laptop) plus a medium range PC seems like a winning combination for me. Given you already have a case/psu/ssd together they can cost as an entry level m3 macbook pro with 8GB of RAM.

Since everyone is sharing, I'm using bookmarklets daily, the main ones being 3x and 2x, which find the video element in the current page and speed it up.

I also made some overlay grid helpers, like 8h and 8v which create some overlay pass through elements and render red grid lines(8px spacing) using css. This is like an alignment tool for every page.

The bad thing is that Chrome and Edge don't highlight bookmarks by default when searching and makes one to type the top arrow key. Firefox used to be better in this regard, with the bookmarklet highlighted and ready to hit. The workaround for Chrome and Edge is to make the bookmarklet a search engine, that makes the bookmarklet selected by default.

Bookmarklets and custom search engines (for various online documentation sites, or plain jira, or aws) can be a productivity booster.

I wouldn't be able to navigate aws ssm without my trusty ssm search engine. Same for the others: EC2, CF, S3.

I should be using bookmarklets more though.

Another great post in the series.

I've been trying to bootstrap my own deep learning framework in Rust for a while now. I'm still stuck at implementing the stuff on the CPU.

But I've always wanted something running on GPU as well. I've poked at OpenCL, and webgpu. But Piet-gpu(more probably piet-hal) seems to be the best starting point in the Rust land.

Once I get to the GPU compute part, I'll start with Piet GPU. Not looking forward to debugging GPU compute kernels, but maybe it will be fun.

PS. It's a bit disheartening to see such buggy Vulkan implementations, since one of the main Vulkan selling points was less bugs in the drivers. I'm not sure what I'll see on Linux.

Also, I hoped that between Vulkan and Metal one could cover all the the major desktop OSes with GPU accelerated software. It's sad to see apple dropping the ball in this regard.

There is also the concern of code reuse and code quality. Doing something complex in one pass would mean a huge kernel. It would be interesting to investigate if something like openai's Triton can be implemented in top of Piet-gpu(HAL). A dsl embedded in Rust that could do kernel fusion, and generate one shader when composed together. I'll poke around at this when I get to Piet-gpu.

I'm vaguely familiar with Factor and hopefully not in process of reinventing it.

I was planning to investigate this space just for fun. Most likely nothing will come out if it and that's ok, it's just a learning opportunity.

Maybe I'm not using the proper FP function term. I don't mean functions as objects that you can store or pass around or higher order functions. Just take some arguments from stack, and return some values.

What I mean is that FORTH words seem to be functions taking stuff from stack and pushing stuff back. The composition is done by chaining them together.

This probably works fine with repl driven development(same as in lisp), you find out during development that your words cannot be composed. But as mentioned by others it may make the code hard to read, having to keep the state id the stack at all times in your head.

I was thinking just adding a light syntax, ex:

let add a b = a b +

// optional type annotation

let square a = a a *

let add-square = add square

Maybe add some type inference, support for structs, match expressions. But at the core keep the low level FORTH nature, with an interpreter and compiler.

It remains to be seen if it's possible to reach a balance here. FP programmers may be disappointed this is not a proper FP language, while FORTH programers may say it's a syntax ridden abomination.

I would say with some tooling (editor support, test framework, god forbid package manager), it can be an interesting option for embedded development.

There's a twitch streamer(tsoding) doing a FORTH like language from first principles.

I've watched a couple of episodes and went started my own. I learned a bit of arm64 assembly, and now I have both an interpreter and a compiler to M1 binaries. Very fun experience.

One day I'll slap a F#/OCaml'esque syntax on top of it with typed functions, structs, interfaces and see if it works.

At their core(it seems to me), concatenative languages are about function (words) composition.

So, instead if having "words", I would make them functions, with typed arguments, and simple argument matching (no more dups and swaps). This would probably make the language less flexible, but will make it more readable.

Anyway, FORTH is nice.

Well, Microsoft became quite a big company off the OS and their business suite.

Of course it will be hard to find corporate clients for a mobile os, but may be doable.

But, you're right, aside from Microsoft and Red hat, and maybe CoreOS some time ago, people don't seem to earn money from OSes, unless I'm missing something.

You have fair points and much more experience than I have in this domain.

Yeah, $5 even at a 1M users is probably not that much, having to maintain drivers, and as you mentioned serving updates, that would require a team.

With regards to the Google apps, I think a company would be better off trying to make something of their own, or collaborate with some app makers, at least for difficult stuff like maps, camera, gallery, browser(not sure about the paid codecs)

As for the app store, I know that's a big project, but that would have to be custom and a core product of the company.

Hope you'll eventually find a way to monetize your ROM.

I was thinking not too long ago about how phones are missing an installable os with a clear business plan.

There is the potential(given an unlocked bootloader), to use webusb to allow people flash the OS from an official website.

And maybe at a later day allow install from phone to phone via a cable, as a lot of people nowadays don't have a PC.

The pricing will be tricky though, $5 per install given a large user base, plus maybe some additional features for sale.

That being said, having an OS also means having an appstore, that alone could generate enough revenue to make the OS free.

By OS I mean an Android fork or a fuchsia based OS if the license allows it.

By ecotypes i meant exotypes(see https://cs.stanford.edu/~zdevito/pldi083-devito.pdf)

Eyesore is a precise word for the local keyword, from what I see the variable names in scripts are usually short, and the local keyword seems to draw more attention than the variable names themselves.

Of course this can be alleviated with syntax highlighting that would slightly mute the local keyword. And, people using Lua they day probably learn to ignore the keyword automatically.

Yep, it makes sense.

With the popularity of Roblox with the young people, and it using Lua for scripting, Lua can see a renaissance of some sorts.

I will definitely give it a try.

Even after many years of not using terralang I still cannot forget what a good of an idea it is.

Nelua seems like a more pragmatic implementation of similar ideas, but generates C code instead of embedding the llvm. And doesn't generate code at runtime. But still, things like ecotypes should be possible.

It will be interesting to play with the compile time lua scripting. Also, as mentioned in the other comments not sure about the GC. But there seems to be a manual memory management option.

But still, looks great, kudos to the author, keep up the great work.

PS: If I would implemented it, I would deviate a bit from Lua and replace local with let. It's highly subjective but I think it would make code "prettier"(whatever that means)

Bought the PDF after reading the free web version, the work definitely has a lot of love put into it and this has to be rewarded.

I've skimmed the web version till the end, but thoroughly read half of it, still learned a lot. Especially the pragmatic approach to code generation via string concatenation. I went a bit further with a small dsl that also does string concatenation.

My C# and Rust implementations are half baked, but still I had a lot of fun doing them, and that's all that counts.

Can't wait to see Bob's next endeavor.

You're right about Swift for Tensorflow. It is an interesting development. I have installed swift on Linux just to play with it. Was dissapointed to see that Swift for Tensorflow was a fork of the Swift compiler, and haven't followed whether it's being refreshed, and whether there are any plans to merge it back.

Ideally an LLVM tool should allow languages that compile to LLVM(I'm mostly interested in Rust and Julia) to leverage it without dramatically changing their compilers.

It would be interesting to see something like JAX in Rust, exposing the AD functionality in the Standard Library, paired with a high performance SIMD/GPU array type. Things could get very interesting.

You might be right, I seem to have confused the pytorch(and tf eager mode) differentiation approaches with numerical methods.

I'm making a Pytorch inspired ML framework, and indeed, each op node, defines also a backward pass, which is a manual definition of a derivative. And going backwards over the ops graph, and combine derivatives for each op via chain rule to get the final gradient, looks indeed like a runtime analytical method rather than a numerical one.

The advantage of an automatic AD is not having to define the backward pass for each op, and the function that calculates the derivative being generated at compile time.

I've left the project marinate a bit, so the little knowledge I had is fading away.

AFAIK, It's mainly used for implementing gradient descent, which is used for training neural networks.

Frameworks like pytorch, tensorflow, probably used back propagation to calculate the gradient of a multidimensional function. But in involves tracing, and storing the network state during the forward pass.

Static automatic differentiation should be faster and should look a lot like differentiation is done mathematically rather than numerically.

Of course there are more applications to AD in scientific computing.

The way I see it, this is what programmers do, they turns things into programs. A good example are hardware description languages, where hardware designers write programs, and at the end a very complex chip design is generated. Which is then taped out. Those are very complex systems with very complex constraints. The output of these are locations of hunderds of billions of transistors with their interconnects.

Assuming you know JavaScript, you get some advantages over doing it via gui:

* Source control support: the model is text and can be stored in git. Multiple people can work incrementally in a branch. I assume cad models are binary, or very complex text. Having readble code will also allow for code reviews.

* Parametric modeling: you can define components (parts) as functions with function parameters to customize the behavior. You can have if else statements inside the functions to change the behavior. For example you can have a bolt function that can generate any type of bolt based on parameters

* Automation : In a programming language you can have loops that can generate any amount of components, and you can customize position, and object parameters in the loop. Making grids if objects is trivial.

* Testing : if you're using a full language like JavaScript, you can write unit tests for your more complicated parametric models. This can help with edgecases, and make your models more resilient.

* Code reuse : you can create a standard library of parametric models, ex: pipes, bricks, bolts and so on, and use them quickly on new projects

* IDE support : programming languages often have ide support that make reafactoring, code navigation, code exploration much simpler

* Familiarity : learning a complex gui app can take time, especially when the GUI is very complex. If the person already knows JavaScript, and can use an ide, with fuzzy searching through the symbols. In my opinion this is much simpler to start and get complex results than searching for some obscure feature in a deeply nested GUI. Of course this can be alleviated by a fuzzy search of UI functionality from a single place, ex: GIMP / command, and the new blender searchable menus.

Anyway, these are a few from the top of my head. There are probably many more. Of course, there are disadvantages, mainly that not everyone can program, and of course the people who use CADs are less likely to be familiar with coding. But capable people can learn quickly.

I'm looking at this as a software developer than wants to design some complex models, but doesn't want to learn a complex UI of a cad system. I'm quite familiar with working on complex systems in code, and find this approach much more suitable for me.

Yes, my plan is to do it in Rust.

I did start it in luajit first, because I thought that it will be a faster feedback loop. I created an OpenGL window, and set up nodemon to restart the luajit app on changes. It is quite fast, but still not as smooth as I imagined it. Doing it Rust from the start, with a TDD approach, might have been a better idea. Things got quite messy quite quickly because of the luajit OpenGL bindings.

The goal is to do difference, addition, intersection first between basic 3d volumes: sphere, cuboids, cylinders. And then yes, some extruded splines. I was thinking about starting with beziers, as this is the only spline I've implement before.

I haven't built a cad before, so I went with a data oriented approach, defined the data needed to represent each of the basic volumes, and 2d shapes, as well as the relationships(diff, union, intersection). Something like a normalized SQL database. A table (array), for positions, rotations, bounding boxes and so on. I'm yet to see how well this scales up to more complex models.

It is very much in the beginning. I've started from first principles, not looking up existing approaches and see what I can come up with. That's the reason it probably won't become something serious, but it's still fun to explore.

Thank you for offering help, might have a closer look at SolveSpace if I get to something serious that I cannot come up with a solution on my own.

I've used it, it's very good.

I've started my own programmable cad after using openscad, I just couldn't look at aliased lines and weird colors anymore, plus making complex models is slow, and bevels are hard.

My own cad is still far from working, probably because I had the brilliant idea to start my own csg kernel. In the meanwhile I've stumbled upon CascadeStudio.

Cascade Studio is fast, looks good, available everywhere via browser. I was able to do a non-trivial model, add bom generation via console.log. It would be great if there was a standalone version as well, so larger projects can be built directly from the filesystem.

Anyway, it's good stuff, whishing the project all the best.

I'm a powershell user for about 7 years now, or more. Have a lot of bindings, helpers in my powershell profile, and use it everyday for work.

That being said, I'll never use it on Linux as my shell. It's just too slow to start. Fish, nushell, bash, all start instantaneously, powershell (legacy and core), all take more than a second to start, on beefy machines.

I've been looking at the powershell core repo in hopes of fixing this with .net core ready to run profile, but they seemed to have something like that in place, but was disabled at that time.

Anyway, PowerShell is good, probably the best you can get on Windows(nushell also takes a bit to start, and it's still new). But, on Linux you can do much better, even if that means having to struggle with Bash/Fish scripting.

For more complex scripts a full language like Lua or Python are most likely better.

Also, last time I checked the docker container for PowerShell Core was easily over 100MB, if I remember correctly. Might work well for a dev machine, where you set it once, but for a CI, it's not ideal.

Geekbench

M1 single threaded ~= 1720

M1 multi threaded ~= 7400

Ryzen 5900h single threaded = 1520

Ryzen 5900h multithreaded = 9325

Ryzen is consuming 45W, M1 is said to consume 13. Ryzen is on TSMC 7nm, M1 on 5nm. It's said that TSMCs 5nm is 1.8x more dense than 7nm, 30% more efficient and 15% faster.

So if both were on 5nm, one could extrapolate that the single threaded performance would be similar, but ryzen would still lose on battery life. But it's a speculation.

As to whether it's fair, Apple can pay premium for 5nm exclusive access, and then charge the users hundreds of dollars per 8GB of RAM or storage upgrades.

AMD is selling the chips to the OEMs which have to make money themselves, which means that using cutting edge nodes might not make sense economically.

But that doesn't matter, both are businesses, it's AMDs fault that they didn't rush to 5nm and are comfortable at staying behind.

The ones to lose will be the premium laptop manufactures, that sell laptops at 1000+ usd. As the agressive Apple marketing will most likely cut into their sales.

It would be interesting to see if the rumors, about Intel using TSMC 5nm this year, are true. Intel had the single threaded perf lead, even on their less dense 14nm node, vs AMD on TSMCs 7nm. Could be that on TSMCs nodes they would be faster than both Apple and AMD, but still probably at a higher TDP than Apple chips.

The description says it's autograd + XLA. So I assumed it always compiles to GPUs via XLA.

But, had a look in the code and jax has cublas and RoCm blas, and it looks like there is a flow where it uses the gpu directly, unless I'm missing something.

Definitely worth having a closer look. Autograd via function reflection should be faster than backprop. And if it's running on AMD GPUs then it's quite intriguing.

I've been playing with OpenCL lately. Is such a shame that the support is so spotty, and even Apple(who I think started it, probably as a CUDA alternative for their AMD cards), are deprecating it.

Even with it not being able to squeeze all the perf from a device, it's still handy, especially being a cross device api.

It would've been ideal for piet, especially since compute kernels can also be run on CPU on all cores, given a CPU driver. And I think some OpenCL implementations support spirv. So it wouldn't have needed any fallbacks, the same kernels would run either on CPU or GPUs.

I'm still going to use it for my project. But, it's a shame that AMD RoCm OpenCL doesn't support running kernels on their CPUs.

Also, Rust might benefit from something like SYCL. There is some attractive convenience in having a DSL built into the language for parallel computing. But one can dream.

One of the few things I remember from watching fastai course a few years ago, is to train a model with smaller data. For example, instead of 2k×2k images, downscale them to for ex 400×400. A network with a better design should still learn fastest on smaller data.

Also, one can train the network to a good accuracy, then change the input layer, and unfreeze the inner layers, that way the network will have a head start.

Not sure how universal this principle is, but it seemed reasonable, if I remember it correctly, of course.

The approach described in the article looks very smart. Also could be handy for integration testing of ML frameworks. I've been working on my own DL framework, and this data set looks like a good way to test the training and inference pipelines E2E.

The other option is to use the best available x86 mobile CPUs, and these at the moment seem to be from AMD.

From what I've seen even current AMD CPUs are competitive with M1. Ryzen 4900HS has a fair lead in multicore performance and close single threaded. Of course, all this at a higher power and fans running as a jet engine.

But AMD is one process node behind, and one architectural node behind their desktops (Zen 2 on 7nm).

So, OEMs can use the perf gains from both the advancements to lower the TDP a bit,(ex 10%), put a larger battery. And this can result in a competitive product, even with the rumored M1X, since Ryzens already have the multicore lead.

That being said, from I read, the next mobile Ryzens will be 5000 series, released in a month, at 7nm. So only a ~20% IPC uplift is expected. Also probably TSMC's 7nm improved a bit. Nothing to scoff at, but not as exciting as a new process node.

Anyway, we'll see soon enough.

Only wishful thinking from my side. As mentioned in the post, I hope that the M1 performance and aggressive apple marketing will push premium pc laptop manufactures in adopting the best performance x86 CPUs. And I also hope that AMD will seize the opportunity to get a good chunk of mobile market.

Of course, next year laptop models are probably being finished right now, and it could be that intel still found a way to bribe premium manufactures from including AMD in their premium models.

In my opinion they'll lose marketshare if that happens. But, there are most likely smart people with a lot of more information than me making these decisions.

You're right, it could be that Nvidia will push for more beefed up cortex CPUs, with the same tricks employed in m1, like larger L1/l2 caches, increasing the die size, and add more complex pipelines, put the ram on the soc, and an Nvidia gpu.

The potential problems are.

- This might take some time, if it's not already in progress. And pc manufactures might not afford lower sales for a few years while this happens.

- Windows seems to still have issues with app portability and emulation. But who knows, maybe MS will push harder

- Making their own chips and also licensing cores to vendors like Qualcomm, might spook them. It's not a good position to compete with someone who owns the IP you depend one. This might be a good thing for RISC-V

But we'll find out in a year or two what is the play. Competition is good for everyone. That's why I don't like apple having exclusive access to new nodes, be it because apple is willing to pay premium, or that other companies avoid low yield.

Anyway, seems like TSMC will have their hands full for a few cycles.

Yep. Mac's look like a comfortable unixy environment. Preinstalled zsh, a fast terminal emulator (item), gnu utils. I'm sure I could make it work as a dev machine.

I have to admit that m1 macbook air battery life is compelling. A long time ago I was thinking about a perfect laptop that would be very light and have a 20+ hours of battery life. This comes close. Even though I have never used a laptop more than 5 hours on battery. And that happens a couple of times a year.

Hope you'll enjoy your m1 air.