stfwn

It seems like this page is updated with the followup questions asked by every visitor. That's an easy way to leak your search history and it's (amusingly) happening live as I'm typing this.

For me, the gradual build up of (1) maximum likelihood estimation to (2) maximum a posteriori estimation to (3) full posterior approximation (or posterior sampling) was helpful to understand where Bayesian methods are in machine learning. Here’s a great video series by Erik Bekkers, who is at the University of Amsterdam. It assumes solid knowledge of calculus & linear algebra and takes you through the math and intuition of all fundamental ML methods: https://youtube.com/playlist?list=PL8FnQMH2k7jzhtVYbKmvrMyXD...

Slight tangent: does anyone know of a good source for more useful blog posts on ML in the wild? Many that I come across are a funnel into some product, are very short, too theoretical, for beginners, or all of the above. This post strikes a nice balance in simply sharing some experiences and opinions, like you would see in blogposts on how to do software engineering well.

That's such a cool nugget of information. The Versailles one is apparently A = 435 Hz, the 'diapason normal'. Nowadays we mostly use concert pitch at A = 440 Hz. The 99% Invisible podcast has a mini-story on it:

https://99percentinvisible.org/episode/mini-stories-volume-8...

They say that after Versailles:

London’s Royal Philharmonic Society would still tune higher to about 439 Hz though. This was because the mandate specified that Concert A should be 435 Hz according to a tuning fork of a specific weight at 15 degrees Celsius. The temperature was specified so that the metal tuning fork could be accurately reproduced, but British orchestras reasoned that their concert halls were warmer than that, and so to compensate they would tune higher.

I wonder if they tested their assumption with a conformant tuning fork.

Conventional methods of rendering 3D objects and spaces rely on specifying geometry and material properties in some format. You then simulate a viewpoint using that info and physics simulations.

A NeRF takes over both the role of the file format and part of the rendering in the form of a neural network. You feed in a world coordinate and a viewpoint and you get an RGB tuple and density out of it. If you interrogate the NeRF enough you can render any traditional 2D or 3D image out of it by combining all the datapoints.

One theoretical benefit is that that a NeRF is a continuous function, so the resolution is only limited by the capacity of the neural network. Another cool thing is that a NeRF is trained on pictures (with info about where they were taken from), so if you train a NeRF successfully in high-res it’s like scanning an object. A major practical challenge is that it is (was?) pretty frickin’ slow to work with. I wrote a more elaborate comment about it on the previous NeRF improvement post [1]. There I closed with:

It would be amazing to have NeRF-based graphics engines that can make up spaces out of layers of NeRFs, all probed in real-time.

Here they’ve taken a major step in that direction by speeding up the rendering 3000X.

[1] https://news.ycombinator.com/item?id=25300283

How do you deal with adversarial/byzantine updates that attempt to degrade performance or even install a backdoor? Do you use plain averaging, or some other aggregation algorithm like Multi-Krum?

Thanks!

The last time a post like this appeared on HN prompted me to write a gist with a simple neural network in Python (with Numpy). It downloads the MNIST dataset for you, trains a fully connected network on it, prints the accuracy on the validation set and plots the loss. It's pretty verbose with plenty of terms and comments to search the web for if you're interested.

https://gist.github.com/stfwn/62e51d86ca4ff155becd3c6a14adf6...

Yes, I'm also using Hetzner and have not run into any limits in what plugins you can install and use yet. They used to call 'storage share' just 'Nextcloud', but moved it to the new name for licensing reasons.

I don't know what type of hardware they're running their Nextcloud instances on, but I've been using calendar/contacts and public file sharing (since Firefox Send went down) on the cheapest plan for a while now and it's fine. I also dabbled a bit in using Nextcloud's maps, video calling, bookmarks and rss reader applications and it all works OK, although I didn't stick to them because they're not the most rich in features. I had to Hetzner email support a few times and got a helpful reply within 1-2 work days, sometimes quicker, which I thought was fine.

In macOS it is not entirely clear to me what category of things the cmd key is for. On one hand it's used for keyboard shortcuts related to window management and the system, but on the other hand it's also used to issue commands to the application itself.

In i3 you assign a $mod key, which then always and exclusively used for 'system and window management'. Opening, closing, moving and resizing windows, switching workspaces and showing toolbars is all done with some key combination that starts with $mod.

Since reading and writing the clipboard is done by the application, in my brain it should not involve $mod. So I use ctrl+c and ctrl+v for all applications. The one exception is the terminal, in which ctrl+c terminates programs. So there I use ctrl+shift+c. I also unified the register vim yanks to with the system clipboard, so for me most copying and pasting that happens in the terminal is done via `y` (yank) and `p` (paste).

A more straight answer to your question is: yes, you can remap everything to bits. You can modify what keypress each keycode should trigger, so there is really no limit to what you can do there.

These are exactly the type of silly applications that are needed on macOS because the OS too locked down, or at least not made for people who like to do this type of stuff.

On Linux, almost all native applications keep their data in regular, non-proprietary files on the file system. This means your data always exists separate from the applications and you can do with it what you want. If you want to run a script every Monday at 3PM that sifts through your ~/downloads folder and uploads the top 10% largest files to a server, except in January, on Easter, when when your drive is below 70% capacity, if it's not hooked up to power, and also not if Kanye West tweeted something in all caps, you can do that. You can make it retry every 42 minutes if conditions weren't met, email Bob and message Alice in Slack about it. You can make it machine learn your behavior and pick a window within 3 hours after sunrise, log the bandwidth usage, and have it play a tune for your friend's MPD server on the other side of the country.

Why you would want to do all of this is beyond me, but odds are you want to do one of these things at some point. Then Hazel won't be able to do it because they didn't consider your use case, that functionality is only available in version 6 (pay up for the upgrade!), Apple didn't provide the right API, or they recently took it away in the name of security. Surely there's another app that can do it, but that one can't do the other thing! Oh well, maybe the next version?

On Linux, you dream up the code in the language of your choice and it will work forever. Your learned skills will transfer one-to-one to programming you do for your job or hobby, instead of being weirdly specific and locked down to this one, overpriced app's clickity click interface. The OSS mentality of interoperability and doing one thing well almost guarantees there are battle-tested tools available that can solve pieces of your specific puzzle, while not locking you in to their way of doing things.

Of course scripting is not exclusive to Linux, but it is the only OS that was and is built with it in mind.

I used to be all in on Apple. On macOS I had a little program called Magnet to snap windows to sides and corners, and on my iPad (with external keyboard) I SSH’d into a VPS to write and run code there. I used Alfred and had all kinds of workflows in there. I thought it was great.

But then during my AI studies I wanted some beefier hardware, which was just not affordable for me within Apple’s ecosystem, plus they only used AMD graphics cards. I built a desktop computer that outperformed the top of the line Mac Pro for a fraction of the cost and turned it into a Hackintosh. Two weeks later Mojave came out, and Apple never approved any Nvidia drivers from then on.

My eyes opened to Linux and i3 in particular, which looked like Magnet taken to the extreme. What had taken me hours to install and configure on macOS (GPU-acceleration for PyTorch, for example) just worked with one package install on Linux. All my expensive apps were replaced with simple and free, much more configurable alternatives. At first I spent a day or two getting things just right. Since then not much has changed because not much needed changing, which I really like.

Now I look at macOS and iOS and cringe how locked down it all is. Users are very creative in their workarounds to make it work, but it is ultimately quite silly that you need to use special URL schemes and workflows to open a text file across different apps.

Thanks, i3!

I studied piano full-time for a short while. My favorite thing about it was how working on my technique or a particularly tricky rhythmic pattern tickled my brain. I often found that I went through a few phases: “ahrg, this is impossible!”, “getting there but sloppy” and finally, often suddenly (after a good night’s sleep) “this is easy! I don’t know how I was ever not able to do this”.

Using vim and learning new features tickles my brain in kinda the same way. I don’t care much for the hacker credits. It’s a fun variety in my work day to practice some little feature that I read about, and it doesn’t look like I’ll run out of new stuff to check out anytime soon.

Fiducial markers are commonly used in photogrammetry to either speed up the process, make the resulting model more accurate or a balance of both depending on what the user is looking for. Good fiducials make for distinct features that can easily be matched across different images.

It works best if you play into the algorithm used to find the point correspondences. One commonly used one is SIFT [1]. It's a multi-step process where each step introduces some invariances, like scale invariance through convolution with gaussian kernels at different standard deviations to create a 'scale space', then doing blob detection in that space by looking at second derivative maxima and minima.

The matching process does of a lot of convolution, which is linear (so you can combine a gaussian and laplacian kernel and do both in one shot) and it can be nicely parallelized. The 8 hours of processing of ~80GB of 24MP images was on a GTX 1080.

I wouldn't say that it's particularly slow considering the amount of data and complexity of the operations, but surely a speedup would be very welcome and useful. It would become much more accessible to game companies, movie studios and even industries that (afaik) don't make much use of 3D models yet -- perhaps archaeology or anthropology would jump at the opportunity of scanning and sharing super high res models.

[1] https://en.wikipedia.org/wiki/Scale-invariant_feature_transf...

I once had to count the number of set bits (the Hamming weight) for an assignment. It had to be done in under 40 bitwise ops, without loops and with constants of up to 8 bits. The naive approach took way too many ops! I was stumped until I found the linked page, in particular 'Counting bits set, in parallel'[1]. Parallelism at the bit level?! That is when I realized low level programming was completely badass.

[1] https://graphics.stanford.edu/~seander/bithacks.html#CountBi...

In June I worked on a comparison of the original NeRF [1] to a state of the art proprietary photogrammetry method.

The photogrammetry method could process ~80GB worth of 24MP photos into a micrometer-level accurate 3D model in about 8 hours, while the fastest NeRF implementations available took the same time to train a model on just 46 pictures at 0.2MP. A funny extrapolation from a handful of datapoints was that it would have taken 1406 hours or about two months to train a NeRF at a resolution of 24MP, assuming it would converge at all. PixelNeRF improves an aspect that was already great (the number of photos required) but does not seem to tackle this complexity problem.

Another problem is this: the representation of the learned scene is entirely abstract, contained within the weights of the neural networks that make up the NeRF. The space itself cannot be meaningfully inspected -- it must be probed and examined by its input/output pairs. The NeRF takes as its input a 3D location plus a viewing direction, and the output is a color radiance in that direction and a density (which depends only on the 3D location). So to generate a 2D image you emit camera rays into the NeRF from a specific hypothetical camera position, direction, focal length and sensor resolution, get the NeRF's output for many different points along all the camera rays and compute an image based on it (volume rendering).

This is fine as long as the NeRF is available and there are no time constraints, but does not seem workable for real-time graphics rendering like in gaming/VR. So the NeRF should probably be rendered into a traditional 3D model ahead of time. Afaik this is an open problem that I've only seen solved by using a combination of marching cubes to extract the scene geometry and then rendering colors from normal vectors. In this process, continuity, spatial density and directional color radiance, three of the most important contributions of the NeRF design, are entirely lost.

I would be very interested to see papers that tackle higher resolution spaces at feasible training times and faster novel view rendering times. It would be amazing to have NeRF-based graphics engines that can make up spaces out of layers of NeRFs, all probed in real-time.

[1] https://www.matthewtancik.com/nerf

A key feature of neural networks is that the outputs are a nonlinear function of the inputs. However, steering a 2D car is simple enough that a linear function is more than sufficient, and neural networks are unnecessary.

This depends entirely on the definition of 'steering a 2D car'. In the model used, throttle is simply proportional to the distance to the nearest wall in front of the car. This means the agent will never accelerate coming out of a corner, because it can't know it has the headroom to steer away from the wall as it's coming out.

Similarly, the model for steering inherently steers the car towards the middle of the track. I would expect the car to wobble from left to right if the road's edges are ragged, make up its own corners if the track edges describe a 'fake' turn on a straight bit, and the car would likely crash if it were to encounter a Y junction or a pit stop. The neural network agents showed smarter behavior here because it is able to capture more complex cross-dependence between different inputs.

On the topic of junctions, if the track were to include them, perhaps it'd be nice if the car chose the quickest route to optimize for lap times. But maybe that stretches the problem statement too much.

Instead of doing anything fancy, my program generates the coefficients at random to explore the space. If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time.

In theory this is more random and less efficient than an evolutionary algorithm, which searches the problem space in a structured way. If the author really wanted to hammer the point home, a least squares method to one-shot the coefficients would be more convincing.

All in all, the author doesn't make any hard claims that are false. But I would nuance the point of "neural networks are unnecessary" to "simpler models will do for simpler objectives".

I spoke to someone who started working in sheet piling when he was 16. This is in the Netherlands, so plenty of that to go around. He stayed with his parents and saved until he could buy a house out of pocket at 25. He was 30, said he was saving to retire at 35. It blew my ‘educated’ mind.

Even if Apple agreed to the demands and made the options available to all developers, Epic would have an advantage.

I don't know that that's the case. Steam, the Humble Store, GOG and Microsoft (just to name a few) would all be happy to compete if the terms were right.

Cool! I really like the (upcoming) idea of watching your own online profiles on HN and such for upvotes and saving those pages. It’s essentially what I already use those upvotes for in my head, but not what they are in reality.

Fwiw, you can run Pi-hole locally just fine. But using the hosts file like Maza does may be a little bit faster than running a DNS-server.

Pruning in general is about trimming larger structures down. This is nice for using a network, but indeed doesn’t help with training it.

Lottery tickets on the other hand are configurations of weights that rival the full network in performance when both are trained from scratch, with the ticket having far fewer parameters to optimize. Pruning is only the method to find the tickets for now, but the ideal would be to have a weight init strategy that can create winning tickets in one-shot. There is work being done on this front, and also to see if winning tickets for one image classification task generalize to image classification as a whole. This would do a lot to reduce the size of networks from the start, and so far results are promising.

But yes, this is all very specific to neural networks. Do you have a blog or other place where you post about some of the things you’re working on? It would be nice to read about.

You might be interested in pruning neural networks. Some networks can be reduced in size by over 90% without performance loss. The lottery ticket hypothesis paper is good place to start, if you don’t already know about it of course.

https://arxiv.org/abs/1803.03635

Some pruning methods have recently been added in the latest feature release of PyTorch.

This is awesome!

I typed in 'jazz' and got a page on Charlie Parker's saxophone, a King Super 20. It has 8 very high quality and nicely composed photos that you can download in TIFF format, some up to ~300MB in size. Each photo has a detailed description in text, as well as the dimensions of what's seen in the shot, info on when/where it was made and by whom and lots more! What a great project.

https://www.si.edu/object/nmaahc_2019.10.1a-g

E: One of the images has already been added to the Wikipedia entry for Charlie Parker a couple of hours ago. How cool is this data set.

Why not demo it in Python?

Not OP, but here: https://gist.github.com/stfwn/62e51d86ca4ff155becd3c6a14adf6...

You should be able to wget the file and run it (Python 3) from start to finish without any set-up and get ~88% accuracy on the test set.

It uses all the data (not one-sixth like in the blog posts) and does 200 iterations by default, so here's the loss plot on the training set if you want to skip all the fun: https://i.imgur.com/F57zmXV.png

For Bandcamp it’s at least 85% based on the fact that their cut is 15 to 10%. Less than half looks pretty bad in comparison I would say.

The _only thing_ Spotify is doing here is distribution. That is some brutal split if you ask me. The only thing holding them back from converting their user numbers into a more silly-level profit margin, like a proper Silicon Valley SaaS, is the power of a handful of major record labels, which in turn hold the artists by the balls.

As in most online industries nowadays, the ones doing the real work get the smallest piece of the pie. The best course of action for them would be to decentralize, which is entirely possible, but deep pockets need to kickstart that process and there is zero incentive for them to do that.

For me, the fact that artists get paid so little of what I would pay Spotify ruins it.

There are various numbers floating around the internet and they are are all in the range of fractions of a cent per play, which is a weird metric to begin with. At $0.00473 per play [0], an artist needs ~100k plays to earn 500 bucks. Then has to be divided up between the label, songwriters, musicians, producers and whoever else was involved in the project.

Compare this to me buying their album on Bandcamp, where anyone can sell their work and keep 85% to 90% of the money [1]. I get to have lossless files on disk forever + streaming while Bandcamp lasts, and the artist gets more money, more control over the distribution and an (opt-in at buy) email list to update their fanbase.

Clearly the era of musicians that are not pop stars earning decent money by selling audio is over, but this is one of the things I like to spend a little more time and money on.

[0] https://www.digitalmusicnews.com/2018/12/25/streaming-music-...

[1] https://bandcamp.com/pricing

There is a Reply All episode about something exactly like what you describe [1]. It's been a while since I've listened to it, but it's something about how making sounds like these is a whole business with resellers upon resellers. They dive into the process and try to track down the person who made a specific recording that also had some background noise that seemed particularly weird.

[1] https://gimletmedia.com/shows/reply-all/mehrar

I may be wrong, but these add-ons seem shady to me. The website pointed to as the homepage is an add-ridden site that is supposedly a community for open-source development, but all the links to 'fork me on GitHub' don't go anywhere. I can't find the source at all. This topic comes up often but these have never been posted and the add-ons have relatively few users (4k, 1.5k, 2k and 0.7k).

I don't know anything about browser add-ons. Perhaps someone who does could take a look and see what's up?

What kind of diagnoses are you interested in learning about? This may be too rudimentary, but I found ‘The Linux Command Line’ a good book to build up skills and knowledge.

Also I would say: don’t try to force it. People write a lot about ricing their Gentoo installs, which tiling window managers use the least RAM and which plugins they use with Neovim. This makes using the terminal effectively seem like a big thing.

But the terminal is exactly not that — all proxies are removed and you just use it for what you want to do, nothing more. Just start by using grep on a text file instead of opening that text file up in your editor.