borramakot

I don't hate FEE, but there's a lot of opinion in this article above and beyond the original source. Maybe link to that? https://www.mckinsey.com/industries/public-and-social-sector...

Given the author mentions multiple cores being available, I'd guess you could use any method, including MPI, to distribute the computation. But whether you used 1 core or 10k cores, it would be nice to have a 20x speedup on each core via this arithmetic/fixed size optimization. Since that's the focus of the article, communication technologies feel pretty unrelated.

I don't know much about MPI, but I thought it related to message passing/distributed workloads rather than arithmetic performance?

The amount of effort and extra hardware required to maintain this reproducibility across different hardware (cpu generations, GPU hardware/drivers, etc) feels insane to me. For example, people seem extremely interested in performance, but insist on using the much slower modes in MKL to maintain the bottom few bits of their golden data.

Do you find this genuinely important? If not, have you had any success encouraging e.g. validating against golden data in a non-bitwise comparison?

If I recall, the zstd binary has a mode that is basically "adapt the compression ratio so that compression isn't the bottleneck", but I didn't have a chance to try it. Have you used that?

The parent article doesn't seem to be trying to be a fully fleshed out survey of compressors, but I found a separate comparison of zstd and Brotli: https://peazip.github.io/fast-compression-benchmark-brotli-z...

tldr: zstd seems to be somewhat more flexible, generally provides slightly better performance across almost all metrics, and decompresses much more quickly than Brotli.

I understand cause of death statistics can be calculated, but do you have an article on what happened with a decapitation being counted as a covid death?

My understanding is that the Texas state requirements used to be borderline insane, but that they have improved over time. Skimming http://ritter.tea.state.tx.us/rules/tac/chapter113/index.htm... , I actually found much less objectionable than I expected to.

There's no way to get the raw data from this site, is there? I have long been interested in doing some analysis of how syllabi have changed over time, but that doesn't seem possible with this interface.

Are are the TPUs in Chisel? I thought only a subset, like edge, were in Chisel?

(I have fairly minor quibbles with some of Nate's modeling ideas, but I broadly mean to be defending him).

I don't mean to imply 2016 was a black swan event- I agree that ~30% was probably as accurate a take as could be achieved (most evidence that seems reasonable to use indicated a lead for Clinton, but that it wouldn't be that surprising for that lead to be overcome). I just mean that the model assumes a fairly normal election environment, without like a huge attack on Election day or something on election day.

The N=3 comment was meant specifically for evaluating their calibration, not the data they use for their model.

I don't think this article provides strong evidence that they are well calibrated on the presidential election specifically (sample size N=3), or that they are correctly accounting for rare black swan events, but it does seem to imply that the criticisms about "538 claims victory no matter what because they always have non-zero probabilities" are oversimplified.

Boy, have I got the incredibly specific article for that exact question: https://fivethirtyeight.com/features/when-we-say-70-percent-...

You say formerly catapult- is catapult no longer being developed?

LZ4 seems really nice for data with a lot of repetition, but not having any symbolic entropy encoder can really kill the compression for some data. I often default to LZ4, and used to fall back to gzip if that doesn't work well, but I've been really impressed with zstd. It's not as commonly used, but if I'm not concerned about interoperability, I'll be trying to use a lot more zstd.

So, the main result of designing a CPU is a series of masks that essentially indicate where to put what. For example, in this layer, inject boron anywhere the mask doesn't block. The masks aren't wafer sized- they are pretty small, and a machine moves the mask from position to position across the wafer to re-use it. But, at least when I was working on this, some masks would be larger than an individual square (die)- maybe the mask could do 2x2 at a time. In that case, maybe the application of the mask would get you one complete die, and three die off the edge.

How so? It seems like lots of businesses run successfully on that model for indefinite periods.

I may not have said it well, but I broadly agree with you. If a workload needs high performance but not consistently (e.g. because you're doing serial tests by swapping bitstreams), predictably (e.g. because you need flexibility for network stuff you can't predict at design time), or with enough volume (e.g. costs in the low millions are prohibitive), an ASIC isn't the right solution.

But my point is that for FPGAs to come to prominence as a major computation paradigm, it probably won't be because it outperforms GPU on one really big workload like bitcoin or genetic analysis or something. It'll have to be a moderately large number of medium scale workloads.

Just to throw in one more complication, I'll assert that the only benefits of FPGAs over ASICs are one time costs and time to market. Those are big benefits, but almost by definition, they aren't as important for workloads that are large scale and stable. So, if you do have a workload that's an excellent match for FPGAs, and if that workload will have lots of long term volume, you should make an ASIC for it.

So, for FPGAs to be the next big thing in HPC, you'd need to find a class of workloads that benefit from the FPGA architecture, for long enough and with high enough volume to be worth the work to move over, and are also unstable or low volume enough that it's not worth making them their own chip.

I agree that's a really striking and suggestive result, but keep in mind the sub-sample size there is four passing on the private side and six failing on the public side. My guess is there's a real and meaningful effect there, but it may not be to the degree that sentence without context would suggest.

I've wondered for a while- are there heuristics on the income a steamer can make per viewer-hour?

Edit: I'd googled this before, but apparently I found the right way to ask: this site says from 1c to $1/hour/viewer. https://wallethacks.com/how-much-do-twitch-streamers-make/#:....

Depending on your definition of cheap, fab access might be pretty cheap right now for old technology nodes, which have totally reasonable performance if you have an architectural advantage.

This article suggests mask tapeout costs are under $1 million in older nodes, sometimes well under. If you have an architectural advantage in a problem domain with tens of millions or more in costs, a simple ASIC can be very worthwhile. That architectural advantage might be hard to find, especially when problem domains aren't fixed for long periods of time (e.g. how many ML accelerators only really work well for dense convolutions?), but I suspect too few companies are making custom chips, rather than too many.

https://www.electronicdesign.com/technologies/embedded-revol...

I don't know much about genomics, but I think there are at least FPGA based accelerators in production, for example, Illumina's DRAGEN.

https://www.illumina.com/products/by-type/informatics-produc...

I've done a startup, a couple of midsize companies (~4k engineers), and AWS. Just my 2c, a lot of the advice I'm seeing here applies more to the midsize companies than at least my corner of AWS.

In my Amazon experience:

* Some very high level project requirements would come from above (e.g. after this date, internal technology X is being deprecated, so you should have a really good reason to put out a project with X).

* Otherwise, decisions were mostly made at a low level, documented, debated with the wider team for an hour, then implemented. This was a little more structured than at the startup, but most of it was that documentation and debate happened before implementation, rather than after implementation at the startup.

* Project managers were somewhat active with the team, but a lot of the features we worked on came from the engineers watching what was used, forum requests, or customer requests through other channels (e.g. conferences).

* There was a focus on getting products out quickly, but tech debt/tests/reliability was a much bigger focus than anywhere else I've been.

* The team was fairly small, and encouraged to make heavy use of other team's internal tooling/native AWS tools for anything that didn't really need to be custom. Interactions with those teams was pretty straightforward and mostly supportive- "We're using your service to do Y, and would like to do Z too, but that doesn't seem possible without some tweaks to your API/service, is that something you can put in the backlog to investigate?"

* An individual team could be quick to change, but the organization as a whole has a lot of cultural momentum in the way things are done, and it's not clear who to talk to to make recommendations. For example, at the startup, I could go to the CEO and express concerns about the newly restrictive information security policy. At Amazon, I'm probably not going to email Jeff Bezos and suggest six-pagers be made available in advance of meetings.

* Transferring teams in Amazon is mostly extremely easy

* Conversely, Conway's law applies hard in AWS- it didn't seem straightforward to offer products or features that weren't obviously under one team's purview without forming a new team.

All the offer's I've gotten have been pretty solidly in the bell curve of what levels.fyi indicated.

Is this a criticism of the ARM chips, or of the TPU's bfloat format?

I've not read Popper directly, so I'd be interested in his actual argument on this.

But, I wonder if you can describe H1 as being a stronger hypothesis than H2 by virtue of withstanding more and higher quality attempts to disprove it?

I found the hash table lecture really interesting, somewhat practical, and it made me sound really smart when I got the inevitable hash table questions during interviews.

Is there data on what tech companies programmers (either generally or new grads) most want to work for? It feels like it used to be Google by a lot, but even outside the HN bubble, I get the sense Google has taken some reputation hit.

They say later that

When a feedback instrument surveys eight colleagues about your business acumen, your score of 3.79 is far greater a distortion than if it simply surveyed one person about you—the 3.79 number is all noise, no signal.

Which implies to me that they believe there is signal there, but that it goes away when aggregated?