mattb314

Super rough summary of the first half: in order to pick out random vectors with a given shape (where the "shape" is determined by the covariance matrix), MASS::mvrnorm() computes some eigenvectors, and eigenvectors are only well defined up to a sign flip. This means tiny floating differences between machines can result in one machine choosing v_1, v_2, v_3,... as eigenvectors, while another machine chooses -v_1, v_3, -v_3,... The result for sampling random numbers is totally different with the sign flips (but still "correct" because we only care about the overall distribution--these are random numbers after all). The section around "Q1 / Q2" is the core of the article.

There's a lot of other stuff here too: mvtnorm::rmvnorm() also can use eigendecomp to generate your numbers, but it does some other stuff to eliminate the effect of the sign flips so you don't see this reproducibility issue. mvtnorm::rmvnorm also supports a second method (Cholesky decomp) that is uniquely defined and avoids eigenvectors entirely, so it's more stable. And there's some stuff on condition numbers not really mattering for this problem--turns out you can't describe all possible floating point problems a matrix could have with a single number.

The idea is that memory-only data systems like HyPer are able to make design decisions that make them significantly faster that disk-based systems (eg postgres), even when the working set fits entirely within cache for the disk-based system. Umbra attempts to act like an in-memory DBMS when the working set fits in memory while degrading gracefully as the working set grows beyond memory. Agree the title doesn’t have enough detail to see this though.

Wonder if this has anything to do with the sliding window:

Sonic only keeps the N most recently pushed results for a given word, in a sliding window way (the sliding window width can be configured)

Default window looks like 1k documents. I read this as saying that super common words are basically dropped from the index (only 1k out of many thousands of docs retained), but I don’t know enough about the internals to be sure. Not sure if this actually hurts search results in practice, seems like an ok trade off for help docs at least.

Joules-Thomson effect is what allows for refrigeration

I don't think this is true. The "simple" model of refrigeration taught in highschool is just a carnot cycle running backwards, and this can be modeled with an ideal gas. The author of the post covers this the section on "the Thermodynamics 101 Answer"[1], where all you need to drop the temperature of a gas is to let it do work on the piston.

That's not to say that JT is not useful, just that we can explain a theoretical refrigerator without it.

[1] https://mattferraro.dev/posts/joule-thomson#the-thermodynami...

Heads up this weights all your scores towards 0. If you want to avoid this, an equally simple approach is to use (x+3)/(y+5) to weight towards 3/5, or any (x+a)/(y+b) to weight towards a/b. It turns out that this seemingly simple method has some (sorta) basis in mathematical rigor: you can model x and y as successes and total attempts from a Bernoulli random variable, a and b as the parameters in a beta prior distribution, and the final score to be the mean of the updated posterior distribution: https://en.wikipedia.org/wiki/Beta_distribution#Bayesian_inf...

(I saw first this covered in Murphy's Machine Learning: A Probabilistic Perspective, which I'd recommend if you're interested in this stuff)

Free fillable forms is hard to use for taxes in general, but this page is specifically for people who want the CTC without having filed taxes (presumably you otherwise would have gotten the CTC when you filed takes). You can see the screenshot says "non-filer sign up tool", where as the main page of the website (for people who want to actually use it to file taxes) has no such warning: https://www.freefilefillableforms.com/#/fd

I doubt it? At least the number times the "last updated" column appears on SQL server stats [1] leads me to believe it collects stats async with updates to the table.

The only system I've heard of that relies on up-to-date statistics for correctness is snowflake (long but interesting talk here [2]), where having accurate max/mins for each micro partition is really helpful for cutting down the amount of data in the large range scan queries common in BI. I'd guess that being a BI system, snowflake can get away with higher row update latency too.

[1] https://www.sqlshack.com/sql-server-statistics-and-how-to-pe...

[2] https://www.youtube.com/watch?v=CPWn1SZUZqE

I think I generally agree with the majority of the comments here that burn in can serve a useful purpose (especially if you can't find a high probability density point to start from), but I also wonder: if burn-in vs no burn-in makes a large difference in your outcome, aren't you likely just not running your chain long enough? Sure, if you choose a bad starting point, your initial samples might not be representative of the overall distribution, but if a handful of non-representative points can massively impact your result, then I'm not sure how stable your result was to begin with (how do you know there isn't some other set of low-probability high-impact points that your sampler just missed through luck?). People tend to have a cognitive bais towards distributions looking pretty (eg not having random chains off to the side as in the article), but I'm not sure it makes a real difference.

That said, I do think burn in is a pretty reasonable way to find a good starting point if you don't have existing knowledge about the distribution. From a practical standpoint, has anyone actually seen a massive difference between runs with/without burn in? kinda curious how often it really matters

If you're new to differential privacy and looking for an introduction, I highly recommend the Dwork and Roth book, especially the first three chapters: https://www.cis.upenn.edu/~aaroth/Papers/privacybook.pdf

Frank McSherry also has some good resources if you enjoy his writing style: https://github.com/frankmcsherry/blog/blob/master/posts/2016...

In particular, I think it's important to keep in mind that differential privacy is as much focused on establishing a framework for measuring information leakage as it is coming up with clever algorithms to preserve privacy (although there are a lot of clever algorithms). I think of it as more analogous to big-O notation (a way of measuring) than to dynamic programming (an implementation technique).

Personally I'm not super bullish on differential privacy outside a couple specific use cases, but correlation attacks and cross referencing against external data are exactly the vectors that differential privacy is intended to protect against: it requires that the results of any query or set of queries would be identical with some probability even if a specific person wasn't present in the dataset.

It's possible I'm misreading, but your paper seems to focus on the very anonymization techniques diff privacy was invented to improve on, specifically because these kinds of attacks exist. While I agree it's no silver bullet, the reason is because it's too strong (it's hard to get useful results while providing such powerful guarantees) rather than not strong enough.

I've found the introduction to this textbook on it to be useful and very approachable if others are interested: https://www.cis.upenn.edu/~aaroth/Papers/privacybook.pdf

I'd also like an answer to this one. It seems like most of these explanations would also apply to plastic or metal getting wet, so why does cloth get so much darker than other materials?

It feels like the fact that water is absorbed has to matter here--wood, dirt, cloth and sand all darken when they absorb water but not when the water remains in a bubble on the surface (as it does on cloth/sand/wood treated with a hydrophobic coating).

This is basically the issue. If a single line from the bottom of left is present at the top of right, it will fast-forward all the way through left and fail to match anything else from right. The hard part with diffing isn't finding differences, it's deciding whether they're inserts, edits, or deletions.

re: efficiency, I think this blog post does a decent job explaining: https://auth0.com/blog/beating-json-performance-with-protobu.... Basically, it doesn't make a huge difference if you're communicating with a Javascript endpoint, but Java to Java (or probably between other non-js backends) you can save a lot of time on serialization/deserialization. It's worth noting that the post uses fairly large blobs (50k people and addresses), and you probably won't see as big of a difference on smaller requests.

I might be missing some context here, but usually the L_infinity norm means the max norm, or the maximum absolute difference over all components (data points). This gives (min + max)/2 as GP suggested.

Wikipedia entry: https://en.wikipedia.org/wiki/Norm_(mathematics)#Maximum_nor...

For what it's worth, I think the confusion of your comment and jules's below comes from the idea that the lp-norm is sum(x_i^p), when it's actually [sum(x_i^p)^1/p] (please forgive my notation). Since raising to the 1/p power is monotonic, it doesn't actually make a difference in minimizing or maximizing the norm, so people often use them interchangeably.

To be honest, I'm a little confused by your answer. Aren't high contention and deadlocking workloads essentially the same thing to a deadlock-detection db? I can imagine some patterns where that wouldn't necessarily be true, but they seem to go together in general. It also seems like your suggestion in the case of deadlocks is to modify your transactions to acquire locks in the same order, which is exactly what (many) avoidance systems do for you in the first place. What's the advantage of putting this burden on the user?

Does anyone know why they use deadlock detection rather than deadlock avoidance (the simplest avoidance algorithm being simply to always acquire locks in the same order)? I can't find a good reference right now, but I was under the impression that deadlock detection, especially distributed detection, is extremely costly and basically precludes high contention workloads, but maybe I'm confused here.

The only two reasons I could think of were 1. You can't know all the locks you want in advance (because the postgres api doesn't require it), or 2. Citus doesn't expect customers to have high contention workloads where deadlocks are likely (or at least they thing they can be reduced primarily to single node deadlocks).

Am I overestimating the cost of deadlock detection?

Even assuming you meant "upper bound" instead of "lower bound", this isn't strictly true in theory. To achieve high throughput with large latency between nodes, all you have to do is group transactions into large batches and reach consensus on how they should be ordered (for example, you can run paxos to decide what a batch consists of, and then use any method you like to order within the batch). Running paxos at interplanetary latencies would obviously take some time, but since you can have multiple rounds in progress at once (or increase the size of each batch to be much larger than the latency), throughput isn't limited by latency. In practice, most people care a lot about latency, and achieving the desired latency puts a limit on throughput. See the Calvin DB paper for a more thorough explanation.

I'm a little confused about the columnar database comment:

Performing queries across billions of metrics looking for labels that only match a few of them (a common scenario with time series data at scale) is really slow in Cassandra. This is because of the way it stores data in columns. This extends to any columnar database including Google's BigQuery which all have a natural disadvantage with time series data.

I've pretty much only heard "columnar database" used as opposed to row store database, and it seems like storing time series data in columns makes much more sense. Could someone clear up exactly how "labels" (which I probably don't understand) are so much harder for column stores to deal with?

Worth noting that (I'm 99% sure) you're absolutely allowed to benchmark Oracle, just not publish the results publicly. It's pretty common for DB papers to compare against postgres and a number of "mystery" databases because the authors' license agreement with Oracle prevents them from naming it.

Could you link/explain how you get those hardware prices, specifically the servers and desktops? Are the available for the average consumer (ie not making a bulk purchase)? I have very little experience buying hardware, but those prices sound 4-8x lower than what I'd expect (and I'd be super happy to find that I'm wrong).

I think the idea of spending to upgrade your helper AI sounds really interesting and potentially novel, but it's important to push back a little bit on the idea of "removing all the rote" from an RTS. day[9] (ex Broodwar pro, current caster of many games) probably makes the argument better than I can here: https://youtu.be/EP9F-AZezCU?t=55, but one of the important implications of having repetitive aspects in an RTS is that they turn the player's attention into a resource. You can either concentrate on microing your units to squeeze the most possible value out of them, or you can focus on hitting every single production cycle back at base, but you can't do both at the same time, so deciding when to focus on what becomes part of the game. Additionally, it allows for some crazy "overcoming the odds" scenarios where a small number of units can pull off a surprising win because the player was willing to donate far more attention to them than usual. Basically, an RTS where you don't feel like you have too much to do all at once is really more like a turn based strategy game (which can be fun too).

I'm certainly not long on uBeam (even after this fragile demo), but what physics dictates that it won't work? Sure, you're certainly going to take an efficiency loss, but wall outlets already output far, far more power than your phone uses while charging, so the real questions are how inefficient is it, how safe is it, and how convenient is it (can a final product survive if it needs line of sight?). While these are certainly very difficult and perhaps intractable engineering problems, I'm not aware of a fundamental physical limit on any of these that would limit an application like phone charging.

Thanks! This talk is pretty cool.

In case anyone else is interested, in addition to the above video, this [1] paper goes into some of the details, including this explanation:

"Reads and writes on SSDs can interfere with each other for many reasons. (1) Both operations share many critical resources, such as the ECC engines and the lock-protected mapping table, etc. Parallel jobs accessing such resources need to be serialized. (2) Both writes and reads can generate background operations internally, such as readahead and asynchronous write-back [6]. (3) Mingled reads and writes can foil certain internal optimizations. For example, flash memory chips often provide a cache mode [2] to pipeline a sequence of reads or writes. A flash memory plane has two registers, data register and cache register. When handling a sequence of reads or writes, data can be transferred between the cache register and the controller, while concurrently moving another page between the flash medium and the data register. However, such pipelined operations must be performed in one direction, so mingling reads and writes would interrupt the pipelining."

Also relevant to the "seek" conversation below: it turns out many SSDs have read-ahead caches built in, so sequential reads are much faster than random reads after an initial warm-up, just like hard disks (the difference, however is closer to ~5x than to 100x difference you see in HDDs).

[1] http://bit.csc.lsu.edu/~fchen/publications/papers/hpca11.pdf

Do you think you could point me towards a source for "mixing reads & writes with a SSD lowers performance"? This seems reasonable, but I've never actually heard it before, and a cursory googling didn't turn much up. I have heard before that small, random writes on SSDs are much more expensive than sequential writes because random writes can produce write amplification and increase fragmentation. It was my impression that SSDs stand to benefit significantly from log structured filesystems despite the mixed read/write load, but I don't follow it that closely and could be wrong.

Slightly related question from a non-expert in ad-blocking: is it possible to block interstitial ads? While the freedom to render a page as you wish seems to come with controlling the browser on your computer, I can't imagine any way to circumvent the ads the obstruct the entire page and force you to wait for a few seconds before seeing content. Obviously you can just choose not to render them, but if the server can just delay serving article content until X seconds after the interstitial loads, it seems like the fundamental advantage here lies with the publisher and not the user. While submarine ads and lobbying are obvious responses to the Ad-blocker proposed above, wouldn't interstitials (which personally bother me much more than sponsored facebook posts) also gain popularity?

Agreed. Especially given that these rate limits seemed to be aimed at stopping something catastrophic like spammers using 100x allotted capacity, a 2x innacuracy shouldn't really matter. The solution was interesting, however, and I could see it being useful in a situation where users are expected to run very close to their rate limits. For example, I could see AWS being fairly careful about not letting anyone use more than their allotted compute/network bandwidth because getting double bandwidth without paying for it is a pretty big deal.

But depending on node failures, couldn't the same client successfully run cas(nil, A) -> cas(A, B) -> cas(nil, C), with all operations succeeding? Say the first two operations only succeed in writing to a single node (as in the post's example). Then if that single node goes down, the third cas will succeed, which is certainly not linearizable. Note: I'm working of the assumption that the system is correctly described by the posts author. The original author of Gryadka has disputed the description in the post, and I haven't read the source.

The source: https://www.glassdoor.com/research/studies/gender-pay-gap/

The methodology: https://www.glassdoor.com/research/how-we-analyzed-the-gende...

I didn't find the editorializing in the cnet article very helpful.

You can actually transmit this information in a single bit: just have your friend send a 0 if you should answer false on everything or 1 if you should answer true on everything. To use the other 11 bits, transmit the first 11 answers first, and then encode whether to guess true or false on all of the remaining 12 questions in the last bit. This method gives 11 + (23-11)/2 = 17 answers correct in the worst case, but (according to the author) this is still suboptimal.

Awesome! I'll keep an eye out.