jackpirate

Coursening is not as elegant as differential privacy. But using coursening is not a "privacy emergency", it is a very-slightly-less-accurate census. And no one knows what the actual economic impacts of this level of accuracy difference are.

I would love to see the more elegant techniques used, and also have the intuition that this is bad policy, but I don't see the "emergency" here. Labeling it as such just histrionic to me.

The name let-go of your programming language is awesome!

I "like" how in their graphic agriculture and cities are both putting water into the lake, and only data centers are removing water from the lake.

The prompter should have redone this image a couple of times until they had all three actually draining the lake.

Also, the racoon it circled isn't in the original.

Edit: Changed this from email because email validation is a can of worms as an example

Email honestly seems much more straightforward than dates... Sweden had a Feb 30 in 1712, and there's all sorts of date ranges that never existed in most countries (e.g. the American colonies skipped September 3-13 in 1752).

Why not have a conversation instead of downvoting. What did I say is wrong?

Your second paragraph is implying that the half of Americans who voted for Trump are "bad Americans". That seems to be sowing the division that your first paragraph warns against (even if it is a reason to dislike Trump).

I don't think either democrats or republicans can claim the moral high ground about sowing division.

What's the origin of XXX? I've seen FIXME/NOTE/TODO all over the place, but never encountered XXX before.

Except those papers are 8ish years old; they actually were among the first 2-3 algs for this task; and they studied the fully general vector space alignment problem. But I agree that naming things is hard and don't have a better name.

We tested all of the methods in the Python Optimal Transport package (https://pythonot.github.io/) and reported the max in most of our tables.

Sorry if I'm being obtuse, but I don't see any mention of the POT package in your paper or of what specific algorithms you used from it to compare against. My best guess is that you used the linear map similar to the example at <https://pythonot.github.io/auto_examples/domain-adaptation/p...>. The methods I mentioned are also linear, but contain a number of additional tricks that result in much better performance than a standard L2 loss, and so I would expect those methods to outperform your OT baseline.

As for the name – the paper you recommend is called 'vecmap' which seems equally general, doesn't it? Google shows me there are others who have developed their own 'vec2vec'. There is a lot of repetition in AI these days, so collisions happen.

But both of those papers are about generic vector alignment, so the generality of the name makes sense. Your contribution here seems specifically about the LLM use case, and so a name that implies the LLM use case would be preferable.

I do agree though that in general naming is hard and I don't have a better name to suggest. I also agree that there's lots of related papers, and you can't cite/discuss them all reasonably.

And I don't mean to be overly critical... the application to LLMs is definitely cool. I wouldn't have read the paper and written up my critiques if I didn't overall like it :)

I hate to be "reviewer 2", but:

I used to work on what your paper calls "unsupervised transport", that is machine translation between two languages without alignment data. You note that this field has existed since ~2016 and you provide a number of references, but you only dedicate ~4 lines of text to this branch of research. There's no comparison about why your technique is different to this prior work or why the prior algorithms can't be applied to the output of modern LLMs.

Naively, I would expect off-the-shelf embedding alignment algorithms (like <https://github.com/artetxem/vecmap> and <https://github.com/facebookresearch/fastText/tree/main/align...>, neither of which are cited or compared against) to work quite well on this problem. So I'm curious if they don't or why they don't.

I can imagine there is lots of room for improvements around implicit regularization in the algorithms. Specifically, these algorithms were designed with word2vec output in mind (typically 300 dimensional vectors with 200000 observations), but your problem has higher dimensional vectors with fewer observations and so would likely require different hyperparameter tuning. IIRC, there's no explicit regularization in these methods, but hyperparameters like stepsize/stepcount can implicitly add L2 regularization, which you probably need for your application.

---

PS.

I *strongly dislike* your name of vec2vec. You aren't the first/only algorithm for taking vectors as input and getting vectors as output, and you have no right to claim such a general title.

---

PPS.

I believe there is a minor typo with footnote 1. The note is "Our code is available on GitHub." but it is attached to the sentence "In practice, it is unrealistic to expect that such a database be available."

It seems like you have some misconceptions about Strassen's alg:

1. It is a standard example of the divide and conquer approach to algorithm design, not the dynamic programming approach. (I'm not even sure how you'd squint at it to convert it into a dynamic programming problem.)

2. Strassen's does not require complex valued matrices. Everything can be done in the real numbers.

As a CS prof, I'd love to have this in my office for students to play with. Looks awesome!

That all make senses to me. But it definitely won't make sense to my intro to programming students. They already have enough weird syntax to juggle.

Building off this question, it's not clear to me why Python should have both t-strings and f-strings. The difference between the two seems like a stumbling block to new programmers, and my "ideal python" would have only one of these mechanisms.

I've seen a lot of examples that use CSS to show the prompt or line number without it becoming part of copied text, and I'm highly in favor of that.

This is unfortunately not compatible with writing the tutorial in markdown to be rendered on github.

I have a minor nit to pick. I actually prefer when tutorials provide the prompts for all code snippets for two reasons:

1. Many tutorials reference many languages. (I frequently write tutorials for students that include bash, sql, and python.) Providing the prompts `$`, `sqlite>` and `>>>` makes it obvious which language a piece of code is being written in.

2. Certain types of code should not be thoughtlessly copy/pasted, and providing multiline `$` prompts enforce that the user copy/pastes line by line. A good example is a sequence of commands that involves `sudo dd` to format a harddrive. But for really intro-level stuff I want the student/reader to carefully think about all the commands, and forcing them to copy/paste line by line helps achieve that goal.

That said, this is an overall good introduction to writing that I will definitely making required reading for some of my data science students. When the book is complete, I'll be happily buying a copy :)

I can talk about concepts like "atoms" or "bacteria" or "black holes" with anyone, and they'll know what they are - even if their knowledge of those subjects isn't in depth.

I'm not convinced this is an unalloyed good. Knowing that a disease is caused by "bacteria" instead of "demons" isn't really helpful if you don't have a deep understanding of exactly what bacteria is. See, for example, all of the people who want antibiotics whenever they're sick for any reason. We've just replaced one set of weird beliefs in the general populace with another and given it a veneer of science.

I think you're wrong.

Suicide does not have stable reporting rates. It was very stigmatized in the past, and so investigators would notoriously report suicides as "unknown cause of death" if they could.

Violent crime, on the other hand, is much more correlated with things like poverty than with mental health.

I think it's quite obviously the case that there are no clear indicators about what "mental health" looked like 100 years ago and there. Any projections into the past will involve a lot of extrapolation and have all sorts of biases.

They very clearly explain why this matters in the "Why should I care?" section. Partially quoting them:

Harry Potter is an innocent example, but this problem is far more costly when it comes to higher value use-cases. For example, we analyze insurance policies. They’re 70-120 pages long, very dense and expect the reader to create logical links between information spread across pages (say, a sentence each on pages 5 and 95). So, answering a question like “what is my fire damage coverage?” means you have to read: Page 2 (the premium), Page 3 (the deductible and limit), Page 78 (the fire damage exclusions), Page 94 (the legal definition of “fire damage”).

It's not at all obvious how you could write code to do that for you. Solving the "Harry Potter Problem" as stated seems like a natural prerequisite for doing this much more high stakes (and harder to benchmark) task, even if there are "better" ways of solving the Harry Potter problem.

The WHO list of essential medicines is not just over-the-counter drugs. It includes things like the chemotherapy drug cisplatin. I happened to need that for testicular cancer ~10 years ago, and the treatment cost was $50k (as "payed" by insurance). That overall seems pretty reasonable to me for the treatment I received, but definitely not something I'd expect the median American to be able to pay out of pocket.

You're wording in this comment (and the twitter/comment video) gives off the same vibes as the google april 1st videos for things like gmail motion (https://www.youtube.com/playlist?list=PLAD8wFTLnQKeDsINWn8Wj...). I honestly thought this was full sarcasm at first.

I don't see how that would have helped in this case. This was not a resource at a known location that was supposed to be only available to logged in users. This was a resource that the admins didn't know about available at an unknown url that was exposed to the public internet due to a configuration error. Are you going to write a test case for every possible url in your server to make sure it's not being exposed?

Something that could work is including a random hash as a first hidden email inside of every client, and then regularly searching outbound traffic for that hash. But that would be rather expensive.

There was rather a lot of NATO coordination in the US-led invasions of both Iraq and Afghanistan. None of the military missions in these countries were in response to the Article V mutual defense clause of the NATO treaty. It's very easy to see how these operations (and therefore the NATO alliance) would be seen as aggressive to these countries.

This is false. Standard sampling algorithms like beamsearch can "backtrack" and are widely used in generative language models.

It is true that the runtime of these algorithms is exponential in the length of the sequence, and so lots of heuristics are used to reduce this runtime in practice, and this limits the "backtracking" ability. But this limitation is purely for computational convenience's sake and not something inherent in the model.

One of the reasons I've intentionally decided not become independently wealthy is that I want to have to explain to other people why I'm doing things. Part of my work is "charity-ish", and by not being able to do things on my own, I'm forced me to improve my communication skills and involve other people in these charity activities. I think that ultimately improves the final outcome, even if the process is immensely more frustrating.

I suspect the numbers would be worse if you looked at households instead of individuals due to declining marriage rates (but I'm not willing to put in the effort to find numbers).

And the next step after this epiphany is that you still have to remember to take the phone with you places, not to leave it behind, and worry about it getting dropped in the toilet by a toddler. Not caring this tool still has a lot of benefits.

Do you happen to have a link to the proposal I can see and share with a class? I'm teaching a few lectures about some "weird" stuff this semester, and this would be a great example.

Then computer security. Unlike the internet or jet engines, these have not panned out as foundational research (except perhaps for some of HIV)

In what word is computer security not a foundational topic? There's lots of reasons to critique the way NSF/NIH/DOD/etc allocate funding, but this is definitely not one of them.

These exercises are writing mathematical proofs that basic machine learning algorithms behave correctly. They are "pen and paper" not because you are manually solving a large equation that a machine would normally solve, but because we don't have automated theorem provers capable of proving interesting machine learning theorems. I would expect a typical 1st year grad student to be using a resource like this.

If you don't understand the purpose of proofs, then this resource is not aimed at you.