ylow

Just for you I will switch my FTP server to run on Port 23.

Most tech startups will be Delaware. And a lot comes down to the definition of "own" which is ambiguous especially as a C-Corp. One may not be CEO, but can be a H1B Co-founder with a non-trivial (for some amount of non-trivial) number of shares. The O-1 far as I can tell allows for startups, and there may be other visa types which I am unaware of.

Really, I am just saying that the statement "you need a GC to own a business" is far too broad a claim to be true.

There is no obligation to provide the public with his life story. Even if provided, few really understand the US immigration process to really comprehend what it means. And finally, does it matter? Even if deportation is fully legally and ethically justified, do the ends justify the means?

Whether he did really have valid work permits, or not, I have no idea. You seem knowledgeable. But I am just generally mildly frustrated by people online jumping to conclusions assuming malice or criminal intent, while knowing nothing about the US immigration process. It is not surprising that people don't know how US immigration works. Why would one need to unless it is something you have to work with? There are so many misunderstands about H1B, GC, etc.

I do agree that really that the core issue is not with this one particular case, but broadly a pattern of how people are treated, and a failure of due process. People make mistakes. Governments are made up of people who also make mistakes. Process is how you catch mistakes and minimize its occurrence. A failure of due process reduces trust that even fully legal aboveboard immigrants will be treated reasonably and fairly. And that is reducing my confidence that I will be staying in this country long term.

This is not true. You do not need to be a US resident to register a company, and anyone own shares in a company. There are a variety of visa options, and ways to navigate the process that will work.

Its not so complicated. Not everyone wants a green card. It triggers international taxation, exit taxes if you give it up, etc. If you can maintain a work permit for 20 years, why not? Until life circumstances change sufficiently that it makes sense to have a green card, the balance of pros and cons may not lean towards it.

The application, method and algorithm needs to be separated. The application is movie recommendation. One of the methods which works pretty well for this is low rank matrix completion. There are several algorithms for this method, one of which is quantum.

The actual problem that is being solved here is well defined mathematically and is matrix completion via low rank matrix factorization. And using a sampling approach for it. (I have not read the paper in its entirety, just skimmed the intro a bit). It is called "recommendation system" largely due to some history around some of its common appplications (Netflix challenge). But this is not addressing the subjective recommendation problem, but a very particular instantiation of it.

This is using statistics to tell a preconceived story. Underlying this a notion that foreign workers are simply “imported” like they are dug out of the ground or something. How do these STEM OPT people find jobs? Guess what. They interview like everyone else does.

1: Every big tech interview I have been in the visa status is not even a question in the interview process. There is just a simple gate that “can you legally work in the US”? The hiring committee is not even thinking about visa (that’s a HR problem)

2: Are there confounders in that foreign workers are less likely to negotiate? Absolutely.

3: are there confounders in that people who come to US for study are likely already a self selected bunch who are striving to succeed? What are the typical grade distributions between foreign STEM students and US STEM students? Is grade a confounding variable? What happens if we control for GPA?

And finally does H1B abuse happen? Absolutely.

There is a lot of nuance that are not captured by surface level statistics. But nuance does not make outrage.

Even just audio transcription can hallucinate in bizarre ways. https://arstechnica.com/ai/2024/10/hospitals-adopt-error-pro...

I think both are necessary. The cdc technique is file format independent. The row group method makes Parquet robust to it.

We are here to help lower that :-) . As we can push dedupe to the edge we can save on bandwidth as well. And hopefully make everyone upload and download faster.

Great question! Rsync also uses a rolling hash/content defined chunking approach to deduplicate and reduce communication. So it will behave very similarly.

I believe Parquet predates Arrow. That's probably why.

Can you elaborate? As I understand Delta Lake provides transactions on top of existing data and effectively stores "diffs" because it knows what the transaction did. But when you have regular snapshots, its much harder to figure out the effective diff and that is where deduplication comes in. (Quite like how git actually stores snapshots of every file version, but very aggressively compressed).

Hi all! Yucheng (CEO XetHub) here, happy to answer any technical questions anyone might have. Our current tech is a significant enhancement over the original Git Is For Data paper we published last year https://www.cidrdb.org/cidr2023/papers/p43-low.pdf . Hope to write more about it soon! (Maybe with follow up paper or at minimum, a blog post)

The dedupe is optimistic and is designed to scale to 1-10 PB range. There is a more complicated architecture blog post we are working on. We can dedupe across repositories but we do not right now largely for privacy reasons so that blocks are not shared across different people as that can cause information leakage.

(Yes 71 + 27 != 100, but that LLMs can't count is a whole other issue)

Indeed this is unsurprising given how LLMs work. I mean if you ask a human to generate a random number, and then reset the universe and all state of the human and ask again, you will get the same number.

But instead if I ask it to generate 100 samples, it actually works pretty well.

"You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Generate 100 samples of either "left" or "right". Do not say anything else. "

I got 71 left, and 27 right.

And if I ask for 50%, 50%. I get 56 lefts and 44 rights.

I think this is because a graph is not a data-structure nor a data-type. It is really an abstraction.

Fundamentally, all I need to define a graph is a set of vertices v \in V and function Neighbors(v). And that really is all is needed for the most foundational set of graph algorithms.

Everything else are case-by-case constraints. Does A->B imply B->A? is the node set partitionable with certain constraints? Are there colors? labels?

To make things even more general I can go up one level and consider the hypergraph. In which case I just have a set of vertices, and a set of sets of vertices. This can be represented in a myriad of different ways depending on what you are interested in. Of which (non-hyper) graph is simply a special case.

An alternative way to think about it perhaps from the database perspective, is that its a query optimization and indexing problem. Depending on what questions you want to ask about the graph, there will be different ways to represent the graph to answer the question better. Just like there is not one way to represent the abstraction called "Table", there is not one way to do "Graph" either. It really depends on the questions you care about.

The issue comes when cells take many minutes or even hours to run (intentionally or not). The ideal is indeed sequential, and this helps me a lot with maintaining the sequential ordering as it simplifies and speeds up the “restart and run all” process.

Author here. Mandala looks really cool. Thanks for the recommendation!

Author here.

The goal in a way is better reproducibility. The memo hashes the contents of the inputs, so if your cell is deterministic (and you cover all the inputs), the memo should give you the right answers. Of course if you want to rerun everything you can always just delete the memos, but properly used it should be a strict performance improvement.

Of course there are many improvements that can be made (tracking if dependent functions have changed etc) and of course there are inherent limitations. This is very much “work in progress”. But is quite useful right now!

Some random historical notes coroutine stuff which are perhaps of interest to some:

1. You can implement "stack-free coroutines" in C with some really entertaining macros https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html

2. Does anyone remembers Cilk? Its still the fastest stackful coroutine models I know of.

Having implemented and used both before (for different purposes), I like both, but with a mild preference for the stackful version. Largely because you don't have function coloring problems, and you can actually get a proper stack trace and is so significantly easier to debug when things go wrong.

It is also good to differentiate coroutines from async (they seem to be very interleaved these days). Coroutines are a mechanic to achieve mutual recursion / generators. And that is fine for expressing certain algorithms and systems in a much cleaner fashion. Note that parallelism is not necessarily implied by "coroutine".

Async is a mechanic to "doing something else" while waiting for IO. Fibers or state machines are both different solutions to this problem. Certain coroutine implementations can help with this, but I think it is massively overused. Rust due to the function coloring problem, ends up requiring almost everything to be marked "async". And some golang code I have read seems to overuse goroutines unecessarily. I think use of async should be narrow and minimized, and localized to only the places where it makes sense.

(Love the Gluster project btw. Thanks for maintaining it!)

Nice! I did look at Ganesha when I was exploring other NFS server implementation I could perhaps borrow and simplify. It looked great, but pretty much all implementations are too "real-filesystem" for me to convert. I needed something which I can make a very simple virtual filesystem interface over, and so wrote my own.

I don't know that its considered a "design mistake". It is not without flaws, but it does make it extremely simple to implement. Its a very acceptable point as a tradeoff between simplicity and performance. Will certainly love to implement NFS4, or even SMB3 at some point though.

Another possibility I am interested in is SMB3. (SMB4 is rather nasty). Similarly, supported by most operating systems, and does not look too horrifying to implement (The protocol documentation is really really hard to read though). And has the advantage that it does not require Windows Pro edition.

Very nice. The mount protocol is pretty minimal, though different OS clients seem to use it differently. The NFSv3 chattiness is definitely a thing, but since its all localhost its not "too bad". v4 will be nice and is something I will love to try to implement. Its quite a bit more complex though.

Interesting. I do not know much about webdav. Great to learn something new! Will take a look.