girfan

Interesting. What would the incentives be for NVIDIA, for example, to opt into this? They are building their own racks now, like the NVL72 etc.

Cool tour. I haven't kept up with their developments; what kind of workloads have they been pushing for? Since they don't seem to have any specialized accelerators in the Compute Sled, I am assuming they are not targeting AI workloads for now?

Go open models!

Partly serious, partly in jest: so type systems are no good?

This is super annoying and imo, really limits the usefulness of this model. It speaks volumes about what Anthropic's position as a company and its priorities will be going forward. I doubt this kind of gatekeeping will prevent open-models or other innovation outside Anthropic to slow down. I would imagine these guardrails, if needed at all, should be done at a legal framework level and students should not be a part of this blanket approach to limiting the usage of these models.

I broadly agree with the premise. As a PhD student in Computer Science, I feel there are some significant upsides to my work routine. LLM access has made many new domains more "accessible" to me which I otherwise would be too hesitant in investing my time in. For example, my area of research is computer systems which involves operating systems, distributed systems and more recently systems for AI. Within these, there is a wide breadth of topics/techniques one can employ and up until now, I have not gone deep into theoretical aspects of things like scheduling etc. But with access to LLMs, I feel like I can at least brainstorm from a high-level about these sub-areas that I am not well-versed in and the responses give me some relevant pieces to start exploring on my own, depending on what interests me more or the amount of time I want to spend on that sub-branch of a larger tree of ideas. However, the one thing I do have skepticism is the lack of awareness of blind-spots when dabbling into areas that I am not an expert in, and taking the LLM's lead in applying such techniques to some systems problems that I am working on. I often feel that I am not aware of what alternatives exist that the LLM has not explored for me, or if the directions it has proposed really do apply or have corner cases/assumptions that break in what I am doing. On the other hand, when working on something I have good intuitions about, I am often correcting the model's assumptions and it back-tracks what it told me. Unfortunately, I cannot do that comfortably with topics I don't have good intuition about which limits my confidence in "if this is the right direction to pursue."

Cool post. Have you looked at slicing a single GPU up for multiple VMs? Is there anything other than MIG that you have come across to partition SMs and memory bandwidth within a single GPU?

This seems very interesting. Timely, given that Yann LeCun's vision also seems to align with world models being the next frontier: https://news.ycombinator.com/item?id=45897271

Right now it seems legal to scrape Reddit. But given their trajectory of making the API fairly expensive to use, do you think it's likely that they would also limit/prohibit scraping (assuming apps like Apollo start scraping as an alternative)?

Even if that is the case, it does not say much about their ability to evaluate what category of users (e.g., those coming from Apollo or their first-party clients) is generating more views/interactions and indirectly more "value" on their platform.

That's great!

There are a few cases where `dlmopen` has issues, for example, some libraries are written with the assumption that there will only be one of them in the process (their use of globals/thread local variables etc.) which may result in conflicts across namespaces.

Specifically, `libpthread` has one such issue [1] where `pthread_key_create` will create duplicate keys in separate namespaces. But these keys are later used to index into `THREAD_SELF->specific_1stblock` which is shared between all namespaces, which can cause all sorts of weird issues.

There is a (relatively old, unmerged) patch to glibc where you can specify some libraries to be shared across namespaces [2].

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=24776#c13

[2] https://patchwork.ozlabs.org/project/glibc/patch/20211010163...

Currently it is, yes. I am not sure how fundamental it is. I tried patching glibc to support more (128 in my case) and it seemed to work fine.

You can't load one so/dll multiple times in some sort of container

I believe you can do that with `dlmopen` in separate link maps. I have worked with multiple completely isolated Python interpreters in the same process that do not share a GIL using that approach.

This is a great analysis; thanks for writing.

I have also been working on running multiple Python interpreters in the same process by isolating them in different namespaces using `dlmopen` [1]. The objective on a high level is to receive requests for some compute intensive operations from a TCP/HTTP server and dispatch them on to different workers. In this case, a thin C++ shim receives the requests and dispatches them on to one of the Python interpreters in a namespace. This eliminates contention for the GIL amongst the interpreters and can exploit parallelism by running each interpreter on a different set of cores. The data obtained from the request does not need to be copied into the interpreter because everything is in the same address space; similarly the output produced by the Python interpreter is also just passed back without any copies to the server.

[1] https://www.man7.org/linux/man-pages/man3/dlmopen.3.html

Can you please share pointers to some of the podcasts etc. you listened to? Looking for something similar for non-Bio expert people.

This looks like a cool project. Is there any support (or plan to support) I/O through kernel bypass technologies like RDMA? For example, the client reads the objects using 1-sided reads from the server given it knows which address the object lives in. This could be really benefitial for reducing latency and CPU load.

Cool product! Any plans to support Azure Functions?

Here is something that has helped me a lot. It's a paper on how to read academic papers. You can extract the general idea and apply to many other technical reading material.

https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPape...

Reading this beautifully written piece brings to mind the recent book about Elizabeth Holmes - Bad Blood. Some of the completely unacceptable behavior by tech CEOs and "higher ups" definitely rings a bell; some startups like Theranos were doing exactly that and ended up getting a lot of negative attention.

Just fyi, €80 billion is $87.55 billion.

Take courses across a breadth of areas in Computer Science, even if you think you don't (or won't) enjoy them much. Don't take up courses only in areas you enjoy (say, Networking) and have no concrete knowledge of areas you think you don't really like (say, Graphics) - Undergrad in my opinion is ideal for building up breadth of Computer Science knowledge. You can always focus on specific areas in grad school or when you start working.

This is absolutely amazing. Thanks for the effort. How do we join the slack group for interactive help/questions?

The 15inch MBP has i7 Quad Core processor, Radeon graphics and double the RAM than the default 13inch models. This is not a minor jump in performance. I agree that they do sometimes prioritize size, thickness and beauty over performance but I don't see what else could go into the 15inch MBP that it currently lacks?

goharirfan94 [at] gmail [dot] com

Open-source e-mail backup utility from POP Servers. Bare minimum features. I couldn't find a free/painless utility to download e-mails from an Outlook Web App (OWA) http://office365.com/owa/ account so I wrote this crude script to do the job.

Or they could just give a countdown timer?

Someone please post it here in the comments. We believe in an open internet!

Is this built on top of the standard Unix "diff" program?

This is beautiful!