pbalcer

This exists in many parts of the world. And is usually reserved for the ultra wealthy, especially if you want it to be near a big city. One famous example is Karuizawa.

Readable code is code that has empathy for the reader and tries to minimize the cognitive load of interpreting it. That's one of the goals of abstraction layers and design patterns.

Yes, it's all subjective, and depends on the reader's expertise and existing familiarity with the codebase. But arguing that code readability isn't at thing, because it's subjective, is an absurd take. Would you claim that Joyce's Ulysses is equally readable as Seuss's The Cat in the Hat?

You'd be surprised how complex a typical memcpy implementation can get to eke out all the performance out of a platform for all the possible scenarios. And while I agree it might not be considered an algorithm in the strictest sense, in response to OP's question, I think memcpy is an apt comparison.

SteamOS is a much more streamlined console-like experience for gaming. Even things as simple as system updates is far less annoying on Linux/SteamOS than it is on Windows. This is especially important in, for example, a set-top box media PC you might want to have for your TV and you don't run every day. And, over time, as SteamOS in its various forms becomes more and more popular, game publishers will be motivated to support it. Many already are, from Steam Deck alone. And, in a few years, it's possible that a "casual PC gamer" will actually prefer the far more plug-and-play SteamOS experience versus the Windows one (which, I, for one, highly dislike, but I understand that's a preferences thing).

One of the reasons why they can do unified memory efficiently is because the CPU/GPU is a single SoC. If you separate them, you end up with a normal PC architecture, with memory having to go through a PCIe bus. This is possible to do with reasonable latency and bandwidth (thanks to CXL), but we haven't seen that in consumer hardware. Even in server space I think only MI300 supports CXL, and even then I don't think it's something AMD particularity promotes.

Personally I think Strix Halo workstations may come with expendable memory, storage and free PCIe slots. But then you have to deal with ROCm...

Disclaimer: I work on this stuff for Intel

At least for Intel, that is just not true. Intel's DPC++ is as open as it gets. It implements a Khronos standard (SYCL), most of the development is happening in public on GitHub, it's permissively licensed, it has a viable backend infrastructure (with implementations for both CUDA and HIP). There's also now a UXL foundation with the goal of creating an "open standard accelerator software ecosystem".

I've been using Gemini-1.5-Pro-2M through Poe for creating e-book recaps and just generally interactively jogging my memory about a prior entry of a book in a series when a new one comes out. It's been working surprisingly well, even for very large books.

Is the alternative "mass hacking"? I thought all this software did was check a box on some compliance list. And slow down everyone's work laptop by unnecessarily scanning the same files over and over again.

I'll just leave this here: https://computeexpresslink.org/wp-content/uploads/2023/12/CX...

Combined with the fact that Intel created both CXL and Optane, it stands to reason that the plan was to combine them eventually. Unfortunately, that was never came to pass :(

Xilinx made triSYCL (https://github.com/triSYCL/triSYCL), so maybe there's some chance AMD invests first-class support for SYCL (an open standard from Khronos). That'd be nice. But I don't have much hope.

Qualcomm and other ARM manufacturers creaming them on AI stuff

That's mostly on Microsoft's DirectML though. I'm not sure whether AMD's implementation is based on ROCm (doubt it).

These are consumer-grade mobile and laptop chips. That are going into products that won't get sold in the US anyway. Oh, and other Chinese companies, that do sale their products in the US, like Lenovo or Oppo, can get these same chips no problem.

So all this is doing is harming Intel and Qualcomm, while incentivizing Huawei to invest even more in domestic chips.

I'm gonna go with Occam's razor. There's no real goal here other than political pandering.

It would require a reverse lookup structure from address to buffer handle, e.g. red-black tree. Maintaining it would no longer be O(1).

Not necessarily. If you are able to map a block of normal memory in a known location relative to the GPU memory that you are managing with an "offset allocator", it should be possible to directly calculate the metadata location for each "offset". This is how most allocators find arenas/buckets (whatever you want to call them) for an allocation by a pointer.

Something like this:

  +-------------------------+ 0x000000 (start of managed memory)
  | Metadata                | 
  |                         |
  |                         |
  | ...                     |
  +-------------------------+ 0x001000 
  | Padding ...             |
  +-------------------------+ 0x010000
  | GPU Memory Block        |
  |                         |
  |                         |
  |                         | ~2MB block
  |                         |
  |                         |
  |                         |
  +-------------------------+ 0x210000

This obviously won't work in all scenarios, but it's a simple and practical way around a map lookup.

Just curious, how does this work out in terms of TCO (even assuming the price of a Groq LPU is 0$)? What you say makes sense, but I'm wondering how you strike a balance between massive horizontal scaling vs vertical scaling. Sometimes (quite often in my experience) having a few beefy servers is much simpler/cheaper/faster than scaling horizontally across many small nodes.

Or I got this completely wrong, and your solution enables use-cases that are simply unattainable on mainstream (Nvidia/AMD) hardware, making TCO argument less relevant?

That's what I have (RX 7900XT on Arch), and ROCm with pytorch has been reasonably stable so far. Certainly more than good enough for my experimentation. Pytorch itself has official support and things are pretty much plug & play.

The reason is physics. NAND just isn't fast enough (and requires firmware wear-leveling, which can mess up latency) for this to be practical.

it's possible to get <1us with Optane PMem (the dimms) in fio, you just need to use the memory-specific engines so that it avoids the fs/block layer.

The point is that on PMem that is simply "sfence", and not a potentially super-expensive "fsync" syscall... Fsync is an fsync, not a memory barrier...

Yup, works just fine. Overall pytorch on ROCm 5.6 has been working very well. I'm impressed with how stable it is, given how much hate AMD driver stack has been getting.

On my machine with AMD RX 7900XT, it takes ~0.17s per image. Are you using SD Turbo Scheduler node?

On a tangentially related note, big companies often say that they perform pay benchmarking by contributing compensation data to third-parties to then determine the market rate for employees. After reading the article, it strikes me as a very similar process, just for a different type of market (housing vs labor).

Did anyone try to challenge pay benchmarking in court?

So? The fact that the most successful Arm server product is a proprietary one designed and used exclusively by one company literally proves OP point. It's much easier to support such deployment if you are its designer and sole user.

There are enterprise database offerings that do something like that with RDMA. I'm almost certain CXL is being looked at for improvements.

+1. This is one of the most underappreciated things about the x86 CPU ecosystem. The documentation is usually solid, and all the Linux drivers are fully open-source and well maintained. Both AMD and Intel are almost always among the top contributors to each Linux kernel release. Hopefully Qualcomm follows suit, and does better than they do with Android... Closed source driver blobs are the worst. I personally wouldn't buy a desktop computer that doesn't at least have an option of running fully with open source drivers.

no, that won't work. You'd have to clflush after every store. And even then, the cacheline might only ever get to the write pending queue (wpq) - and that you can't control.

Even in 2022, gaming was still the largest revenue generating sector for Nvidia...

When you want a clean room non-GPL implementation of something GPL that already exists, you ask the developers not to look at the original. I don't see how this is any different.

I'm not arguing that Sony are saints and Microsoft is the devil. I agree that Gamepass is awesome value and forced Sony to improve their PS+ offerings (competition is awesome!). However, there's a stark difference between Sony buying and investing in game studios after a long partnership (like in the case of Naughty Dog or Insomniac Games), which allows those studios to create bigger and more ambitious games, and Microsoft snapping up multiple large, well funded and profitable, game publishing companies. All I'm really trying to say is that the former deploys capital to fund new games in order to gain market share by competing on quality of the product (which is clearly working for Sony), whereas the latter is deploying capital to just snap up market share directly...

Sony has created or acquihired many of its first party studios that develop original IP specifically for PlayStation. Or they just fund third-party studios to create new exclusive games. You could argue that, if not for Sony, many of those games wouldn't even exist. In recent times, from what I can recall, the only large studio that Sony acquired was Bungie. Which is tiny compared to Bethesda or Activision, and continues to support and release (Marathon) new games on Xbox and Windows.

Now compare this to what Microsoft is doing. They mostly failed to create good first party games on their own (apart from a few gems), and are now resorting to buying up not just individual game studios but whole publishers in order to make their games exclusives to their own platforms.

What Sony is doing is creating real value to players. What Microsoft is doing is primarily meant to create value for shareholders. As a gamer, I strongly prefer the former.