mratsim

In my case with vLLM on dual RTX Pro 6000

gpt-oss-120b: (unknown prefill), ~175 tok/s generation. I don't remember the prefill speed but it certainly was below 10k

Nemotron-3-Super: 14070 tok/s prefill, ~194.5 tok/s generation. (Tested fresh after reload, no caching, I have a screenshot.)

Nemotron-3-Super using NVFP4 and speculative decoding via MTP 5 tokens at a time as mentioned in Nvidia cookbook: https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemo...

Exact instruction challenge https://www.youtube.com/watch?v=cDA3_5982h8

Years ago I started a collection of convolution optimization resources: https://github.com/mratsim/laser/wiki/Convolution-optimisati...

Also checked and apparently Nvidia Cutlass now supports generic convolutions: https://github.com/NVIDIA/cutlass

The Internet Engineering Task Force, which in particular standardizes TLS, has acknowledge my contributions for my contributions to the very generic and globally useful hashing-to-curve primitive: https://datatracker.ietf.org/doc/html/draft-irtf-cfrg-hash-t...

And BLS signatures are standards as well: https://www.ietf.org/archive/id/draft-irtf-cfrg-bls-signatur...

BLS signatures are general purpose, the only Ethereum specific thing is choosing which elliptic curves are used for public keys and which for signatures as well as a prefix before hashing inputs.

Similarly KZG polynomial commitments are generic, the Ethereum specific thing is the trusted setup ceremony which needs to be specialized per protocol.

The finite field backends and the elliptic curves supported are all generic as well: https://github.com/mratsim/constantine/blob/master/constanti...

Great question, first of all I did extensive research https://github.com/mratsim/constantine/wiki/Constant-time-ar....

Then the library defines its own types SecretBool and SecretWord throughough library that cannot be mixed with regular Nim types and are based on uint32/uint64 so the compiler cannot assume 0/1 and do boolean logic.

Assembly is used to avoid compilers getting too smart and reintroducing branches but the no-assembly fallback should still compile without branches.

By convention, all variable-time cryptographic functions are suffixed _vartime.

There is a plan to add automated constant-time checks as well https://github.com/mratsim/constantine/issues/7.

Author here, adding an accompanying post focused on performance: https://ethresear.ch/t/releasing-constantine-v0-1-0-a-modula...

According to the US, an US citizen just have to open a bank account anywhere in the world and the law applies, regardless of where the bank is hosted.

Reminded me of Bastiat's candlestick maker's petition:

http://bastiat.org/en/petition.html

Why is it different?

People don't have to comply to GDPR but if they want to serve EU folks then they don't have a choice.

What would be the 7 dimensions?

That was my reasoning for this "restriction".

Author here,

Nim supports variadic generic, it's an arbitrary limitation so that shape and stride small vectors that describe a tebsor can be stack-allocated and fit in a cacheline.

Also at the time, Nim default heap allocator was not compatible with OpenMP.

Edit: it can be configured via a compile-time flag to 8 or 10 or anything: https://github.com/mratsim/Arraymancer/blob/master/src%2Farr...

They always take two months at the very minimum as well because buyers as to go to many banks or a loan broker. You have notaries involved and right to step away from a sale, no question asked for 10 days iirc.

Overpromise and underdeliver.

And lots of funding for then vaporware

idiv is not constant-time on any ISA anyway ;)

Percival cache attacks on Hyperthreading.

I go over some examples here: https://github.com/mratsim/constantine/issues/358#issuecomme...

I've written and given a lot of references to attacks, countermeasures, fighting the compilers and even non-cryptographic side-channel attacks to uncover someone's Twitter identity:

- https://github.com/mratsim/constantine/wiki/Constant-time-ar...

Venmo, CashApp, Paypal "Friends or Family" and gift cards (Apple, Google Pay, Steam, ...) are all irreversible.

The same problem as with passwords that has been plagging the security industry for decades.

Passwords, credentials and other high-risk high value intangible data are not associated with high-risk high-value AND our memory is not designed for random junk.

Maybe an anthropologist can study this phenomenon further.

We already have hints of that: our memory has evolved for thousands year to easily memorize places and spatial navigation, in thr past for food, water and danger. Today you go in a foreign place, just one visit and you know where everything is. At the same time, people can usually hold only ~7 concepts/numbers/objects in their head.

In memory competitions people leverage our spatial prowess with a technique called the memory palace, that was already used in ancient Greece and Roman Empire to recall anything, from bard tales to the Iliad and Odysseus.

https://en.m.wikipedia.org/wiki/Method_of_loci

Or for some there is a national base price.

It is a French company so maybe they have extra French datasets?

I've been quite disappointed by French LLMs on Huggingface when I tried a month ago.

In a way, I think you're absolutely correct that it's harder to change anything, but not because the method is wrong; it's because it addresses the fundamental issue of traditional budgeting and it's difficult and tough to deal with the consequences: you can only budget the money you actually have. You can't make up numbers about what a month usually looks like.

You can predict what your month will look like based on your rent, utilities and how expensive are your groceries from the past months.

The unpredictable is accident and sickness but that's what insurances and emergency funds are for.

When there is an emergency to break into a house, the police needs to get a mandate from a judge.

GPU drivers provide an event system:

- Cuda: https://github.com/mratsim/weave/issues/133 - OpenCL: https://github.com/mratsim/weave/issues/134

GPTs are a little better, but I still don't see the product market fit.

If ChatGPT doesn't have product-market fit, what actually has?

What would that check do? Check IPs?

The fact that it was not enforced doesn't mean there wasn't a sword of Damocles above your head.

But what happens if the company you're trading with has non-US entities? Or traveling salesmen?

The issue here is the complexity of complying and closing all loopholes that would allow the government to bring the full weight of the legal system against a library writer.

I disagree.

To fully comply with this you would need as a library provider to fully KYC your clients so that there is a firewall between their US and non-US entities, and that travelling people don't bring out an encryption library at the same time.

It would be a operational nightmare.