withoutboats3

I don't remember the details with encryption but it was basically you have to ship a breakable version for the rest of the world, and you generally sometimes ship a backdoored version.

I do remember the details: the result of Bernstein v. United States was that you have a First Amendment right to publish code because it is a speech act and so the USGOV cannot prevent you from publishing effective encryption algorithms. Will model weights be afforded the same protection? What about serving a model without publishing its weights? We shall see.

Your anti-semitism is vile and you should be shunned by any upstanding person in society.

This is indeed a great quote (one of many gems from Sir Tony) but I think the context that follows it is also an essential insight:

The first method is far more difficult. It demands the same skill, devotion, insight, and even inspiration as the discovery of the simple physical laws which underlie the complex phenomena of nature. It also requires a willingness to accept objectives which are limited by physical, logical, and technological constraints, and to accept a compromise when conflicting objectives cannot be met. No committee will ever do this until it is too late.

(All from his Turing Award lecture, "The Emperor's Old Clothes": https://www.labouseur.com/projects/codeReckon/papers/The-Emp...)

We can tell you think that when we use your app.

Cuckoo filters outperform bloom filters and allow dynamic insertion and deletion (unlike bloom filters, which only allow insertion). The trade off is that insertion can fail if the table is too full and then would need to expand or store those entries some other way to avoid a false negative.

The problem in this conversation is that you are equivocating between "fixing memory safety bugs" and "preventing memory safety bugs statically." When this blog post refers to "memory safety skeptics," it refers to people who think the second is not a good way to expend engineering resources, not your imagined flagrantly irresponsible engineer who is satisfied to deliver a known nonfunctional product.

genuinely so sad to me that you are still grinding this axe. if your fantasy design works so much better - go build it then!

Initial comments as I write this are all negative, and also responding to something the blog post didn't claim. The only time it says faster than C is when talking about a language targeting the GPU; it is not controversial that GPUs perform better than CPUs for many workloads.

On the other hand, the results of this true fact are already seen in practice in real systems. Rust's iterator adapters, for example, allow high level "functional style" with closures to be compiled to code that is exactly the same as hand rolled C loops without unamortized bounds checks - despite guaranteeing the absence of out of bound accesses even in the face of programmer error - because the constraints of these interfaces and the design of the language enable tons of optimizations by LLVM. This has been true since Rust 1.0 more than a decade ago, it's not a promise of the future.

The alternative to C++ that I meant was Rust, which statically prevents data races.

The occurrence of data races depends on the specific non-deterministic sequence of execution of concurrent codepaths. Just because you have 100% code coverage does not mean you've covered every potential execution sequence, and its almost never practical to actually execute every possibility to ensure the absence of data races. Depending on the probability that your data race will occur, it could indeed be something you have to make stars align for TSAN to catch.

Not to talk my own book, but there is a well-known alternative to C++ that can actually guarantee the absence of data races.

This is exactly the attitude this blog post spends its first section pretty passionately railing against.

The IO aspect is not a surprising flaw but it's disappointing to learn that Erlang lets any process enumerate all the other ones.

Joe Armstrong goes to lengths to describe the benefits of "privately addressable" actors in his thesis (though he uses different terminology). As far as I'm aware, Erlang actors are also privately addressable. cf:

System security is intimately connected with the idea of knowing the name of a process. If we do not know the name of a process we cannot interact with it in any way, thus the system is secure. Once the names of processes become widely know the system becomes less secure. We call the process of revealing names to other processes in a controlled manner the name distribution problem— the key to security lies in the name distribution problem. When we reveal a Pid to another process we will say that we have published the name of the process. If a name is never published there are no security problems.

Thus knowing the name of a process is the key element of security. Since names are unforgeable the system is secure only if we can limit the knowledge of the names of the processes to trusted processes.

https://erlang.org/download/armstrong_thesis_2003.pdf (page 24-25)

100%. So tiring that the discourse around this is based on 15 minute demos and not actual understandings of the trade offs. Varun Gandhi's post that you link to is great.

Based on my experience with Rust, a lot of what people want to do with its "constant generics" probably would be easier to do with a feature like comptime. Letting you do math on constant generics while maintaining parametricity is hard to implement, and when all you really want is "a trait for a hash function with an output size of N," probably giving up parametricity for that purpose and generating the trait from N as an earlier codegen step is fine for you, but Rust's macros are too flexible and annoying for doing it that way. But as soon as you replace parametric polymorphism with a naive code generation feature, you're in for a world of hurt.

Better newtypes are the answer.

Consider Java for example. In Java, interfaces are even more restrictive than traits: only the package which defines the class can implement them for that class, not even the package which defines the interface. But this is fine, because if you want to implement an interface for a foreign class, you create a new class which inherits from it, and it can be used like an instance of the foreign class except it also implements this interface.

In Rust, to the extent this is possible with the new type pattern it’s a lot of cruft. Making this more ergonomic would ease the burden of the orphan rule without giving up on the benefits the orphan rule provides.

Implementations are not imported at all because they are not named. Like I wrote, named implementations (ala ML modules) is a valid alternative, but one with a much greater annotation burden.

You could imagine having named impls that are allowed to be incoherent as an additional feature on top of coherent unnamed impls, but to use them you would need to make any code that depends on their behavior parameterized by the impl as well as the types. In fact, you can pretty trivially emulate that behavior in Rust today by adding a dummy type parameter to your type and traits.

Again, it's all a set of trade offs.

Your conclusion is correct. I'm very happy with the two that Rust picked and tired of people pretending that there will be a magical pick three option if we just keep talking about it.

Implementations are not exported or public at all: they are used in functions and those functions are exported. For correctness, you want those implementations to be resolved consistently (this is what coherence is). This post gives the example of unioning two sets: you need to know that they're ordered the same way for your algorithm to work.

So the problem isn't that the implementation is public, it's that its used somewhere by a function which is public (or called, transitively, by a public function). For a library, code which is not being used by a public function is dead code, so any impl that is actually used is inherently public.

You might say, okay, well can binaries define orphan impls? The problem here is that we like backward compatibility: when a new impl is added to your dependency, possibly in a point release, it could conflict with your orphan and break you. You could allow users, probably with some ceremony, to opt into orphan impls in binaries, with the caveat that they are accepting that updating any of their dependencies could cause a compilation failure. But that's it: if you allow this in libraries, downstream users could start seeing unsolvable, unpredictable compilation failures as point releases of their dependencies introduce conflicts with orphan impls in other dependencies.

This post is written by a fan of implicits, so it frames it as "better" than traits, though at the end it admits it is in fact a complex trade off, which is the truth. In my opinion, the trade off favors traits, but others may feel differently.

The core difference between traits (also called type classes) and ML modules is that with traits the instance/implementation has no name, whereas for ML modules they do. The analogy here is between Rust/Haskell's traits/typeclasses and ML's signatures and between Rust/Haskell's impls/instances and ML's structures. In Rust/Haskell, implementations are looked up by a tuple of types and a trait to determine the implementation. The advantage of this is that you don't need to name the impl and then invoke that name every time you use it; since we usually don't think of "Hash for i32" as something which has a meaningful name beyond the relationship between Hash and i32, this is quite nice.

But coherence requires that instances resolve consistently: if I hash an integer in one code location to insert into a map and then hash it again in a different location to do a lookup on the same map, I need to hash integers the same way each time. If you care about coherence, and the correctness property it implies, you can't allow overlapping impls if impls aren't named, because otherwise you aren't guaranteed a consistent result every time you look up the impl.

This introduces another problem: you can't see all the impls in the universe at once. Two libraries could add impls for types/traits in their upstream dependencies, and the incoherence won't be discovered until they are compiled together later on. This problem, called "orphan impls," causes its own controversy: do you just let downstream users discover the error eventually, when they try to combine the two libraries, or do you prohibit all orphan impls early on? Rust and Haskell have chosen different horns of this dilemma, and the grass is always greener.

Of course with implicits, this author intends a different solution to the problem of resolving instances without naming them: just allow incoherence (which they re-brand as "local coherence"). Instead, incoherence impls are allowed and are selected in a manner based on proximity to code location.

As the post eventually admits, this does nothing to solve the correctness problem that coherence is meant to solve, because code with different nearest impls can be compiled together, and in Rust such a correctness problem could become a memory safety problem, and how you figure out if the impl you've found for this type is actually the nearest impl to your code is left as an exercise to your reader. But sure, since you've rebranded incoherence to "local coherence" you can do some juxtaposed wordplay to call coherence a "local maxima" because achieving it has the downside that you can't have arbitrary orphan impls.

[flagged]

[flagged]

I freely admit I’m frustrated by the discourse around async Rust! I’m also very frustrated because I feel I was iced out of the project for petty reasons to do with whom I’m friends with and the people who were supposed to take over my work have done a very poor job, hence the failure to ship much of value to users. What we shipped in 2019 was an MVP that was intended to be followed by several improvements in quick succession, which the Rust project is only now moving toward delivering. I’ve written about this extensively.

My opinion is that async Rust is an incredible achievement, primarily not mine (among the people who deserve more credit than me are Alex Crichton, Carl Lerche, and Aaron Turon). My only really significant contributions were making it safe to use references in an async function and documenting how to interface with completion based APIs like io-uring correctly. So it is very frustrating to see the discourse focused on inaccurate statements about async Rust which I believe is the best system for async IO in any language and which just needs to be finished.

To be fair, I’m not sure if there exists any zero cost IOCP library.

The main way people use IOCP is via mio via tokio. To make IOCP present a readiness interface mio introduces a data copy. This is because tokio/mio assume you’re deploying to Linux and only developing on windows and so optimize performance for epoll. So it’s reasonable to wonder if a completion based interface can be zero cost.

But the answer is that it can be zero cost, and we’ve known that for half a decade. It requires different APIs from readiness based interfaces, but it’s completely possible without introducing the copy using either a “pass ownership of the buffer” model or “buffered IO” model.

Either way, this is unrelated to the issue this blog post identifies, which is just that some io-uring libraries handle cancellation incorrectly.

Hmm, because the cancel CQE will have a reference to the CQE it was supposed to cancel? Yes, that could work.

It's true, there's a necessary layer of abstraction with io-uring that doesn't exist with epoll.

With epoll, the reactor just maps FDs to Wakers, and then wakes whatever Waker is waiting on that FD. Then that task does the syscall.

With io-uring, instead the reactor is reading completion events from a queue. It processes those events, sets some state, and then wakes those tasks. Those tasks find the result of the syscall in that state that the reactor set.

This is the difference between readiness (epoll) and completion (io-uring): with readiness the task wakes when the syscall is ready to be performed without blocking, with completion the task wakes when the syscall is already complete.

When a task loses interest in an event in epoll, all that happens is it gets "spuriously awoken," so it sees there's nothing for it to do and goes back to sleep. With io-uring, the reactor needs to do more: when a task has lost interest in an incomplete event, that task needs to set the reactor into a state where instead of waking it, it will clean up the resources owned by the completion event. In the case of accept, this means closing that FD. According to your post, monoio fails to do this, and just spuriously wakes up the task, leaking the resource.

The only way this relates to Rust's async model is that all futures in Rust are cancellable, so the reactor needs to handle the possibility that interest in a syscall is cancelled or the reactor is incorrect. But its completely possible to implement an io-uring reactor correctly under Rust's async model, this is just a requirement to do so.

My sincere apologies if I read this uncharitably, I interpreted your comment about sarcasm to itself be sarcastic. I will not be publishing any such post.

To clarify what you seem not to understand: the code in the blog post depends on a library called monoio, and based on the post's description of the behavior of that code (I haven't actually read the source for monoio), monoio contains a bug. I don't dispute that this code behaves incorrectly, just that it has anything to do with Rust's facility for async IO rather than with the specific IO library employed in the post.

[flagged]

[flagged]

If you want the language to encode a liveness guarantee that you do something meaningful in response to an accept rather than just accept and close you do need linear types. I don't know any mainstream language that encodes that guarantee in its type system, whatever IO mechanism it uses.

You're right. Looking at my actual code, instead I stored the accept to be yielded next time you call accept and only cancel an accept call if you drop the entire listener object mid-accept.

The solution proposed in this post doesn't work, though: if the accept completes before the SQE for the cancellation is submitted, the FD will still be leaked. io-uring's async cancellation mechanism is just an optimization opportunity and doesn't synchronize anything, so it can't be relied on for correctness here. My library could have submitted a cancellation when the future drops as such an optimization, but couldn't have relied on it to ensure the accept does not complete.