llogiq

The problem with C++ vs. unsafety is that there is really no boundary: All code is by default unsafe. You will need to go to great lengths to make it all somewhat safe, and then to even greater lengths to ensure any libraries you use won't undermine your safety.

In Rust, if you have unsafe code, the onus is on you to ensure its soundness at the module level. And yes, that's harder than writing the corresponding C++, but it makes the safe code using that abstraction a lot easier to reason about. And if you don't have unsafe code (which is possible for a lot of problems), you won't need to worry about UB at all. Imagine never needing to keep all the object lifetimes in your head because the compiler does it for you.

So professional work during a worldwide pandemic with all the hardship that entailed suffered...from working remotely? I don't buy that.

That very much depends on the code you're compiling. The factors that come into play are monomorphization (which means the compiler builds one copy of the code per type it is called for), procedural macros (which need to be fully compiled before being able to expand code using them), whether complex type shenanigans are used, etc. etc. Absent that, Rust will compile roughly as fast as C nowadays.

There is so much wrong with this article. Throw a little bit of ML pixie dust on everything for more hype? Check. Compare wildly different things as if they were the same? Double check.

digdugdirk has the right idea, and AFAIR, there is some work on that front (https://www.fornjot.app/).

Also the Fiat 500 goes 100km on about 4l of gas, while the Ford F150 uses 7l. No clue where the author gets the idea that the Fiat would get worse mileage, perhaps he's dividing by weight?

The rest, I don't even.

Very nice diagrams, they make the article really easy to follow! This really drives the point home that vector search isn't only a qantitative (as in faster), but a qualitative evolutionary step.

Makes one wonder what other use cases are lurking that would need just another small modification and haven't even been thought of yet because they used to be impossible to implement.

I totally agree that latency of this solution leaves a lot of room to improvement. But that's totally besides the point of the article, which is that people can get a no-cost semantic search for their personal website using those services. They can also use other solutions, of course.

Also I'm experimenting in further integrating things to reduce latency and most likely will publish another article within the month. Stay tuned.

Finally I somewhat agree that many of the players in the vector DB space try to push their cloud offerings. Which is fine, how else should they make money? And if latency matters that much to you, Qdrant offers custom deployments, too. I believe running Qdrant locally will handily beat your LanceDB solution perf-wise unless you're talking about less than 100k entries. We have both docker containers and release binaries for all major OSes, why not give it a try?

The initial version of this actually used Mighty, but I didn't find any free tier available, so I switched to Cohere to keep the $0 pricetag.

I also found while writing the article but after I had already done my research that cargo-lambda has grown some additional functionality that could have removed the need for the AWS CLI, but I wanted to get the article out, so I didn't test-drive that.

Thank you for the suggestion! I actually thought about using terraform, but I wanted to keep the experiment somewhat minimal regarding technologies and as I had already added AWS Lambda and Rust to the tech stack, I wanted to stay as close to the metal as possible. Besides, this is not for commercial applications, so I don't think high availability is in scope.

When starting this project I thought the same thing, but having done it I honestly cannot tell that much of a difference. Yes, there are two more steps in setting up the Lambda function, but in the end you still write an HTTP server and have them serve it.

Author here. This was a fun exercise to produce a semantic search using only free-tier services (which in the case of Cohere means non-commercial use only and in the case of AWS Lambda is limited to one year).

It also marks my first foray into using cloud services for a project. I've long been a cloud sceptic, and doing this confirmed some of my suspicions regarding complexity (mostly the administrative part around roles and URLs), while the coding part itself was a blast.

Nick's articles are always a delight to read, and this one is no exception. Openly discussing failure is one part, but coming up with so many ideas and having the gumption to actually try them out on a compiler is really fascinating.

This is a nice article for people new to async/await in Rust.

Well, if the programmer writes the type annotation to actually have their code checked, how would a gradually typed language differ in practice from a static language?

Yes, you can still do dynamic typing. But I'd argue that using `dyn Any` you can do so in Rust, which is a statically typed language if I ever saw one.

Otherwise I completely agree about the languages growing closer together.

Yeah, that one hasn't aged too well. We've all seen the backswing to statically typed languages. Yes, some of them (e.g. typescript) run on top of dynamic langs, or allow for VMs (hi, WASM!). Why? Because ironically the same famed flexibility that makes it oh so easy to whip up a prototype is biting us in the ass when it comes to make a production-grade piece of software while staying on top of the ever changing requirements. So while we have a lot of python in ML (where most things haven't left prototyping stage), a lot of code nowadays is written in languages like Rust, TypeScript, Swift and others.

Not (only) because that's faster to run, but because it's faster to change while still remaining somewhat working correctly. And the current crop of compilers not only can produce stunningly fast code, but also awe-inspiringly great error messages that put the 90's and oughties' cryptic error messages to shame. Try that with a dynamic language!

What would that buy you? You train until you push your loss under a certain threshold, then check the external criteria and if they don't hold you train again? Your external criteria would essentially become another part of your loss function, but the whole training would become vastly more inefficient.

I'm saying that we shouldn't expect the models to come up with things we didn't train them to come up with.

Again, there is a lot of words to describe the fact that machine learning is just lossy compression for a bunch of data with the possibility to interpolate between data points and get somewhat plausible results. This means data points may get lost during compression/training, and certain things will look off, whether it be a preference for banana pairs, even numbers of fingers or certain weasel words in verbiage.

Having seen state of the art neural networks in the 70s and 80s, I am both in awe at the progress that has been made since and deeply unconcerned about AGI.

Yes, storage capacity and compute have made major leaps, but it's still only lossily compressing a semantic space. It won't generate things from thin air, and it's certainly not sentient, no matter how far you scale it up.

That said, it may be a useful tool in some cases, and we've also seen its numerous limitations, only some of which may be alleviated in the future.

As always, the problem is not artificial intelligence, but natural stupidity.

This looks like a fun way to get into programming. I'll have my kids check it out!

I sometimes wonder what CPU designers think when building such weird instructions? They must have some programs in mind that could be run faster with them, or else the additional transistors are just lost weight. But then compilers and language runtimes might or might not use those instructions. Add to that the fact that modern CPUs are basically their own compilers (going from "machine code" to microcode) and you have weirdness atop of more weirdness. But perhaps this is just for business sake; adding more instruction sets to provide a barrier to the competition, because programs using this run faster but are no longer portable to competitors' CPUs.

I used to work on the Rust compiler itself on a Chromebook with a 1.1GHz dual core, 4G RAM and 32G of disk. That's about as far from a high-end PC you can get. Most mid-range phones nowadays have more processing power and memory than that. And the Rust compiler has been sped up considerably since then. Even with a 4 year old mid-tier PC you can get a complete Rust compile in half an hour. Roughly half of that is building LLVM.

So you can of course compile your Rust compiler. If you are used to compile clang or gcc, it's not that much of a hassle. And the benefits have already been shown. If you only want to compile Rust code, and not develop it, mrustc might also be a good choice for you (it doesn't implement borrowck, just what's needed for codegen).

Finally, if you don't want to use Rust drivers, you can simply configure them out and don't need to build Rust. It'll be quite a long while until Rust will arrive in the kernel outside of drivers (which tend to benefit most from Rust anyway).

If matthieum hadn't done it, someone else would have (and has; some dupes have appeared and were subsequently removed).

You know there's multi-reddits? In your specific example, https://reddit.com/r/RoastMe+toastme

Not all platforms have atomics; you'll mostly encounter this on embedded systems. That said, use them if you can!

Regarding Rc, the overhead is more in memory than time, which usually has a non-linear relation to performance, so any overhead is really hard to guess or estimate.

Looks can be deceiving. You'll have to try it to see if you like it.

And if not, that's totally OK.

Well, I have been writing both high and low level code in Rust, and I must admit I've been happily falling into that "trap". The nice thing about Rust for high level software is that it lets us specify interfaces in a way that makes them actually usable without delving into the low level details.

A lot of things that are hand waved away in C++ (who owns this? How long does this need to be alive? Can I mutate this?) are completely explicit in a Rust interface. And Rust's type system is strong enough to express a lot of rules that no one would want to write down in C++ (because overriding them is just one cast away), allowing us to write hard-to-misuse libraries.

Coding on top of such libraries is a really fun experience. That's why I call Rust an all level language.

Oh, right! Thank you for bringing it up, I'll fix this.

Well, titles are hard.

All of those things are possible in Rust. But they're not easy or approachable.

Author here. I blog about Rust as a hobby (and have been doing so since 2015 on my personal blog), and LogRocket is so nice to pay me to have my content hosted on their blog. And yes, their product is only tangentially related (if you use Rust on the frontend with WASM).

We should be able to write a MIR lint that gives us linear types wherever we want them. I hear they are most useful with unsafe code.