jamwt

It's a surprisingly tricky problem. I know all too well.

If you want to minimize drama, it's worth still paying for Dropbox.

Makes sense to me

IME, engineers and "the discourse" when we argue on the Internet often conflate "ease of use" and "ease of learning". When we say "ease of use", we usually mean "ease of learning."

If you're a hobbyist, yeah, you should heavily value "ease of learning." If you're a professional, the learning curve is worth it if the tool's every day leverage is very high once you're ramped up. Too many developers don't put those 3-4 months in, in part due to the over-emphasis on "ease of learning" in our discussions/evaluations of things.

I was a part of a very large go project (https://news.ycombinator.com/item?id=11282948) and go-based company infra generally some years ago, and go is emblematic of the classic tool that is amazing at ease of learning, and quite mediocre at "ease of use" as time goes on.

I personally end up resenting those tools because I feel tricked or condescended to. (This is a little silly, but emotions are silly.)

I'd wager this is also why Rust is a perennial "most loved" winner in the surveys: it gets better as your relationship with it deepens, and it keeps its promises. Developers highly value integrity over trickery, and hard-earned but deep value feels like integrity and wins in the long run. (other examples: VIM, *nix, git)

Agree. If you want a fully reactive database (and not just hierarchy-based reactivity like Firebase) but want to keep your ACID + relational data modeling, check out https://convex.dev .

Plug notice: this is my company.

Hi there. Another Convex cofounder here.

SiteGuide (https://siteguide.ai/) was the first to do vector embeddings with Convex, built by integrating Convex + Pinecone. This combination has been an increasingly common pattern over the last few months. So we put a template project to demonstrate how this is usually done:

https://github.com/ianmacartney/embeddings-in-convex

We're strongly considering building in vector search a little further down the road, but this is the recommended approach for now.

Yes, way less effort

doesn’t explain what the data model it is proposing actually is in concrete terms.

Fair enough. The goal of the post wasn't necessarily to comprehensively describe the product–so the specifics of what we provide are found elsewhere in our docs, etc.

Really, the big thing that I see here is the transaction retry logic,

These transactions run as embedded functions inside Convex in a runtime where it is impossible to have side effects.

Article author here.

Agree this is a necessary part of the change. Convex is working on our OSS strategy this year. Thanks for the feedback!

Article author here.

It's not new "in the world", but often new to our users, who understand it less intuitively than pessimistic locking.

Unless they're using Haskell or probably other very exotic environments with strict understanding of mutability and so on, they know Mutexes but not STM or other OCC patterns. So we lean heavily on explaining it. Not because it is original, but because it is unfamiliar.

Also, yes, great pickup re: transactional memory. We talk about this internally all the time, this is the inspiration.

Haskell: IO -> STM -> (pure)

Convex: (The browser / Convex Actions / other effectful environments) -> Mutations -> queries

All the same benefits re: retry and memoization.

Steal steal from Haskell, so many great ideas there.

Hi there Michael! We continue to love nomad.

SQL is the C ABI of querying for sure. BI tools will never adapt to use Convex directly, and nor should they.

So... yes, Convex actually had a prototype SQL adapter for the read side of things back in the early few months when we were figuring things out. Convex's read semantics are very compatible with SQL.

We've kept this adapter on ice in part because of point #3 in the article -- we don't want to overpromise things which are a bad idea.

Meaning, if we exposed SQL on the thing as-is, this would presumably be for more analytical type queries involving patterns normal Convex queries can't express. Right now that would be a Bad Idea because your website would slow down just like every other database system allows you to.

So the current recommended practice is use our Airbyte Egress connector (https://airbyte.com/connectors/convex) and get yourself into an offline Clickhouse/MySQL/Snowflake whatever and jam SQL over there to your heart's content. That's basically what we do.

We may one day abstract this away by embedding some sort of mirrored column store sql thing (maybe DuckDB based? who knows) so you can do your analytical work without impact on your OLTP or integrating 3 more systems. But the team hasn't invested in that yet and probably won't for some time.

Folks for whom TypeScript is a big part of their project are going to be the most natural fit for Convex.

But no, not necessarily exclusively TypeScript. For example, the existence of the Python client library is due to developer demand. Some users had ML jobs that are triggered by Convex applications or reported outcomes into Convex.

The embedded functions are always TypeScript. The experience is pretty smooth in other languages as well-but the degree of this is largely dependent on how easily the type systems map from e.g. Python to TypeScript and how our client library can infer things in a way that feels native or requires little of your involvement.

Article author here.

It may actually be fine to just require everything to be typescript, but the idea that you'd require your application be written in the same language as your data store, and thus implement a different data store for each language you might want to write an app in (and not share between languages)... would formerly be thought of as pretty ridiculous?

The vision is definitely aspirational, and is reflecting on the fact that JS/TS is becoming overwhelmingly the most popular language for projects. With things like React Native, for many teams even mobile apps are taken care of all on one language.

There will obviously always be apps written in other languages, and in those cases, a bit more type mapping will be necessary, because (for now) Convex functions are always side-effect free, automatically cached JS/TS running in a very particular runtime. But we'll work on making that translation as smooth as possible in our client libraries.

We have a Python client library out now ( https://pypi.org/project/convex/ ) and a Rust one coming soon, just to keep us honest in this respect.

I guess we'll see if Convex can get popular enough to spawn lots of "what I wish I knew about convex a year ago" and "Goodbye Convex, hello Postgresql" posts...

Well, that's the dream, isn't it?

Article author here -- DuckDB is indeed cool.

Another interesting bit of work in this space is LINQ on the microsoft side of things. And even list comprehensions end up having a scanning/summing language kind of feel to them that would be interesting to see translated into a new database query language.

Article author here, yep, that was the intended point. Agree the wording was unclear, and there's a footnote now clarifying that. Thanks!

Article author, here. Good point -- but the article actually proposes document relational, not hierarchical. Relational is definitely good!

Exercise hard in the morning.

At 8:30p: Last big glass of water. Put phone on charger in non-bedroom and don’t touch until 7a. Keep the lights low and warm. Do low stimulation activities like reading fiction or watching semi-boring TV or chatting with your family. Lay down when you can’t keep your eyes open anymore.

Big change for me. The phone especially is poison for evening tranquillity.

Yep and every lock is a spinlock. ;-)

(Founder / CEO of Convex here)

Hi Shawn. I don't remember you bringing this up when we spoke in person recently, but it's a great question.

In our opinion, the very best way to evaluate yourself as a backend developer is how directly you solve problems for frontend developers. We believe in the merit of customer obsession, and the customers are not buying queues. They're buying the product as they see it: its surfaces, workflows and experience. And that's what the frontend developers, PMs, and designers are creating.

Historically, all these backend technologies that only interoperate with each other are only useful so long as they make product creation and improvement easier, more reliable, etc. We strongly believe as soon as you don't need them anymore, you should toss them out. They're complex and not proprietary to your product.

Convex (and serverless in general) is just the next step in providing more powerful abstractions that allow companies to double down on frontend engineering (work that adds product value) instead of reimplementing the same backend/devops plumbing the users never see (work that, at best, merely sustain product value).

So, given that we recognize this need, I respectfully disagree that we're not well equipped to solve these problems for frontend developers! Most of our team's recent our work has been designing synchronization and storage platforms to enable product development, including work on web, desktop, and mobile libraries/SDKs. We feel like we have both a lot of empathy and experience for this space, and we're very proud of our early product and the enthusiasm from the web dev community.

Convex | https://convex.dev | San Francisco 3/2 hybrid ONSITE| Front-end engineers, full-stack engineers

Convex is a rapidly growing startup aiming to change the way the web is built. Specifically, we're creating a Global State Management platform--a new way to build apps without backends, databases, VPCs, kubernetes, etc.

Featured recently here: https://www.forbes.com/sites/kenrickcai/2022/04/27/convex-se...

More background on the vision: https://a16z.com/2022/04/27/investing-in-convex/ https://blog.convex.dev/unfulfilled-functional-ideals/

We're looking for people passionate about developer platforms, web development, mobile development, distributed systems and databases--all the pieces necessary to create a holistic replacement for the backends of years past.

Our stack is TypeScript + Rust. The Convex team has been building critical systems in Rust for nearly 10 years:

https://www.wired.com/2016/03/epic-story-dropboxs-exodus-ama... https://dropbox.tech/infrastructure/rewriting-the-heart-of-o...

...so if you're passionate about learning in depth how to take Rust into production, join up!

Feel free to either reply to me or apply directly on https://convex.dev/jobs

Or, if you just want a beta key to learn more about what Convex is first, let me know. I'll hand those out too.

Pretty different approach, but with many application architectures today you can move beyond "hosting" into pure serverless with a stack like Vercel + Convex (https://convex.dev)

In the news today: https://www.forbes.com/sites/kenrickcai/2022/04/27/convex-se...

(disclosure: Convex co-founder)

Unregulated capitalism. Regulation limits how much information/expertise disparity can tip the scales. It doesn't prevent it altogether (nor is that a constructive objective), but limits it so the distribution between the winners and losers ensures still reasonable outcomes for the loser.

All that's happening in crypto is the same exploitative games/vehicles that happened in other markets before regulation protected the least informed and least Machiavellian players from the worst outcomes. There's no real innovation in that respect.

As one of those other team members of James's, I'd profess that if it's literally less than 50 lines of code, I'm probably still faster in Python (22 years of programming python). Otherwise, once things hit 200-300 lines, I can now sort of feel the overhead of more fragile refactors and longer debug cycles kicking in. So, for me personally, it doesn't take much of a codebase before the tipping point.

Convex | San Francisco (3/2 hybrid ONSITE) | Full-Time | https://convex.dev

We're a rapidly-growing team building a next-generation development platform for dynamic serverless applications. We have a ton of interesting challenges in the distributed databases space as well as in modern web development frameworks and SDKs. The founding team includes infrastructure/storage/database leads from Dropbox and engineering lead from Google Photos ( https://www.linkedin.com/company/convexinc/people/ ).

We have very strong funding from top investors, great feedback on early prototypes, and now it's time to build out our early team!

Looking for:

1. Web frontend/fullstack engineers who are well versed in the modern web development ecosystem. This includes familiarity with and strong opinions about TypeScript, React, npm/yarn, etc.

2. Backend engineers who love solving tricky systems problems, ideally with Rust experience.

Come join us! Email us at: join@convex.dev

Zerowatt Inc. | Lead Web Developer / Evangelist | Full-time | San Francisco (3/2 Hybrid ONSITE)

We're a brand new startup building a serverless platform--a new kind of "Cloud Operating System", composed of storage + compute abstractions that will power future Internet companies. We just raised a seed round led by Neo, with additional investments from a16z, Sequoia, Kleiner Perkins, General Catalyst, and lots of other great folks.

We're looking for an individual who's really deep on the web/react stack and community who can help shape how APIs and SDKs should work in this new environment. You would be hire #1 with the founding team--we're formerly three of Dropbox's most senior technical people, that lead most of dropbox's storage + database systems for the last ~8 years.

Our backend is a pretty radically new way to store + compute built in 100% Rust. It's fun and potentially hugely disruptive stuff.

You can email me at jamwt@zerowatt.io if you're interested or just have any questions/ideas for us! Thanks.

Heh, and the last time the podcast published an episode about an Auth0 outage, Auth0 had another outage the next day. Now Slack follows shortly after an episode. How eerie!

This is crazy timing -- my co-host and I just released a Podcast episode yesterday sifting through the details about the Auth0 database-related outage from 2018 ( https://downtimeproject.com/ ).

I'll be curious to see how much overlap or not there is with that previous outage. They wrote up a nice post-mortem back then, so hopefully we'll get another one this time.

That's an unusual claim

It's really... not. For people that have actually written big things in Rust. We keep repeating it, but we try to cautiously, tread carefully, lest we be accused of being part of this dreaded Evangelism Strike Force we keep hearing so much about.

Ease of use != ease of learning.

Rust is hard to learn (unless you have a deep ML/Haskell + C background). But once you're up the curve, it's a very expressive + productive language. Refactoring and maintaining large projects confidently is a breeze, and creating powerful abstractions that can be safely leveraged by large teams of programmers... just works.

And if I write a "Rust: The Bad Parts", believe me, I'll be mentioning panicking.

This is why `panic=abort` should be the default, and the whole unwind-and-try-to-keep-the-world-sound path should be opt-in. Then panic is truly like `assert` and I'm guessing most of his objections would be gone.

My guess about default-panic behavior being unwind is rust's origins in the servo project. When you're part of a very large monolith that should try very hard not to crash (a browser), you will put some work in to try to make this unwinding okay. Yes, tests still want panic to unwind, but you could opt in to this to, or change the default in a `[test]` context, or a bunch of other things I'm sure smarter folks could argue in an RFC. But getting correctness right in prod should be goal #1 IMO, so it should bias toward abort.

For most places rust is probably actually used today (server-side), crashing is the safer and simpler behavior, and things like lock poisioning are not things you need to reason though.

I know the article is about Haskell, so not trying to derail it, but I have a really similar Haskell -> Rust path in my background, so a lot of the rest of Michael's reactions here are just +1 for me. For example, yes, exactly this about partial functions.

And, IMO, laziness, which he hints at in this section. The default should be the other way. Nothing worse than an `error` that's fired in some unexpected place/time due to lazy evaluation, and some thunk landed somewhere technically correct, but infuriating. Trying to figure out what the heck is going on can be really challenging/frustrating (as of my prod experience in Haskell 8-11 years ago, not sure what's gotten better/worse since then in ghc-land.)

I learned a ton from Haskell, and am so glad I used it in depth for awhile (ditto ML). But these days, to actually build something I want to build with long-term value in mind, either individually or as part of a team, I just use Rust. I get most of what I loved about Haskell without the annoyances.

People are happy to pay AWS

Many of them are not.

We have serious vendor lock-in now, where a very few companies are gatekeepers to almost any business that runs on the internet.

And their margins are _enormous_ on this business. It ends up costing much, much more to pay them to run our machines for us.

And increasingly, the expertise to do this is being consolidated in these companies, so the talent available to pursue any other way is diminishing as new grads never learn about the magic places their code runs.

The reliability outcomes are nearly the same, despite the deferral to their expertise.

Labor savings b/c you don't to learn about provisioning your own machines? Not much. AWS is so sophisticated you need to develop a nearly equivalent amount of (non-portable) expertise to actually operate it well. Remember, the alternative isn't just rack your own, it's... dedicated hosting! And lots of other options with less lock-in and more standards.

It's sort of frightening how complicit the broader technology industry is in this power consolidation.