HN user

stephen

1,672 karma

stephen.haberman@gmail.com draconianoverlord.com (the domain name was available :-)) https://joist-orm.io/

Posts26
Comments396
View on HN
joist-orm.io 2y ago

Show HN: Joist, the Best ORM, Ever?

stephen
5pts1
github.com 2y ago

Prettier Plugin for SQL Files

stephen
1pts0
twitter.com 2y ago

Static Hermes: Statically Compiled JavaScript

stephen
3pts0
regolith-desktop.com 4y ago

Regolith Desktop 2.0 Released

stephen
2pts0
spin.atomicobject.com 6y ago

Flavoring: Flexible Nominal Typing for TypeScript

stephen
2pts0
www.pg-structure.com 6y ago

Get PostgreSQL Database Structure as a Detailed JavaScript Object

stephen
158pts29
www.pg-structure.com 6y ago

Get PostgreSQL Database Structure as a Detailed JavaScript Object

stephen
3pts0
www.draconianoverlord.com 8y ago

Are Classes and Prototypes That Different?

stephen
3pts0
medium.com 8y ago

Manual Typing Is No Fun: Introducing TypeWiz – Uri Shaked – Medium

stephen
4pts0
remotedebug.org 12y ago

RemoteDebug: Initiative to unify remote debugging across browsers

stephen
1pts0
dev.bizo.com 13y ago

What Makes Spark Exciting

stephen
1pts0
draconianoverlord.com 13y ago

DHH vs. Fowler

stephen
7pts1
ncees.org 13y ago

Professional Software Engineer Exam coming in April 2013

stephen
61pts58
robey.lag.net 14y ago

Type safe config in Scala

stephen
6pts1
git.kernel.org 14y ago

Git can now correctly push merges to svn

stephen
2pts0
draconianoverlord.com 14y ago

Sane AJAX Testing in Selenium

stephen
1pts0
jschema.org 14y ago

Schema for JSON APIs

stephen
1pts0
draconianoverlord.com 14y ago

Why No One Uses Scala’s Structural Typing

stephen
3pts0
draconianoverlord.com 14y ago

A GWT Developer’s Reaction to Dart

stephen
4pts1
www.draconianoverlord.com 15y ago

Working with git? These aliases will save you even *more* hours

stephen
1pts1
ssw.jku.at 15y ago

The Dynamic Code Evolution VM

stephen
5pts0
www.draconianoverlord.com 15y ago

Why I Don't Like Mocks

stephen
1pts0
guidewiredevelopment.wordpress.com 15y ago

New Feature Literals in Gosu (first class data binding)

stephen
3pts0
github.com 15y ago

S3fsr - FUSE file system for S3

stephen
2pts0
www.draconianoverlord.com 16y ago

Dammit, MySQL

stephen
141pts98
draconianoverlord.com 16y ago

Caller Side Structural Typing

stephen
1pts0

I get why folks use tmux/herdr, but I already use i3wm/Hyprland for window/workspace management, and want to have "shared first class windows" instead of dual binds of "super-based binds for i3 windows ... oh wait control-a based binds for tmux panes".

Has anyone got a tool/setup that is tmux-like but the remote terminals/panes are all local/native windows?

Disclaimer I just edited this into my OP comment, but "generating boilerplate INSERTs" is not the main reason I use ORMs -- it's business rule enforcement.

I.e. regardless of how easy it is to write `INSERT authors (...) VALUES (...)`, with an appropriately cute/ergonomic query builder to bind the variables/POJOs ... where does your business logic actually go?

Whenever you insert an author, are you always enforcing the same validation logic? Whenever you update a book, are you always updating the derived fields that need updated?

Getting the business rules right is "the actual hard stuff" imo, and nothing I've seen a query builder help with; it's always left as an exercise to the reader to reinvent their "business logic wrapped around POJOs" adhoc in their codebase.

You're right, that has been another "pro ORM" pitch that has gone awry and, taken to the extreme, is wrong imo.

My nuanced articulation is "you don't have to write the _boilerplate_ SQL for the 90% of just-do-some-CRUD endpoints in your enterprise SaaS application, but you 100% need to 'know SQL' for the last 5-10% of ~reporting/analytics queries that the ORM is going to mess up".

Fair point, both "pro ORM" and "anti ORM" camps are prone to extreme stances.

I definitely don't agree with the "all queries must be executed through the ORM", and think that dogmatic stance has done a lot of damage to the ORM brand. :-/

I'm admittedly an ORM apologist [1], but a few of his points articulated as "deal breakers" aren't that bad imo:

- "the pernicious use of foreign keys [...] links between classes are [...] foreign keys" ==> that just sounds like schema normalization, which is usually a good thing?

- "bending over backwards [...] to generate SQL that runs efficiently" ==> the huge majority of ORM-driven queries are "select * from table where id in ..."; for the queries that are more complicated than that, then yes use SQL! That's allowed!

Folks who dislike ORMs seem to have this false dichotomy that "the ORM _must_ be used for all queries", which is a self-imposed/unpractical restriction.

- "dual schema dangers" ==> he's exactly right that database should own the schema definition, but then just codegen the entities from the db schema? That's your singular source of truth, no drift. You can do this with Hibernate, ActiveRecord, Joist, many ORMs.

- "Identities" ==> ironically I think ORMs (that use the unit of work pattern) actually have net-better DX here b/c you can hook up a graph of entities with just references.

I.e. hook up a book to its author w/o knowing their ids yet, which explicitly avoids the annoyance he mentions of doing a partial commit/going to the db to figure out "what value should I INSERT into in the book.author_id column?" (but my author is new) in the middle of your business logic that just wants to "create books".

- transactions ==> agreed that "transactions via annotations" ala JPA/Hibernate are terrible, but afaiu all "internet scale" apps these days do reads outside of transactions, and just use op-locking during the singular flush/commit step to the db.

Disclaimer I am sure I won't change anyone's minds :-)

Edit: in the HN comments, we're debating "the best way to generate SQL", which is fine, but imo it overlooks the biggest value for ORMs: enforcing business invariants.

I.e. yes a simple INSERT is trivial is write, "why have the ORM to that!", but are you going to enforce the same business logic in the 10 places you do `INSERT authors` in your codebase? And if the answer is "I write an single `insertAuthor` abstraction to enforce this" then you're half-way to writing an adhoc half-specified, bug-riddled version of what a reactive ORM like Joist will do for you. [2] :-)

[1] https://joist-orm.io/

[2] https://joist-orm.io/modeling/why-entities/

You're already using a new runtime with tsgo -- it's golang at build time -- but still running Node in prod, so the same could work here. :-)

Agreed I would not want all Typescript users forced to use /this/ runtime, but if the TS team shipped tsc as "oh now it's uses a special fast JS runtime" (just like tsgo is a different runtime) I'd love to at least have the option of using the same special fast runtime in my own still-written-in-TS apps.

Seems I've either struck or a nerve, or miscommunicated, given the insta down votes.

"beyond a junior level" -- I doubt this will change your mind, but this post is from the author of Tachyons, a "pre-Tailwinds" competitor that didn't get the same traction:

https://mrmrs.cc/writing/scalable-css/

And the tldr is that he downloaded and read the CSS for several major websites at the time (post is from 2016) and they were all hodge-podge of terribleness.

Maybe all the devs writing that CSS were junior, but imo it's more than CSS just doesn't have the abstractions to match the level of OCD/bespokeness that designers spec into every Figma -- move this box by _this_ much / _that_ much / etc.

Per "how to handle dynamic queries", it's admittedly pretty different b/c we're an ORM (https://joist-orm.io/) that "fetches entities" instead of adhoc SQL queries, but our pattern for "variable number of filters/joins" looks like:

const { date, name, status } = args.filter;

await em.find(Employee, { date, name, employer: { status } });

Where the "shape" of the query is static, but `em.find` will drop/prune any filters/joins that are set to `undefined`.

So you get this nice "declarative / static structure" that gets "dynamically pruned to only what's applicable for the current query", instead of trying to jump through "how do I string together knex .orWhere clauses for this?" hoops.

Hello! Yeah, I totally get Dagger is more "hey client please create a DAG via RPC calls", but just making something up in 30 seconds, like this is what I had in mind:

https://gist.github.com/stephenh/8c7823229dfffc0347c2e94a3c9...

Like I'm still building a DAG, but by creating objects with "kinda POJOs" (doesn't have to be literally POJOs) and then stitching them together, like the outputs of 1 construct (the build) can be used as inputs to the other constructs (tests & container).

Right, my point is that this:

https://docs.dagger.io/cookbook/services?sdk=typescript

Still looks like "a circa-2000s Java builder API" and doesn't look like pleasant / declarative / idiomatic TypeScript, which is what aws-cdk pulled off.

Genuinely impressively (imo), aws-cdk intermixes "it's declarative" (you're setting up your desired state) but also "it's code" (you can use all the usual abstractions) in a way that is pretty great & unique.

I thought Dagger had/has a lot of potential to be "AWS-CDK for CI pipelines".

I.e. declaratively setup a web of CI / deployment tasks, based on docker, with a code-first DSL, instead of the morass of copy-pasted (and yes orbs) CircleCI yaml files we have strewn about our internals repos.

But their DSL for defining your pipelines is ... golang? Like who would pick golang as "a friendly language for setting up configs".

The underlying tech is technically language-agnostic, just as aws-cdk's is (you can share cdk constructs across TypeScript/Python), but it's rooted in golang as the originating/first-class language, so imo will never hit aws-cdk levels of ergonomics.

That technical nit aside, I love the idea; ran a few examples of it a year or so ago and was really impressed with the speed; just couldn't wrap my around "how can I make this look like cdk".

Thanks for the reply! That all makes sense!

As a potential user, I'd probably be thinking through things like: if I have a ~small-fleet of 10 ECS tasks serving my REST/API endpoints, would I run `client.query`s on these same machines, or would it be better to have a dedicated pool of "live query" machines that are separate from most API serving, so that maybe I get more overlap of inherited queries.

...also I think there is a limit on WAL slots? Or at least I'd probably want not each of my API servers to be consuming their own WAL slots.

Totally makes sense this is all "things you worry about later" (where later might be now-/soon-ish) given the infra/core concepts you've got working now -- looking really amazing!

Can you description the deployment setup, somewhere in the docs/maybe with a diagram?

I get this is a backend library, which is great, but like does it use postgres replication slots? Per the inherited queries, do they all live on 1 machine, and we just assume that machine needs to be sufficiently beefy to serve all currently-live queries?

Do all of my (backend) live-queries live/run on that one beefy machine? What's the life cycle for live-queries? Like how can I deploy new ones / kill old ones / as I'm making deployments / business logic changes that might change the queries?

This is all really hard ofc, so apologies for all the questions, just trying to understand -- thanks!

These two suggestions are fine, but I don't think they make fixtures really that much better--they're still a morass of technical debt & should be avoided at all costs.

The article doesn't mention what I hate most about fixtures: the noise of all the other crap in the fixture that doesn't matter to the current test scenario.

I.e. I want to test "merge these two books" -- great -- but now when stepping through the code, I have 30, 40, 100 other books floating around the code/database b/c "they were added by the fixture" that I need to ignore / step through / etc. Gah.

Factories are the way: https://joist-orm.io/testing/test-factories/

I still use my index finger; I've just gotten used to moving my hand ~slightly over from j to the nub.

I would definitely prefer their trackpoint module be "flipped upside down" so the nub was on top, directly next to the H key, so I could move "just the index finger", and not my palm, but it's really not a big deal now that I'm used to it.

They seem to get this feedback a lot, b/c they have an FAQ entry about (nub location), which asserts the current thumb location is due to space/engineering constraints. But, dunno, I kinda wonder if that was for the smaller UHK60? B/c just looking at my UHK80, it really seems like the nub could be by the H if they wanted it to. :-)

So not "perfect perfect" but still really amazing imo, and so glad I switched over -- I'm like 10 years late to split keyboards, custom layers for movement / programming binds, everything the cool kids have been doing forever, but I couldn't give up a trackpoint. But here we are, finally! :-)

(Also fwiw I held off on the UHK80 for about a year b/c they were having firmware issues on initial release, repeated/missed keys, that sort of thing, but its been rock solid for me; literally zero issues.)

Great to hear you're using postgres.js in prod/large deployments! That sort of real-world-driven usage/improvements/roadmap imo leads to the best results for open source projects.

Also interesting about a potential v4! I'll keep lurking on the github project and hope to see what it brings!

Oh hello! Very happy to hear from you, and even happier to be wrong about your "AWOL-ness" (since I want to ship postgres.js to prod). :-)

My assumption was just from, afaict, the general lack of triage on GitHub issues, i.e. for a few needs we have like tracing/APM, and then also admittedly esoteric topics like this stack trace fixing:

https://github.com/porsager/postgres/issues/963#issuecomment...

Fwiw I definitely sympathize with issue triage being time-consuming/sometimes a pita, i.e. where a nontrivial/majority of issues are from well-meaning but maybe naive users asking for free support/filing incorrect/distracting issues.

I don't have an answer, but just saying that's where my impression came from.

Thanks for replying!

I really want to use pipelining for our "em.flush" of sending all INSERTs & UPDATEs to the db as part of a transaction, b/c my initial prototyping showed a 3-6x increase:

https://joist-orm.io/blog/initial-pipelining-benchmark/

If you're not in a transaction, afaiu pipelining is not as applicable/useful b/c any SQL statement failing in the pipeline fails all other queries after it, and imo it would suck for separate/unrelated web requests that "share a pipeline" to have one request fail the others -- but for a single txn/single request, these semantics are what you expect anyway.

Unfortunately in the TypeScript ecosystem, the node-pg package/driver doesn't support pipelining yet, instead this "didn't quite hit mainstream adoption and now the author is AWOL" driver does: https://github.com/porsager/postgres

I've got a branch to convert our TypeScript ORM to postgres.js solely for this "send all our INSERTs/UPDATEs/DELETEs in parallel" perf benefit, and have some great stats so far:

https://github.com/joist-orm/joist-orm/pull/1373#issuecommen...

But it's not "must have" for us atm, so haven't gotten time to rebase/ship/etc...hoping to rebase & land the PR by eoy...

I want to use/try Niri, but have been staying on Hyprland from my safety blanket of Omarchy [1], and really liking its hyprscrolling plugin:

https://github.com/hyprwm/hyprland-plugins/tree/main/hyprscr...

Imo Hyprland should merge this hyprscrolling plugin into the main project, and just ship it as the default (only?) layout option -- it just scales to "more than 4 windows" so much better than either of Hyprland's master/dwindle layouts.

[1] I tried vanilla arch + archinstall + sway/niri/etc but really couldn't make it work from scratch, vs. the contrast of Omarchy which was "wow this all works" :shrug:

I've been using Ghostty, and other GPU-based apps like Alacritty / WezTerm / Zed, because they're ofc better/faster...

Ironically they've all made my DX worse, by highlighting how terrible the nvidia drivers actually worked on both my old Regolith i3wm/compositor-less or new sway/wayland setup.

Like it's ridiculously terrible.

I've tried every magical env flag that Claude can think of, and 4 of the various 550/560/575/580 driver versions--they all suck at screensharing, or resume from sleep, or just out-right buginess/crashes/segfaults in the nvidia drivers.

It must have always been this bad, but I just didn't notice, b/c I never actually used my GPU? lol

Microservices are a design pattern for organisations as opposed to technology ... breakout into multiple teams

I agree, but just saying "multiple teams" has led many eng directors to think "I have two squads now --> omg they cannot both be in the same monolith".

When both squads are 5 people each.

And the squads re-org (or "right size") every 9 months to re-prioritize on the latest features.

Five years go by, 7 team/re-org changes, all of which made sense, but thank god we didn't microservice on the 2nd/3rd/4th/5th/6th team boundaries. :grimmacing:

We should stay "stable, long-lived teams" -- like you need to have a team that exists with the same ownership and mandate for ~18 months to prove its a stable entity worth forming your architecture around.

Do queue operations (enqueue a job & mark this job as complete) happen in the same transaction as my business logic?

Imo that's the killer feature of database-based queues, because it dramatically simplifies reasoning about retries, i.e. "did my endpoint logic commit _and_ my background operation enqueue both atomically commit, or atomically fail"?

Same thing for performing jobs, if my worker's business logic commits, but the job later retries (b/c marking the job as committed is a separate transaction), then oof, that's annoying.

And I might as well be using SQS at that point.

Working on v2 of our n+1-proof/reactive TypeScript ORM, Joist (https://joist-orm.io/), that moves to using the new-ish postgres.js driver (instead of knex/node-pg), so that we can leverage postgres.js's statement pipelining within transactions.

I'm anticipating a really sweet perf increase (as shown by some proof-of-concepts), but now that everything is actually working on the v2 branch, I'm putting together benchmarks that show the benefit in practice.

Love to have anyone poke around/ask questions/hang out on discord.

Same! We've also been using the polyfill in production for a ~year or so.

We recently configured the node-pg driver to leave the postgres DATE / TIMESTAMP / TIMESTAMPTZ dates as "just strings" (instead of its default parsing to JS dates), and then we go straight from strings -> temporals at our ORM layer, without the jank/hop of it first being a JS Date.

...shoot, we need to get the Temporal out-of-the-box support updated in our Joist (https://github.com/joist-orm/joist-orm/) docs. :-/

But it's been great, having `DATE` in the db => `PlainDate` in the TS backend => `Date` in GraphQL API => `PlainDate` again in the TS/React FE. Very clean.