bruth

Unfortunately the LLM is leading you astray :)

(disclaimer: I am the VP of Prod/Eng at Synadia)

NATS - An application connectivity technology (L7). It was originally designed for low-latency M:N messaging, and that is still true today. In 2018, native multi-tenancy, clustering options, etc. got introduced. The persistence subsystem (JetStream) was introduced in 2021. It has a completely different design than Kafka, but with overlapping use cases. For better or worse, we get compared to Kafka a lot and virtually everyone who engages realizes the advantages and opportunities. NATS is much more flexible for application use cases, for example, it provides KeyValue and ObjectStore abstractions on top of the core persistent stream. There are a plethora of other details, but that is the basic gist. Overall, it has a lot of batteries included for building everything from embedded applications to large scale cloud-to-edge systems.

Synadia - The founder (Derek) created NATS. We are the primary maintainers of the project. We build products and services on top of NATS including a global SaaS where you can sign up and use "hosted NATS" with additional features. We offer a BYOC model, one of which we manage for you, or a Kubernetes-based self-service one that you deploy yourself. We also support fully self-hosted for customers that need to run in their own data centers or at the edge.

Regarding the comment re: the website, there are improvements we have in the works. Happy to engage and help clarify anything that is confusing.

I don't know Simon personally, I do know he has been blogging for a very long time and many of his posts are for his own benefit of recall, not necessarily intended for an external audience. Also He did not post this, a different person did.

(Disclaimer: I am a NATS maintainer and work for Synadia)

The parent comment may have been referring to the fact that NATS has support for durable (and replicated) work queue streams, so those could be used directly for queuing tasks and having a set of workers dequeuing concurrently. And this is regardless if you would want to use Nex or not. Nex is indeed fairly new, but the team on is iterating on it quickly and we are dog-fooding it internally to keep stabilizing it.

The other benefits of NATS is the built-in multi-tenancy which would allow for distinct applications/teams/contexts to have an isolated set of streams and messaging. It acts as a secure namespace.

NATS supports clustering within a region or across regions. For example, Synadia hosts a supercluster in many different regions across the globe and across the three major cloud providers. As it applies to distributed work queues, you can place work queue streams in a cluster within a region/provider closest to the users/apps enqueuing the work, and then deploy workers in the same region for optimizing latency of dequeuing and processing.

Could be worth a deeper look on how much you could leverage for this use case.

If a learning resource is lacking or confusing, Slack or GitHub issues/discussions is a way to engage and provide feedback so they can be improved. If the docs were confusing, there are other channels to get help to unblock folks. The outcome of that interaction would lead to improvements in the docs.

You're in your own little bubble and expect people to be mind readers.

Out of curiosity, have you asked questions in Slack or Github? If so, and you had a bad experience with the interaction, I get the sentiment and would offer help. But this comment is not constructive without context.

I'm guessing that the documentation is bad so one has to pay for support.

The NATS project has been open source for around 12 years, and part of the CNCF since 2018. This is an incorrect statement and very poor assumption to make because the documentation doesn't make sense for you.

I 100% agree it can be improved and we are working on a new docs site, but it is not quite ready.

In case its helpful, there is an increasing collection of examples on https://natsbyexample.com with new JetStream client examples among others. If you have specific requests, feel free to open issues in the corresponding repo: https://github.com/ConnectEverything/nats-by-example

This post is about a product for NATS.

I presume you are talking about the roadmap after the 2.10 release?

Yes that reads correct. The `sub` would a NATS user public nkey, the `iss` would be the NATS account public nkey (either the issuer nkey in config-mode or existing nkey in decentralized auth).

As long as it can verify the chain of trust for the user JWT that is returned, it should work.

The three schema types are shown here: https://docs.nats.io/running-a-nats-service/configuration/se...

auth request comes in -> generate user jwt, sign + encode -> respond with auth response.

As long as the necessary bits of the response and user JWT conform, it will work.

Auth callout is post-NATS client authentication, so it would not solve the "auth web flow" for authentication. Instead, the resulting token from that would be set as a cookie that then would be passed into the nats.ws client connection. The auth callout service would use that token to map to the concrete NATS user. The mechanism of doing that is up to the implementation. One option is to manage NATS claims into the OIDC provider (for the user authenticating) and then the auth service would decode that source JWT and extract the NATS claims and generate the NATS user JWT in the response.

Seems like I need to write a service utilizing NATS which talks to the OIDC server.

Auth callout was designed to be a generic extension point to delegate authentication and generate dynamic a user JWT that NATS understands (permissions, limits, etc). It enables an arbitrary backend to be integrated with, not tied specifically to OIDC. But indeed, this requires implementing a service that does this integration.

NATS auth story is a complicated one, and now with auth callout it's even more complicated.

There is a spectrum of auth options, starting with simple config-based, token or user/pass leading up to decentralized auth for use cases that need it. Auth callout is an opt-in thing, so it should only be adopted if it is truly necessary.

you have to have dedicated functions which you would call in the request handler [...] instead of being able to define a chain of middleware functions

I don't quite understand this statement. Wrapping a NATS handler is the same approach as wrapping an HTTP handler (within the same client app). The function would take a handler and return a handler. There can be inspection of the message within that function and the choice of calling the next handler, responding early, doing some external call, or doing nothing.

If you want to distribute this middleware, then you would need to have separate, explicit subjects that each middleware would subscribe to and then publish to for the next element in the chain (for a choreography approach).

There is also the "message slip pattern" where the ingest component sets the path as metadata (e.g. headers) that each middleware component uses that to know (at request time) which subject to publish the result to next.

If messages were frequent enough, and/or snapshots took a while to compute/transfer, you might never have a roll-up succeed.

Yes, good point. The "snapshotting up to a lagging sequence" could be achieved with two separate subjects to reduce contention, but is a bit more work.

It sounds like, in Drift's case, the snapshot effectively brings up the tail (snapshot), but the head can still be appended to with new events.

#1: Correct, I putting together an example this week to show what this looks like. Pretty straightforward.

#2: You can combine rollup and expected last sequence header to prevent this, unless I am missing another subtle detail?

(I am enjoying this thread FWIW :)

#1: This isn't a hard requirement to achieve the desired permissions. For the DriftDB use case, my understanding is that all members in the room have full pub/sub k/v permissions, so that could be achieved by declaring a new permission pinned to the room when the room is created or joined (this can be done dynamically without a config file reload).

#2 Publishes do support optimistic concurrency control using the `Nats-Last-Expected-Sequence` (stream level) or `Nats-Last-Expected-Subject-Sequence` for the subject-level. This ensures to concurrent publishes will be serialized and all but one is rejected with "conflict wrong sequence" error. For example headers in Rust[0] and WS[1]

[0] https://docs.rs/async-nats/latest/async_nats/header/index.ht...

It is a top priority. I do appreciate the constructive criticism and conversation. Feel free to reach out to me (byron at synadia dot com) or on the NATS Slack (https://slack.nats.io).

Would appreciate it if you disclosed your affiliation with Synadia and NATS before advertising it.

Fair point, but I did not mention Synadia. FWIW, I have been a NATS user for seven+ years prior to joining Synadia so I was speaking on behalf of myself and experience with the tech.

Also fair point that the nats.io website does not highlight this strongly. The NATS maintainers are aware (nearly all employed by Synadia) and we are working on it.

I disagree that simply because it is not advertised as a "edge" technology that it is not one of the best-in-class techs for edge. It simply means, we are doing a poor job at awareness.

Why don't you start comparing yourself to products and technology that serve the edge

The vast majority of people and customers compare NATS to Kafka and the variety of variants out there. Once the push on edge occurs, I suspect comparison to these other tech will occur.

To be clear, I am not looking for a "winner" in this discussion, rather my original comment was to correct a gap in understanding of what NATS is capable of.

Even with NATS jetstream, NATS has a focus on service communication.

It indeed excels at service communication as well. However, a core use case for NATS is the edge, be it your definition (browsers and mobile), but also in cars, factories, tractors, low-orbit satellites, etc, whether it is running on Kubernetes, k3s, or bare metal.

The issue you called out is a Firefox-specific issue, but it will be addressed and not indicative of an inherit limitation of NATS.

Check out this playlist of a live event I organized last fall with a variety of live demos: https://youtube.com/playlist?list=PLgqCaaYodvKY6xRbvB6ffON0_...

Was not aware of Plane, nice! Regarding the two points:

- A unique room ID/subject is a form of authentication. Essentially anyone having that unique identifier can join, akin to a token. This is straightforward to setup in NATS avoiding the ">" for all problem (which I may now need to write a blog post about ;-)

- Rollups are supported on a per-subject basis. Each room could be modeled as a subject and individually rolled up.

In fact it does all of these things now properly! STAN (NATS Streaming) was deprecated two years ago in favor of a new embedded subsystem called JetStream: https://docs.nats.io/nats-concepts/jetstream released in March 2021.

Nice, have you come across NATS? https://nats.io. The server natively supports WebSockets. There are many clients including Deno, Node, WebSockets, Rust, Go, C, Python, etc.

In addition to stateless messaging, it supports durable streams, and optimized API layers on top like key-value, and object storage.

The server also natively supports MQTT 3.1.1.

Check out SimpleIoT: https://github.com/simpleiot/simpleiot

If you have a low-volume, non-performant, non-critical database, k8s is fine. If you need it to perform and/or need built-in ops (managed backups or replication), use a managed service.

k8s _can_ do stateful, but if a managed service exists for this workload, use it. It is not about _can_ I run it on k8s, it is a _should_ question. Is it worth the cumulative effort required to achieve the same degree of quality.

Changelog podcast with the author: https://changelog.com/podcast/482

You may be thinking of NATS Streaming which was a predecessor to JetStream. It was indeed a standalone thing that used NATS under the hood. JetStream is now baked into the NATS server.

The Children's Hospital of Philadelphia | Bioinformatics Scientist | Philadelphia, PA | Full-time | Remote or Onsite

This role will work with bioinformatics scientists and software engineers to operationalize a set of pipelines used to align and annotate whole exome and genome sequencing data. This role will also collaborate with data archivists and metadata librarians to advise as a subject matter expert for genomic research data contributions into Arcus.

We are looking for highly creative people who share our mission to advance child health and who will thrive in a continuous learning environment, acquiring and applying both new technical skills and biomedical domain knowledge.

Apply: https://careers.chop.edu/job/Philadelphia-Bioinformatics-Sci...

If you have questions or would like a more direct overview of the position, please feel free to reach out ruthb [at] chop [dot] edu.

They aren't solving the same problem. An event sourced entity (should) have a very small scope. It acts as a consistency boundary around some bit of state. Each event in "the stream" for that entity represents a state change. This is analogous to a database row which represents the _current_ state of the entity being modeled for that table.

Like a table, there can be an arbitrary number of rows (>millions), thus you could have millions of event streams. You can't have millions of Kafka partitions. The scope/scale of a Kafka partition is not designed for that small granularity.

Again, I have no issue with Kafka and I think Pulsar is superior in the event streaming/platform space. I also the Kafka ecosystem is impressive and I love listening Confluent Cloud podcast! But it is not designed for this use case.

I have not watched that clip, but as I said above, Confluent isn't a good resource for defining this term since Kafka cannot be used to do ES. I would suggest an article like: https://domaincentric.net/blog/eventstoredb-vs-kafka

Also refer to the list of great resources @oskar_dudycz posted in another comment: https://github.com/oskardudycz/EventSourcing.NetCore#1319-th...

The title here is being overlooked, its not _terrible_, its _hard_. Also this quote from the article:

The bulk of these probably fall under "he obviously didn't understand X," or "you should never do Y!" in which case you would be absolutely right. The point of this is that I didn't understand the drawbacks or pain points until I'd gotten past the "toy" stage.

Most of what they discussed was indeed not great, but that was due to lack of understanding. I don't think anyone who does or promotes ES says it's _easy_. However, I would argue it _conceptually_ simple to understand (herein lies the fallacy people make simple != easy).

Like any pattern or tech that you are new too, it's going to be hard to build and operate in production (beyond the toy). Everyone takes for granted the decades we've had great tooling and operational patterns for state-oriented databases (mostly RDBMS').

And to be clear, event sourcing is "persistence strategy". What is mostly shown/discussed in the article is "event streaming" along with CQRS. They concepts are largely conflated. Once you understand, you won't be bamboozled.

Not to throw Confluent under the bus, but they have unfortunately written or promoted many articles for "event sourcing" use cases and this has confused everyone. You _can't_ do event sourcing with Kafka. It doesn't support optimistic concurrency control at the key level (where as something like EventStoreDB does natively or even NATS now with JetStream). Kafka if fantastic for what it is designed to do.. event streaming.

I appreciate the response! My first thought and assumption was indeed there are backend costs given the size limitations you mentioned. But I suppose if that is sufficient for some folks, then they can benefit from that without requiring a subscription.

Why is it 3.99 for a lifetime payment?

My cynical self tells me the product won't improve or your intention is generate enough hype to make a quick exit and the product will go away.

You should make this is a recurring cost.

The Children's Hospital of Philadelphia | Bioinformatics Scientist | Philadelphia, PA | Full-time | Remote/Onsite

This role will work with bioinformatics scientists and software engineers to operationalize a set of pipelines used to align and annotate whole exome and genome sequencing data. This role will also collaborate with data archivists and metadata librarians to advise as a subject matter expert for genomic research data contributions into Arcus.

We are looking for highly creative people who share our mission to advance child health and who will thrive in a continuous learning environment, acquiring and applying both new technical skills and biomedical domain knowledge.

Apply: https://careers.chop.edu/job/Philadelphia-Bioinformatics-Sci...

If you have questions or would like a more direct overview of the position, please feel free to reach out ruthb [at] chop [dot] edu.