chuckhend

Great work Lev!

IMO, Python rode some of the growth from the data and scientific computing computing. It was lower friction for many people partially due to the amount of blogs, and open source projects that were available. Pandas and Numpy made it quite easy to get up and running with a lot of analytics. pytorch and tensorflow were also there to facilitate this, made people able to get the benefits of the optimized C code but most users did not have to learn C. Eventually FastAPI came out for those looking to build products out of their data, and FastAPI had fantastic documentation and guides, which helped these same folks coming from data/scientific compute build working software. I suppose what I am describing are some of the network effects.

Helpful read, thanks for sharing! We have been (slowly) working on some fairness related queueing features over in pgmq for a bit too https://github.com/pgmq/pgmq/pull/442. It does get complicated on Postgres IMO.

Love SQLx for my Rust projects. I would like to figure out a great way to use the compile time checks in python or js projects, but haven't explored it yet.

LiteLLM is quite battle tested at this point as well.

it reimplements provider interfaces rather than leveraging official SDKs, which can lead to compatibility issues and unexpected behavior modifications

Leveraging official SDKs also does not solve compatibility issues. any_llm would still need to maintain compatibility with those offical SDKs. I don't think one way clearly better than the other here.

Very relevant talk if anyone is interested in learning a bit more about how the project is so fast. https://www.youtube.com/watch?v=gSKTfG1GXYQ

Automating this with linter and formatter is great. It moves the argument over style and format to a one liner change to a lint config instead of mingling it with the with the main code change.

Congrats to the paradedb team!

you can send or read a single message at a time or as many as you want in a batch.

https://github.com/tembo-io/pgmq?tab=readme-ov-file#read-mes...

pop() or read() in a loop, yes. can read 1 message or many messages at a time.

what we do at Tembo in our infrastructure is pause for up to a few seconds if a read() returns no messages. when there are messages, then we read() with no pause in between. when the queues are empty it amounts to less than one query per second. there is not much cost to reading frequently if you use a client side connection pool, or a server side pool like pgbouncer.

The client libs are a nice convenience, but most users write the sql directly when integrating with other SQL statements, something like:

begin; select * from my table group by... select pgmq.send(); commit;

There is a long poll, https://tembo-io.github.io/pgmq/api/sql/functions/#read_with...

We have been talking about a push using Postgres 'notify', or even via an http, but we don't have a solid design for it yet.

This is exactly how pgmq is implemented, + the usage of VT.

I agree. But it can be useful to have a guarantee, even for a specified period of time, that the message will only be seen once. For example, if the processing of that message is very expensive, such as if that message results in API requests to a very expensive SaaS service. It may be idempotent to process that message more times than necessary, but doing so may be cost prohibitive if you are billed per request. I think this is a case where using the VT to help you only process that message one time could help out quite a bit.

IMO, it is most valuable when you are looking for ways of reducing complexity. For a lot of projects, if you're already running Postgres then it is maybe not worth the added complexity of bringing in another technology.

If the message never reaches the queue (network error, database is down, app is down, etc), then yes that is a 0 delivery scenario. Once the message reaches the queue though, it is guaranteed that only a single consumer can read the message for the duration of the visibility timeout. FOR UPDATE guarantees only a single consumer can read the record, and the visibility timeout means we don't have to hold a lock. After that visibility timeout expires, it is an at-least-once scenario. Any suggestion for how we could change the verbiage on the readme to make that more clear?

pgmq.archive() gives us an API to retain messages on your queue, its an alternative to pgmq.delete(). For me as a long-time Redis user, message retention was always important and was always extra work to implement.

DLQ isn't a built-in feature to PGMQ yet. We run PGMQ our SaaS at Tembo.io, and the way we implement the DLQ is by checking the message's read_ct value. When it exceeds some value, we send the message to another queue rather than processing it. Successfully processed messages end up getting pgmq.archive()'d.

Soon, we will be integrating https://github.com/tembo-io/pg_tier into pgmq so that the archive table is put directly into cloud storage/S3.

This is exactly how we do this in our SaaS at Tembo.io. We check read_ct, and move the message if >= N. I think it would be awesome if this were a built-in feature though.

We talk a little bit in https://tembo.io/blog/managed-postgres-rust about how we use PGMQ to run our SaaS at Tembo.io. We could have ran a Redis instance and used RSMQ, but it simplified our architecture to stick with Postgres rather than bringing in Redis.

As for scaling - normal Postgres scaling rules apply. max_connections will determine how many concurrent applications can connect. The queue workload (many insert, read, update, delete) is very OLTP-like IMO, and Postgres handles that very well. We wrote some about dealing with bloat in this blog: https://tembo.io/blog/optimizing-postgres-auto-vacuum

PGMQ doesn't give you a way to deliver the same message to concurrent consumers the same way that you can with Kafka via consumer groups.

To get this with PGMQ, you'd need to do something like creating multiple queues, then send messages to all the queues within a transaction. e.g. `begin; pgmq.send('queue_a'...); pgmq.send('queue_b'...); commit;`

Simplicity is one of the reasons we started this project. IMO, far less maintenance overhead to running Postgres compared to RabbitMQ, especially if you are already running Postgres in your application stack. If PGMQ fits your requirements, then you do not need to introduce a new technically.

There's definitely use cases where PGMQ wont compare to RabbitMQ, or Kafka, though.

I think it would be tough to compare. There are client libraries for several languages, but the project is mostly a SQL API to the queue operations like send, read, archive, delete using the same semantics as SQS/RSMQ.

Any language that can connect to Postgres can use PGMQ, whereas it seems River is Go only?

Take a look at https://github.com/tembo-io/pg_vectorize. It makes it a lot easier to get started. It runs on pgvector, but as a user, its completely abstracted from you. It also provides you with a way to auto-update embeddings as you add new data or update existing source data.

check out https://github.com/tembo-io/pg_vectorize - we're taking it a little bit beyond just the storage and index. The project uses pgvector for the indices and distance operators, but also adds a simpler API, hooks into pre-trained embedding models, and helps you keep embeddings updated as data changes/grows

I've heard of people having success with methods like this. Would be awesome if we found a way to build that into this project :)

We are working on a 'self-hosted' alternative to OpenAI. The project already has that for the embeddings. i.e. you specify an open-source model from hugging face/sentence-transformers, then API calls get routed to that service that you're self hosting in a container next to Postgres. This is how the docker-compose example in the project readme is set up. We'll be doing the same pattern but for chat completion models.

On Tembo cloud, we deploy this as part of the VectorDB and RAG Stacks. So you get a dedicated Postgres instance, and a container next to Postgres that hosts the text-to-embeddings transformers. The API calls/data never leave your namespace.

There's an API that abstracts vector search only. vectorize.search() and that part is not unique to LLMs but it does require selection of an embedding model. Some people have called embedding models LLMs.

vectorize.rag() requires selection of a chat completion model. Thats more specific to LLM than vector search IMO.

One difference between the two projects is that pg_vectorize does not run the embedding or chat models on the same host as postgres, rather they are always separate. The extension makes http requests to those models, and provides background workers that help with that orchestration. Last I checked, PostgresML extension interoperates with a python runtime on same host as postgres.

PostgresML has a bunch of features for supervised machine learning and ML Ops...and pg_vectorize does not do any of that. e.g. you cannot train an XGboost model on pg_vectorize, but PostgresML does a great job with that. PGML is a great extension, there is some overlap but architecturally very different projects.

To oversimplify, its more like sending a block of text to an LLM and asking it to answer a query based on that block of text.

Its certainly up for debate and there is a lot of nuance. I think it can simplify the system's architecture quite a bit of all the consumers of data do not need to keep track of which transformer model to use. After all, once the embeddings are first derived from the source data, any subsequent search query will need to use the same transformer model that created the embeddings in the first place.

I think the same problem exists with classical/supervised machine learning. Most model's features went through some sort of transformation, and when its time to call the model for inference those same transformations will need to happen again.