tullie

Fair enough. We're releasing our first bring-your-own-cloud (BYOC) offering in April. We're working with a big e-commerce platform in Germany that has data sovereignty requirements so totally get the constraint and excited to offer something like this. We're starting with AWS then will do GCP at end of year. Full on-premise will still be awhile though to be honest.

For the cloud platform though (console.shaped.ai), i'd recommend just testing with some synthetic or deanonymized data or our demos and then if you're interested in BYOC reach out after April!

Makes sense. Implementation simplicity was part of the reason we didn't go for this. Currently the language is a transpiler, which maps SELECT and FROM into the retrieval stages, WHERE as post-filter stage, ORDER BY as score stage and REORDER BY to the final stage reranker. Because the text lookup is pushed through to the retriever we put it there. In the future we'll have more logic in the language compilation process which means that we can move things around a bit and under the hood they all get pushed to where they need to anyway.

I've been loving all these projects that are integrating LLMs/encoding directly into the language. There's so much power there.

Someone shared with me these the other day and we're inspired to add more remote LLM calls directly into ShapedQL now: https://github.com/asg017/sqlite-rembed https://github.com/asg017/sqlite-lembed

This line comes from a specific customer we migrated from Elastic Search, they had 3k lines of query logic, and it was completely unmaintainable. When they moved to Shaped we were able to describe all of their queries into a 30 line ShapedQL file. For them the reducing lines of code basically meant reducing tech-debt and ability to continue to improve their search because they could actually understand what was happening in a declarative way.

To put it in the perspective of LLMs, LLMs perform much better when you can paste the full context in a short context window. I've personally found it just doesn't miss things as much so the number of tokens does matter even if it's less important than for a human.

For the turbopuffer comment, just btw, we're not a vector store necessarily we're more like a vector store + feature store + machine learning inference service. So we do the encoding on our side, and bundle the model fine-tuning etc...

You can think of Shaped more like a vector store + feature store + ML inference combined into one service. This bundling is what makes it so easy to get state-of-the art real-time recommendations and search performance.

E.g imagine trying to build a feed with pgvector, you need to build all of the vector encoding logic for your catalog, then you need to build user embeddings, the models to represent that and then have a service that at query time encodes user embeddings from interactions does a lookup on pgvector and returns nearest neighbor items. Then you also need to think about fine-tuning reranking models, diversity algorithms and the cold-start problem of serving new items to users. Shaped and ShapedQL bundles all of that logic into a service that does it all as one in a low-latency and fault-tolerant way.

Subprocessors are here: https://docs.shaped.ai/docs/v2/support/security

Thanks for the feedback on the privacy policy, let me see if we can get that changed. For what it's worth we don't share personal information with anyone, this is likely just overly defensive legal writing on our part.

Makes sense! Agreed on the diversity for agents being a bit contrived here.

All of the retrievers do support pre-filtering, you just add the where clause within the retriever function. We're working on more query optimization to make this automatic also.

Would love to chat about it, and talk about dbt integration. There's a few use cases that have come up where this would be really helpful. I'll PM you.

1) So we do actually have a python and typescript API, it's just the console web experience is SQL only as it feels the best for that kind of experience. The most important thing though is that it's declarative. This helps keep things relatively simple despite all the configuration complexity, and is also the best for LLMs/agents as they can iterate on the syntax without doc context.

2) Yeah exactly, joins is something we can't do at the moment, and i'm not sure the exact solution their honestly. Under the hood most of Shaped's offline data is built around Clickhouse, and we do want to build a more standard SQL interface just so you can do ad-hoc, analytical queries. We're currently trying to work if we should integrate it more directly with ShapedQL or just keep it as a separate interface (e.g. a ShapedQL tab vs a Clickhouse SQL tab).

3) We didn't really want to create a new SQL dialect, or really a new database. The problem is none of the current databases are well suited for search and recommendations, where you need extremely low latency, scalable, fault-tolerance, but also the ability to query based on a user or session context. One of the big things here is that because Shaped stores the user interactions alongside the item catalog, we can encode real-time vectors based on those interactions all in an embedding query service. I don't think that's possible with any other database.

4) I haven't looked into mindsdb too much, but this is a good reminder for me to deep dive into it later today. From taking a quick pass on it, my guess is the biggest difference is that we're built specifically for real-time search, recommendations and RAG, and that means latency, and ability to integrate click-through-rate models and things becomes vital.

Thanks so much for the playground syntax, have some follow up questions but i'm going to pm you if that's okay. Agreed on the being able to see which columns exist.

Hey everyone,

I’m one of the founders of Shaped. We’ve spent the last few years building relevance infrastructure.

Over the time we've noticed a convergence happening: The stack for Search (usually Elastic/OpenSearch) and the stack for Ranking (Vectors + Feature Stores) are merging. Maintaining two stacks for the same math is inefficient.

We just launched Shaped 2.0 to collapse this stack. It treats Relevance like a database problem:

1. Ingest: Connect to Snowflake/Postgres/Kafka. 2. Index: We handle the embeddings (ModernBERT, etc.) and statistical feature engineering. 3. Query: We built a SQL-like language (ShapedQL) to retrieve, filter, and rerank candidates in <50ms.

Instead of writing Python glue code to merge BM25 and Vector results, you can do it in a single query:

SELECT title, description FROM semantic_search("$param.query"), keyword_search("$param.query")

ORDER BY -- Combine semantic relevance (ColBERT) with a personalized model

  colbert_v2(item, "$param.query") +
  click_through_rate_model(user, item)

Try it here: https://console.shaped.ai/register

Yeah exactly. I was really worried about reducing the serendipity that HN provides (as it's arguably why I've used it for so long as well) but the configurability allows it so that everyone can tweak their level of personalization to get their perfect goldilocks level.

Yeah I definitely agree. I think the nuance is that personalization doesn't have to completely overrun the current ranking algorithm, it more just slightly boosts known similar content you might like. Should be the best of both worlds really.

The other direction that isn’t explicitly mentioned in this post is the variants of SASRec and Bert4Rec that are still trained on ID-Tokens but showing scaling laws much like LLMs. E.g. Meta’s approach https://arxiv.org/abs/2402.17152 (paper write up here: https://www.shaped.ai/blog/is-this-the-chatgpt-moment-for-re...)

The performance for us was best when we evaluated a couple of options, both in terms of scale and latency. I also like the arrow/dataframe interface. We use arrow everywhere else at Shaped so it was a natural integration.

LanceDB! We love it.

Yes when integrating Shaped you connect up the data sources needed to ingest: interactions, items and users. The Shaped interface then allows you to select which exact fields should be used for creating a Shaped model. We provide a full SQL interface to do this, which gives a lot of flexibility.

Our dashboard provides monitoring to help understand what data is ingested and view data quality over time. We expect customers to monitor this but also have alerts on our side and jump in to help customers if we see anything unexpected.

The dashboard also shows training metrics over time (how well does the model predict the test set after each retrain?) and online attribution metrics (how well does the model optimize the chosen objective?).

Customers can disable retraining if they want (which is essentially pinning the model version to current), we can do model version rollbacks on our side if we see an issue or if requested but it's not a self-serve feature yet. Because we've made it easy to create or fork a Shaped model, we've seen customers often create several models as fall-backs that rely on more static data sources or are checkpoints of a good state.

Australia represent! Although we're based in NYC we still are a mostly Aus/international team over here, it's great!

The biggest change is some of the less sexy stuff, like scale and security. E.g. we're now able to scale to 100M+ MAU companies with 100M+ items, and we have a completely tenant isolated architecture, with security as a top priority.

We've also made the platform more configurable and lower levels and we've found that people like choosing their own models and experimenting rather than just relying on our system.

Finally, we launched search only a couple of months ago and are currently heavily focused on building a best-in-class experience there.

Thank you! Would love to catch up sometime assuming you're in NYC with the rest of the Pinecone team!

Yes by 100M+ users we definitely mean end-users, wasn't intentional to mislead so thanks for flagging -- we'll update.

Yes play.shaped.ai! We just opened that up in a gateless way for this post. Let me know what you think. I should also mention that these demo models are on our cold-tier so that it doesn't break things, in production there's a big speed up.

Compared to Vespa, we're much easier to get setup on. A big part of this is that we have real-time and batch connectors to all leading CDPs and data warehouses. E.g. if you're on Amplitude it takes < 10mins to stream data directly to Shaped and start seeing initial results.

Being quicker to setup, also means it's quicker to build and experiment with new use-cases. So you can start with a feed ranking use-case the first week and then move to an email recommendation use-case the next week.

In terms of actual performance and results, we've never gone head-to-head in an A/B test so i'm not sure the specifics there honestly!

The short answer is: we're better at recommendations and personalization and lean towards more technical teams (e.g. even with data/ML experience). They're better at traditional search and, these days, lean towards less technical teams.

Longer answer is in our blog post about it: https://www.shaped.ai/blog/shaped-vs-algolia-recommend :)

We have a library with about 100 algorithms which you can choose from or by default we automatically choose based on your objective.

Majority of them are open source models we've forked and improved. Just as an example, we integrated in gSASRec last week: https://github.com/asash/gSASRec-pytorch, and added a couple of improvements on scale and the ability use language and image features. We use LLMs for the encoding of unstructured data, and we host these our self, although OpenAI and Gemini are used for error message parsing and intelligent type inference, things not on the real-time path.

More info here: https://docs.shaped.ai/docs/overview/model-library

Thank you!

Would love to chat, we've had several customers come over from Algolia and they've seen significant uplift. I can share more if you want to message me at tullie@shaped.ai.

Our pricing is competitive with Algolia's to give you an idea there. We really wanted to get pricing calculator done get before this post but ran out of time. Keep an eye out over the next month for it to come up!

The build vs buy decision does come up, but like you mentioned, the product direction of Shaped is to be primitives for search and recommendation, allowing users that want to build use Shaped to empower them to build quicker (e.g. integrated behind their psudo-recommendation engine). In truth we have multiple abstractions to Shaped allowing more technical teams to integrate like this, or less technical ones to have more of an end-to-end integration experience.

The other related market trend we think about here: recommendation is going through a similar journey to what search did 10 years ago. Search at some point was more build leaning, but over time the technology became democratized and then companies like Elastic and Algolia had offerings that pushed search to lean towards buy. We're seeing recommendations going through the same revolution now that the technologies and system design (e.g. 4 stage recommenders) are more solidified. It's the data that makes these systems unique between companies not the infrastructure or algorithms.

Thanks for the first question!

We run online A/B tests to objectively measure quality against our ranking algorithms and other baselines. As you mentioned it's crucial that the measure of quality for these tests chosen is fair and correlates with the topline business objective. E.g. if you just evaluate clicks then the system will show click-baity content and overall perform worse.

To handle this, we make it really easy to define different objectives and experiment with how it changes results. So although we don't claim to solve the issue directly, we believe that if users can quickly experiment with different proxy objectives, that'll be able to find the one that correlates with their topline objective quicker.

Great to see OpenAI finally being open :)

Love it! Wish I had this when practicing for our YC interviews

Congrats to the team at Search.io! Looking forward to seeing what Algolia does with vector search.

We just launched Shaped's Model API, which allows you to configure a multi-stage recommendation system for your ranking use-case. Excited to see what people make with it.

I’m fairly new to using AWS Lambdas so loved this post. Lots of tangible insights on how we can improve things.

Love vast.ai. Only wish is for it to be easier to work with persistent disks over time. I only use it for one off expensive jobs now.