dandraper

Sorry I was imprecise.

Query logging would tell you what queries were executed. E.g the following would tell you that 10 user records were returned with name and email but not which users.

SELECT name, email FROM users LIMIT 10;

CipherStash can tell you the ID of every row that was returned, like:

users;email,name;1 users;email,name;2 users;email,name;3 etc

If you need to know the actual values instead of just the IDs (say when doing an investigation), you can do a query to join the audit data with the users table.

Assuming audit logs are in a table called audit with a column called record_id and a timestamp:

-- full forensic analysis SELECT name,email FROM audit LEFT JOIN users ON users.id = audit.record_id WHERE timestamp BETWEEN $1 AND $2;

And because the data is encrypted, that query will ALSO be audited. Not even admins escape the auditing :p

All of this is important for "materiality" investigations. If you suspect some data has been accessed inappropriately, knowing how much and exactly which records is very useful - especially if deciding if customer/regulator notification is required.

What does the risk profile look like in a full leak of the encrypted database?

We actually have a number of different schemes which trade leakage for performance/storage overhead/compatibility. Each one has its own type in Postgres so you just create a column with whatever type you want.

The weakest (most leakage) is OPE - order preserving encryption. We use an encoding scheme that works like scientific notation so an attacker with a DB snapshot would learn relative order but not the size of the gaps between values. Its major benefit is compatibility - it works everywhere.

The strongest is Block ORE - Order Revealing Encryption. A DB snapshot reveals nothing more than standard randomized encryption (IND-CPA2 semantic security). The tradeoff is that BlockORE values are much bigger: 32-bit integer goes to 384 bytes. On Postgres its still very fast and works with standard B-trees.

If you had a column of integers are you able to order them without decrypting?

Yes. For either scheme, its just an EQL function:

SELECT * FROM foo ORDER by eql_v3.ord_term(x);

Check that values are the same?

-- $1: encrypt(query) SELECT * FROM foo WHERE eql_v3.eq_term(x) = eql_v3.eq_term($1);

Identify null values?

NULL is just like any value for OPE/ORE - encrypt it and use that to query -- $1: encrypt(NULL) SELECT * FROM foo WHERE eql_v3.eq_term(x) = eql_v3.eq_term($1);

NULL values can be encrypted with either scheme. Very safe to do so with the ORE scheme (fully randomized values). If encrypting strings, NULLs would reveal length but you can always pad if not leaking value length is important to your security model.

In the OPE scheme, NULL would encrypt deterministically so if you expect a lot of NULLs in your data this could leak the distribution.

We are working on a capability now called Leakage Tuned Domains. The goal here is not to eliminate leakage but scope it to a specific data type (e.g. birthdays) such that any leakage cannot be an advantage to an attacker. Think k-anonymisation or differential-privacy on steroids.

Some links if you want to learn more: https://github.com/cipherstash/encrypt-query-language https://cipherstash.com/blog/fixing-a-1-in-256-bug-in-cllw-o...

Because queries are encrypted, too. And you need to be authorized to encrypt a query term. That's the whole crux.

TBH, I'm so close to this now I forget that part isn't obvious! You literally can't do:

select * from blah where secret_col == 10

And get meaningful results. You can only do something like:

select * from blah where secret_col == Encrypt(10);

And in order to generate that encrypted query, you must be authorized. So an adversary can't simply do trial and error queries to guess records.

Let’s chat dan@cipherstash.com

Applications do not have access - individual authenticated users have access. A user must be authenticated & authorised in order to be able to create encrypted query terms. This is enforced by CipherStash's key server (ZeroKMS), not the application.

Having a copy of the production database and a local copy of the application and a debugger is insufficient to be able to encrypt/decrypt any values or run any queries involving encrypted search terms: you'd still need to authenticate with ZeroKMS.

Having root access to a production server running an application is a different matter because you'd have access to the entire address space of the app that real authenticated users are interacting with. A sufficiently motivated adversary would be able to intercept auth tokens allowing impersonation of an authorised user.

The application server process is a trust boundary in this model and thus requires appropriate access controls.

To address this last point, we now support decryption in edge workers like Cloudflare or Supabase Edge Functions. We also have browser-based encryption/decryption in the works. Both of these approaches mean data is never decrypted in the app, not even in production.

Thanks for all the comments.

CipherStash isn't something that we just threw together on a weekend. We've spent years developing it. I've personally invested almost every waking moment for the last 6 years working on this (and the prior 2 preparing to quit my job so I could work on it full time).

I get that it seems like snake oil to some. But that's just because we don't do a good enough job of explaining what we've created. We're working on that - we're engineers, not marketers.

There have been prior attempts at making tech like this work, and maybe because those approaches were limited, its tainted how folks see encryption tech. We've studied many of the previous attempts and invested in the engineering required to overcome the problems.

Key management - we made a 2 party key system that is faster and has a better security model than AWS KMS alone. Not because we thought it was fun, not on a whim but because it was the only way to solve the real security challenges we faced when it comes to protecting data.

Searchable encryption - that is less secure than homomorphic encryption but FAST. Like under a few milliseconds for most queries (homomorphic would take literally minutes or worse). And a whole lot better than no encryption at all. It uses a key per value.

Encrypt query language - a framework that makes searchable encryption work in standard postgres. No exotic index schemes, no custom builds. Just SQL functions.

Typescript and Rust libraries to make this _almost_ drop-in for Prisma Next, supabase-js and Drizzle.

Identity integrations with Supabase, Clerk, Auth0, and Okta.

A postgres proxy that automatically encrypts and decrypts data for authorized requests - for when using the SDK isn't possible/convenient.

And an audit system that records every access and can send data to anything you can connect to AWS firehose or an S3 bucket.

This is real and IMHO is exceptional engineering...anyone know a great technical marketer?!

https://github.com/cipherstash/stack https://github.com/cipherstash/benches https://github.com/cipherstash/proxy https://github.com/cipherstash/encrypt-query-language

lol yeah. Just a side chain that I found on a weekend. 8 years and $10m later. Couldn’t possibly be real now could it?

Yep, we’ve read the papers. Naveed et al and Grubbs et al primarily.

This doesn’t have anything to do with being able to change the code. Devs can code and change things as they need. The database is also the same as it always is.

What CipherStash does is let you specify specific columns that you want to encrypt. The application (via our SDK), encrypts values for that field before it’s saved and optionally decrypts it again when it’s read. Decryption is performed when the user provides a valid credential (JWT).

This would allow you to give your devs full access to the production DB. You don’t have to give them permission to decrypt any or specific fields if they don’t need it.

So for your use case that would possibly save a lot of effort - no need to create a santized copy of the database (which in itself is fraught).

What James was referring to was the attacker threat model. Decryption happens in the application code. So an attacker who can steal encryption keys as well as be a valid JWT would be able to decrypt data for as long as the JWT is valid. That scenario is not something CipherStash can fully prevent.

What it does prevent is unauthorised access to data. Whether via the database directly or via the application.

What’s important here is that it’s the data itself that protected. If you move encrypted values from your prod DB to test or dev, the same rules apply. A user who can’t read “email” values that were encrypted in prod would still not be able to read those values if you moved them somewhere else.

Does that clarify things?

No, to derive a key you need a client key (controlled by the app) and key-seeds for each value which can only be retrieved from the key server with a valid JWT. The JWT is time bound (15 mins).

Now, if an attacker could gain access to the client key AND a valid JWT then they would be able to decrypt until the JWT expired. Blast radius reduced but we don't stop the attack entirely. An attacker with that much advantage is pretty hard to stop dead. But note that 2 things are required for the adversary to do this - where as in most systems, getting a JWT or a decryption key alone would be enough.

If you're not a cryptographer, then I'd suggest you reserve judgement about what flaws you think might be in the system. We have cryptographers on our team, have connections with universities in the US and Australia and our work is based on published, peer-reviewed papers. E.g. Lewi-Wu 2016 and Syalim et al 2011 and 2021.

Encrypted*.

It is indeed the case.

No, not by default. You could but as you said, that would be a A LOT of data.

It depends on your setup. If you're using Supabase, one way is to send the logs to Clickhouse and use the Clickhouse partner integration to query the audit logs and join it to the actual data.

Keeping only the ids in the audit log means you need to stitch the data together later. This means you don't accidentally leak data via your audit trail!

Not just each column, each value.

The answer to your question is yes.

Explanation: The identifier is actually for the key that encrypts the value (1 unique key per value).

1. When the value is encrypted for the first time, it gets an ID. 2. When its decrypted, the application requests the key for that ID from the key-server (key-server records that the data was accessed)* 3. When updating, the same data key is used so the ID is persistent*

* Technically the key server doesn't return a key, it generates partial key material that can be used to derive the data key in the app. It can do this at up to 10,000 keys per second. * You can also tag each value with the table/column name and the row-id to link everything together

The 3 values form the "descriptor" of a value: `table/column/id`.

CipherStash - specifically the key service. There is a lot of data for sure but we only record an identifier for each value and (optionally) the user ID. It compresses well.

If we do, I'll be sure to attribute it to this thread :p

See above but some additional points:

Keys are not stored in the database or in the application. Every data key is derived at query time via a 2-party system: 1. by the key server which manages root-key material (stored in an HSM or traditional KMS) 2. keys derived via a user credential at the time of the query

SQL Injection case: An adversary signs up to your app, pulls of a SQLi and you might think that the app will just decrypt the values. It won't because the adversary's auth cred can't derive keys to decrypt anything but the data they are explicitly allowed to access.

RCE/shell compromise: Because keys are derived on demand these kinds of attacks are largely inert. If the attacker could read memory from the running app on-demand then they could conceivably see credentials in flight but each key only ever decrypts one value so the blast radius is limited.

On the cost of decryption: Queries typically only return a subset of the values in a table and, importantly, don't need to decrypt anything at all. Querying and decryption are independent. If you DO need to decrypt a large volume of data, CipherStash can do it at 10,000 values per second.

Query performance is under 1ms for simple queries and no-more than 200-300ms for more complex queries.

Full benches here: https://github.com/cipherstash/benches

No security tool is perfect, nor "gold plated" but this tech meaningfully reduces blast area of attacks, makes many attacks inert and gives you an incredibly reliable audit trail.

I've replied above

Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.

This is all started when I was the CTO of a health-tech and the engineers all had access to patient data. They needed DB access (data migrations, managing backups, reading non-sensitive fields) but not access to the sensitive values. 10m+ patient records - it made me nervous.

Row-level Security (RLS) was a nightmare, it was slow, easy to mess up and could be overridden by DBAs anyway. Plus the application connected to the DB using a service account so RLS was useless for end-user access control.

CipherStash solves the following:

* Hide sensitive data from direct database accessors (authorized or not!)

My original use-case. Direct access to the DB doesn't reveal sensitive data (unless the user also has permission to decrypt a specific record).

* Support or even replace application access controls

Decryption happens in the application layer and is based on end-user identity. That means a gap in application defenses is mitigated by the encryption. If the user can't decrypt the value then they can't access the data.

* Access auditing

Every sensitive data access is recorded (and by who) which is very useful for compliance and incident response. SQL query auditing is great for knowing what queries were run but it doesn't tell you what data was actually returned. Its also blind once data actually leaves the DB and can inadvertently leak sensitive data itself. Using key accesses with non-identifying value IDs to audit decryptions makes the auditing more reliable and super granular.

We have customers who use the tech to prove to their users (often large enterprise) that no internal staff can access the data.

* Agent access to data

The encryption shifts access control decisions down to the data layer on a per-value basis. This is incredibly useful for gating access to agents that need access to data. Every decryption requires authentication (e.g via Supabase Auth, Auth0 etc) and combining it with agent identities (creds issued to agents) means you can not only limit what agents can access, you can also audit exactly what _was_ accessed.

The searchable encryption is the enabler: value level encryption is what makes all of this possible. Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.

Hope that helps! I'm so in the weeds I don't know if I explain things that well sometimes!

Also the proxy runs in your infra not ours (docker container).

Hey, CipherStash founder here: So we actually have 2 kinds of encryption: * standard encryption which can be decrypted (we call this "source" encryption) * what we call SEM: searchable encrypted metadata (cannot be decrypted)

Queries are performed by generating SEM which is "compared" to values in the table and the source encrypted values are returned to the caller (usually an application or agent) which does the actual decryption.

This is crucial because if you have a table of millions of rows, you don't want to decrypt every one to find the results. SEM works with built in postgres indexes (B-tree, GIN) so a query over millions of rows executes in literally milliseconds. Then the resultset (typically only 10s or 100s or rows) can is all that needs to be actually decrypted.

You're right that we use OPE and ORE. OPE is necessary on Supabase right now due to some limitations on their side but that's being addressed. ORE is more secure and almost as fast but requires "CREATE OPERATOR FAMILY" permissions in postgres to work.

We also use encrypted bloom filters and structured encryption (STE) for queries over JSON objects.

CipherStash founder here.

If a column is encrypted using standard encryption (like AES-GCM) then the values are non-deterministic and fully randomized. That means that if you encrypt the same value twice, you'll get 2 different ciphertexts.

So the query: select * where secret_col == 10

Would actualy be: select * where secret_col == encrypt_aes(10);

And values in secret_col will never match (because the output of encrypt_aes will be different every time, even for the same input).

A common way around this is to use deterministic encryption which eliminates the randomization at the cost of a slightly weaker security model. What leaks is the ability to see if any 2 plaintexts are equal (because they have the same ciphertext) - what you actually want in the case of search.

You have to be careful implementing deterministic encryption though: don't use AES-GCM with a fixed nonce because the scheme completely breaks. You can use CBC mode but then you lose authenticity. We use AES-GCM-SIV (synthetic IV which retains authentication but is secure under a fixed nonce) and HMAC (keyed hashing).

But there are approaches to solving queries like: -- range/order SELECT * FROM foo WHERE x > 10; SELECT * FROM foo ORDER BY x; -- fuzzy text SELECT * FROM foo WHERE name ~ "dan";

These capabilities are all based on public research: For example, order/range uses: https://eprint.iacr.org/2016/612.pdf

Our docs are quite limited at the moment (fixing as quickly as we can!) but you can see the current list of supported queries here: https://cipherstash.com/docs/stack/cipherstash/encryption/qu...

CipherStash founder here: I'll cop it about the website - we're a small team so we lean on AI for marketing copy but this is a reminder that we need to do better.

The tech is the result of 6 years of work. We're a team of 8 engineers and have been working on it full time thanks to VC funding.

I'm obviously biased but its legit tech. We've invested in making it fast, secure and able to support most common query patterns.

Our github repos might be better than our website but I'll certainly admit that our docs/marketing needs some work!

Checkout out: https://github.com/cipherstash/stack https://github.com/cipherstash/encrypt-query-language https://github.com/cipherstash/proxy

Hey, CipherStash founder here.

Our model is that data is encrypted in the application before being saved to the database. In order to encrypt, decrypt or query, you need to use the SDK (@cipherstash/stack). Connecting via psql, pgadmin etc, you'll only ever be able to see encrypted values (those tools don't know how to decrypt).

Proxy connects to the database and then you connect to the proxy (with psql or whatever). Proxy can now perform queries and decryptions on your behalf so that you can still access data if you need - hence "escape hatch".

The proxy authenticates using your credentials (or an access key) and interacts with the key management server.

Hope that helps!

CipherStash founder here: FHE isn't the only option here. Specialized searchable encryption schemes exist and are much faster than FHE. Different flavours can be combined to create a comprehensive search system which is very close to the performance of plaintext information retrieval. FHE remains an option for generalized computation but can be reserved for small datasets that have been narrowed down using fast searchable encryption.

I don't think FHE is the solution to PIR but it might well form a part of it when combined with more practical approaches.

Protect.js makes encrypting sensitive fields in Node apps like Next.js simple for devs without the need for expertise in cryptography. It supports searchable encryption, integrates with Auth0, Okta and Clerk and works with the performance-focused ZeroKMS key management service.

Clearing memory of secrets is a black art. This post is a summary experiments I ran to convince myself that memory in Rust _really_ is being Zeroized.

As with any secure system, it depends on your threat model but the early versions of OPE are woefully insecure. The attack you're describing sounds like the one on the Boldyreva scheme from 2009. While all OPE schemes leak some information, the more recent schemes are much more secure and often quite appropriate for many use cases (arguably at least).

ORE has different security properties, particularly Lewi-Wu 2016. There are also now hybrid schemes like the EncodeORE scheme of 2020 which is both more efficient and more secure than previous OPE and practical ORE schemes.

Note that this paper is about the CLWW scheme of 2015. The Lewi-Wu Scheme of 2016 (Block Order Revealing Encryption) has entirely different security properties. Storage of the right ciphertexts (as they describe in the paper) is semantically secure.

That's true except that if that session key is lost or exfiltrated, the scope of the breach is everything that key was used to encrypt (all of the user's data in your example).

The other consideration is how to safely cache the data key? What if the cache is popped?