Announcing Google Cloud Spanner as a Vault Storage Back End

https://cloudplatform.googleblog.com/2018/02/announcing-Google-Cloud-Spanner-as-a-Vault-storage-backend.html

by petercooper • 8 years ago

126 43 8 years ago

cloudplatform.googleblog.com

manojlds• 8 years ago

Seth joining Google seems to be beneficial for both the companies!

casey_lang• 8 years ago

I had to do a double take on the author name. I didn't realize Seth had moved but it certainly seems like a great fit.

jlgaddis• 8 years ago

Isn't Spanner outrageously expensive?

ISTR reading to basically avoid touching Spanner if you're just experimenting or playing around, unless you are prepared to receive a very large bill (something like ~$700/mo. just for the lowest-end setup).

cobookman3• 8 years ago

If you want to use spanner but don't have a large data workload I would recommend looking into firestore as it is built on top of spanner.

http://firebase.googleblog.com/2017/10/cloud-firestore-for-r...

jey2• 8 years ago

Isn't it actually built on top of Datastore?

perfmode• 8 years ago

I would love to be able to use some cheap, basic DB under the Spanner API for ease of migration.

odammit• 8 years ago

I think it’s for the convenience of people that are already using spanner and don’t want an additional data store.

sethvargo4• 8 years ago

Spanner is relatively more expensive than something like CloudSQL or BigTable, but brings added reliability, availability, and consistency. There are definitely more cost-effective solutions for Vault storage, such as the Google Cloud Storage backend, but at the time of this writing, they do not provide the same guarantees as Spanner.

Ultimately it's about your organization's risk tolerance. If a service is non-critical, non-SPOF, perhaps Spanner is overkill. On the other hand, if you have a service that requires immediate global consistency and availability, such as global secrets management, Spanner is a great choice.

To be clear, this post is not saying "use Spanner with Vault"; it's saying "Spanner is now supported as a Vault storage backend for users who need the features it provides".

FD: I work for Google Cloud

toomuchtodo2• 8 years ago

Can you comment on how Spanner differs from AWS’ DynamoDB Global Tables?

klodolph• 8 years ago

To me, spanner’s main draw is automatic sharding and consistency across replicas. If you’re spending less than $700/mo on your database you probably don’t need those features. Why pay for automatic sharding if your workload fits on one machine? Why pay for global consistency if, again, you can fit everything on one machine (and therefore local consistency = global consistency)?

puzzle2• 8 years ago

It's not outrageously expensive if you have hit the limits of of Postgres or MySQL or need multi-region replication, but yes, for small amounts of data there's too much overhead from all the redundancy.

Using it for Vault makes more sense if you are already storing other, larger amounts of data. Then the incremental cost is much smaller and more than worth it.

hjnilsson• 8 years ago

Yes, the pricing for Spanner is fair for what it offers in production (and test). But I really think there should be a cheaper way to use it for dev purposes (and for trying it out). Either running an emulator locally, or on a micro instance equivalent in the cloud.

qaq7• 8 years ago

A Spanner setup that will outperform PostgreSQL on a hefty x86 box will run you close to 150-200K a month.

sam0x17• 8 years ago

Totally. I'm hoping eventually they add a free or cheap tier of some sort for testing. Otherwise they are locking out a huge number of organizations simply because it is impractical to use something so expensive in development mode.

seanieb3• 8 years ago

Some stellar marketing. GC should be offering Vault as a service. No one should be running a critical service like this unless you have a team large enough that can support it. Frustratingly AWS KMS + Credstash is as close to a managed vault that's available right now.

sethvargo4• 8 years ago

Thanks for the feedback. I'll be sure that both our marketing team and product management team receive it :)

FD: I work for Google Cloud

krallja• 8 years ago

Have you tried https://azure.microsoft.com/en-us/services/key-vault/? I haven’t played with it much yet, but it seems to tick the boxes.

seanieb3• 8 years ago

Thanks! I'll check it out.

willejs2• 8 years ago

AWS SSM pretty decent as a middle ground.

seanieb3• 8 years ago

Oh wow. I had no idea about uts credential store functions. Thanks!

djhworld• 8 years ago

AWS has a solution that awkwardly tucked under the "SSM" product, it's called the Parameter Store.

https://aws.amazon.com/systems-manager/features/#Parameter_S...

I really don't know why they don't spin it out into a separate product, I honesly had no idea it existed until my colleague just happened to stumble across it

paulfurtado• 8 years ago

Does this remove the limitation that vault reads may only go through the master on the non-enterprise vault version? The global replication is much less useful if reads must go through a single server in a single region.

sethvargo4• 8 years ago

As of Vault 0.9.4, this is still the case. The leader services all read and write responses. The Spanner backend does enable Vault to run in HA mode, but only a single server will respond to requests.

If, in the future, Vault allows for multi-leader or at least read-only followers to respond, the Spanner backend would support that. Unfortunately this is a limitation in Vault's current architecture, and not something a new storage backend can fix.

Announcing Google Cloud Spanner as a Vault Storage Back End

Related Stories