clarkbw

Neon has pg18 ready to use on the day it's released, just like last year.

Sadly one of the most interesting features, Async I/O, is going to take some work at the file system level for us to support. For now all operations are still sync. And we'll keep updating on our progress there.

I think this was actually trying to say, Neon prices were high. Because otherwise I agree it doesn't make sense.

And Neon will be lowering prices dramatically... a day from now?

cold starts are 500ms on average, and that's only for the first call that wakes up the db from hibernation. people still seem to think that this latency happens for every call (see other threads here) but once the service has woken up (cold start over) you're back to regular (sub 10ms) latency timings and the service continues to run that way. you'll only hit a cold start again if (you have this option turned on) your service goes idle for > 5 min. You can turn scale-to-zero off and you'll run 24/7, have zero cold starts.

$19 plan is going away, will launch a better $5 plan soon.

It is!

And we plan to have 18 out as quickly as we did 17; on the day of release.

is it only the auth that's a problem? you could try stack auth or other oss auth providers out there.

(neon employee)

there isn't something like Vitess for Postgres yet but there needs to be. Migrations are painful in general and they become very painful at scale. i haven't used gel yet, i know it manages migrations but i don't know to what extent. most of my experience is with prisma, drizzle, and atlas.

neon is working on some plans to solve migrations at scale. we think our custom storage layer will allow us to optimize certain paths, like setting a default value in a new column for a table with millions of rows. this alter command can take a lock on a table for hours. but ultimately we need better tooling.

ideally there is a client and a hosted service such that you can use the client to run migrations on your own from the CLI and integrate it into your dev workflow. the hosted service version allows you to push up your schema change from the client to an API. from there you can manage the migration rollout from an operational dashboard that helps you tune resourcing.

when i was at github we used vitess to roll out a migration that took 3 weeks to complete. a long time to wait but that's a better tradeoff compared to a migration that takes down production for 6 hours.

(neon employee)

we have multi-region disaster recovery and replicas coming in 2025. Switch over time will be greater than this active-active system but the overall latency for Neon should be much less on a consistent basis.

(neon employee) and Neon will be available on Azure very soon ( https://neon.tech/blog/neon-is-coming-to-azure )

oh, i see I'm getting downvoted; this isn't a sales pitch for my "managed postgres platform".

every year a new postgres release occurs and i want to try out some of the features i have to find a way to get it. usually nobody has it available.

here's the list of homebrew options I see right now:

brew formulae | grep postgresql@ postgresql@10 postgresql@11 postgresql@12 postgresql@13 postgresql@14 postgresql@15 postgresql@16

maybe you're seeing otherwise but i updated a min ago and 17 isn't there yet. even searching for 'postgres' on homebrew doesn't reveal any options for 17. i don't know where you've found those but it doesn't seem easily available.

and i'm not suggesting you use a cloud service as an alternative to homebrew or local development. neon is pure postgres, the local service is the same as whats in the cloud. but right now there isn't an easy local version and i wanted everyone else to be able to try it quickly.

It's never available on homebrew the same day so we all worked hard to make it available the same day on Neon. If you want to try out the JSON_TABLE and MERGE RETURNING features you can spin up a free instance quickly.

https://neon.tech/blog/postgres-17

(note that not all extensions are available yet, that takes some time still)

Some awesome quality-of-life improvements here as well. The random function now takes min, max parameters

SELECT random(1, 10) AS random_number;

Agreed! A common ORM pitfall is column rename which often doesn't get implemented as a rename as much as it does a DROP and ADD which will affect the data in a surprising way :-D

My tests running ALTER varied from ~20 seconds to ~1 min for the changes.

Current CI/CD practices often make it very easy for software developers to commit and roll out database migrations to a production environment, only to find themselves in the middle of a production incident minutes later. While a staging deployment might help, it's not guaranteed to share the same characteristics as production (either due to the level of load or monetary constraints).

(neon.tech employee here)

This is where branching databases with production data helps quite a bit. Your CI/CD environment and even staging can experience the schema changes. When you build from a seed database you can often miss this kind of issue because it lacks the characteristics of your production environment.

But the author rightly calls out how staging isn't even enough in the next paragraph:

The problem is, therefore (and I will repeat myself), the scale of the amount of data being modified, overall congestion of the system, I/O capacity, and the target table's importance in the application design.

Your staging, even when branched from production, won't have the same load patterns as your production database. And that load and locks associated will result in a different rollout.

This has me thinking if you can match the production environment patterns in staging by setting staging up to mirror the query patterns of production. Mirroring like what's available from pg_cat could put your staging under similar pressure.

And then this also made me think about how we're not capturing the timing of these schema changes. Unless a developer looks and sees that their schema change took 56 seconds to complete in their CI system you won't know that this change might have larger knock on effects in production.

42

Yes, we have great APIs ( https://neon.tech/docs/reference/api-reference ) and a Terraform provider ( https://registry.terraform.io/providers/terraform-community-... )

(neon product)

hey all, i've only been with neon for a very short time but i'm super excited to be here because i believe neon can deliver an amazing developer experience that databases have been lacking. while we are simply postgres on top the neon platform is something special that unlocks a lot of new possibilities.

this ga is simply marking readiness for the platform and the team that has been building it. there's a lot more to do going forward.

as we're looking forward, post-GA, i'd love to hear what you think neon needs to focus on next. here's what i'm seeing so far: - improved gh actions integration - more extensions - better developer extension support - autoscaling communications - metrics / logs integrations

what else?

i'd love to learn more about what you're doing! if you haven't already, please send a message. definitely want to hear lessons learned with your k8s operator.

Congrats to the Supabase team! You've all done some impressive work to be proud of.

TY, fixed. Neon does have a public Discord https://discord.gg/92vNTzKDGp

Thank you! Can you tell me more about the stack you're using? [pure curiosity]

(neon product)

woof, that does not sound good. Can you say more about how you were connecting? What was slow / gitchy and laggy? Feel free to reach to over DM to me with your project details.

(neon empl)

Product Manager here.

This is about customer obsession, relentlessly following the needs of our audience. I know the engineers are always complaining that without vegetables we'll create significant tech debt but there's no time for that right now. Maybe next quarter. The customer wants desert and the customer should get it.

While I admit this current strategy does not directly lead to a health business, we're aiming for scale right now and scale up is what we're gonna do. Did I mention that "stick things in the outlets" is coming next quarter?

Yes, we use EBS SSD on the backend so we can scale up storage separately from the instance. Our Cloud performance metrics are based on this backend so the short answer is no it doesn't constrain perf. The constraint I see right now is that we are currently mostly GP2 with a planned migration to GP3 which will allow for new independent controls of IOPS and throughput. There are certain, uncommon, situations where customers need to bump up performance beyond what the normal GP2 perf steps allow.

To tie GP2/3 back into the serverless vs. DBaaS concepts we are looking at auto-scale for IOPS/Throughput performance while also allowing more direct access such that a customer could control performance via APIs to manage on your own.

(timescaler here)

For the DBaaS services the problem is more of simplicity and developer experience; they are more of a transparent box which can make things more complicated than necessary. You should be able to have a clear paved path for an excellent general database experience with automatic backups, scale, and functionality to improve developer ergonomics. While at the same time, under the hood having this DBaaS like experience which allows you to take more control as needed. Of course it is not easy to offer both simplicity and flexibility, that's a fine line to draw but that is our goal.

As an ex-pat what bothers me most is the lack of representation, I'd keep paying taxes if we actually had some virtual state. Citizens abroad are essentially gerrymandered by their last state of residence meaning there is no useful representation for those abroad. However with an estimated 9 million+ citizens living abroad ex-pats all together would represent the 10th or 11th most populace state. 9 million US citizens is similar to the population of New Jersey which has 2 senators and 12 representatives working for them. Where's my Cory Booker?

Yes, most people in this situation liquidate them. Dual US / Canadian citizen here living in Canada.

While you can keep a TFSA around you'll end up paying taxes on the gains back to the states as well you need to file additional paperwork come tax time. If your TFSA gains exceed the filing requirements it might still be worth keeping. Historically you needed to file a TFSA as a foreign trust however recently the IRS has deemed them a foreign disregarded entity which reduces the filing requirements thus making it cheaper.

In your situation you could open up a US based fund with that money and Canada won't care so that's likely your best option.

... make it possible to select a pg database based on async or sync replication

Can you explain why you doing this? Is this application data vs. time-series?

Keeping them updated as the code changes is difficult. I’m finding as docs are separate repos disconnected from the source. Not sure how to easily keep it all in sync

Data transfer between GHCR and Actions is free. We're building out a much tighter integration between your source code and the artifacts created. The effects of this will create a stronger supply chain link and work toward better reproducibility.

We recommend using GHCR for the development and test workflows and then publishing to the "Cloud *CRs" for your production images such that they can be pulled directly from there.

Bandwidth between all GitHub Packages services (GHCR included) is free.