gbartolini

See for example: https://infohub.delltechnologies.com/en-us/t/edb-postgres-ai...

It all depends on the storage underneath. If you have got good storage, with CNPG you have comparable results you get on bare metal PostgreSQL.

Barman Cloud was a convenient choice for CloudNativePG (CNPG) because it was developed by the same team that created Barman originally (I am part of both teams). When we started CNPG, we never anticipated it would become so popular, which has obviously resulted in some technical debt. The issue you mentioned concerns our decision not to integrate pgBackRest into CNPG's core, as we aimed to develop a pluggable interface (CNPG-I).

As a community, we have decided to support volume snapshot backups and offer the Barman Cloud plugin to ensure we provide the same level of service. Our aim is to encourage other organisations or developers to create plugins for their preferred backup solutions.

Currently, as maintainers of CNPG, we must concentrate on the core capabilities and allow the ecosystem to grow with both community and, potentially, commercial solutions based on CNPG-I.

As a founder and maintainer of CNPG, it is exciting to see it used here. I would love to know more! Thanks.

I don’t agree here. There are operators like the one I’m a maintainer of (CloudnativePG) which works directly with Kubernetes, teaching it how to handle Postgres clusters as a coordinated set of instances. Enormous improvements have been done in the last couple of years, and we are particularly focused in working together with storage groups in Kubernetes to handle database workloads, such as for example declarative support for tablespaces and volume snapshots.

I suggest you to read CloudnativePG documentation as well as this article: https://www.cncf.io/blog/2023/09/29/recommended-architecture...

Also watch the video of my talk at last Kubecon in Chicago about handling very large databases.

I hope this helps.

I actually do not understand the point here. And maybe you are not very familiar with the concept of transactions. Backups can only account for committed transactions.

However, we are talking about Postgres, here, not a generic database. PostgreSQL natively provides continuous backup, streaming replication, including synchronous (controlled at transaction level), cascading, and logical. You can easily implement with Postgres, even in Kubernetes with CloudNativePG, architectures with RPO=0 (yes, zero data loss) and low RTO in the same Kubernetes cluster (normally a region), and RPO <= 5 minutes with low RTO across regions. Out of the box, with CloudNativePG, through replica clusters.

We are also now launching native declarative support for Kubernetes Volume Snapshot API in CloudNativePG with the possibility to use incremental/differential backup and recovery to reduce RTO in case of very large databases recovery (like ... dozens of seconds to restore 500GB databases).

So maybe it is time to reconsider some assumptions.

I'm the author of this blog article in which I try to recap why running Postgres in Kubernetes is a good thing: https://cloudnativenow.com/kubecon-cnc-eu-2022/why-run-postg...

The reason could be the last 4 years of evolution in Kubernetes. Have you heard of DoK Community (Data on Kubernetes)? Might be a good place where to start.

Why not dedicate some worker nodes using taints/tolerations/labels, even on bare metal, with locally attached storage? I wrote this many years ago now but that's the reason why we started CloudNativePG (OpenEBS might not be the answer today, but there are many storage engines now, including topolvm which brings LVM to the game): https://www.2ndquadrant.com/en/blog/local-persistent-volumes...

It is ultimately your choice. I am a big fan of shared nothing architecture for the database. (I am a maintainer of CloudNativePG)

If you are interested, I suggest this article I wrote that covers the current state of major upgrades with CloudNativePG: https://www.enterprisedb.com/blog/current-state-major-postgr...

I agree with your view point and concerns.

That is why we took the approach to reduce the number of components and integrate everything in Kubernetes, especially with logging (we directly log in JSON to standard output) and the usage of application containers, which enables us to cover the case of troubleshooting via the fencing mechanism (your pods are up, you can access storage, but Postgres is down, giving you the possibility to check even possible data corruption issues).

Also, the status is directly available in Kubernetes, so in our view easier for Kubernetes administrators.

Finally, the source code is open source and directly available for inspection - if you want to understand what is happening.

Maintainer of CloudNativePG here.

Back then, we evaluated Crunchy Operator's source code. Being primarily imperative and using an external tool for failover, where the two main reasons we decided to start a new project in 2019 which was entirely declarative and purely based on the Kubernetes API server for cluster status. Such project was released open source last April under the name CloudNativePG and hopefully it will enter the CNCF Sandbox soon (fingers crossed).

Thank you!

Can you please elaborate about obfuscation?

Regarding being opinionated I believe that it is what we expect from an operator. An operator simulates what human DBAs in this case would do. I am a maintainer of CloudNativePG, and I have been running and supporting PostgreSQL in production for 15+ years, creating also another open source software for backups (Barman). In CloudNativePG we have basically translated our recipes into Go code and tests.

Many people believe that databases should not run in Kubernetes. I not only believe the opposite, I believe that running Postgres in Kubernetes represents the best way, potentially, to run Postgres out there.

As maintainer of the project, I suggest looking at CloudNativePG, which is production ready (cloudnative-pg.io).

One clarification about CloudNativePG. It is not EDB's anymore (or EnterpriseDB if you prefer).

EDB is the original creator. The software is now entirely owned by a vendor neutral community, openly governed. We have applied for the CNCF sandbox and waiting for the approval at this stage.

CloudNativePG Maintainer and VP of CloudNative at EDB here.

We decided to go even further with the CloudNativePG operator.

EDB as original creator has decided to donate the intellectual property of the source code to the community, open sourced the existing operator under Apache License 2.0 to apply for the CNCF Sandbox. The project not only includes the operator, but also the PostgreSQL operand images - which can be customized (we provide details on how images should be).

We genuinely welcome other vendors to participate in the community and contribute to the project, including by offering professional services around it. Our multi-year commitment is to become a graduated CNCF project.

For more information: https://cloudnative-pg.io/

That's another good example of why CNCF is important, as in that ecosystem live technologies that are becoming standard for everything related to infrastructure, including monitoring and alerting. Prometheus is an example, Open Telemetry is another one. When you are able to "interface" with one of these components, you've done your integration job.

From a Postgres standpoint, IMO, it is also important the concept of cluster - as that's what applications connect too - the operator hides the underlying complexity of managing the single instances. And .. also important monitoring sets of clusters from an infrastructure management PoV.

Got it. That's an interesting point of view. I think it makes sense in the data workloads, especially in the database one. Consider for example how database workloads are still lagging behind in Kubernetes, and - most importantly - viceversa (Kubernetes is a hard topic IMO for traditional DBAs).

Operators are there to help in both cases, and this is the goal of our CloudNativePG operator.

IMO CNCF represents an opportunity for vendors of the same database engine to converge in a healthy community and provide the best operator for their platform. This is probably the first stage.

The second stage could be to study those operators (of different engines, like Postgres, MySQL, ...) and generalize commonalities in a sort of standard spec. Just an idea. Is this in line with what you are thinking?

Thanks!

Can you please elaborate on this: "Could this reasoning be extended to other databases and stateful applications?"?

There's definitely commonalities between operators for databases, not only with the same engine, but also different ones and similar primary/standby architecture.

I agree this is something we could somehow work together as a standard kind of spec, but probably we are still at an early stage of the process. My 2 cents.

The project has just been submitted to the CNCF Sandbox, it hasn't yet been accepted. There's a lot of road to be done. If we get accepted, we'll be honoured to adapt and contribute to the project until incubation.

However, what is important here though is that this is the first operator for Postgres whose IP is now owned by a vendor neutral and openly governed community - or at least aims to be based on the public GOVERNANCE policies. We look upon the CNCF for principles and values.

This is our commitment.

Disclaimer - I'm one of the maintainers of the project

Thanks for sharing the news!

The project is hosted on github.com/cloudnative-pg/cloudnative-pg and it is distributed under Apache License 2.0.

It manages PostgreSQL primary/standby architecture clusters, providing self-healing, high availability through automated failover, continuous backup, point-in-time recovery, native Prometheus exporter, JSON standard output logging, rolling updates, and so on.

It relies on the Kubernetes API server to keep the status of the cluster, detect issues/changes, and react.

It is production ready, originally created by EDB, which is one of the major contributors to the open source PostgreSQL project.

Some food for thought. These are the first cases that came to my mind:

1) Zero data loss backups (as far as I know, Barman is the first backup tool for PostgreSQL to implement it)

2) Monitoring ('check' command)

3) Hybrid WAL shipping (archiving + streaming)

4) Incremental backups

5) Faster recovery time (backups are stored as plain files, you can have incremental recovery and you can use 'get-wal') - also we are working on parallel backup and recovery

6) Backup from a standby

In terms of management, Barman is very low maintenance, but the outcomes you get are relevant.

If you have multiple PostgreSQL servers, you can concentrate backups in a single Barman host. Every server can have its own settings allowing you to implement hybrid solutions (you can mix versions, backup methods such as rsync or pg_basebackup, WAL archiving, WAL streaming, and so on).

As said above by "amenonsen", with hook scripts you can also relay WAL files to S3 very easily, and create a backup of your backup.

That is why I think having a separate backup server has more pros than cons.

I hope this helps.

You are right. I confirm, currently it is only WAL compression (documentation is up to date). I hope that we can add one between incremental backups and retention policies in version 1.1. Which are part of the most wanted features still. :)

By then, we'd love to have a Debian package too.

Thank you.

As core team member of Barman (and manager), I want to start answering from the bottom of your request. I actually thank you for giving me the chance to answer to this question and to talk about Barman in general.

2ndQuadrant has adopted the open-source business model, and we have a long history of contributions to the PostgreSQL project and its evolution (I encourage you to verify my words). This applies to satellite projects too, including repmgr (High Availability) and Barman (Disaster Recovery). With Barman we have a precise statement: it has to be open-source and to remain open-source, under GNU GPL 3. Being a business, we need to fund the development of features through our consulting services in Disaster Recovery or through feature sponsorships. But our intention is that everything will be released open-source (otherwise, why bother releasing this software in the first place?).

On a more technical point of view, one of the main characteristics of Barman is that it does not require to be physically located where the Postgres server resides. It performs remote backup (if you can, you can do local backups too, but I prefer sharing nothing between a backup server and a production server - points of view?). If you think about it carefully, this also gives more flexibility in terms of architecture, especially in large master-slave scenarios (you do not need to install Barman on every Postgres server or share the same file system, for instance).

Barman can compress transactions/WAL files (bzip2, gzip or custom) and has a parallel management of the WAL archive and periodical backups (I suggest you read this article where we describe the association between a WAL segment and a backup http://blog.2ndquadrant.com/management-wal-archive-barman/).

Currently it does not automatically compress periodical backups, but gives a clear and simple interface to archiving (to give you an idea, we have a customer that regularly - every week - transfers on tapes a 6TB database with Barman). Current development version has already hooks for pre and post backup operations. Next boundaries will be incremental backup and retention policies (what you probably call backup maintenance) - as I said, depending on fundings we receive.

Another important aspect of Barman is that you can have a backup server managing multiple PostgreSQL servers, from one single place, and having a catalogue of periodical backups.

Then ... I'd say remote recovery, data directory and tablespace relocation during recovery, Point in Time recovery, diagnostics, etc.

If you have more information, you can join the IRC channel too or the mailing list (details on www.pgbarman.org).