b-man

I think your post misses the point of the DBMS centralization: managed consistency.

It is not about ops cost in infrastructure, but ops cost in debugging consistency errors.

  Location: Brazil (UTC-3, overlaps US hours)
  Remote: Yes
  Willing to Relocate: Yes (US preferred)
  Technologies: SQL Server, DBT, SQL Mesh, PostgreSQL, SQL, OLTP & OLAP, cloud optimization, query optimization, large-table migrations, F#, Clojure, Python, C#, C, Java, Ruby, Rust, Prolog, OCaml, Haskell
  Email: ebellani at gmail

My work focuses on high-risk database migrations, dangerous schema remediation, hot-path optimization, and making existing systems scale without rewriting the product.

Open to consulting or full-time roles where data is central and performance matters.

  Location: Brazil (UTC-3, overlaps US hours)
  Remote: Yes
  Willing to Relocate: Yes (US preferred)
  Technologies: SQL Server, DBT, SQL Mesh, PostgreSQL, SQL, OLTP & OLAP, cloud optimization, query optimization, large-table migrations, F#, Clojure, Python, C#, C, Java, Ruby, Rust, Prolog, OCaml, Haskell
  Email: ebellani at gmail

Recent work: re-architected two multi-terabyte OLTP tables (~2TB and ~1TB) handling 200+ writes/sec, improving scalability and reducing application-level complexity.

My work focuses on high-risk database migrations, dangerous schema remediation, hot-path optimization, and making existing systems scale without rewriting the product.

Open to consulting or full-time roles where data is central and performance matters.

https://fred.stlouisfed.org/graph/?g=1WPjJ

have you tried just using emacs? They have an emacs mode https://github.com/coalton-lang/coalton-labs

Great refs. I think one can find more accessible sources for this knowledge, for instance this course:

https://www.youtube.com/playlist?list=PLoYRQl2t0w0EjRIb9Jr1y...

or Feser's articles such as

- https://www.firstthings.com/article/2013/04/kurzweils-phanta...

- https://edwardfeser.blogspot.com/2019/03/artificial-intellig...

The Agile example makes this worse, not better. Yes, Agile was overhyped and badly implemented in many places. But using that to indict the entire movement as Girardian ritual is precisely the logical move the author claims to be critiquing: take some real failures, blame them on a paradigm rather than specific implementations, declare the whole thing rotten. He scapegoats Agile to validate his theory about scapegoating

I don't think the author did that at all. He was fair to interactive development. He specifically points out the scapegoating of waterfall, where the methodology was misrepresented in order to create the space for agile.

fwiw, your career page seems broken (https://supabase.com/careers)

   Location: EST
   Remote: Yes 
   Willing to relocate: Yes (US preferred)
   Technologies: PostgreSQL (partitioning, performance, OLTP architecture), SQL, F#, C#, C, Java, Clojure, Common Lisp, Scheme, Emacs Lisp, Python, Ruby, AWS, Linux
   Email: ebellani at gmail

Recent work:

Re-architected two multi-terabyte OLTP tables (~2TB and ~1TB) receiving 200+ writes/sec. I focus on “rescue architecture” work: fixing dangerous schemas, stabilizing hot paths, removing app-level complexity, and making Postgres scale without rewriting the product.

Open to consulting or full-time roles where data is core to the business and performance/architecture matters.

Résumé: https://www.linkedin.com/in/eduardo-bellani/ https://ebellani.github.io/

ocation: EST Remote: Yes Willing to relocate: Yes (US preferred) Technologies: PostgreSQL (partitioning, performance, OLTP architecture), SQL, F#, C#, C, Java, Clojure, Common Lisp, Scheme, Emacs Lisp, Python, Ruby, AWS, Linux

Email: ebellani at gmail

I work on high-throughput PostgreSQL systems, especially when they’ve grown into a state where migrations, performance, or schema design have become limiting factors.

Recent work:

Re-architected two multi-terabyte OLTP tables (~2TB and ~1TB) receiving 200+ writes/sec. I focus on “rescue architecture” work: fixing dangerous schemas, stabilizing hot paths, removing app-level complexity, and making Postgres scale without rewriting the product.

Open to consulting or full-time roles where Postgres is core to the business and performance/architecture matters.

Résumé: https://www.linkedin.com/in/eduardo-bellani/ https://ebellani.github.io/

Location: EST Remote: Yes Willing to relocate: Yes (US preferred)

Technologies: PostgreSQL (partitioning, performance, OLTP architecture), SQL, F#, C#, C, Java, Clojure, Common Lisp, Scheme, Emacs Lisp, Python, Ruby, AWS, Linux

Email: ebellani at gmail

I work on high-throughput PostgreSQL systems, especially when they’ve grown into a state where migrations, performance, or schema design have become limiting factors.

Recent work:

Re-architected two multi-terabyte OLTP tables (~2TB and ~1TB) receiving 200+ writes/sec. I focus on “rescue architecture” work: fixing dangerous schemas, stabilizing hot paths, removing app-level complexity, and making Postgres scale without rewriting the product.

Open to consulting or full-time roles where Postgres is core to the business and performance/architecture matters.

Résumé: https://www.linkedin.com/in/eduardo-bellani/ https://ebellani.github.io/

This is the reason for the push-back against it.

Do you have evidence for that? From memory, it was basically because it was associated with the java/.net bloat from the early 2000s. Then ruby on rails came.

Location: EST Remote: Yes

Willing to relocate: Yes (US preferred)

Technologies: PostgreSQL (partitioning, performance, OLTP architecture), SQL, F#, C#, C, Java, Clojure, Common Lisp, Scheme, Emacs Lisp, Python, Ruby, AWS, Linux

Email: ebellani at gmail

I work on high-throughput PostgreSQL systems, especially when they’ve grown into a state where migrations, performance, or schema design have become limiting factors.

Recent work:

Re-architected two multi-terabyte OLTP tables (~2TB and ~1TB) receiving 200+ writes/sec. I focus on “rescue architecture” work: fixing dangerous schemas, stabilizing hot paths, removing app-level complexity, and making Postgres scale without rewriting the product.

Open to consulting or full-time roles where Postgres is core to the business and performance/architecture matters.

Résumé: https://www.linkedin.com/in/eduardo-bellani/ https://ebellani.github.io/

I have written the entire backend of a fintech using nothing but postgresql, integration over http and webhook receival included (the last bit was with postgrest, but you get the point)

Location: EST

Remote: Yes

Willing to relocate: Yes (US preferred)

Technologies: PostgreSQL (partitioning, performance, OLTP architecture), SQL, F#, C#, C, Java, Clojure, Common Lisp, Scheme, Emacs Lisp, Python, Ruby, AWS, Linux

Email: ebellani at gmail

I work on high-throughput PostgreSQL systems, especially when they’ve grown into a state where migrations, performance, or schema design have become limiting factors.

Recent work:

Re-architected two multi-terabyte OLTP tables (~2TB and ~1TB) receiving 200+ writes/sec. I focus on “rescue architecture” work: fixing dangerous schemas, stabilizing hot paths, removing app-level complexity, and making Postgres scale without rewriting the product.

Open to consulting or full-time roles where Postgres is core to the business and performance/architecture matters.

Résumé: https://www.linkedin.com/in/eduardo-bellani/ https://ebellani.github.io/

Not to mention that perfectly normalizing a database always incurs join overhead that limits horizontal scalability. In fact, denormalization is required to achieve scale (with a trade-off).

This is just not true, at least not in general. Inserting on a normalized design is usually faster, due to smaller index sizes, fewer indexes and fitting more rows per page.

fixed

Indeed. I was trying to make that point on my concluding paragraph.

That may be. What's not specified there is the immense, immense cost of driving a dev org on those terms

I'm happy that we agree on the solution, but disagree only if it is cost worthy. About the cost, I took that into consideration when I wrote the conclusion:

FAANG-style companies are unlikely to adopt formal methods or relational rigor wholesale. But for their most critical systems, they should. It’s the only way to make failures like this impossible by design, rather than just less likely.

There is an actionable plan in the article. It is possible to run teams like these. It is an economical decision of upper management to run the risk of having these outages vis-a-vis this alternative.

a fully normalized relation is one where the SQL (say) table in question represents one and only one predicate of your business rules.

It is literally impossible for that to be done automatically. Someone needs to look at the resulting code and confirm that that was the case.

Normalization cannot be done by machines, because it depends on expressing the (and only the) predicate that corresponds to the business rule in question.

It requires apprehending the essence of the situation, something a machine cannot do.

btw, I do reference an article that is rich in experiences of formal verification: https://dl.acm.org/doi/10.1145/3624728

Is it more expensive? The answer depends on the criticality of the component.

In this case in particular, having a 'not null' directive on the table at hand would have suffice. And that is something everyone can do.

what you need is to add temporality to your tables. Then your logs/dependencies will work just fine

Thanks, I didn't knew that. Can you link to one such text?

Lots of peole got hung up on the example, which I thought would be be helpful on the discussion, but certainly should not replace the main point, which is:

Relations, attributes and tuples are logical. A PK is a combination of one or more attributes representing a name that uniquely identifies tuples and, thus, is logical too[2], while performance is determined exclusively at the physical level, by implementation.

So generating SKs for performance reasons (see, for example, Natural versus Surrogate Keys: Performance and Usability, Performance of Surrogate Key vs Composite Keys) is logical-physical confusion (LPC)[3]. Performance can be considered in PK choice only when there is no logical reason for choosing one key over another.

https://www.dbdebunk.com/2018/04/a-new-understanding-of-keys...

Depends on the universe of discourse adopted.

I still wouldn't use that as a primary key on the citizen table, though.

Why not?

Trying to tamp out these ambiguities adds an unbounded number of data model epicycles that add a lot of complexity and performance loss

If you can talk about a business rule, you have a predicate. If you have a predicate, you can make it 5 or 6 normal form, since all that means is that your relation expresses only and completely the predicate.

It seems that your definition of normalization is not the one that I am using above. What is it?

Memory, and CPU, and even storage eventually, those would be the main practical examples of where having a key that's composed of something very small saves you space and thus, time.

Say we want to use a bigint key vs a VARCHAR(30)? depending on your big key you might be talking about terabytes of additional data, just to store a key (1t rows @ bigint = 8TB, 1T rows at 30 chars? 30TB...). The data also is going to constantly shuffle (random inserts).

> Joins, lookups, indexes

I don't see how what you brought up has anything to do with these.

But the main point is being missed here because of a physical vs logical conflation anyhow.