On #3 (and #4) its all about moderation and making informed decisions based on specific load. My initial wording was pretty slanted and typed in the context and bias of some work I was dealing with in our DB at the time. I hedged it a long while back. De-normalization can get you massive performance wins in a very large database by preventing joins or reducing query counts on SELECTs. The maintenance side of de-normalization can indeed be painful so you have to carefully consider each usage. Sometimes it doesn’t matter (very static data), sometimes you can have the DB handle it automatically with triggers or similar, and occasionally handling it in code isn’t too bad.

I finally did adjust the wording on #4 just now to prevent further confusion. My point is joins don’t work when your tables are HUGE. We run OLTP loads (big social networks specifically) with hundreds of thousands of users and at peak times thousands of transactions a second on a single decent Dell server. We have lots of tables and partitions that are in the 30million+ row range. Databases handle joins in memory and unless you have the RAM to set work_mem (or non-Postgres equivalent) very high they get done on disk for joining of very large tables. Once that happens they run at crippling speed. Joins are wicked fast when they fit in memory and as a general rule of thumb are the way to go (the PostgreSQL planner will even convert queries to joins under the hood when it makes sense) unless you are working with massive tables and result sets.

So I think both points are valid, but my initial language was far too broad. They make sense when you have a lot of data and are still using an RDBMS to handle it. Partitioning can help delay the usage of those techniques by keeping the tables sizes smaller but I find it hard that you could avoid both in a very large system.

Thanks again for commenting and perhaps point those fellas in that sqlservercentral forum at this response so they have some context before assuming I don’t know what I am talking about

I’m no PostgreSQL guy, what means I don’t know about the truth of all those tips for PostgreSQL. Nevertheless, I know other database systems and most tips sound good and logical to me.

Anyway, are #3 and #4 are really true for PostgreSQL? If yes, for which kind of database? De-normalization is a technique to design selective reporting and warehousing databases, yes. De-normalization is a no-go for an OLTP database.

These “tips” are fine for a warehouse database which is filled by a small count of ETL jobs or very simple systems with only one front end application. My experiences showed me, data redundancy in a large operational database always(!) ends up in data inconsistency. To keep a de-normalized database in sync within a enterprise system is way too complex and will (not might) fail.

Again, all other tips sound fine to me and are even generally correct to all database systems.

Greets
Flo

The conceptual understanding that needs to be applied is the concept of a primary data record and all copies as being permutations of that. As long as you can at any time perform the duplication logic from primary data record to produce copies and get what you expect, then you are in safe heaven. You never update the copies except from the duplication logic.

I am willing to bet in 99% of the cases throwing down as many indexes as you need is the right way to go. I am not saying to just create indexes for no reason – just create one for every common SELECT. If you have a use case or project that falls in that 1% of super heavy writes would love to hear about it (honestly, love hearing about unique setups and problems).

As an example our OLTP database handles a lot of writes (millions a day) but in the overall picture writes are less than 5% of our queries so it simply doesn’t make sense to punish the other 95%. If we have to do a lot of writing for a big migration or to bring a new customer’s lists online we drop the indexes on the relevant partitions and then recreate them when done. Single record inserts caused by your application often aren’t going to feel any slower because the code-level stuff is going to hog most of the perceived time anyway.

Most of the time it doesn’t matter though. Throw down as many indexes as you need.”

I’ve seen plenty of times when having too many indexes mattered quite significantly. An index is effectively another copy of the data, and must be updated whenever the underlying data are updated. If having more indexes didn’t actually matter, PostgreSQL features like Heap-Only Tuples wouldn’t have any significant performance impact, which clearly isn’t the case.