How YouTube scaled MySQL in 2007
Lessons from Paul Tuckfield’s keynote on keeping YouTube’s databases ahead of its growth.
From the archive: in 2012 I shared “Extreme makeover: database, or MySQL@YouTube (MySQL Conference 2007 notes)” by Colin Charles, on Paul Tuckfield’s keynote on my old blog. This page is a short summary in my own words; the original belongs to its author.
At the 2007 MySQL Conference, Paul Tuckfield, a long-time sysadmin and DBA at YouTube, told the story of keeping the site’s databases ahead of its explosive growth. The lessons that stuck with me:
- The fastest query is the one you don’t run. Caching (memcached in front of MySQL) did more than any database tuning.
- Reads far outnumber writes, so replication, with many read copies of one primary, carried them a long way.
- Replication has limits: lag between primary and replicas, and every replica still has to apply every write. Eventually the data has to be split (sharding).
- Know the layers: RAID controllers, the operating system’s cache and the database’s own cache can fight each other. Understanding where data actually sits in memory matters.
I ran large real-time systems myself around the same time, and the same truth held: plan for the busiest night, keep the setup simple enough to fix under pressure, and don’t ask the database for anything you can avoid.
Read the original: Extreme makeover: database, or MySQL@YouTube (MySQL Conference 2007 notes), by Colin Charles, on Paul Tuckfield’s keynote.