Web scale backups for MySQL, Алексей Копытов (Percona)
2 likes1,312 views
Доклад Алексея Копытова на HighLoad++ 2014.
Web scale backups for MySQL, Алексей Копытов (Percona)
- 1. Webscale backups for MySQL Alexey.Kopytov@percona.com October 29, 2014
- 3. Êòî ÿ? Àëåêñåé Êîïûòîâ <alexey.kopytov@percona.com> I Percona Server I Percona XtraBackup (ðóêîâîäèòåëü ïðîåêòà) I Percona XtraDB Cluster
- 4. Òèïû ðåçåðâíûõ êîïèé Ëîãè÷åñêèå I äàííûå â âèäå SQL, ïîëó÷åííûå îò ñåðâåðà I òðåáóþò èìïîðò äëÿ âîññòàíîâëåíèÿ (ìåäëåííî!) I âîññòàíîâëåíèå ïîâðåæä¼ííûõ äàííûõ I upgrade/downgrade Ôèçè÷åñêèå I êîïèÿ ôàéëîâ äàííûõ ñ äèñêà I êëîíèðîâàíèå I âîññòàíîâëåíèå âñåé áàçû öåëèêîì Îáû÷íî èñïîëüçóþòñÿ îäíîâðåìåííî
- 5. Óòèëèòû äëÿ áýêàïîâ MySQL I mysqldump I mydumper I mylvmbackup I TokuDB Hot Backup I MySQL Enterprise Backup I Percona XtraBackup I íå ñ÷èòàÿ ¾íàäñòðîåê¿
- 6. mysqldump I ëîãè÷åñêèé áýêàï I + âõîäèò â ïàêåòû MySQL/MariaDB/Percona Server I íåò èíêðåìåíòàëüíûõ áýêàïîâ I îäíîïîòî÷íûé I Î×ÅÍÜ ìåäëåííîå âîññòàíîâëåíèå
- 7. mydumper “How MySQL DBA support engineer would imagine mysqldump ” I ëîãè÷åñêèé áýêàï I + ìíîãîïîòî÷íàÿ âåðñèÿ mysqldump I + îäíà òàáëèöà – îäèí ôàéë I íåò èíêðåìåíòàëüíûõ áýêàïîâ I òîëüêî ëîêàëüíûå áýêàïû (íåëüçÿ ñäåëàòü mydumper | ssh remotehost) I ìåäëåííîå âîññòàíîâëåíèå
- 8. mylvmbackup I ôèçè÷åñêèé áýêàï I + ëþáûå òèïû òàáëèö (InnoDB, TokuDB, etc.) I íåò èíêðåìåíòàëüíûõ áýêàïîâ I òîëüêî ëîêàëüíûå áýêàïû
- 9. TokuDB Hot Backup I áèáëèîòåêà, ïåðåõâàòûâàþùàÿ I/O âûçîâû I ôàêòè÷åñêè LVM snapshot â userspace
- 10. TokuDB Hot Backup I ôèçè÷åñêèé áýêàï I + ëþáûå òèïû òàáëèö (òåîðåòè÷åñêè) I íåò èíêðåìåíòàëüíûõ áýêàïîâ I òîëüêî ëîêàëüíûå áýêàïû I – âñÿ çàïèñü â áàçó äóáëèðóåòñÿ â ïðîöåññå áýêàïà
- 11. MySQL Enterprise Backup I ôèçè÷åñêèé áýêàï I ïðîïðèåòàðíîå ÏÎ, äîñòóïíî ïî ïëàòíîé ïîäïèñêå I + ¾ãîðÿ÷èé¿ áýêàï äëÿ InnoDB I + èíêðåìåíòàëüíûå áýêàïû I + áýêàïû íà óäàë¼ííûå õîñò I + áûñòðîå âîññòàíîâëåíèå âñåé áàçû I - áîëüøîé ðàçìåð íà äèñêå I - íå ïîääåðæèâàåò íåêîòîðûå ðàñøèðåíèÿ XtraDB (Percona Server, MariaDB) I - íå ïîääåðæèâàåò TokuDB
- 12. Percona XtraBackup I ôèçè÷åñêèé áýêàï I ñâîáîäíàÿ, áåñïëàòíàÿ àëüòåðíàòèâà MySQL Enterprise Backup I ïðàêòè÷åñêè 100% ñîâìåñòèìîñòü â ôóíêöèîíàëüíîñòè (+/- íþàíñû ðåàëèçàöèè) I ïîääåðæèâàåò MySQL, MariaDB, Percona Server, WebscaleSQL, . . .
- 13. ×òî âûáðàòü? mydumper + Percona XtraBackup ïîêðûâàþò áîëüøèíñòâî ñëó÷àåâ I ñòàäàðò äå-ôàêòî äëÿ Percona Managed Services TokuDB? Òîãäà mylvmbackup ! I èñïîëüçóåòñÿ â Percona Cloud Tools Åñòü ïîäïèñêà íà MySQL Enterprise? I MySQL Enterprise Backup òîæå âàðèàíò! :)
- 15. Áîëüøèå îáú¼ìû äàííûõ: mysqldump I mysqldump --innodb-optimize-keys â ïàêåòàõ Percona Server 5.1/5.5/5.6 I óáèðàåò âòîðè÷íûå èíäåêñû èç InnoDB òàáëèö I ïåðåñòðàèâàåò ïîñëå èìïîðòà äàííûõ “áûñòðûì” ìåòîäîì I çíà÷èòåëüíî óñêîðÿåò èìïîðò I ïîêà íå ðåàëèçîâàíî â MySQL (Bug #49120) I ïîêà íå ðåàëèçîâàíî â mydumper
- 16. Áîëüøèå îáú¼ìû äàííûõ: PXB/MEB ¾Êîìïàêòíûå¿ áýêàïû â Percona XtraBackup innobackupex --compact /backup/dir/ ¾Skip unused pages¿ â MySQL Enterprise Backup mysqlbackup --skip-unused-pages /backupdir Ìåíüøå ðàçìåð íà äèñêå, íî áîëüøå âðåìåíè íà âîññòàíîâëåíèå!
- 17. Áëîêèðîâêè Âñå óòèëèòû îáåùàþò ¾ãîðÿ÷èé¿ áýêàï. Íà ñàìîì äåëå èñïîëüçóþò áëîêèðîâêè äëÿ: I ñîãëàñîâàííîñòè ñ íåòðàíçàêöèîííûìè òàáëèöàìè I ñîãëàñîâàííîñòè .frm ñ äàííûìè I ñîãëàñîâàííîñòè master/slave êîîðäèíàò ñ êîïèåé äàííûõ
- 18. Áëîêèðîâêè: FLUSH TABLES WITH READ LOCK  ÷¼ì ïðîáëåìà íà âûñîêîíàãðóæåííûõ ñåðâåðàõ? 1. Ñáðîñèòü query cache 2. Çàáëîêèðîâàòü âñå íîâûå çàïèñè 3. Äîæäàòüñÿ çàâåðøåíèÿ âñåõ âûïîëíÿþùèõñÿ çàïèñåé 4. Äîæäàòüñÿ çàâåðøåíèÿ âñåõ SELECT 5. Çàáëîêèðîâàòü COMMIT
- 19. Áëîêèðîâêè: Backup locks â Percona Server 5.6 LOCK TABLES FOR BACKUP: I áëîêèðóåò DDL è çàïèñè â íåòðàíçàêöèîííûå òàáëèöû LOCK BINLOG FOR BACKUP: I áëîêèðóåò COMMIT
- 20. Áëîêèðîâêè: FTWRL vs. Backup locks Percona XtraBackup: FLUSH TABLES WITH READ LOCK 1. ñêîïèðîâàòü InnoDB òàáëèöû 2. FLUSH TABLES WITH READ LOCK; 3. ñêîïèðîâàòü .frm, MyISAM, è ò.ä. 4. ïîëó÷èòü master/slave êîîðäèíàòû 5. çàâåðøèòü êîïèðîâàíèå REDO log 6. UNLOCK TABLES; Backup locks 1. ñêîïèðîâàòü InnoDB òàáëèöû 2. LOCK TABLES FOR BACKUP; 3. ñêîïèðîâàòü .frm, MyISAM, è ò.ä. 4. LOCK BINLOG FOR BACKUP; 5. çàâåðøèòü êîïèðîâàíèå REDO log 6. UNLOCK TABLES; 7. ïîëó÷èòü master/slave êîîðäèíàòû 8. UNLOCK BINLOG;
- 21. Áëîêèðîâêè: FTWRL vs. Backup locks mysqldump --single-transaction I íå áëîêèðóåò ñåðâåð I íå ðàáîòàåò, åñëè åñòü MyISAM è/èëè DDL mysqldump --lock-all-tables I áëîêèðóåò ñåðâåð I ðàáîòàåò ñ MyISAM è DDL Percona Server: mysqldump -single-transaction --lock-for-backup: I íå áëîêèðóåò ñåðâåð I ðàáîòàåò ñ MyISAM è DDL
- 22. ×àñòè÷íûå êîïèè
- 24. Èíêðåìåíòàëüíûå áýêàïû Ðåàëèçàöèÿ èíêðåìåíòàëüíûõ áýêàïîâ â Percona XtraBackup è MySQL Enterprise Backup: I ñêîïèðîâàòü âñå ñòðàíèöû, äëÿ êîòîðûõ LSNîáíîâëåíèÿ LSNïîëíîãî áýêàïà Ïðîáëåìà äëÿ áîëüøèõ îáú¼ìîâ äàííûõ: I òðåáóåò ïîëíîãî ñêàíèðîâàíèÿ âñåõ äàííûõ
- 25. Èíêðåìåíòàëüíûå áýêàïû Percona Server: innodb_track_changed_pages=1 I ñîõðàíÿåò bitmap èçìåí¼ííûõ ñòðàíèö â îòäåëüíûõ ôàéëàõ Percona XtraBackup: I àâòîìàòè÷åñêè èñïîëüçóåò bitmap ôàéëû âìåñòî ïîëíîãî ñêàíèðîâàíèÿ
- 26. Èíêðåìåíòàëüíûå áýêàïû Percona Server: innodb_log_archive=1 I äóáëèðóåò REDO ëîã â îòäåëüíûõ ôàéëàõ Percona XtraBackup: I xtrabackup --prepare --log-arch-dir=/data/archived-logs/ ïðèêëàäûâàåò èçìåíåíèÿ áåç ïîëíîãî ñêàíèðîâàíèÿ
- 27. Ïðîâåðêà öåëîñòíîñòè Ïðîáëåìà: êàê ïðîâåðèòü, ÷òî ñ áýêàïîì ¾âñ¼ â ïîðÿäêå¿? I ïðîâåðêà êîíòðîëüíûõ ñóìì, ñïèñêà ôàéëîâ, è ò.ä. íå ãàðàíòèðóåò öåëîñòíîñòü I íå ñ ÷åì ñðàâíèâàòü:
- 28. Ïðîâåðêà öåëîñòíîñòè: ðåøåíèå Ïðåäëîæåíî ïîëüçîâàòåëåì â ñïèñêå ðàññûëêè: 1. Ñîçäàòü slave èç áýêàïà 1. pt-table-checksum äëÿ ïðîâåðêè öåëîñòíîñòè ðåïëèêàöèè 2. profit!
- 29. Îáëàêà Êòî ïîääåðæèâàåò cloud backups? I mysqldump/mydumper + ñòîðîííèå óòèëèòû (s3cmd) I MySQL Enterprise Backup ñ âåðñèè 3.10.2 I Percona XtraBackup 2.3 (ïîêà alpha)
- 30. Îáëàêà MySQL Enterprise Backup I ñåé÷àñ: Amazon S3 I â ïëàíàõ: ? Percona XtraBackup: I ñåé÷àñ: OpenStack I â ïëàíàõ: Amazon S3, Amazon Glacier
- 31. Îáëàêà: Percona XtraBackup Ñîçäàíèå áýêàïà: innobackupex --stream=xbstream /tmp | xbcloud put --storage=swift --swift-container=testbackup --swift-user=user --swift-password=password --swift-url=http://my.cloud.com/ --swift-key=testkey --parallel=10
- 32. Îáëàêà: Percona XtraBackup Âîññòàíîâëåíèå èç áýêàïà: xbcloud get --storage=swift --swift-container=testbackup --swift-user=user --swift-key=testkey --swift-url=http://my.cloud.com/ --parallel=10 | xbstream -x /data/backup innobackupex --apply-log /data/backup innobackupex --copy-back /data/backup
- 33. Âîïðîñû!