[InnoDB系列] -- 实测ibbackup vs mysqldump

前言:随着数据量的增加,备份问题会越来越突出,尤其是当前没有免费热备工具的情形下。本文针对 InnoDB 的备份需求,采用商业的 ibbackup 以及免费的 mysqldump 2种工具进行对比,让大家有个大致的了解。

 

1. 准备

ibbackupInnoDB 提供的收费工具,它支持在线热备 InnoDB 数据,主要有以下特性:

  • * Online backup of InnoDB tables — the backup takes place entirely online, without preventing queries or updates.
  • * Online backup of MyISAM tables — during the backup of InnoDB tables, read and write access is permitted to MyISAM tables. While the MyISAM tables are being copied, updates (but not reads) to the MyISAM tables are precluded.
  • * Compressed backups — the backup of InnoDB files can be compressed at various levels, saving as much as 70% or more of the storage required.
  • * Partial backups — you can selectively backup all or only some of your InnoDB tables.
  • * High performance — backup time is comparable to file copy, applying logs for recovery is even faster.
  • * Unlimited database size — no practical limit to database size or number of tables.
  • * Broad platform support — runs on Linux, Windows and leading Unix platforms.

innodb官网 申请了个试用版测试下,发现效果还是不错的,不过在我们自己的独特应用环境下,ibbackup 的优势并没有多少。具体的测试结果就不发上来了,有兴趣的同学自己测试看看吧。
ibbackup 只需要识别简单的几个 InnoDB 相关参数即可。它在备份时需要用到2个配置文件,一个是指定在线运行的信息,一个是备份相关信息。例如:
online-my.cnf 告诉 ibbackup 当前正在运行的mysql数据文件所在目录

[mysqld]
datadir  = /home/mysql
innodb_data_file_path = ibdata1:512M:autoextend
innodb_data_home_dir = /home/mysql
innodb_log_file_size = 256M
innodb_log_files_in_group = 3
innodb_log_group_home_dir = /home/mysql

 

backup-my.cnf 告诉 ibbackup 备份文件存储的目录

[mysqld]
datadir  = /home/hotbackup
innodb_data_file_path = ibdata1:512M:autoextend
innodb_data_home_dir = /home/hotbackup
innodb_log_file_size = 256M
innodb_log_files_in_group = 3
innodb_log_group_home_dir = /home/hotbackup

 

运行 ibbackup --help,就能看到以下几个主要选项:

[yejr@imysql.cn]# ibbackup --help
Usage:
ibbackup [--sleep ms] [--suspend-at-end] [--compress [level]]
[--include regexp] my.cnf backup-my.cnf
or
ibbackup --apply-log [--use-memory mb] [--uncompress] backup-my.cnf
or
ibbackup --restore [--use-memory mb] [--uncompress] backup-my.cnf

 

我一般只需要用到 --compress 以及 --uncompress,告诉 ibbackup 压缩/解压缩的级别。

2. 备份

执行以下命令:

[yejr@imysql.cn]# ibbackup --compress 5 online-my.cnf backup-my.cnf
InnoDB Hot Backup version 3.0.0; Copyright 2002-2005 Innobase Oy
License A13002 is granted to imysql Inc. (imysql@imysql.cn)
(--apply-log works in any computer regardless of the hostname)
Licensed for use in a computer whose hostname is 'yejr.imysql.cn'
Expires 2008-9-1 (year-month-day) at 00:00
See http://www.innodb.com for further information
Type ibbackup --license for detailed license terms, --help for help
Contents of online-my.cnf:
innodb_data_home_dir got value /home/mysql
innodb_data_file_path got value ibdata1:512M:autoextend
datadir got value /home/mysql
innodb_log_group_home_dir got value /home/mysql
innodb_log_files_in_group got value 3
innodb_log_file_size got value 268435456
Contents of backup-my.cnf:
innodb_data_home_dir got value /home/hotbackup
innodb_data_file_path got value ibdata1:512M:autoextend
datadir got value /home/hotbackup
innodb_log_group_home_dir got value /home/hotbackup
innodb_log_files_in_group got value 3
innodb_log_file_size got value 268435456
ibbackup: Found checkpoint at lsn 8 804380391
ibbackup: Starting log scan from lsn 8 804380160
080716 19:50:47  ibbackup: Copying log...
080716 19:50:47  ibbackup: Log copied, lsn 8 804380391
ibbackup: We wait 1 second before starting copying the data files...
080716 19:50:48  ibbackup: Copying /home/mysql/ibdata1
ibbackup: Progress in MB: 100 200 300 400 500 600 700 800 900 1000
080716 19:51:21  ibbackup: Copying /home/mysql/test/sbtest.ibd
ibbackup: Progress in MB: 100 200 300 400
080716 19:51:34  ibbackup: Copying /home/mysql/test/geom.ibd
080716 19:51:34  ibbackup: Copying /home/mysql/test/t.ibd
080716 19:51:34  ibbackup: Copying /home/mysql/test/sbtest1.ibd
ibbackup: Progress in MB: 100 200 300 400
080716 19:51:50  ibbackup: Copying /home/mysql/yejr/yejr.ibd
ibbackup: Progress in MB: 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300
............
ibbackup: A copied database page was modified at 8 804394835
ibbackup: Scanned log up to lsn 8 804394835
ibbackup: Was able to parse the log up to lsn 8 804394835
ibbackup: Maximum page number for a log record 209
080716 19:53:03  ibbackup: Full backup completed!

 

就会在 /home/hotbackup 下产生相应备份文件,例如

[yejr@imysql.cn]# ls -l hotbackup/
total 34368
-rw-r-----  1 root root     1024 Jul 17 17:40 ibbackup_logfile
-rw-r-----  1 root root 35128069 Jul 17 17:38 ibdata1.ibz
drwxr-x---  2 root root     4096 Jul 17 17:39 yejr
drwxr-x---  2 root root     4096 Jul 17 17:38 test

 

在备份过程中,还可以持续的提交事务,例如增、删、改数据等,ibbackup 会以最后一次提交的事务为准,取得最新备份。
压缩级别视各自的实际情况而定了。一般需要通过测试,得到一个中间折中值。

3. 还原

[yejr@imysql.cn]# ibbackup --uncompress 5 --apply-log backup-my.cnf
InnoDB Hot Backup version 3.0.0; Copyright 2002-2005 Innobase Oy
License A13002 is granted to imysql Inc. (imysql@gmail.com)
(--apply-log works in any computer regardless of the hostname)
Licensed for use in a computer whose hostname is 's01_tc_10.11.54.189_CNC'
Expires 2008-9-1 (year-month-day) at 00:00
See http://www.innodb.com for further information
Type ibbackup --license for detailed license terms, --help for help
Contents of backup-my.cnf:
innodb_data_home_dir got value /home/hotbackup
innodb_data_file_path got value ibdata1:512M:autoextend
datadir got value /home/hotbackup
innodb_log_group_home_dir got value /home/hotbackup
innodb_log_files_in_group got value 3
innodb_log_file_size got value 268435456
ibbackup: Uncompressing data file '/home/hotbackup/ibdata1.ibz'
ibbackup: Progress in MB: 100 200 300 400 500 600 700 800 900 1000
ibbackup: Uncompressing data file '/home/hotbackup/test/sbtest.ibz'
ibbackup: Progress in MB: 100 200 300 400
ibbackup: Uncompressing data file '/home/hotbackup/test/geom.ibz'
ibbackup: Uncompressing data file '/home/hotbackup/test/t.ibz'
ibbackup: Uncompressing data file '/home/hotbackup/test/sbtest1.ibz'
ibbackup: Progress in MB: 100 200 300 400
........
ibbackup: Uncompressing data file '/home/hotbackup/yejr/yejr.ibz'
ibbackup: Progress in MB: 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300
........
080717 17:49:43  ibbackup: ibbackup_logfile's creation parameters:
ibbackup: start lsn 8 872965632, end lsn 8 872966026,
ibbackup: start checkpoint 8 872966026
InnoDB: Doing recovery: scanned up to log sequence number 8 872966026
InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 Setting log file size to 0 268435456
InnoDB: Progress in MB: 100 200
Setting log file size to 0 268435456
InnoDB: Progress in MB: 100 200
Setting log file size to 0 268435456
InnoDB: Progress in MB: 100 200
ibbackup: We were able to parse ibbackup_logfile up to
ibbackup: lsn 8 872966026
ibbackup: The first data file is '/home/hotbackup/ibdata1'
ibbackup: and the new created log files are at '/home/hotbackup/'
080717 17:50:10  ibbackup: Full backup prepared for recovery successfully!

 

然后把还原出来的数据文件 "冷" 拷贝到mysql的 datadir 下,启动mysqld,即可正常使用了。
从测试结果来看,ibbackup 备份和恢复是基于物理块级,把每个表空间文件做一个干净的备份,并且支持备份同时有事务提交,以得到最新的数据备份。
因此 ibbackup 产生的备份文件如果不经压缩(不管是备份时压缩,还是事后压缩),那么占用的空间会比较大。
mysqldump 可以用 single-transaction 来得到一致性备份,但是备份的数据是发起操作的瞬间快照,之后提交的事务都不会在备份
中体现。mysqldump 可能会导致 log purge 停滞很长时间,使得mysql的响应变得非常慢,这点 ibbackup 就可以避免。mysqldump 产生的备份文件相对 ibbackup 生成的小多了。
还原时,ibbackup 相比重新导入sql文件会快不少,在这方面,ibbackup 有非常大的优势。

技术相关:

评论

[root@host backup]# ./ibbackup mysql.cnf backup.cnf
InnoDB Hot Backup version 3.0.0; Copyright 2002-2005 Innobase Oy
License A18220 is granted to huf (huf@china.org.cn)
(--apply-log works in any computer regardless of the hostname)
Licensed for use in a computer whose hostname is 'host.test.com'
Expires 2010-3-1 (year-month-day) at 00:00
See http://www.innodb.com for further information
Type ibbackup --license for detailed license terms, --help for help

Contents of mysql.cnf:
innodb_data_home_dir got value /server/mysql5.1/data
innodb_data_file_path got value ibdata1:5M;ibdata2:200M;ibdata3:500M:autoextend
datadir got value /server/mysql5.1/data
innodb_log_group_home_dir got value /server/mysql5.1/data
innodb_log_files_in_group got value 3
innodb_log_file_size got value 947912704

Contents of backup.cnf:
innodb_data_home_dir got value /server/backup/db
innodb_data_file_path got value ibdata1:5M;ibdata2:200M;ibdata3:500M:autoextend
datadir got value /server/backup/db
innodb_log_group_home_dir got value /server/backup/db
innodb_log_files_in_group got value 2
innodb_log_file_size got value 947912704

ibbackup: Found checkpoint at lsn 7 3058397889
ibbackup: Error: log file size specified in my.cnf is 947912704,
ibbackup: but file size function returns /server/mysql5.1/data/ib_logfile0 size as 0 5242880
ibbackup: (print format here: high 32 bits, low 32 bits).
ibbackup: Check that you have specified the size right in the my.cnf

为什么我备份的时候总是报这个错误? my.cnf中却没有关于这个修改的选项。
求高手指点下。

检查下2个配置文件中:

innodb_log_files_in_group got value 3
innodb_log_file_size got value 947912704

innodb_log_files_in_group got value 2
innodb_log_file_size got value 947912704

很明显对不上,这个测试我很早时做的,有点忘了,呵呵

我后来将配置改了。
[root@host backup]# ./ibbackup mysql.cnf backup.cnf
InnoDB Hot Backup version 3.0.0; Copyright 2002-2005 Innobase Oy
License A18220 is granted to huf (huf@china.org.cn)
(--apply-log works in any computer regardless of the hostname)
Licensed for use in a computer whose hostname is 'host.test.com'
Expires 2010-3-1 (year-month-day) at 00:00
See http://www.innodb.com for further information
Type ibbackup --license for detailed license terms, --help for help

Contents of mysql.cnf:
innodb_data_home_dir got value /mysql/mysql5.1/data
innodb_data_file_path got value ibdata1:50M;ibdata2:50M:autoextend
datadir got value /mysql/mysql5.1/data
innodb_log_group_home_dir got value /mysql/mysql5.1/data
innodb_log_files_in_group got value 3
innodb_log_file_size got value 947912704

Contents of backup.cnf:
innodb_data_home_dir got value /mysql/backup
innodb_data_file_path got value ibdata1:50M;ibdata2:50M:autoextend
datadir got value /mysql/backup
innodb_log_group_home_dir got value /mysql/backup
innodb_log_files_in_group got value 3
innodb_log_file_size got value 947912704

ibbackup: Found checkpoint at lsn 0 46409
ibbackup: Error: log file size specified in my.cnf is 947912704,
ibbackup: but file size function returns /mysql/mysql5.1/data/ib_logfile0 size as 0 5242880
ibbackup: (print format here: high 32 bits, low 32 bits).
ibbackup: Check that you have specified the size right in the my.cnf
ibbackup: you give as an argument to ibbackup.
改了还是不行。 
but file size function returns /mysql/mysql5.1/data/ib_logfile0 size as 0 5242880
这句错误提示我需要修改那里才能保持一致?
谢谢