{"id":95064,"date":"2022-11-19T17:13:23","date_gmt":"2022-11-19T17:13:23","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=95064"},"modified":"2022-11-19T17:13:23","modified_gmt":"2022-11-19T17:13:23","slug":"backing-up-mysql-part-3-mysqlpump","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/blogs\/backing-up-mysql-part-3-mysqlpump\/","title":{"rendered":"Backing up MySQL Part 3: mysqlpump"},"content":{"rendered":"<p><span style=\"font-weight: 400\">In the MySQL world, there are a couple of ways to take backups of your data. Backups in MySQL can be categorized into two distinct categories: logical or physical. MySQL also comes with a lot of tools helping us achieve our backup objectives: <a href=\"https:\/\/www.red-gate.com\/simple-talk\/blogs\/backing-up-mysql-part-2-percona-xtrabackup\/\">we have walked you through one of the most popular offerings in this space &#8211; Percona XtraBackup &#8211; in the second part of this series<\/a>, and <\/span><a href=\"https:\/\/www.red-gate.com\/simple-talk\/databases\/mysql\/backing-up-mysql-part-1-mysqldump\/\"><span style=\"font-weight: 400\">the first part of this series covered one of the main logical MySQL backup tools called <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><\/a><span style=\"font-weight: 400\">. As far as backups are concerned, though, we are far from done \u2013 for example, <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\"> alone has a brother called <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\">. Since the two CLI-based backup tools are frequently confused with each other, the aim of this blog is to clear the confusion up a little.<\/span><\/p>\n<h2><span style=\"font-weight: 400\">What is mysqlpump?<\/span><\/h2>\n<p><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> is a backup utility that is used via the command-line interface. The tool is very similar to <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\"> in that it provides us with the ability to take logical backups, but also different at the same time \u2013 the goal of <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> is to be an extendable, parallel-supporting replacement of <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\">. <\/span><a href=\"https:\/\/dev.mysql.com\/blog-archive\/introducing-mysqlpump\/\"><span style=\"font-weight: 400\">In their blog from 2015<\/span><\/a><span style=\"font-weight: 400\">, MySQL team said that one of the primary aims of introducing <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> was not be forced to implement legacy functionality that is provided by <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\">.<\/span><\/p>\n<p><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> was introduced to the world of MySQL in 2015 \u2013 the tool was added to MySQL 5.7.8.<\/span><\/p>\n<h2><span style=\"font-weight: 400\">Using mysqlpump<\/span><\/h2>\n<p><span style=\"font-weight: 400\">The most basic use case of <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> looks like this (the tool can be used in the same fashion both on Linux and on Windows):<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-95066\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2022\/11\/mysqlpump-1.png\" alt=\"\" width=\"392\" height=\"113\" \/><\/p>\n<p><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> is designed in such a way that it will only dump tables that were created by a user \u2013 in other words, it will refrain from backing up tables that were created internally by MySQL (it won\u2018t touch any of the internal tables.)<\/span><\/p>\n<p><span style=\"font-weight: 400\">The one feature that is exclusive to <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\">, though, is the ability to issue backups using multiple threads at once \u2013 to figure out how many threads your system has, follow these steps:<\/span><\/p>\n<ol>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Linux users \u2013 issue a command like <\/span><code><span style=\"font-weight: 400\">lscpu<\/span><\/code><span style=\"font-weight: 400\"> or <\/span><code><span style=\"font-weight: 400\">lscpu | egrep \u2018Thread|CPU\\(s\\)\u2019<\/span><\/code><span style=\"font-weight: 400\"> and look at the output (you should see something similar to \u201cThread(s) per core.\u201d)<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Windows users \u2013 open up task manager and look at the number next to \u201cLogical processors\u201d \u2013 they represent the amount of threads in the system:<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-95067\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2022\/11\/taskmgr.png\" alt=\"\" width=\"285\" height=\"288\" \/><\/span><\/li>\n<\/ol>\n<p><span style=\"font-weight: 400\">Once you know the number of threads in your system, think about the number of threads you wish (and can) allocate to <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> \u2013 things might seem difficult and complex, but your thought process shouldn\u2019t take more than a couple of seconds \u2013 <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> is usually quick to complete too.<\/span><\/p>\n<p><span style=\"font-weight: 400\">Once you know the number of threads in your infrastructure, configure the number of threads that can be used together with <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\">. Threads are to be configured using the <\/span><code><span style=\"font-weight: 400\">--default-parallelism<\/span><\/code><span style=\"font-weight: 400\"> and <\/span><code><span style=\"font-weight: 400\">--parallel-schemas<\/span><\/code><span style=\"font-weight: 400\"> options. Also, consider providing the username and password that you\u2019re using to access MySQL itself through my.cnf for secure access: <\/span><i><span style=\"font-weight: 400\">passwords provided through the CLI can be observed by accessing the history of issued commands<\/span><\/i><span style=\"font-weight: 400\">. If you don\u2019t want to do that, feel free to provide a username and a password, but be aware that <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> itself will issue you warnings if you do so:<\/span><\/p>\n<p><code><span style=\"font-weight: 400\">mysqlpump --default-parallelism=[threads] &gt; data.sql<\/span><\/code><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-95068\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2022\/11\/mysqlpump-2.png\" alt=\"\" width=\"928\" height=\"130\" \/><\/p>\n<p><span style=\"font-weight: 400\">Since the \u201cspecialty\u201d of <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> is to back up data using multiple threads, we can also define the number of threads that we want the tool to use to dump a specific database \u2013 this functionality of <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> can be exceptionally useful if we\u2019re working with bigger sets of data, but need a logical backup to be taken.<\/span><\/p>\n<h2><span style=\"font-weight: 400\">mysqlpump \u2013 the Details<\/span><\/h2>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">To specify more threads to dump a large database and less threads for smaller databases, look at the option named <\/span><code><span style=\"font-weight: 400\">parallel-schemas<\/span><\/code><span style=\"font-weight: 400\">. If you define your parameters like so, 6 threads will be used for larger databases, and 2 for smaller ones:<\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><code><span style=\"font-weight: 400\">mysqlpump --parallel-schemas=6:large_db1,large_db2 --parallel_schemas=2:small_db [--default-parallelism=x] &gt; backup.sql<\/span><\/code><span style=\"font-weight: 400\"><br \/>\n<\/span><span style=\"font-weight: 400\">Note the <\/span><code><span style=\"font-weight: 400\">--default-parallelism<\/span><\/code><span style=\"font-weight: 400\"> option: a default number of threads when dumping other databases (databases that are not defined) can also be specified.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">To back up only specific databases, define your query like so (here db_1 and db_2 represent two separate databases):<\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><code><span style=\"font-weight: 400\">mysqlpump \u2013-databases db_1 db_2 &gt; databases.sql<\/span><\/code><span style=\"font-weight: 400\"><br \/>\n<\/span><span style=\"font-weight: 400\">You can also accomplish the same goal like so:<\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><code><span style=\"font-weight: 400\">mysqlpump --include-databases=db_1,db_2 --result-file=dump.sql<\/span><\/code><\/li>\n<li style=\"font-weight: 400\"><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> can also perform an \u201cempty\u201d dump (meaning that it can only back up the schema of the database, but not the data contained within):<\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><code><span style=\"font-weight: 400\">mysqlpump --include-databases=data_1,db2,db3 <\/span><b>--skip-dump-rows<\/b><span style=\"font-weight: 400\"> --result-file=mydata.sql<\/span><\/code><\/li>\n<li style=\"font-weight: 400\"><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> also gives us the ability to use wildcards inside of any parameter. That means that if we have a lot of databases, backing a part of them (only those starting with the letter x, for example) is a breeze:<\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><code><span style=\"font-weight: 400\">mysqlpump <\/span><b>--include-databases=x%<\/b><span style=\"font-weight: 400\"> --result-file=s_databases.sql<\/span><\/code><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We can also <\/span><i><span style=\"font-weight: 400\">exclude<\/span><\/i><span style=\"font-weight: 400\"> databases from being backed up like so (the following query would exclude all databases starting with <code>test_<\/code> and back up all the rest):<\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><code><span style=\"font-weight: 400\">mysqlpump <\/span><b>--exclude-databases<\/b><span style=\"font-weight: 400\">=test_% --result-file=data.sql<\/span><\/code><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We can also work with entire patterns (the following query would <\/span><i><span style=\"font-weight: 400\">exclude<\/span><\/i><span style=\"font-weight: 400\"> all tables matching the <code>demo<\/code> pattern from the beginning):<\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><code><span style=\"font-weight: 400\">mysqlpump <\/span><b>--exclude-tables=__demo<\/b><span style=\"font-weight: 400\"> --result-file=backup.sql<\/span><\/code><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">And last but not least, <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> is also able to work with events and routines \u2013 all the same, just specify events or routines instead of databases and tables:<\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><code><span style=\"font-weight: 400\">mysqlpump --[include|exclude]-[events|routines]=title1,title2 --result-file=verycoolbackup.sql<\/span><\/code><\/li>\n<\/ul>\n<p><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> can do a number of other things as well \u2013 all of the information on <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\">, as with everything MySQL-related, can be found at <\/span><a href=\"https:\/\/dev.mysql.com\/doc\/refman\/8.0\/en\/mysqlpump.html\"><span style=\"font-weight: 400\">the documentation<\/span><\/a><span style=\"font-weight: 400\">.<\/span><\/p>\n<h2><span style=\"font-weight: 400\">The Downsides of mysqlpump<\/span><\/h2>\n<p><span style=\"font-weight: 400\">However, as powerful as <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> is, it\u2019s also not without its weaknesses. <\/span><a href=\"http:\/\/datacharmer.blogspot.com\/2015\/09\/mysql-57-playing-with-mysqlpump.html\"><span style=\"font-weight: 400\">Research made by Giuseppe Maxia back in September 2015<\/span><\/a><span style=\"font-weight: 400\"> suggests that <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> is faster than <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\">, but only slightly \u2013 the blogger provides an example where <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\"> takes 3 minutes and 33 seconds to execute, whereas <\/span><span style=\"font-weight: 400\">mysqlpump<\/span><span style=\"font-weight: 400\"> takes 2 minutes and 55 seconds \u2013 the difference is there, but we presume that the blogger has expected it to be way bigger than it was.<\/span><\/p>\n<p><span style=\"font-weight: 400\">The research was made with approximately 20 million rows inside of the tables in the database \u2013 a figure big enough for both scripts to handle, but not large enough to invoke <\/span><code><span style=\"font-weight: 400\">SELECT * INTO OUTFILE<\/span><\/code><span style=\"font-weight: 400\"> (a command used to back up bigger sets of data.) This suggests that <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> is good if we find ourselves needing to use specific options not available to <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\">, but doesn\u2019t provide much of an upside otherwise.<\/span><\/p>\n<h2><span style=\"font-weight: 400\">Summary<\/span><\/h2>\n<p><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> entered the MySQL scene in 2015 with the release of MySQL 5.7.8. At first it was thought that it could be a reliable replacement for <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\"> and also offer a couple of \u201cexotic\u201d options not available to its counterpart, but as time went by <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> proved that it still needs more refinement to be considered a reliable replacement for <\/span><span style=\"font-weight: 400\">mysqldump<\/span><span style=\"font-weight: 400\">.<\/span><\/p>\n<p><span style=\"font-weight: 400\">Both <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\"> and <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> are part of the MySQL\u2019s ecosystem to this day, yet people steer towards <\/span><code><span style=\"font-weight: 400\">mysqldump<\/span><\/code><span style=\"font-weight: 400\"> more than they do towards <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> \u2013 while <\/span><code><span style=\"font-weight: 400\">mysqlpump<\/span><\/code><span style=\"font-weight: 400\"> has its use cases, it is thought that its speed is not significant enough to outweigh its counterpart.<\/span><\/p>\n<p><span style=\"font-weight: 400\">We hope that you\u2019ve enjoyed reading this article, <\/span><a href=\"https:\/\/www.red-gate.com\/simple-talk\/databases\/\"><span style=\"font-weight: 400\">stay around the Redgate blog to learn more about everything database-related<\/span><\/a><span style=\"font-weight: 400\">, and until next time.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this blog, we walk you through the brother of mysqldump &#8211; mysqlpump &#8211; and tell you everything about it. Have a read!&hellip;<\/p>\n","protected":false},"author":339547,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2],"tags":[5564,4168,5854,6011],"coauthors":[146040],"class_list":["post-95064","post","type-post","status-publish","format-standard","hentry","category-blogs","tag-backups","tag-database","tag-mysql","tag-software-tools"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/95064","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/339547"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=95064"}],"version-history":[{"count":4,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/95064\/revisions"}],"predecessor-version":[{"id":95073,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/95064\/revisions\/95073"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=95064"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=95064"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=95064"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=95064"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}