Migrating Data Into mLab

• Migrating from an existing MongoDB deployment
  • Use mLab’s migration tool
  • Using replica set replication
  • Using mongodump / mongorestore
  • Using copydb
• Importing a JSON, CSV or TSV file
• Pre-filled import/export command-line strings

Migrating from an existing MongoDB deployment

There are several different ways to migrate data into mLab. The method you choose depends on your particular data source and your uptime requirements.

If you have any questions about which process makes the most sense for your application, don’t hesitate to contact support@mlab.com for help.

Use mLab’s migration tool

Not available for Sandbox databases.

If you are currently running a MongoDB instance that you would like to migrate over to a mLab-hosted deployment, you have the option to migrate with almost no downtime using mLab’s migration tool.

Pre-requisites for Source Deployment

1. Must be running version 2.6.x, 3.0.x, 3.2.x, or 3.4.x
   • Some restrictions apply if running 2.6.x but < 2.6.10
2. Must be running as a Replica Set
3. Must be able to supply mLab with an admin database user of the Replica Set that has least the privileges of the backup role
4. Must be able to allow inbound network access from a mLab-hosted deployment

High-level Migration Steps

1. Create your new deployment on mLab.
   • If you’re migrating into one of our Dedicated plans, we recommend creating a larger plan than you think you’ll end up on to help with the initial bulk load. We pro-rate charges by the day and downgrades are seamless on our Dedicated Cluster plans via our rolling node replacement process.
   • If you would like some help sizing your new deployment, please ask support@mlab.com for help.
2. Supply mLab with the credentials for a admin database user that has at least the privileges of the backup role.
3. mLab will start the migration process and inform you when your mLab-hosted deployment is fully in-sync with your existing deployment.
   • The migration tool will take an initial copy of your deployment and restore it into your mLab-hosted deployment.
   • After that it will keep up-to-date with your deployment by tailing the source deployment’s oplog.
4. You will confirm that you can connect and authenticate to your new mLab-hosted deployment.
5. When you are ready to finalize the migration to mLab:
   • You will stop writes to your application.
   • You will roll out a connection string change.

Contact support@mlab.com for more details on this migration method and/or if you’d like to get started.

Using replica set replication

Available for Dedicated plans only.

If you are currently running a replica set that you would like to migrate over to one of mLab’s Dedicated plans, you have the option to minimize downtime by using MongoDB’s internal replication mechanisms.

Pre-requisites for the source deployment

1. Must be running version 2.6.x+, 3.0.x, 3.2.x, or 3.4.x
2. Must be running as a replica set
3. If running in “auth” mode, must be able to supply mLab with the replica set’s key file (the value for the --keyFile option)
4. Must be able to allow network access from and to a mLab-hosted deployment

Using mongodump / mongorestore

MongoDB’s mongodump and mongorestore utilities are command-line tools that provide fine-grained data export and import capabilities. To download these utilities visit the MongoDB download page.

When you are ready to copy data from one MongoDB deployment to another, follow these instructions:

1. Log in to the mLab management portal.
2. If you have not already done so, create a new mLab deployment as your target.
   • Be sure to create database user(s) on the new deployment.
3. To prevent data inconsistency, stop any processes running against your source deployment.
4. Take a backup your source deployment by running the mongodump command in a terminal window.
   • See below for a single-database example.
5. Restore the backup into your target deployment by running the mongorestore command.
   • See below for single-database example.
   • Also, mLab provides pre-filled command-line strings for import/export commands to help.
6. Update your application with the target’s connection URI and start writing to your new deployment.

Example

To mongodump a single database from your source deployment:

% mongodump -h source_host:source_port -d dbname -u source_dbuser -p source_dbpassword -o dump_dir

To mongorestore this backup to your target deployment:

% mongorestore -h target_host:target_port -d dbname -u target_dbuser -p target_dbpassword dump_dir/*

Using copydb

Available for Dedicated plans only

MongoDB’s copydb command allows you to copy a database directly from a remote, source instance to the current, destination instance.

This method is faster than mongodump/mongorestore, but you must have a Dedicated plan with mLab in order to run this command, since it requires full administrative privileges on the destination instance.

1. Log in to the mLab management portal.
2. If you have not already done so, create a new mLab deployment to host the destination database.
   • The destination database itself does not need to be created; the command will implicitly create it if it does not exist.
3. If you have not already done so, create an admin database user for the destination deployment.
4. To prevent data inconsistency, stop any processes running against your source database.
5. In a terminal window, connect to the destination instance using the admin database user’s credentials.
6. Run the db.copyDatabase() helper method.
   • The expected order of arguments is somewhat confusing so take caution as you construct the command.
   • The duration of the copy process can vary widely, depending on the amount of data, number of indexes, network latency, etc..
7. When the copying has completed and you have updated your application with the new connection URI, you can start writing to your new database.

Importing a JSON, CSV or TSV file

If you need to import or export data in a flat file (non-binary) format such as JSON, CSV, or TSV, the mongoimport and mongoexport commands are the appropriate MongoDB command-line tools to use. To download these utilities, visit the MongoDB download page.

Once MongoDB is installed successfully, follow the instructions below to import a JSON, CSV, or TSV file into your mLab-hosted database:

1. Log in to the mLab management portal.
2. If you have not already done so, create a new mLab deployment as your target.
   • Be sure to create database user(s) on the new deployment.
3. Use the mongoimport command to import your text file(s) into your new deployment.
   • Take advantage of mLab’s pre-filled command-line strings for import/export commands and choose the JSON- or CSV-specific line to get you started with constructing the command.

Example

To mongoimport a JSON file to a collection in your target deployment:

% mongoimport -h ds012345.mlab.com:56789 -d dbname -c collectionname -u dbuser -p dbpassword --file filename.json

To mongoimport a CSV file to a collection in your target deployment:

% mongoimport -h ds012345.mlab.com:56789 -d dbname -c collectionname -u dbuser -p dbpassword --file filename.csv --type csv --headerline 

• The first row of the CSV input file must specify the field names; each subsequent line in the file must represent a single document.
• The --headerline option in the command is what tells mongoimport to interpret the first line of the CSV file as the field names and not as a document.
• For a TSV file, all of the above also applies, the only difference is that --type tsv should replace --type csv in the command.

Pre-filled import/export command-line strings

MongoDB commands can be tricky to write and typos aren’t any fun so we’ve tried to make it easier for you by constructing pre-filled command-line strings that you can copy and paste directly into your terminal. To obtain these strings, follow these instructions:

1. Log in to the mLab management portal.
2. From your account’s Home page, navigate to the deployment with the database that you are going to import or export.
   • If necessary, navigate to the desired database after navigating to the deployment.
3. Click the “Tools” tab.
4. Click the “import/export” link and find the command you want to use.
5. Replace all placeholders (i.e., the values enclosed in angled brackets).
   • Make sure the database user is NOT a read-only user.
   • The <input db directory> for mongorestore is not the same as the <output directory> for mongodump. The <input db directory> should be one level below the <output directory>. For example, if the mongodump <output directory> was “dump_dir” for the database “foo”, the <input db directory> for mongorestore would be “dump_dir/foo”.
6. In your terminal window, run the adjusted command.