Backup and Recovery

Overview

Taking regular backups and knowing what to do to recover from data loss and corruption events are both very important. A multi-node replica set cluster helps protect against system failure, but it will not protect your data from human error (e.g., accidentally deleting data or deploying bad code). This is why backups are such a crucial part of any production database deployment.

In addition, taking and restoring backups is often a useful way to migrate known datasets between environments.

You can implement a backup and recovery strategy for your mLab-hosted deployment by using:

Using mLab’s backup system

mLab’s backup system offers an easy way, through the mLab management portal, for users to create a one-time backup or schedule recurring backups.

In addition, mLab’s backup system offers a easy ways to restore a backup into an mLab-hosted deployment.

Supported backup methods

There are two different methods used by mLab’s backup system to take a backup: using the mongodump tool or taking a block storage snapshot (i.e., filesystem snapshot).

For replica set cluster plans (Shared or Dedicated), mLab’s backup system exclusively takes backups from secondary members to minimize the impact of backups.

mongodump

For Sandbox or Shared plan deployments, the mLab backup system uses MongoDB’s mongodump tool. This utility queries your database deployment and creates BSON files which can then be used by the mongorestore tool to populate a database deployment.

Dedicated plan deployments can choose to use the mongodump method or use block storage snapshots as described in the following section.

mongodump is great for small MongoDB deployments because the resulting backup is both portable and space-efficient. However, it is not a good backup and recovery strategy for larger deployments because of how time- and resource-intensive the strategy can be; most importantly, mongorestore must rebuild indexes as part of the restore. Also, note that mongodump backups do not include the local database which includes the operations log (oplog).

When using the mongodump method through the mLab backup system, backups can be stored with mLab or in custom Amazon S3 buckets.

Block storage snapshot of underlying data files

Available for Dedicated plans only

Another way to create a MongoDB backup is to snapshot MongoDB’s underlying data files using a file system or “block-level” backup method.

On Dedicated plans, mLab’s backup system takes block storage snapshots to create backups of a MongoDB deployment at an exact moment in time. The mechanics of how these block storage snapshots are taken and how they can be used will vary by cloud provider. For example, we use Amazon EBS Snapshots on AWS, and we use Microsoft’s blob snapshot features on Azure.

Block storage snapshots are preferred for larger MongoDB deployments because they tend to be orders of magnitude faster to both take and restore than the mongodump/mongorestore method. In addition, block storage snapshots include the local database which holds the oplog while the mongodump method does not back up this operations log.

Creating backups

You can use mLab’s backup system to take a one-time backup or set up a backup plan to take backups on a recurring schedule.

Step-by-step guide

Backing up your Sandbox or Shared plan deployment
When you request a backup for your Sandbox or Shared plan deployment, the mLab backup system will use MongoDB’s mongodump utility to create it.

Follow these instructions to create backups for your Sandbox or Shared plan deployment:

  1. Log in to the mLab management portal
  2. From your account’s Home page, navigate to the deployment from which you will be taking a backup
  3. Click the “Backups” tab
  4. For a one-time backup, click the “Take one-time mongodump” button
    • To take automatic backups on a regular basis, click the “Schedule recurring backup” button instead
  5. In the form that appears, select the desired options for each of the following:
    • Schedule (for recurring backup plans only)
    • Target (i.e., where you want your backup to be stored)
  6. Click “Submit” to start the backup process
    • For recurring backup plans, this button is labeled “Create backup plan”

Backing up your Dedicated plan deployment
Creating backups for your Dedicated plan deployment has an option not available with other plan types: the ability to choose between block storage snapshots or mongodump as the backup method.

By default, all newly provisioned Dedicated plans come with one daily recurring backup plan using the block storage snapshot method. You can delete this backup plan if you want and/or create additional backup plans to suit your needs.

Follow these instructions to create backups for your Dedicated plan:

  1. Log in to the mLab management portal
  2. From your account’s Home page, navigate to the deployment from which you will be taking a backup
  3. Click the “Backups” tab
  4. For a one-time backup, click the “Take one-time backup” button
    • To take automatic backups on a regular basis, click the “Schedule recurring backup” button instead
  5. In the form that appears, select the desired options for each of the following:
    • Schedule (for recurring backup plans only)
    • Backup type or strategy: mongodump, block storage, or hybrid
      • Hybrid is for recurring backup plans only - it uses mongodump if dataSize is less than 50GB, otherwise uses block storage
    • Target (i.e., where you want your backup to be stored)
  6. Click “Submit” to start the backup process
    • For recurring backup plans, this button is labeled “Create backup plan”

Storage options

Backups taken using mongodump

mongodump-based backups are stored as compressed archives (.tgz) of mongodump outputs. Backups of this type can be stored in either:

  1. A secure mLab-owned container
    • On AWS, mongodump-based backups to mLab-owned containers are stored in the same AWS region as the database.
    • mLab-owned containers are encrypted at rest.
  2. Your own Amazon S3 bucket
Backups taken as block storage snapshots

Available for Dedicated plans only

Block storage snapshots are file system or “block-level” snapshots of MongoDB’s underlying data files. Backups of these types can only be stored in mLab’s cloud account:

EBS Snapshots (applicable to AWS only) are still owned and managed by mLab’s AWS account even if they are shared with another AWS account . If you’d like them to be owned and managed by your own AWS account, you can copy the EBS Snapshot to your own AWS account either manually or programatically via AWS’s API.

If you’re having trouble finding the EBS Snapshots that we have shared with your account through AWS’s management console, be sure to pull down the drop-down such that it says “Private Snapshots” in the same region that your mLab deployment is hosted.

Accessing backups

All completed backups appear under the “Backups” tab of each deployment with informative details such as when the backup started/ended, the size of the original source, the size of the resulting backup, and more.

Backups for deployments that have already been deleted can be accessed by going to the Account Settings page for your account and clicking on the “Backups” tab.

The system-level backups mLab takes for disaster recovery purposes will not appear under the “Backups” tab. See the Frequently Asked Questions section below if you need access to one of these.

Backups taken using mongodump
For backups taken using mongodump, the icons on the far right side of each row allow you to delete, download, or view the log file for the backup. The backup itself is compressed into a .tar file and when extracted, the backup-related files are in the form of a binary dump which is the standard output of MongoDB’s mongodump utility.

You do not need to retrieve the backup if you want to use your mLab-created backup to create a new mLab deployment. However, if you need to retrieve your mongodump backup created by mLab’s backup system for other purposes, follow these instructions:

  1. Log in to the mLab management portal
  2. From your account’s Home page, navigate to the deployment with the backup you want to retrieve
  3. Click the “Backups” tab
  4. Under the “Backups” header, find the row with the backup that you want to download
    • Click anywhere on the row to see additional details about the backup (e.g., backup ID, start and end time, type, size, etc.)
  5. Click the download icon (a down arrow) on the right side of the row with the backup you want - the compressed file will be automatically downloaded to your local machine
    img-backup
  6. Extract and decompress the contents into your desired directory using the following command:
    tar -xvf myfilename.tgz
  7. You are now ready to restore your database or individual collection(s) from your backup using the mongorestore utility.

If you need to programatically access your backups, one option is to store your backups in your own Amazon S3 bucket and use the boto python interface to access them.

Backups taken as block storage snapshots
For Dedicated plan deployments that use this method, block storage snapshots can be restored very quickly. See the Restoring a backup section below.

Additionally, for EBS Snapshots (AWS only), there is an icon on the far right side of the row which, when clicked, allows you to share the EBS Snapshot with another AWS account.

Retention policies

Depending on the type of backup and where it is stored, the retention policy for backups taken using mLab’s backup system is based on the following rules:

Backup event Stored in your Amazon S3 bucket Stored in mLab’s account 1
One-time backup Backups are never deleted Backups are deleted after 30 days
Created from recurring backup plan Retention based on “number to keep” value Retention based on “number to keep” value 2
Recurring backup plan deleted Backups are never deleted Backups are kept for 8 days beyond expected deleted date

If a deployment is deleted, the retention policy is the same as if the recurring backup plan were deleted (as specified above). Backups for deleted deployments can be accessed by going to the Account Settings page for your account and clicking on the “Backups” tab.

Restoring backups

Supported restore methods

There are two ways to restore a backup taken by mLab’s backup system:

  1. Create a new deployment from a backup
  2. Replace an existing deployment with a backup
Create a new deployment from a backup

If you want to use your mLab-created backup to create a new mLab deployment, follow these instructions:

  1. Log in to the mLab management portal
  2. From your account’s Home page, click the “Create from backup” button
  3. Select the backup source (all eligible backups will be listed here)
  4. Complete the remaining options to configure your new deployment
  5. Click “Create new mLab deployment” and your new plan will be created using your selected backup
Replace an existing deployment from a backup

Not available for Sandbox databases

When you restore a backup into an existing deployment, it is require that the target deployment be empty before we can replace it with a backup. You can either delete all the databases in your target deployment before attempting to do an in-place restore, or you will be prompted to delete it before the backup restore happens.

If you want to use your mLab-created backup to take the place of an existing mLab deployment, follow these instructions:

  1. Log in to the mLab management portal
  2. From your account’s Home page, navigate to the deployment into which you want to restore a backup
  3. Click on the “Restore from backup” button
  4. Select the backup source (all eligible backups will be listed here)
  5. If you have not deleted all the databases in your deployment, you will be prompted to delete it before the backup restore happens.
    • If you want to delete all the databases in your deployment, you must manually enter your replica set ID as shown in the pop-up window.

This deployment will not be available for the duration of the restore. Also, all changes (i.e., inserts/updates/deletes) that occurred since the backup will be lost.

Restore considerations

mLab’s restore features will only allow options/actions that apply to a given backup. Therefore, it is important to keep in mind the following limitations and restrictions when restoring a backup. If you continue to have questions or problems while restoring your backup, contact our support team at support@mlab.com, and we’d be happy to help.

General limitations and restrictions

Restoring from a block storage snapshot

In general mLab recommends restoring using a block storage snapshot whenever possible, as block storage snapshots are generally orders of magnitude faster to restore. That being said, there are certain restrictions specific to block storage snapshots that you must keep in mind:

Restoring from a mongodump

When it’s not possible to restore from a block storage snapshot, you can restore from a mongodump. Note the following restrictions when restoring from a mongodump:

Recommendations for common restore use cases

Below are some common backup restore use cases and mLab’s recommendations how to address them.

mLab always recommends restoring a block storage snapshot over a mongodump unless it’s not an option to restore a mongodump.

Fees

mLab does not charge for restoring backups. However, fees may apply when taking and storing backups (see below).

Backup fees for Sandbox and Shared plan deployments
Custom backups (i.e., backups taken using mLab’s backup system) incur a fee, charged on a per-run basis with fees determined by overall data size and storage option. For example, the table below shows the estimated monthly cost based on 30 days (i.e., 30 runs), calculating what you might pay for different data sizes and storage options.

Data Size Your cloud container ($0.50 + $0.05 per GB) mLab’s cloud container ($0.50 + $0.07 per GB)
500 KB $15.00 ($0.50 per run) $15.00 ($0.50 per run)
500 MB $15.73 ($0.52 per run) $16.03 ($0.53 per run)
1.0 GB $16.50 ($0.55 per run) $17.10 ($0.57 per run)
5.0 GB $22.50 ($0.75 per run) $25.50 ($0.85 per run)

The per GB charge is based on the dataSize value from the MongoDB db.stats() function for your database(s). With for-pay plans, MongoDB’s system databases “admin” and “config” are also included in the overall data size calculation.

Backup fees for Dedicated plan deployments
Custom backups for Dedicated plan deployments are free of charge.

Using mongodump/mongorestore

mLab’s backup system is great for creating and storing backups easily. However, there are times when it is necessary to use a more manual method to create or restore a database.

MongoDB’s mongodump and mongorestore utilities are command-line tools that provide fine-grained data import and export capabilities. To download these utilities visit the MongoDB download page.

The mLab management portal includes a command-line helper tool that provides pre-filled command-line strings specific to your database. You can copy and paste them directly into a terminal session, filling in any placeholders before executing the command.

To minimize the possibility of error, the versions of your target database, source database, and mongodump/mongorestore utility should match. For example: use mongodump 2.6 to restore a backup taken from MongoDB 2.6 into a MongoDB 2.6 database.

Note that the version of your mongodump/mongorestore utility must match the version of MongoDB you have deployed. If you have multiple MongoDB versions installed, be sure to use the correct one.

Basic method

Before you take a backup of a single database, you must first stop writes to that database.

Then, to take a backup of that database, run a command in a terminal window that looks similar to this:

% mongodump -h ds012345.mlab.com:56789 -d dbname -u dbuser -p dbpassword -o dumpdir

To restore this backup, run a command that looks similar to this:

% mongorestore -h ds023456.mlab.com:45678 -d dbname -u dbuser -p dbpassword dumpdir/*

Point-in-time method

Applicable to Dedicated plans only

If you have a Dedicated plan, you can take server-wide mongodumps to export all of the databases on the server.

This method is useful because it allows you to use the the --oplog and --oplogReplay options to mongodump and mongorestore (respectively). The options allow for a point-in-time snapshot of the server by also including the oplog in the dump. This oplog is then replayed when you use the --oplogReplay option upon restore.

When doing a point-in-time server-wide dump or restore you must use credentials to the admin database.

To take a backup of an entire server, run a command in a terminal window that looks similar to this:

% mongodump -h ds012345-a0.mlab.com:56789 --authenticationDatabase admin -u admindbuser -p admindbpassword -o dumpdir --oplog

To perform a point-in-time restore of all the databases in this backup, run a command that looks similar to this:

% mongorestore -h ds023456-a0.mlab.com:45678 --authenticationDatabase admin -u admindbuser -p admindbpassword dumpdir --oplogReplay

All mongodump backups taken by mLab’s backup system are server-wide except for Sandbox plan backups. Use the --oplogReplay option with mongorestore to restore to the point in time when the mongodump completed.

Frequently Asked Questions

Help - I lost my data and do not have a backup!

Rest assured, we take daily system-level backups of the databases that we host for disaster recovery purposes. These backups are stored for a limited time (approximately one week), so in the event that you need to restore from a system-level backup, contact support@mlab.com and we will let you know if it’s available. If a backup is available, we will do our best to restore it for you.

It typically takes anywhere from three to 18 hours to restore a system-level backup. If you feel that you will need to self-service backup restores, we encourage you to create a custom backup plan.

I cannot mongodump because of a “system.users: not authorized” error

If you are running MongoDB 3.0 or above and trying to take a mongodump of a specific database you might encounter the following error:

 not authorized on mydb to execute command { count: "system.users", query: {} }

To avoid this error, try using the --excludeCollection=system.users option with mongodump. Adding this option will allow you to exclude legacy database users leftover when the user authorization schema was upgraded in release 2.6.

Why can’t I restore my EBS Snapshot into a smaller plan?

The reason why our management console is restricting what plans you can restore into is that AWS requires that EBS Volumes created from EBS Snapshots be at least as large as the original snapshot.

If your source deployment is on our High Performance line, note that the plans on this line utilize in-chassis/local disks for the electable nodes but also include a third, hidden node that is EBS-backed and designated for EBS Snapshot backups. This backup node often utilizes very large General Purpose SSD (gp2) EBS Volumes because performance on this volume type scales with volume size, and we need these backup nodes to keep up with the electable nodes. Therefore, if you’re restoring from an EBS Snapshot taken from our High Performance line, you will likely need to restore into the same High Performance plan as your source deployment or a Single-node High Storage M5+ plan.

I cannot find the EBS Snapshot that was shared

If you have shared your EBS Snapshot backup with your AWS account but cannot find your shared EBS Snapshot when you log into your AWS account, make sure that you are in the Amazon EC2 console and in the same region that your mLab deployment is hosted. Finally, make sure you have selected the drop-down option “Private Snapshots” from the snapshot filter list.

For more information on how to view EBS Snapshots in AWS, see Viewing Amazon EBS Snapshot Information.

How do I restore a shared EBS Snapshot in my own AWS account?

If you want to restore an EBS Snapshot that was shared with your own AWS account onto a self-hosted MongoDB deployment, here are the high-level steps:

  1. Create an AWS Instance in the same region as the EBS Snapshot
  2. Create an EBS Volume from your EBS Snapshot (in same Availability Zone as your Instance)
  3. Attach this EBS Volume to the Instance
  4. After you’re ssh’d into the Instance, mount the EBS Volume
  5. Install the MongoDB Community Edition version that matches your database (e.g., 3.4.x)
  6. When you start the mongod process set storage.directoryPerDB to true. Set the storage.engine to mmapv1 or wiredTiger depending on the storage engine of the source deployment.




  1. “Stored in mLab’s cloud account” refers to either mongodump backups stored in mLab’s cloud container or block storage snapshots taken in mLab’s account with the cloud provider that your deployment is hosted on (e.g., AWS, Azure, or Google). 

  2. Backup plans resulting in block storage snapshots stored in mLab’s account have a “number to keep” of 8 backups. This number is not customizable.