Migroose: MongoDB Migrations
Migroose provides database / data-structure migrations for MongoDB, with NodeJS and MongooseJS models / schemas.
Changing your document structure in MongoDB is no different than changing a table structure in a relational database, when it comes to migrating data. The migration needs to be done - you need your new data structure to be populated with the data from the old structure.
Migroose will help you get there by providing a structure and framework in which you can migrate from your old MongoDB collections and document structures, in to your new one.
Start by installing migroose in your project:
npm install --save migroose npm install -g migroose-cli
Create A Migration Script
To create a migration, you will want to use migroose-cli. This is a command line tool to generate and run you migrations.
migroose some example migration
This will create a
where "########" is a timestamp. The contents of this file will
be a barebones migration that does nothing more than a console.log.
The migration will also have an ID passed in to the Migration constructor function. This is used to ensure idempotency within a given database / system. Running a migration more than once will only do the work once, based on the ID.
Working With The Migration API
The first thing you need to know: Every method in the migration API creates a new step to execute. Steps are always run in the order in which they are defined.
The net effect of this: If you drop a table first and then try to load data from it, there will be no data to load!
So be careful. Be sure to specify the steps in the order you need. As a recommended starting point, use steps in this order:
- Data Load
- Custom Migration Steps
- Data Remove
- Collection Drop
If you are migrating away from an old data structure, and no longer have a model that represents this structure, you can use the data load feature.
After creating a migration instance, call the
.load method, passing in an
object literal with key / value pairs. The key will be used to reference the
data that is returned. The value will be the name of the collection from which
data is loaded.
When the migration is run, each of the collections specified in the
configuration will result in a data set being made made available to the steps.
If you need to limit the data that is loaded, from within a given collection, you can specify any standard MongooseJS query, as shown in the above example. If you do not need to limit the data returned, and want to retrieve the entire collection, specifying the collection name directly will do that.
The data loaded by the
load command returns a raw MongoDB "collection" and
an array od "documents" on the named data parameter.
By returning the raw collection and document array, you can manipulate the documents in a manner that would not be possible using MongooseJS models. However, this does limit your migration code in that you can't use methods and features of MongooseJS models and schemas.
If you wish to manipulate the data in your documents that are loaded with the
load feature, you can do so. To add / update or otherwise save things to the
database, see the MongoDB Collection documentation
Custom Migration Steps
Now that you have a migration and have optionally specified a collection of
documents to load, you can define steps for your migration. Any given migration
can be built with 1 or more steps, using the
.step method. This method receives
a single argument of a callback function. The callback function receives a
data parameter, and a
stepComplete function parameter.
data parameter will contain the named collections that were previously
defined by the
When your step is complete, call the
The step definition function is where you will do the real work of transforming your previous collection and document structure, in to your new MongooseJS model structure. Be sure to require any MongooseJS model you need, so that you can manipulate the data correctly.
var MyModel = ;migration;migration;
In this example, a single model will be retrieved form the
that was previously loaded. This model is used to create a new model, which is
then saved. After saving the model, the step is completed.
A second step is also defined in this example. Steps are run in the order in which they are defined in the file. This allows you to have multiple steps that potentially deal with multiple collections, or to have processes that are a little more involved be split apart.
Remove Old Data
Having migrated your data, you may wish to remove data from old collections (models)
from your database. This can be with the
remove configuration, which works
the same was as the
In this example, both the
collections are removed from the database. In the case of
all documents are removed. In the case of
anothercollection, however, only
documents that match the query will be removed.
Note that the query for removing data can be any valid MongooseJS query, the
same as the
Drop Old Collections
If you are completely removing a collection from your database, you may wish
to drop the collection entirely after migrating data out of it. To do that,
you can specify a
drop configuration with a list of collections.
Drops will run last in the migration process.
Run The Migration
Having written this complete script, you can now run the migroose command line with no parameters, to execute your migrations.
Please see the migroose-cli documentation for information on how to configure the migroose-cli tool and connect it to your database.
Once you have connected migroose to your database, you can use the migroose command line tool to run your migrations:
This will run the migrations that you have created, and not yet run. Running migrations mutliple times will result in the work being done only once, due to the ID passed in to the Migration constructor.
View Prevously Run Migrations
If you would like to view the list of migrations that have been run on your app instance, you can do that in two different ways.
- Run the
- Examine the
To run the MigrationModel's find method, require Migroose in your script and then execute the find method as you would any other MongooseJS model find method.
var Migroose = ;MigrooseMigrationModel;
This will print out a list of all migrations that have been run in the current app database. You are free to use any MongooseJS methods to find Migrations, using the MigrationModel - it is a standard MongooseJS model / schema.
If you wish to examine the
migrootions collection in your MongoDB
instance directly, you may do this however you wish. It will show you the same
information as the MigrationModel.find method.
Migroose is ©2015 Muted Solutions, LLC. All Rights Reserved.
You may distribute and use Migroose under the MIT License.