Mongo Migrations in Clojure

I recently had to run database migrations for a MongoDB database in Clojure. I couldn’t find a good example readily available so I put togther this quick sample.

This functionality is also available as part of our Clams framework.

Design

For a migration to work, we need

  1. A list of migrations to run
  2. A way to store the migration state
  3. The ability to run each migration

Ragtime does most of this and Monger will store the state, so all we’ll need to do is run each migration and wire it all together.

We’ll take the approach that each Mongo migration is a Clojure file that we will just read and evaluate using load-string. Each migration will be responsible for gettng a connection to the running Mongo databases.

mongomigrate

We’re just going to replicate the logic in ragtime.jdbc for our Mongo migration. You can follow along in mongomigrate.clj

ragtime.jdbc defines a load-files method based on the file extension. We’ll call all our migration files *.mongoclj and have our load-files return :up and :down fields that load the file:

(defmethod jdbc/load-files ".mongoclj" [files]
  (for [[id files] (group-by (comp first mongoclj-file-parts) files)]
    (let [{:strs [up down]} (group-by (comp second mongoclj-file-parts) files)]
      {:id (basename id)
       :up (gen-load-fn up)
       :down (gen-load-fn down)})))

The gen-load-fn generates a function that evals the file when called. The db handle is passed in, but we’ll ignore that.

(defn- gen-load-fn
  "Generate a function the loads the mongo migrations.  The migration
   is responsible for connecting to the mongo instance."
  [urls]
  (fn [_]
    (doseq [u urls]
      (load-string (slurp u)))))

The monger code already implements the pieces to save the migration state in monger/ragtime.clj so all we need to do is create a config and call the ragtime migrate function:

;; The hardcoded config
(def mongo (mg/connect-via-uri "mongodb://127.0.0.1/rtexample"))
(def config
  {:database (:db mongo)
   :migrations (jdbc/load-resources "migrations/mongo")})

(defn -main
  "Run the migrations or rollbacks"
  [& args]
  (println "Running mongo migrations")
  (case (first args)
    "migrate" (repl/migrate config)
    "rollback" (repl/rollback config (if-let [n (second args)]
                                       (str->int n)
                                       1))
    (println "usage: rtexample.mongomigrate {migrate,rollback}")))

That’s it.

You can find some examples migration files in the rtexample repo mentioned below.

rtexample

A working example (including all the above code) is on github at https://github.com/ctdean/rtexample

One nice thing about this example is you can run the migrations either from lein or from an uberjar. See the README.md file for instructions on how to run the migrations from the uberjar.

Other systems

It’s also worth looking at https://github.com/juxt/joplin for another perspective.

Published

08 Jul 2015


Previous Archive Next