Grant Fritchey

30 July 2020

Flyway: Naming Patterns Matter

Frequently when we demonstrate how our tools work, we take the happy path. Everything is configured correctly. Nothing goes wrong. We introduce a simple change and, ta-da, everything works. However, it’s important to know just how things work when the path isn’t so happy. For example, what if, like I did recently, you mess up your file naming within your Flyway project?

Flyway Naming Patterns

The default naming patterns within Flyway are documented clearly. You have a prefix that determines the type of file, whether a versioned migration (V), an undo migration (U), or a repeatable migration (R). That goes at the front of the file name. Next, you have a version number. This can be almost any format that you like, but it has to be unique for any given migration (although a versioned migration and an undo migration must share a common version number) and it has to be logically ordered. Then, you add two underscores, in order to separate the functional naming aspects of the file from the purely descriptive. After that, it’s just text. You can use an underscore between words and it will be translated as spaces. There’s even a neat graphic describing this:

Simple to understand and implement, right?

Well, not necessarily. I’ve found that it’s shockingly simple to mess this up. Then, the path is not so happy.

The Unhappy Path

I was recently working on some practice deployments in order to better understand the tool and dive deeper into the various options so that I could produce blog posts, like this, showing off how well the tool works. Here are my *.SQL files for deploying a PostgreSQL database:

Everything is in order right? I’ve got repeatable migrations, undo migrations and versioned migrations, neatly stacked. I’ve got logical processing order on all my version numbers, using 2.0, 2.1, 3.0, etc. So, why then, when I tried to deploy was I constantly faced with this little gem of a message:

Notice, it’s only deploying to version 2.0. You can see my command doesn’t include any flags to limit the deployment. What I should be seeing is my full set of scripts deployed up to v4.1. What’s going wrong? Well, look at the file names. Heck, look at the sentence above this one where I talk about v4.1. Notice anything disparate between my file names and the naming pattern defined above? Yeah, capitalization. My code was not being deployed because I was using a lower case value on the names, which didn’t match the naming standard.

Of course, being me, I had to make all this worse and contact the development team to complain about it. Happily, they’re supportive and helpful people. I was given a command to help understand what was going on. It’s simple really. I added -X (again, not the capitalization, it matters) to my migrate command. This adds debug output. So now my migration output looks like this:

In other words, because of my own sloppy standards with the file names, many of my scripts were ignored.

How To Prevent the Unhappy Path

The obvious answer is to just do the naming patterns correctly. However, in this case, I’ll admit, after 30+ years working mostly, but not exclusively, on an OS where ‘V’ and ‘v’ are the same, I can be very lax. For the sake of argument though, let’s assume that you’re not lax like I am. However, it’ still pretty easy to miss a key stroke. So, what can you do? Two things.

First, as a safety, you can modify your flyway.conf file that you use to control your deployment. Add the following:

flyway.validateMigrationNaming=true

This will ensure that Flyway will check the names on your files to ensure that you are correctly following the pattern. However, we’re going to make one more addition to the process. Instead of just attempting to migrate, without checking if my migrations will work, I’m going to run the following:

flyway -enterprise validate

The output on my system is as follows:

So, I’m going to clean up the file names and try again:

You’ll note I still have two problems. Let’s talk about the second. Upon closer examination, when I named the file V4.1_AddView.sql, I missed an underscore. There should be two between the version definition and the file description parts of the name. Easily fixed.

The first problem is a little more of an issue. I’ve kept a full script that can recreate my database changes along with those changes. It’s just a way to be able to set up tests of different kinds. However, now it’s always going to show up as an error in my files. So, in my case, I’m going to move that file to a different location. You’ll have to figure out what’s best in your situation.

Conclusion

While it is possible to get off the happy path, there are tools in place to ensure that you can get back onto the happy path. Also, worth noting, you can adjust all the defaults. If you find you’d prefer a small ‘v’ for your naming patterns, you can adjust the defaults. Flyway has a lot of flexibility built-in, as well as support to help you stay on that happy path.