Managing Credentials Securely with Flyway Resolvers
One of the problems that creeps up on you as a Flyway project grows is how to manage securely the different credentials Flyway needs to access all the databases it needs to migrate. It's easy enough to get started by storing them in the TOML config files, but that approach isn't viable once you're dealing with sensitive data, multiple environments, or stricter security requirements. Fortunately, Flyway's Property Resolvers give us several ways to pull credentials, connection information and other config details from secure stores, such as secrets managers, at runtime.
Where should I keep my database credentials?
For a small-scale development, the answer is easy: put them in the user area TOML config file. By small-scale, I mean a small database with a single developer, where real data isn’t being used for testing, and the RDBMS doesn’t allow OS access.
Flyway uses two TOML configuration files to store project-level and user-level config for a Flyway project:
- flyway.toml – this file is used for shared team preferences and project settings. It can be stored in the root Flyway project directory and should be checked into Git
- flyway.user.toml – used to store URLs, credentials and personal preferences for each environment. If storing credentials, this file should be in the secure user area (and never checked into Git)
This separation helps ensure that sensitive information is kept out of version-controlled project files and managed securely on a per-user basis.
However, the issue with using the user area for credentials is that it could well fall foul of your organization’s security rules. When Flyway runs under your user account, it inherits your permissions to read files in your user area. Basic operating system access controls protect these files. However, they are not encrypted, so the information can be easily read directly from the drive.
Also, if Flyway executes a script callback (e.g., PowerShell or Bash), it will run with the privileges of your user ID. This means that the callback will have access to files in your user area, environment variables (including credentials), and will be able to connect directly to databases. A small malicious block of code inserted into a PowerShell callback could do a lot of damage, if undetected (see Simple Safeguards for PowerShell Scripting with Flyway).
This level of security will be frowned upon by your security team if you are developing using ‘live data’ containing personal or sensitive information. They will probably insist you store credentials in a directory service or a secure vault. This is exactly what Property resolvers allow you to do.
Using Property Resolvers
Property Resolvers allow you to reference credentials and other sensitive configuration values without storing them directly in your configuration files. Instead, your TOML files contain resolvers that tell Flyway which secure store to retrieve the values from, such as a secure secrets manager.
This also makes it much easier to manage the same project configuration across different environments, each with its own credentials and connection details.
To use a resolver in your flyway.toml configuration file, simply reference it using the syntax ${<resolver-name>.<resolver-key>}
.
For example, to retrieve a password from Vault, we’d use:
password = "${vault.db_password}"
Or to get the user from an environment variable:
user = "${env.DATABASE_USER}"
Property Resolvers work within the project TOML file or user TOML file, but not currently a TOML file specified as a parameter in -configFiles
.
Supported Property Resolvers
Flyway supports several built-in property resolvers, including:
- Dapr Secret Store Resolver: Fetches secrets from a Dapr (Distributed Application Runtime) secret store. This is designed for cloud-native applications.
- Google Cloud Secret Manager Resolver: A service that retrieves secrets, such as API keys, usernames, passwords, and certificates from Google Cloud’s Secret Manager.
- HashiCorp Vault Resolver: Accesses secrets stored in HashiCorp Vault, which uses identity-based security to automatically authenticate and authorize access to secrets and other sensitive data.
- Environment Variables Resolver: Pulls configuration values straight from environment variables you’ve defined.
- Git Resolver: Extracts configuration from Git repositories.
- Redgate Clone Resolver: Integrates with Redgate Clone for database provisioning to allow the use of temporary servers with auto-generated IDs and Passwords.
- Azure Active Directory Interactive Resolver: Facilitates interactive authentication with Microsoft Entra AD. (formerly known as Azure Active Directory)
- Local DB Resolver: A specialised resolver designed to simplify connections to SQL Server LocalDB instances.
- Local Secret Resolver: Reads secrets on a development machine from that machine’s encrypted secret store. The type of store that Flyway uses depends on the operating system of the workstation
Note that some of these resolvers require internet access.
Some resolvers, such as HashiCorp VAULT, require additional configuration before Flyway can use them to get a secret. The following example specifies the database user ID in the flyway.user.toml file because that is part of the definition of the ‘environment’. However, it gets the password from HashiCorp Vault:
[environments.production] url = "jdbc:postgresql://localhost:5432/prod_db" user = "admin" password = "${vault.db_password}"
To get this to work, we also need to add a resolvers.<name>
table to the TOML file to provide extra config details needed for Flyway to connect to the store (in this case, url
, token
, engineName
, and engineVersion
):
[environments.production.resolvers.vault] url = "https://vault.example.com" token = "s.1234567890abcdef" engineName = "secret" engineVersion = "v2"
Restrictions on the use of some Resolvers
Traditional Staging environments are usually in a DMZ, so it is unlikely that you can use any resolver that requires internet access, such as Dapr, Google Cloud Secret Manager or HashiCorp Vault.
The reason for these draconian rules is not just to avoid intrusion, but also to prevent the delivery of a payload to the dark side from a rogue script. This level of security is necessary when using real data from a backup for final pre-deployment checks; at this point, you must be completely confident there is no possibility of data leakage. Nowadays, firewalls are reliable enough to allow specific access, so check with your Ops people.
If you can’t run Flyway in your Staging environment, you can use its –dryRun
feature to prepare the migration script in the development environment, as long you have a database at the current release version. You then execute this script manually in staging, using whatever authentication method is permitted there.
Standardizing secrets management from development to release
If you want consistency in how you manage connection details, from development to release, but some environments are in a DMZ, then you’ll need a solution that works offline. Here are some of your options, depending on your environment:
- Avoid storing credentials entirely by using integrated security – e.g., Kerberos authentication with a local Active Directory inside the DMZ.
- Use the operating system’s secure credential store – such as Windows Credential Manager or Linux Secret Storage (e.g. GNOME Keyring or pass). These options work fully offline.
- Store secrets in an encrypted local file – use a GPG-encrypted TOML or JSON file and decrypt them only in memory at runtime.
- Run a local secrets vault – such as a self-hosted HashiCorp Vault instance inside the DMZ. Flyway can fetch credentials from the vault using a resolver.
- Provision credentials in a temporary in-memory location – such as a memory-mapped file (tmpfs on Linux, RAM disk on Windows), which is wiped after use.
For a Windows environment, the obvious choice is Kerberos authentication with Active Directory to avoid password storage entirely. However, many database development tools required in a Flyway pipeline still need explicit credentials. It’s worth verifying which tools are involved in the Staging or Release pipeline before deciding how to manage credentials.
Where avoiding local credential storage isn’t practical, the operating system’s Credential Manager is often the best option, for example, using the Windows credential store. My previous article, Storing Credentials Securely on a Windows-based Flyway Installation, described how to fetch login credentials using Get-WindowsCredential
and pipe them securely to Flyway using the configFiles=-
parameter. However, we can now use a built-in localSecret
Property Resolver to fetch credentials directly, simply by referencing them in the Flyway TOML file (see below).
For storing an entire TOML file in encrypted form, by contrast, you’d need Windows Security. See my article Using Windows Security and Encryption with Flyway for details.
For more general use across other operating systems, I prefer GPG-encrypted secrets with a dongle such a Yubikey, but a local Vault would be a good alternative. The cross-platform GPG approach (Gnu Privacy Guard) allows you to store database connections in a single encrypted TOML file. This file is decrypted at runtime and fed into Flyway via STDIN using -configfiles=-
. See: Pipelining Configuration Information Securely to Flyway for more details. Ensure that no one uses your workstation if the dongle is still plugged in. GPG is a good approach when Active Directory isn’t available or can’t be used by your RDBMS, as it keeps information local and secure at rest.
Saving credentials to Windows Credentials Manager
First, we use PowerShell to extract and save the secrets Flyway needs to connect to the relevant database. In this example, we’ll store the Flyway connection details (user, password, and JDBC URL) securely in Windows Credential Manager as separate secrets.
Each secret is stored under a unique name based on the project, database type, branch, and server, making it easy to retrieve them later using Flyway’s localSecret
resolver, avoiding the need to pass credentials via environment variables or STDIN.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
<# Windows credentials were designed specifically for userID/Passwords but we need to specify and hide connection details too. There could be other secrets as well. That means that we put every secret into the password field. so that we can have a whole lot of secrets stored, we need a unique name for each. I use ... #> $Project = 'Pubs' # for working directory and the name of the credentials file $Branch = 'Main' # the branch. We use this just for suitable login credentials $Server = 'Philf01' #the name of the server $RDBMS = 'SQLServer' #the name of the SQL database provider $Username = 'biggles' #not used by flyway, the name of the user who has the secret # -- we now put these together as a secret, along with the name of the secret. $connection = "$($Project)$($RDBMS)$($Branch)$($Server)" # this is the array of the names of the secrets $Type=@('User','Password','Url') #you may need other secrets #Now we can simply delete all existing secrets for this connection $Type|foreach{ Remove-storedCredential -Target "$($connection)$($_)" -ErrorAction Ignore} #These are the values we want to put into the secret store $ConnectionInfo=@( "PhilipFactor",#user "CunningPasswrd",#password "jdbc:sqlserver://MyServer\sql2017;databaseName=PubsMain;encrypt=true;trustServerCertificate=true;"#url ) #for each of our secrets, turn it into a secure string $secrets=($ConnectionInfo|foreach { ConvertTo-SecureString $_ -AsPlainText -Force})|foreach{ New-Object -TypeName PSCredential -ArgumentList $UserName, $_ } #and now we create the credential $Secrets|foreach {$ii=0}{ "$($connection)_$($type[$ii])" New-StoredCredential -Target "$($connection)$($type[$ii])" -Credential $secrets[$ii] -Type Generic -Persist LocalMachine $ii++ } #and read themback out, just to check that we have them OK. $Type| foreach{ $Variable= "$($connection)$($_)" $cred = Get-StoredCredential -Target $Variable [pscustomobject]@{'code'=$Variable; 'value' = $cred.GetNetworkCredential().Password} # Use this to retrieve the password } |
Connecting to Flyway securely using Resolvers
Once the secrets are stored, we can reference them from either the user or project TOML file using the localSecret
resolver.
It’s generally better to use the user TOML file, since the resolver points to a local, user-specific resource. If you put these references in the shared project TOML file, then every user would need to create and name their secrets the same way. The revolvers in my case look something like this:
[environments.develop] url = "${localSecret.Pubs_SQLServer_Main_Philf01_url}" user = "${localSecret.Pubs_SQLServer_Main_Philf01_user}" password = "${localSecret.Pubs_SQLServer_Main_Philf01_password}" schemas = [ "dbo", "classic", "people", "accounting" ]
Flyway retrieves the secrets from the credential store and substitutes them for the resolver placeholders at runtime.
Conclusion
Managing database credentials securely in a Flyway pipeline becomes more challenging as project complexity and security requirements grow. For simple development setups, storing credentials in the user TOML file may be sufficient, but for environments involving sensitive data, compliance demands, or multiple users, stronger protections are needed.
Flyway’s built-in property resolvers allow credentials to be fetched securely from external sources such as HashiCorp Vault, Azure Key Vault, and Windows Credential Manager. This avoids plaintext credentials in config files while supporting automated, reliable access to connection details.
In highly secure environments, local Vault instances, encrypted local storage, or memory-mapped files may be appropriate. On Windows, Kerberos authentication with Active Directory is ideal, as it eliminates the need to store passwords entirely. Where that’s not possible, Credential Manager is a solid fallback. Cross-platform options like GPG-encrypted secrets with a hardware token (e.g., YubiKey) provide strong local security.
Ultimately, the goal is to strike a balance between security, maintainability, and compatibility with your tools. Resolver-based approaches offer a practical way to achieve this, keeping credentials encrypted and accessible only to the processes that need them.