Flyway – Implementing Java-based Migrations and Callbacks
Flyway’s SQL-script based database migration is more than powerful enough for most use cases. But sometimes, you must take it one step further to adapt your existing data to the new database schema. E.g., you might need to extract data from blobs or read JSON documents so that you can fill the newly added database columns. In these cases, Flyway’s Java migration and callback methods provide an easy and powerful way to implement the necessary migration logic.
Let’s implement a Java migration step first. As you will see, this is quite simple. And after you implement it, you can use it in the same way as the SQL migration steps I showed you in the previous post of this series.
Implement Complex Migrations in Java
When searching for available migration steps, Flyway not only searches for migration scripts. It also picks up implementations of the Callback interface from the db/migration package. If you want to use a different package, you can configure it in the flyway.locations property.
An easy way to implement the Callback interface is to extend Flyway’s BaseJavaMigration class. It handles all the technical complexity of a migration step and allows you to focus on the actual migration. When you do that, you need to use a class name that follows Flyway’s naming schema V<VERSION>__DESCRIPTION.java. Flyway then picks up your migration step, checks if it needs to be executed, and does that if necessary.
Here’s an example of a simple migration class that updates the database to version 2.0. The goal of this migration is to store the author of the book in a separate table. This requires the following operations:
- Create a new author table and sequence
- Read all records from the book table and get the id of the book and the name of the author
- Persist each author as a new record in the author table
- Set the id of the author as the foreign key in the book table
public class V2__extract_author extends BaseJavaMigration {
@Override
public void migrate(Context context) throws Exception {
Connection connection = context.getConnection();
// create author table
Statement st = connection.createStatement();
st.execute(
"CREATE TABLE author(id bigint NOT NULL, firstname character varying(255), lastname character varying(255), CONSTRAINT author_pkey PRIMARY KEY (id));");
st.execute("CREATE SEQUENCE author_seq");
// add fk_author to book table
st.execute("ALTER TABLE book ADD COLUMN fk_author bigint REFERENCES author (id);");
// migrate author information
final PreparedStatement psAuthor = connection
.prepareStatement("INSERT INTO author (id, firstname, lastname) VALUES (?, ?, ?)");
final PreparedStatement psBook = connection.prepareStatement("UPDATE book SET fk_author = ? WHERE id = ?;");
ResultSet rs = st.executeQuery("select id, author from book");
Statement idSt = connection.createStatement();
while (rs.next()) {
// get data from book table
Long bookId = rs.getLong("id");
String author = rs.getString("author");
String[] name = author.split(",");
// get author id from sequence
ResultSet authorIdRs = idSt.executeQuery("select nextval('author_seq');");
authorIdRs.next();
Long authorId = authorIdRs.getLong(1);
// write new author
psAuthor.setLong(1, authorId);
psAuthor.setString(2, name[1]);
psAuthor.setString(3, name[0]);
psAuthor.execute();
// update book
psBook.setLong(1, authorId);
psBook.setLong(2, bookId);
psBook.execute();
}
rs.close();
psAuthor.close();
// add fk_author to book table
st.execute("ALTER TABLE book DROP COLUMN author;");
st.close();
}
}
As you can see, this requires almost no Flyway-specific code. You just need to implement the migrate method of the JavaMigration interface. Within this method, you can use the provided Context object to get a java.sql.Connection to the database. Using this Connection, you can then define and execute the required SQL statements.
This approach gives you full flexibility to read data from your database, transform it in any way you need, and store it in your database. That makes Java-based migration steps a great option for implementing complex, multi-step migration operations.
When you now run your migration, Flyway will detect the current database version, scan for all SQL and Java migration steps and execute the required ones. You can see the log output of these operations below.
15:42:53,864 INFO BaseDatabaseType:37 - Database: jdbc:postgresql://localhost:5432/test-flyway (PostgreSQL 10.14)
15:42:53,925 INFO DbValidate:37 - Successfully validated 2 migrations (execution time 00:00.023s)
15:42:53,966 INFO JdbcTableSchemaHistory:37 - Creating Schema History table "public"."flyway_schema_history" ...
15:42:54,038 INFO DbMigrate:37 - Current version of schema "public": << Empty Schema >>
15:42:54,049 INFO DbMigrate:37 - Migrating schema "public" to version "1 - create database"
15:42:54,097 INFO DbMigrate:37 - Migrating schema "public" to version "2 - extract author"
I triggered the migration on an empty database, and Flyway found the migration steps for versions 1 and 2. The 2nd one was the migration step I implemented as a Java class, which you saw in the previous code snippet.
After Flyway successfully executes a migration step, it adds a record to the flyway_schema_history table.
As you have seen, a Java migration step is used the same way as an SQL script and fully integrates into your migration process. So, when you’re in the situation that you can’t describe the required migration in SQL, you just need to implement the JavaMigration interface and follow Flyway’s naming convention.
Use Callbacks for Repetitive Tasks
Another useful feature for complex migration scenarios is Flyway’s callback mechanism. It allows you to execute an SQL script or a Java class when one of the lifecycle events defined in the Event enum gets triggered within Flyway. A few examples of these events AFTER_BASELINE, AFTER_CLEAN, AFTER_EACH_MIGRATE, AFTER_EACH_MIGRATE_ERROR, AFTER_UNDO, and AFTER_MIGRATE. You can find a list of all supported events in official javadoc.
We didn’t discuss Flyway’s Callback feature in any of the previous articles. So, let’s also take a quick look at SQL callbacks before I get into more details about Java callbacks.
SQL Callbacks
The implementation of an SQL callback is straightforward. You only need to add an SQL script with the name of the lifecycle trigger you want to use in your migration directory. The migration directory is either the sql folder of the Flyway command-line client or your Java application’s src/main/resources/db/migration folder.
So, if you want to execute a SQL script after Flyway migrated your database, you need to put all SQL statements into a file with the name afterMigrate.sql and copy it to the sql or src/main/resources/db/migration folder.
Java Callbacks
If your callback operation is too complex for an SQL script, you can implement it in Java.
A callback implementation is very similar to the previously discussed migration step. You need to implement Flyway’s Callback interface and add your class to the db/callback package or the package configured by the flyway.callbacks property.
The easiest way to implement the Callback interface is to extend Flyway’s BaseCallback class. It provides all the required technical boilerplate code so that you can concentrate on implementing the callback operation.
For each Callback implementation, Flyway calls the handle method for each previously described event. When doing that, Flyway provides an Event enum value and a Context object of the current migration. Similar to the previously described implementation of a migration step, you can use the Context object to get a Connection to the database and perform the operations of your callback.
I use that in the following example to implement a callback that adds some example data if the database is empty. To do that, I first check if the book table contains any data. If it doesn’t, I insert a record into the author and the book table.
public class FillDatabaseAfterMigrate extends BaseCallback {
Logger log = Logger.getLogger(FillDatabaseAfterMigrate.class.getSimpleName());
@Override
public void handle(Event event, Context context) {
if (event == Event.AFTER_MIGRATE) {
log.info("afterMigrate");
Statement st;
try {
st = context.getConnection().createStatement();
ResultSet rs = st.executeQuery("SELECT count(id) FROM book");
rs.next();
if (rs.getInt(1) == 0) {
st.execute(
"INSERT INTO author (id, firstname, lastname) VALUES ((SELECT nextval('author_seq')), 'Thorben', 'Janssen');");
st.execute(
"INSERT INTO book (id, publishingdate, title, fk_author, price) VALUES ((SELECT nextval('book_seq')), '2017-04-04', 'Hibernate Tips - More than 70 solutions to common Hibernate problems', 1, 9.99);");
log.info("Database was empty. Added example data.");
} else {
log.info("Database contains books. No example data needed.");
return;
}
} catch (SQLException e) {
throw new MigrationException(e);
}
}
}
public class MigrationException extends RuntimeException {
public MigrationException(Throwable cause) {
super(cause);
}
}
}
That’s all you need to do to implement a callback. When you now start your application and trigger the database migration, Flyway will call the Callback implementation.
The following log output shows that Flyway called our callback implementation after it completed the migration. The callback implementation then initialized the empty database with 2 example records.
16:06:27,515 INFO BaseDatabaseType:37 - Database: jdbc:postgresql://localhost:5432/test-flyway (PostgreSQL 10.14)
16:06:27,605 INFO DbValidate:37 - Successfully validated 2 migrations (execution time 00:00.030s)
16:06:27,659 INFO JdbcTableSchemaHistory:37 - Creating Schema History table "public"."flyway_schema_history" ...
16:06:27,745 INFO DbMigrate:37 - Current version of schema "public": << Empty Schema >>
16:06:27,760 INFO DbMigrate:37 - Migrating schema "public" to version "1 - create database"
16:06:27,822 INFO DbMigrate:37 - Migrating schema "public" to version "2 - extract author"
16:06:27,893 INFO DbMigrate:37 - Successfully applied 2 migrations to schema "public", now at version v2 (execution time 00:00.162s)
16:06:27,909 INFO FillDatabaseAfterMigrate:19 - afterMigrate
16:06:27,919 INFO FillDatabaseAfterMigrate:30 - Database was empty. Added example data.
Summary
I showed you in the previous post of this series that Flyway provides an easy but powerful approach to implementing a version-based migration process. You only need to provide a script with the required SQL statements to update your database structure and migrate your data.
In my experience, you should be able to implement almost all migrations within these SQL scripts. But as you’ve seen in this post, you’re not limited to that approach. If you need more flexibility to perform complex migration operations, you can implement them in Java.
And for all repetitive tasks, like recompilation of stored procedures, database initialization with sample data, or dynamic creation of database triggers, you can implement lifecycle callbacks in SQL scripts or Java classes.
Combining all this gives you a powerful toolset to implement a version-based database migration approach.