BIMserver Database Migration

BIMserver has a built-in schema migration system. This system works by keeping track of the current schema version in the database and also in the sofware itself. On startup, both version are compared and the server either starts up without a problem (RUNNING), goes into (MIGRATION_REQUIRED) mode when a migration is required, or to (MIGRATION_IMPOSSIBLE).

The next few sections describe the database upgrade system, but please make sure to read the last chapter as well because that describes when the database migration system DOES NOT WORK

MIGRATION_IMPOSSIBLE

Reverse migrations are never possible, so when your database for example has version 15 and the software has version 14, BIMserver will always go into MIGRATION_IMPOSSIBLE state. The only solution for this is to download the right version of the software.

MIGRATION_REQUIRED

BIMserver goes into this state when the database version is lower than the software version of the schema. BIMserver will not automatically migrate the database, because you should always (manually) MAKE A BACKUP OF YOUR DATABASE FIRST. When you have made a backup, you can either:

• Type "migrate" into the console, this only works when you actually have a console into BIMserver (local dev)
• Use the API call AdminInterface.migrateDatabase
• For BIMservers running on an application server (only Tomcat is tested), you can also add a parameter to your context called "autoMigrate". This will actually try to auto-migrate your database on startup when necessary. This is not recommended.

How it works

For every database schema change in BIMserver we keep track of what has changed in migration steps. Each new schema version has a file called "StepXXXX", where XXXX is the schema version. Example: https://github.com/opensourceBIM/BIMserver/blob/master/BimServer/src/org/bimserver/database/migrations/steps/Step0038.java.

It might be obvious or not, but once a database migration has been published (by releasing a version of BIMserver), you cannot change a migration anymore without breaking the upgrade-system. The only way to revert changes is by adding another migration that reverses your earlier changes.

No migration possible

There are a few reasons why database upgrades do not always work:

• Not all database changes can be described in the migration steps. A migration step can only change the schema (add a class, add an attribute, remove an attribute etc...). Some database changes do not change the schema, but the way the data is stored in the key-value store.
• Not all database changes are cleanly documented in migration steps. For example, a while back we migrated from IFC4 to IFC4Add2. This was done by changing Step0013. When a (already published) migration step is changed, this breaks the migration system.

Current policy

Major releases should be migratable to newer versions within that major release. For example 1.4 should be upgradable to 1.4.1 without problems. Major releases are usually not upgradable to new major releases (1.4 to 1.5 for example is not possible).

Important: The current 1.5 releases are still marked "Pre-release", the description says "Developers release". As long as a major version number is not officially released (1.5), we put in no effort whatsoever to keep migations working. As soon as 1.5 is officially released, we intend to support that version for a while and only do bug fixes that don't require incompatible database changes.

Could we put more effort into making sure databases are upgradable? Yes. At the moment though, we think fixing bugs and adding features etcetera is more important.

What to do if you want to upgrade, but no upgrade-path is available?

So for example when you are running an unreleased 1.5 BIMserver and you want to upgrade to a newer (unreleased) version.

Depending on your situation, you might get away with the following:

1. Keep your old BIMserver running
2. Setup a second BIMserver with the version you would like to run
3. Write a piece of code that connects with both BIMservers
4. Query the first BIMserver for the information you want to migrate (for example Projects, Users, etc...)
5. Use the BIMserver API of the second BIMserver to insert that data

This solution has worked for a few people migration 1.3 to 1.4. You might get a headstart by studying https://github.com/opensourceBIM/BIMserver/blob/master/BimServer/src/org/bimserver/BimServerImporter.java. Which is non-maintained code, but at least it still compiles...

The most important part of most BIMserver databases would be the models. Since sometimes bugs exist in older version of BIMserver, you might want to consider getting the original IFC file (stored in "{homedir}/incoming" and checking those into your new server instead of serializing those models in the original server.