Difference between revisions of "Database migration"

From Dragon Age Toolset Wiki
Jump to: navigation, search
(a big update to this page in anticipation of the new toolset version coming up soon)
Line 1: Line 1:
<div style="float:left;">[[Image:Ambox_warning_yellow.png]]</div> '''Caution''': When installing a new version of the toolset over the old version, make sure to tell the installer not to overwrite the existing database unless you want to wipe the resources there and start fresh. If you want to preserve your existing work, you'll need to leave the existing database intact and then after installing you'll need to run a migration script to bring the database format up to date.
+
<div style="float:left;">[[Image:Ambox_warning_yellow.png]]</div> '''Caution''': When installing a new version of the toolset over the old version, make sure to tell the installer not to overwrite the existing database unless you want to wipe the resources there and start fresh. If you want to preserve your existing work, you'll need to leave the existing database intact.
  
Note also that the migration script may not work correctly on Vista when UAC is enabled. A workaround for this issue has not been found; if you're running Vista you may need to disable UAC (which requires a reboot) before you can successfully run the migration script.
+
See [[#Backing up your database|Backing up your database]] for instructions on how to back up your database before proceeding.
  
See http://support.microsoft.com/kb/929907 for more detail on this issue.
+
== Migrating from verison 1.00 to version 1.01 ==
  
== Database version 170 to 175 ==
+
Unfortunately there have been changes in the builder-to-builder format of the toolset in the most recent version (to fix the root cause of the [[Plot GUID bug]]) so it will not be possible to migrate old content directly to the new toolset version using a builder to builder export/import. Instead, there's a particular sequence of steps you'll need to follow to bring forward any content you want to preserve while also gaining the corrected core resources in the new database that comes with the installer.
  
The tools needed for this are found on the sftp site:
+
=== Step 1 - Install version 1.01 of the toolset but keep your old database ===
  
*MigrateProcess.bat - Gets the Database Server Name from the registry and then calls ExecuteMigrationScript.bat passing as parameters the server name and the migration.sql file name
+
[[File:Installer - don't restore the default database.png]]
*ExecuteMigrationScript.bat - takes two parameters, Server name and migration script.sql name
+
*Migration-1.0.0.170-1.0.0.175.sql - this name will change with each version we release.
+
  
If you've installed your toolset with the default location and default database name, updating your database is very simple; just double-click on the "MigrateProcess" batch file.
+
This will give you the new toolset but keep your old database, with all the content you want to preserve still stored within it.
  
If you've used a non-default server name you may need to run the "ExecuteMigrationScript" batch file with the server name and migration script as parameters:
+
=== Step 2 - [[Builder-to-builder]] save your work ===
  
<code>ExecuteMigrationScript %ServerName% Migration-1.0.0.170-1.0.0.175.sql</code>
+
Now that you've got your resources accessible in a new version of the toolset that uses the new builder-to-builder file format, you can save your work in a [[DADBDATA]] file that the new version of the toolset will be able to understand. See the [[builder to builder]] page for information on how to select and save the resources you want to keep. Don't save the old core resources, and especially not the old core plots, as these resources are updated in the new database you'll be installing in step 3.
  
If you change the database name you will have to change the Migration Script sql file.
+
=== Step 3 - Replace the old database with the new one ===
 +
 
 +
The toolset installer will have placed a backup copy of the new database in the following location:
 +
 
 +
C:\Program Files\Dragon Age\tools\dbbak\bw_dragonage_content.bak
 +
 
 +
or wherever your Dragon Age game was installed, if not Program Files. This database has corrected core plot GUIDs in it, and installing this backup will resolve the [[Plot GUID bug]]. For information on how to restore this database backup, see [[#Backing up your database|Backing up your database]], below.
 +
 
 +
=== Step 4 - Builder-to-builder load your work ===
 +
 
 +
Now that you have the corrected core resources in your database, recreate the modules that you saved your work from and use [[builder to builder]] to load your resources back into them.
 +
 
 +
== Will we need to do this rigamarole each time the toolset is updated? ==
 +
 
 +
The process above is due to an unfortunate combination of needing to provide you with a brand new set of core resources (to correct the [[plot GUID bug]]) and also changing the format used to store information in DADBDATA files (to correct the root ''cause'' of the plot GUID bug, which was that the builder-to-builder process didn't preserve GUIDs for plot resources when loading them). In the future, assuming we don't find any more fundamental bugs like this one that require the builder to builder format to change again, DADBDATA files should be compatible across toolset versions. Furthermore, unless once again we find more fundamental bugs in the core resources as distributed, it should not be necessary to replace your database in future updates.
 +
 
 +
So barring unforseen circumstances future updates should be a lot less complicated than this one.
  
 
== Backing up your database ==
 
== Backing up your database ==
 +
 +
=== Using the Backup and Restore batch files ===
 +
 +
A pair of batch files have been included with the toolset install to ease the process of making backups. They can be found at:
 +
 +
C:\Program Files\Dragon Age\tools\DatabaseUtilities\Backup_Restore
 +
 +
Both the "Backup" and "Restore" scripts are controlled by settings stored in the config.ini file. If you're using the default database location, the only settings you'll need to change are the filenames that backups will be stored to or read from.
 +
 +
Use caution when restoring your database, as this will overwrite everything in your existing database.
 +
 +
=== Using SQLCMD manually ===
  
 
Should you want to back up your database, you'll need to enter a few commands to the database server.
 
Should you want to back up your database, you'll need to enter a few commands to the database server.
Line 38: Line 64:
  
 
The exported version exists as a file in the override directory. This directory gets emptied by default when a new version is installed.  If you haven't backed up the directory's contents you will need to reexport the resources into game format.
 
The exported version exists as a file in the override directory. This directory gets emptied by default when a new version is installed.  If you haven't backed up the directory's contents you will need to reexport the resources into game format.
 
== Behind the scenes ==
 
 
In a nutshell, a database is just a single file. In the case of SQL Express there is also an engine that needs to run to interface with that file. When you install our toolset it installs the database engine and creates a new file. When you run the installer again it is smart enough to know that you have the engine already installed and will not install it again. You have the option of unchecking this part of the install. You also have the option to create a new file if you want.
 
 
If you do not create a new file then you have to run the migration script on your old file for the new toolset executable to work with your old file.
 
 
Uninstallation is done by the Microsoft uninstaller so we can't say for certain what will be kept around.
 
 
By default the database name we use is bw_dragonage_content and that is what the migration script is keyed off of. If you change it then you will have to modify the migration script to reflect that.
 
 
If you run the toolset configuration tool, the file location for the Win32 game build is where the final game resources exist. The other tab is used to specify which database server and database file to point to.
 
  
 
[[Category:Database]]
 
[[Category:Database]]

Revision as of 17:59, 2 December 2009

Ambox warning yellow.png
Caution: When installing a new version of the toolset over the old version, make sure to tell the installer not to overwrite the existing database unless you want to wipe the resources there and start fresh. If you want to preserve your existing work, you'll need to leave the existing database intact.

See Backing up your database for instructions on how to back up your database before proceeding.

Migrating from verison 1.00 to version 1.01

Unfortunately there have been changes in the builder-to-builder format of the toolset in the most recent version (to fix the root cause of the Plot GUID bug) so it will not be possible to migrate old content directly to the new toolset version using a builder to builder export/import. Instead, there's a particular sequence of steps you'll need to follow to bring forward any content you want to preserve while also gaining the corrected core resources in the new database that comes with the installer.

Step 1 - Install version 1.01 of the toolset but keep your old database

Installer - don't restore the default database.png

This will give you the new toolset but keep your old database, with all the content you want to preserve still stored within it.

Step 2 - Builder-to-builder save your work

Now that you've got your resources accessible in a new version of the toolset that uses the new builder-to-builder file format, you can save your work in a DADBDATA file that the new version of the toolset will be able to understand. See the builder to builder page for information on how to select and save the resources you want to keep. Don't save the old core resources, and especially not the old core plots, as these resources are updated in the new database you'll be installing in step 3.

Step 3 - Replace the old database with the new one

The toolset installer will have placed a backup copy of the new database in the following location:

C:\Program Files\Dragon Age\tools\dbbak\bw_dragonage_content.bak

or wherever your Dragon Age game was installed, if not Program Files. This database has corrected core plot GUIDs in it, and installing this backup will resolve the Plot GUID bug. For information on how to restore this database backup, see Backing up your database, below.

Step 4 - Builder-to-builder load your work

Now that you have the corrected core resources in your database, recreate the modules that you saved your work from and use builder to builder to load your resources back into them.

Will we need to do this rigamarole each time the toolset is updated?

The process above is due to an unfortunate combination of needing to provide you with a brand new set of core resources (to correct the plot GUID bug) and also changing the format used to store information in DADBDATA files (to correct the root cause of the plot GUID bug, which was that the builder-to-builder process didn't preserve GUIDs for plot resources when loading them). In the future, assuming we don't find any more fundamental bugs like this one that require the builder to builder format to change again, DADBDATA files should be compatible across toolset versions. Furthermore, unless once again we find more fundamental bugs in the core resources as distributed, it should not be necessary to replace your database in future updates.

So barring unforseen circumstances future updates should be a lot less complicated than this one.

Backing up your database

Using the Backup and Restore batch files

A pair of batch files have been included with the toolset install to ease the process of making backups. They can be found at:

C:\Program Files\Dragon Age\tools\DatabaseUtilities\Backup_Restore

Both the "Backup" and "Restore" scripts are controlled by settings stored in the config.ini file. If you're using the default database location, the only settings you'll need to change are the filenames that backups will be stored to or read from.

Use caution when restoring your database, as this will overwrite everything in your existing database.

Using SQLCMD manually

Should you want to back up your database, you'll need to enter a few commands to the database server.

Starting from a DOS prompt, type:

sqlcmd –S.\bwdatoolset

When the prompt comes up 1>, enter

backup database bw_dragonage_content to disk="<Directory & File Name>"

GO

Also, bear in mind that some resources (such as art resources) are not included in your database; they are stored as files on your hard drive. Keep track of where you've put these and, if they're in a directory that might be affected by reinstallation of the toolset, make backups of these too.

The exported version exists as a file in the override directory. This directory gets emptied by default when a new version is installed. If you haven't backed up the directory's contents you will need to reexport the resources into game format.