Skip to main content
Pentaho Documentation

Prepare DI Migration Plan

Overview

Helps you create a migration plan.

Planning essential for a successful migration experience. As you formulate your strategy, consider these questions.

Should You Migrate To Version 5.x.x?

As you determine whether to upgrade your system to this version of the software, consider these questions.

What Resources Do You Need?

Resources are a key component in an upgrade plan. Consider both technical and human resources.

  • Do your systems meet the hardware and software requirements indicated in the Supported Components matrix?
  • Who should be involved in the upgrade process? Smaller systems that are housed on a single computer can sometimes be upgraded by one person. But, larger or geographically distributed systems might involve more people. For example, the web application server and security might be handled by one group while reporting databases might be handled by another. List those who can provide needed information or help troubleshoot problems. Remember to include stakeholders.
  • Who will help verify that data has been migrated correctly? Keep in mind that some information, such as user favorites, are sometimes best verified by end users.
  • You will need to download the Migration Tool from our site. You also need a text editor, and a tool to compare the differences between text files.

How Much Time Will The Migration Process Take?

The amount of time it takes to perform the upgrade process varies based on how much data you have and how much you have configured the 4.4.x system. Allot time to install and configure the licenses for the 5.x.x system, run the Migration Tool, perform manual configuration tasks, and to test and troubleshoot the system. After testing, allow time to configure the 5.x.x system, deprovision the 4.4.x server, and provision the 5.x.x one.  Use these time estimates as a rough guide.

Table 1. Migration Process Time Estimates
Task

Time Estimate

Description
Installing 5.x.x system, licenses, and the Migration Tool 1 to 1 1/2 Hours If you decide to use the installation wizard to 5.x.x server, allot about one hour to run the installation wizard and the license, and to also download and unpack the Migration Tool.  If you need to customize your DI Repository database, web application server, or both, set aside anywhere from an hour to an hour and a half to work through the custom installation procedures, install licenses, and to download and unpack the Migration Tool.
Run Migration Tool 30 minutes to several hours. 1 second job, transformation, and shareable object. One second per job, transformation, and shareable object present in your system.
Verify Migration 1 Hour to Several Days Plan to spend anywhere from a few hours to several days for validation testing.   The amount of time that you spend depends on your content, the numbers of transformations, shared objects, and jobs that you have, and how exhaustively you want to test the system.

When Should You Migrate?

Once you have identified your resources and have estimated how much time it might take to upgrade your system, consider appropriate timeframes for upgrade.

  • Some parts of the process require that the 4.4.x and 5.x.x servers be offline. How much downtime does your organization allow?
  • Consider performing the migration process during off-peak hours.  
  • Consider organizational events and seasonal reporting needs as you select a timeframe.

What Data Will Be Migrated by the Migration Tool?

It is important to understand what types of data is migrated from the 4.4.x system to the 5.x.x one. Much of the data from the 4.4.x server can be migrated automatically using the Migration Tool, but some must be migrated manually.  The Migration Tool is a standalone utility that you can download from our website. If you use the Migration Tool, it will attempt to migrate all of the data in the following DI Data Migrated by the Migration Tool table.

Table 2. DI Data Migrated by the Migration Tool
Type of Data Data Migrated with the Migration Tool

Exceptions

Transformation and Jobs All versions of transformations and jobs, all comments related to each version. N/A
Shareable Objects All enterprise repository shareable objects and associated connection information. This includes database connections and connection information (such as connection name), partitions, slave servers, and clusters. File based shareable objects that are stored in the shared.xml file, database shareable objects.
Schedules Name of schedule, frequency, username associated with it, parameters needed to run it. N/A
System Configuration Information in these files: pentaho.xml, applicationContext-security-ldap.properties, settings.xml, slave-server-config.xml, web.xml, context.xml. JNDI data sources named "Hibernate" or "Quartz" will not be migrated. Only the following web.xml settings are migrated: Solution-path, Base-url, Fully-qualified-server-url, Locale-language, contextClass, contextConfigLocation, pentahoObjectFactory, encoding, hsqldb-databases, Proxy Trusting Filter, TrustedIpAddrs. No custom servlets or filters will be migrated.  
Users and Roles Usernames and roles. N/A

As indicated in the previous table, some configuration files are processed by the migration tool. Each configuration file is processed in one of four ways.

  • Overwrites 5.x.x File: The Migration Tool exports a copy of the file from the 4.4.x system, then saves it to the migration bundle. Then, when you import migration data, it replaces the 5.x.x version of the file with the 4.4.x version of the file. The Migration Tool processes configuration files in this way when there were no changes made from version 4.4.x to version 5.x.x of the file, but the file might have been customized by the customer.
  • Adds New Properties to 5.x.x File: The Migration Tool exports a copy of the file from the 4.4.x system, then saves it to the migration bundle. Then, when you import migration data, it finds the properties that exist in the 5.x.x version of the file and adds them to the copy of the 4.4.x version of the file that is in the migration file bundle. The existing 4.4.x version file comments are preserved, in property name order. The Migration Tool processes configuration files in this way when new properties were added to the 5.x.x version of the file.
  • Updates 5.x.x File with 4.4.x Changes: The Migration Tool exports a copy of the file from the 4.4.x system, then saves it to the migration bundle. Then, when you use the tool to import migration data, the Migration Tool adds the 5.x.x properties and code to the 4.4.x version of the file. This 5.x.x file then replaces the 5.x.x version of the file that was in the server’s file system, prior to the migration process. The Migration Tool processes configuration files in this way when there were significant changes made to the version 5.x.x configuration files.
  • Updates 4.4.x file with 5.x.x Changes: The Migration Tool exports a copy of the file from the 4.4.x system, then saves it to the migration bundle. Then, when you use the tool to import migration data, the Migration Tool adds 4.4.x properties and code to the 5.x.x version of the file. This file then replaces the 5.x.x version of the file that was in the server’s file system, prior to the migration process. This approach was taken when there were few changes made to the version 5.x.x configuration files.

The following table indicates how the Migration Tool has processed specific configuration files.

Table 3. PDI Configuration Files Migrated by the Migration Tool and Migration Strategy
Path 4.4.x DI Server 5.x.x DI Server Migration Strategy
system\ applicationContext-security-ldap.properties applicationContext-security-ldap.properties Add New Properties to 5.x.x File
system\ pentaho.xml pentaho.xml Update 4.4.x File with 5.x.x Changes
system\ pentaho-spring-beans.xml security.properties Update 5.x.x File with 4.4.x Changes
system\kettle\ settings.xml settings.xml Overwrite 5.x.x File
system\kettle\ slave-server-config.xml slave-server-config.xml Overwrite 5.x.x File
webapps\pentaho-di\WEB-INF\classes log4j.xml log4j.xml Update 4.4.x File with 5.x.x Changes
webapps\pentaho-di\WEB-INF\ web.xml web.xml Update 5.x.x File with 4.4.x Changes
webapps\pentaho-di\META-INF\ context.xml context.xml Update 5.x.x File with 4.4.x Changes

What Must Be Migrated Manually?

Data that must be migrated manually includes:

  • JDBC Drivers for your data sources.
  • Memory settings for the server.
  • Any additional JSPs, servlets, custom libraries, plugins, java API extensions that require a change to the pentaho-objects-spring.xml file, themes, or other resources integrated as part of the custom pentaho system.
  • Pentaho system localization information.
  • Content for deleted or otherwise inactive users.

Where Are The Components In The 4.4.x System Located?

Identify the paths to the these components so that you can supply them when you export data from the 4.4.x server.

  • Web Application Server
  • DI Repository Database
  • pentaho-di web application directory on the web application server

You might also want to obtain the location of design tools that are not on the server.

Where Will The Components In The 5.x.x System Be Located?

Identify the paths to the these components so that you can supply them when you import data into the 5.x.x server.

  • Web Application Server
  • DI Repository Database
  • pentaho-di web application directory on the web application server

Additionally, obtain the location of design tools that are not on the server.

Note: After you migrate the data from the 4.4.x server into the 5.x.x one, you must verify that the data in both systems match. This is easier to do if you can see the user interfaces for the 4.4.x and 5.x.x versions of the server at the same time.

How Will You Prepare Users For The Migration?

Consider how you will communicate with users, obtain feedback, and provide training.

  • How will you handle customer feedback about the upgrade process?
  • How you will communicate with internal and external customers?
  • How much advanced notice will you provide?
  • How will you train users? Note that tutorials and documentation are available in the Infocenter. Additionally, Pentaho offers professionally-led training classes on many aspects of the 5.x.x system. See the Pentaho website for details.

How Will You Document Your Migration Plan?

Documenting your responses to the questions raised in this section can be an invaluable troubleshooting aid should you need to contact technical support.

  • How will you document upgrade plan?
  • Who will have access to the plan and where will it be located? How will this plan be updated throughout the process?