If you have a Hadoop distribution not supported by Pentaho, or you have modified your Hadoop Installation in such a way that it is no longer compatible with Pentaho, you may need to create a new Hadoop configuration.
Changing which version of Hadoop PDI can communicate with requires you to swap the appropriate jar files within the plugin directory and then update the plugin.properties file.
- Identify which Hadoop configuration most closely matches the version of Hadoop you want to communicate with. If you compare the default configurations included the differences are apparent. Copy this folder, then paste and rename it. The name of this folder will be the name of your new configuration.
- Copy the jar files for your specified Hadoop version.
- Paste the jar files into the lib/ directory.
- Change the
active.hadoop.configuration=property in the plugins/pentaho-big-dataplugin/plugin.properties file to match your specific Hadoop configuration. This property configures which distribution of Hadoop to use when communicating with a Hadoop cluster and must match the name of the folder you created in Step 1. Update this property if you are using a version other than the default Hadoop version.