Pentaho connects to Hadoop clusters using a compatibility layer called a driver. These drivers are Apache Karaf archive files (.kar format) that connect Pentaho services to Hadoop clusters. You must use a driver to connect each vendor and version of your Hadoop clusters for the following Pentaho products:
- PDI client
- Pentaho Server
- Interactive Reports
- Pentaho Report Designer (PRD)
- Pentaho Metadata Editor (PME)
Pentaho ships with drivers for Hortonworks, Amazon EMR, and Cloudera. However, you do need to install the drivers on the PDI client or Pentaho Server before using them. When drivers for new Hadoop versions are released, you can add them to Pentaho, then install them to connect to the new Hadoop distributions.
- If you add multiple drivers, you need to allocate 8 to 10 GB of disk space for each driver to your runtime cache.
- If you are running an environment that manages multiple instances (executions) of the PDI engine, then allocate enough space for each runtime cache. For example, environments running 50+ concurrent instances require 75GB or more of cache space during peak processing times.
Add a new driver
Perform the following steps to add a new driver.
Download the new driver from the Pentaho Customer Support Portal
Log in to the Pentaho Customer Support Portal home page using the Pentaho Support user name and password provided to you in your Pentaho Welcome Packet.
Click Downloads, then click Pentaho 9.0 GA Release in the 9.x list.
On the bottom of the Pentaho 9.0 GA Release page, browse the folders in the Box widget to find the new driver file and download it.
In the PDI client, select the View tab of your transformation or job.
Right-click the Hadoop clusters folder and click Add driver.The Add driver dialog box appears.
Click Browse and navigate to where you saved the .kar file you downloaded from the Pentaho Customer Support Portal.
Select the new driver file, click Open, then click Next.The Congratulations dialog box appears, notifying you that you must restart the PDI client and the Pentaho Server.
- If you are connected to a repository, the driver is copied to the Pentaho Server driver directory.
- If you are not connected to a repository, the driver is copied to the PDI client driver directory.