Skip to main content

Pentaho+ documentation has moved!

The new product documentation portal is here. Check it out now at docs.hitachivantara.com. 

 
Version Data Integration and Analytics
Hitachi Vantara Lumada and Pentaho Documentation

Pentaho Worker Nodes

  1. Last updated
  2. Save as PDF

Parent article

Pentaho Worker Nodes (PWN) allow you to execute Pentaho work items, such as KTRs and KJBs, with parallel processing, dynamic-scalability, load-balancing, and dependency-management in a clustered environment.

Pentaho Worker Nodes can be deployed in physical, virtual, and Cloud environments. The hybrid deployment model allows the user to manage the base and peak time processing demands for the workloads of hundreds of work items with different processing requirements. Through this model, additional nodes can be scaled onto Cloud resources, on-demand, to distribute the load in cases where on-premise resources are insufficient.

Pentaho Worker Nodes, using tools such as Docker, Chronos, and Mesos, allows the Pentaho Server to scale the execution of work items in a worker node environment. You can manage system scaling by adding or removing instances (servers or virtual machines connected to the cluster) to the system and by specifying which services run on an instance. A single-instance system is useful for testing and demonstration purposes, while a multi-instance system is recommended for use in a production environment.

Worker Nodes base times and peak times diagram
Parent Topic
Child Topics

Audience

Worker Nodes documentation is intended for selected 9.0 and later Hitachi Vantara customers who are interested in exploring this feature. Contact your Hitachi Vantara sales associate for more information.

Worker node installation and configuration documentation is intended for IT administrators who know where data is stored, how to connect to it, details about the computing environment, and how to use the Linux command line.

Parent Topic

Requirements
You must meet the following requirements to set up Worker Nodes:
  • Pentaho 9.0 or later installation. See Pentaho Installation.
  • Pentaho Worker Nodes distribution file.
  • A 64-bit Linux distribution
  • Docker version 1.13.1 (Minimum version required) or later
Parent Topic

Process Overview

The following topics explain how to set up your system and Pentaho to use worker nodes:

  1. Read the Pentaho Worker Nodes system best practices. We can recommend several hardware, networking, and operating systems for creating and running your version of the Pentaho Worker Nodes Product on one or more instances.
  2. Installing the Pentaho Worker Nodes product. Walk through how to install your custom Pentaho Worker Nodes Product package.
  3. Setting up mounted volumes on Pentaho Worker Nodes to manage your configuration.
  4. Setting up Pentaho Worker Nodes. Provide access to Pentaho work items, including PDI jobs and transformations.
  5. Enabling secure communication for Pentaho Worker Nodes. Secure your Pentaho Worker Nodes configuration.
Once you have Pentaho Worker Nodes set up and configured, you can then Run Work Items on Pentaho Worker Nodes and Administer Pentaho Worker Nodes. You can also review points to remember for Scalability and High Availability (HA) on Pentaho Worker Nodes.
Parent Topic