Skip to main content
Pentaho Documentation

Adaptive Execution Layer

Pentaho uses the Adaptive Execution Layer (AEL) for running transformations in different engines. AEL adapts steps from the transformation you develop in PDI to native operators in the engine you select for your environment.

For example, AEL currently supports a Spark engine, which is better suited for running big data transformations in a Hadoop cluster. Additional engines will be added to AEL in future releases.

Set Up AEL

AEL must be configured before using a supported optional engine in your transformation's run configuration. Refer your Pentaho or IT administrator to Setting Up the Adaptive Execution Layer for more details.

Use AEL

Once configured, you can select the AEL optional engine for the transformation. See Run Configurations for more details.

Feature Parity

As a general rule, most PDI transformation steps function as documented while running under AEL. Some PDI steps are optimized for a specific engine, while other steps have known exceptions when you run a transformation under AEL. See the following articles for more information:

Advanced Topics

The following topics help to extend your knowledge of the Adaptive Execution Layer beyond basic setup and use:

  • Specify Additional Spark Properties
    You can define additional Spark properties within the application.properties file or as run modification parameters within a transformation.
  • Configuring AEL with Spark in a Secure Cluster
    If your AEL daemon server and your cluster machines are in a secure environment like a data center, you may only want to configure a secure connection between the PDI client and the AEL daemon server.

Troubleshooting

See our list of common problems and resolutions.