Skip to main content
Pentaho Documentation

Configuring AEL with Spark in a secure cluster

Parent article

The AEL daemon works in an unsecured cluster by default. You can secure communication channels between the PDI client and the AEL daemon server and also between the AEL daemon server and the Spark driver using SSL (Secure Sockets Layer), Kerberos, or both. If your AEL daemon server and your cluster machines are in a secure environment like a data center, you may only want to configure a secure connection between the PDI client and the AEL daemon server.

Authentication with Kerberos

To enable security, you can configure the AEL daemon to work in a secure cluster using impersonation. Kerberos authentication can be used with AEL in two ways: with the connection from the client to the AEL daemon and with the Spark submit process.

Setup a secure client connection

Complete the following steps to set up a secure connection from the PDI client to the AEL daemon:

Procedure

  1. Download and install Kerberos server. Refer to Set Up Kerberos for Pentaho for further details on installing the Kerberos server.

  2. Create a keytab and principal to use for your client access.

  3. Open the PDI client and choose Edit Edit the kettle.properties file.

  4. Add the properties KETTLE_AEL_PDI_DAEMON_KEYTAB and KETTLE_AEL_PDI_DAEMON_PRINCIPAL and set the values to the location of the keytab and principal, respectively.

  5. Restart the PDI client.

Setup a secure server connection

Complete the following steps to set up a secure connection from the AEL daemon to the cluster:

Procedure

  1. Create a keytab and server principal to use for your server access.

  2. Navigate to the adaptive-execution/config/application.properties file and open it with a text editor. Set the values for your environment as in the following table:

    ParameterValue
    keytabLocationPath to the keytab used for the Kerberos principal.
    kerberosPrincipalPath to the Kerberos service principal that has the authority to impersonate another user.
    disableProxyUserThe AEL daemon can impersonate a proxy user when authenticating to your secure cluster. Set to true to disable the proxy user. The acting user will then be the Kerberos service principal. The default value is false.

Next steps

You can now test your AEL configuration by creating a run configuration using the Spark engine. Refer to Run configurations for more details.

Using SSL encryption

Complete the following steps to set up SSL connections for the PDI client and the Pentaho Server:

Procedure

  1. Set up SSL security by following the instructions in the article Enable SSL in the Pentaho Server with a certificate authority.

  2. Import your certificate to the Java keystore on the machine where the PDI client is installed. If the Pentaho Server is installed on a different machine, import the certificate to the Java keystore on that machine.

  3. At the following prompts, enter a new password and enter Y:

    Enter keystore password: 
    Trust this certificate?

Results

The certificate is now trusted by the PDI client and the Pentaho Server.

Configure the daemon for SSL

Complete the following to configure the AEL daemon for SSL:

Procedure

  1. Navigate to the adaptive-execution/config/application.properties file and open it with a text editor.

  2. Set the values for your environment as in the following table:

    ParameterValue
    websocketURLThe fully-qualified domain name of the node where the AEL daemon is installed. For example,

    websocketURL=wss://localhost:${ael.ssl.port}

    ael.ssl.enabledtrue
    ael.ssl.key-store/users/myusername/pentaho/mycertificate.p12
    ael.ssl.key-store-typePKCS12
    ael.ssl.key-store-passwordThe SSL keystore password. This must be set to your keystore password.
    ael.ssl.key-passwordThe SSL key password. This must be set to your key password.

Results

You can now test your AEL configuration by creating a run configuration using the Spark engine. Refer to Run configurations for more details.

Next steps

The first time you start the AEL daemon, it will prompt you to enter the SSL keystore and key passwords.