AEL logging
You can view logging information for transformations and jobs which are executed on the Adaptive Execution Layer (AEL). AEL logging provides information on details such as a step’s tuning options, global settings, and what cluster the step ran on. AEL logging uses the Spring Boot framework which can be customized for your environment. For instructions on customizing the Spring Boot framework, see the Spring Boot Reference Documentation.
Activating AEL logging
Procedure
Log on to the cluster and stop the AEL daemon as described in Step 6 of Configure the AEL daemon for YARN mode.
Navigate to the design-tools/data-integration/adaptive-execution/config directory and open the application.properties file.
Locate the following line of code:
logging.config=./config/logback.xml
Add comment tags for this line of code.
Save and close the file.
Restart the AEL daemon as described in Step 6 of Configure the AEL daemon for YARN mode
Configuring AEL logging
When a log file reaches its maximum size, a new log file is created and the logging data is written to the new file. When the maximum number of log files is reached, the oldest log file is deleted and a new one is created.
The maximum log size is 10MB by default and the maximum number of stored logs is 10 by default. You can modify the log rotation parameters in the logback.xml file.
Modify the XML file
Procedure
1. Log on to the cluster and stop the AEL daemon as described in Step 6 of Configure the AEL daemon for YARN mode.
2. Navigate to the design-tools/data-integration/adaptive-execution/config directory and open the logback.xml file.
3. Edit the following settings:
maxFileSize
The rotation file size. The maximum size of each log file is 10MB (default).
minIndex and maxIndex
The number of log files in use. The maximum number of log files is 10 (default). The minimum number of log files is 0.
5. Save and close the file.
Restart the AEL daemon as described in Step 6 of Configure the AEL daemon for YARN mode