Skip to main content

Pentaho+ documentation has moved!

The new product documentation portal is here. Check it out now at docs.hitachivantara.com

 

Hitachi Vantara Lumada and Pentaho Documentation

MQTT Consumer

Parent article

The PDI client can pull streaming data from an MQTT broker or clients through an MQTT transformation. The parent MQTT Consumer step runs a child transformation that executes according to the message batch size or duration, allowing you to process a continuous stream of records in near real-time. The child transformation must start with the Get records from stream step.

Additionally, from the MQTT Consumer step, you can select a step in the child transformation to stream records back to the parent transformation. This capability allows records processed by an MQTT Consumer step in a parent transformation to be passed downstream to any other steps included within the same parent transformation.

Select an engine

You can run the MQTT Consumer on the Pentaho engine or on the Spark engine. Depending on your selected engine, the transformation will run differently. Select one of the following options to view how to set up the MQTT Consumer step for your selected engine.

For instructions on selecting an engine from your transformation, see Run configurations.