streamsets/transformer repository overview

⁠StreamSets Transformer using Docker

⁠Basic Usage

docker run --restart on-failure -p 19630:19630 -d --name streamsets-transformer streamsets/transformer

The default login is: admin / admin.

⁠Detailed Usage

• You can specify a custom config by mounting them as a volume to /etc/transformer or /etc/transformer/
• Configuration properties in transformer.properties can also be overridden at runtime by specifying them env vars prefixed with TRANSFORMER_CONF *, For example, http.port would be set as TRANSFORMER_CONF_HTTP_PORT=12345
• You should at a minimum specify a data volume for the data directory and stage libraries. The default configured location is /data for $TRANSFORMER_DATA. You can override this location by passing a different value to the environment variable SDC_DATA. Creating a volume for additional stage libraries is described in more detail below.
• You can also specify your own explicit port mappings or arguments to the streamsets command.

For example to run with a customized sdc.properties file, a local filsystem path to store pipelines, and statically map the default UI port you could use the following:

docker run -v $PWD/sdc.properties:/etc/transformer/transformer.properties:ro -v $PWD/transformer-data:/data:rw -p 19630:19630 -d streamsets/transformer transformer

⁠Creating Data Volumes

To create a dedicated data volume for the pipeline store issue the following command:

docker volume create --name transformer-data

You can then use the -v (volume) argument to mount it when you start the transformer.

docker run -v transformer-data:/data -P -d streamsets/transformer transformer

Note: There are two different methods for managing data in Docker. The above is using data volumes which are empty when created. You can also use data containers which are derived from an image. These are useful when you want to modify and persist a path starting with existing files from a base container, such as for configuration files. We'll use both in the example below. See Manage data in containers⁠ for more detailed documentation.