Skip to main content

Environment

Env specific variables

Starlake allows you to use variables almost everywhere in the domain and job files. For example, you may need to set the folder name to watch to a different value in development and production environments. This is where variables may help. They are enclosed inside ${} or {{}}

Assuming we have a sales domain as follows:

load:
name: "sales"
directory: "{{root_path}}/sales"
ack: "ack"

We create a file env.DEV.sl.yml in the metadata folder

env:
root_path: "/tmp/userguide"

and the file env.PROD.sl.yml in the metadata folder

env:
root_path: "/cluster/userguide"

To apply the substitution in the DEV env set the SL_ENV variable before running Starlake as follows:

export SL_ENV=DEV

In Production set it rather to:

export SL_ENV=PROD

Global Variables

To define variables across environments, declare them in the env.sl.yml file in the metadata folder.

Global variables definitions may be superseded by the env specific variables files.

Predefined variables

The following variables are preset to the current date/time and may be used anywhere:

Date Time VariableFormat
sl_dateyyyyMMdd
sl_datetimeyyyyMMddHHmmss
sl_yearyy
sl_monthMM
sl_daydd
sl_hourHH
sl_minutemm
sl_secondss
sl_milliSSS
sl_epoch_secondNumber of seconds since 1/1/1970
sl_epoch_milliNumber of milliseconds since 1/1/1970