axibase/atsd-sandbox repository overview

⁠ATSD Sandbox Docker Image

⁠Overview

The ATSD Sandbox Docker Image contains Axibase Time Series Database⁠, Axibase Collector⁠, and companion tools.

Collector is pre-configured to send data into the local ATSD instance.

⁠Image Contents

• Axibase Time Series Database⁠
• Axibase Collector⁠
• collectd⁠
• Python, version 3
• ATSD API Python Client⁠

⁠Launch Instructions

⁠Run Container

docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 axibase/atsd-sandbox:latest

Watch the start log for the All applications started message.

docker logs -f atsd-sandbox

⁠Docker Statistics

To collect container statistics from the Docker host, start the container with mounted /var/run/docker.sock.

The Collector automatically starts the Docker Job⁠ if the docker run command mounts docker.sock into the container.

docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  axibase/atsd-sandbox:latest

⁠Exposed Ports

• Port 8443: ATSD web interface at https://docker_host:8443/
• Port 9443: Axibase Collector web interface at https://docker_host:9443/.
• Port 8081: ATSD network commands⁠ receiver.

⁠Default Credentials

Default user credentials are shown below:

• Username axibase
• Password axibase

Open the user account page in ATSD by clicking on the account icon in the upper-right corner of the screen to modify credentials after intial login.

⁠Container Parameters

⁠Path Formats

This path format is used in ATSD_IMPORT_PATH, COLLECTOR_IMPORT_PATH, COLLECTOR_CONFIG and EMAIL_CONFIG variables

1. URL address of the file:

   docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
     --name=atsd-sandbox \
     --volume /var/run/docker.sock:/var/run/docker.sock \
     --env ATSD_IMPORT_PATH='https://example.org/atsd-marathon-xml.zip' \
     --env COLLECTOR_IMPORT_PATH='https://example.org/marathon-jobs.xml' \
     axibase/atsd-sandbox:latest
   

   Copy

2. Absolute path on the container file system to the file:

   docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
    --name=atsd-sandbox \
    --volume /var/run/docker.sock:/var/run/docker.sock \
    --volume /home/user/atsd-marathon-xml.zip:/atsd-marathon-xml.zip \
    --volume /home/user/marathon-jobs.xml:/marathon-jobs.xml \
    --env ATSD_IMPORT_PATH='/atsd-marathon-xml.zip' \
    --env COLLECTOR_IMPORT_PATH='/marathon-jobs.xml' \
    axibase/atsd-sandbox:latest
   

   Copy

3. Relative path to the file. In this case the file is in the /import directory on the container file system.

   mkdir /home/user/import
   cp atsd-marathon-xml.zip /home/user/import
   cp marathon-jobs.xml /home/user/import
   docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
     --name=atsd-sandbox \
     --volume /var/run/docker.sock:/var/run/docker.sock \
     --volume /home/user/import:/import \
     --env ATSD_IMPORT_PATH='atsd-marathon-xml.zip' \
     --env COLLECTOR_IMPORT_PATH='marathon-jobs.xml' \
     axibase/atsd-sandbox:latest
   

   Copy

⁠Configuration File Import

⁠Import Parameters

ATSD_IMPORT_PATH and COLLECTOR_IMPORT_PATH variables must be specified using the following format: path_1,path_2,...,path_N where each path can refer to either an XML file or zip/tar.gz archive. See Path Formats⁠.

⁠Variable Substitution

Environment variable substitution is performed in each of the imported files. Placeholders must be identified with ${ENV.NAME} format, where NAME is the name of the environment variable.

The launch command declares a variable NAMESPACE and the imported jobs.xml file contains a corresponding placeholder ${ENV.NAMESPACE}.

 --env NAMESPACE=Axibase \
 --env COLLECTOR_IMPORT_PATH=jobs.xml \

<attr>${ENV.NAMESPACE}</attr>

The imported file is modified by substituting ${ENV.NAMESPACE} with the value of the NAMESPACE variable:

<attr>Axibase</attr>

To ensure that the XML file remains valid after the variable substitution, wrap the placeholder with CDATA.

<attr><![CDATA[${ENV.NAMESPACE}]]></attr>

If no corresponding environment variable is defined for a placeholder specified in the file, the placeholder remains unchanged.

⁠Server URL

The SERVER_URL variable determines the URL, including the protocol, the DNS name and port, at which the database is accessible.

Usage example:

docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
  --env SERVER_URL=https://atsd.example.org:8443 \
  axibase/atsd-sandbox:latest

⁠Incoming Webhooks

WEBHOOK variable specifies a list of templates for creating incoming webhook⁠ URLs.

The list of available templates:

• aws-cw
• github
• jenkins
• slack
• telegram

Example:

docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
  --name=atsd-sandbox \
  --env SERVER_URL=https://atsd.example.org:8443 \
  --env WEBHOOK=github,telegram \
  axibase/atsd-sandbox:latest

Generated URLs, including user credentials, are displayed in the start log:

...
Webhook user: github
Webhook URL: https://github:[email protected]:8443/api/v1/messages/webhook/github?exclude=organization.*;repository.*;*.signature;*.payload;*.sha;*.ref;*_at;*.id&include=repository.name;repository.full_name&header.tag.event=X-GitHub-Event&excludeValues=http*&debug=true
...
Webhook user: telegram
Webhook URL: https://telegram:[email protected]:8443/api/v1/messages/webhook/telegram?command.message=message.text

⁠Outgoing Webhooks

The following environment variables configure outgoing webhooks for Slack and Telegram notifications.

Example:

docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081
  --name=atsd-sandbox \
  --env SLACK_TOKEN=xoxb-265141031111-QIw323gJUWaX5Fl311111111
  axibase/atsd-sandbox:latest

Alternatively, configuration files can be used instead of environment variables.

SLACK_CONFIG and TELEGRAM_CONFIG variables specify path to the files with configuration parameters for Slack⁠ and Telegram⁠ respectively. File format is the same as for EMAIL_CONFIG variable.

• Slack:

  Configuration properties for SLACK_CONFIG:

  Property	Variable
  token	SLACK_TOKEN
  channels	SLACK_CHANNELS

  Contents of /home/user/import/slack.properties file:

  token=xoxb-************-************************
  channels=general,devops
  

  Copy

  Container run command:

  docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
    --volume /home/user/import:/import
    --name=atsd-sandbox \
    --env SLACK_CONFIG=slack.properties
    axibase/atsd-sandbox:latest
  

  Copy

• Telegram

  Configuration properties for TELEGRAM_CONFIG:

  Property	Variable
  bot_token	TELEGRAM_BOT_TOKEN
  chat_id	TELEGRAM_CHAT_ID

  bot_token=*********:***********************************
  chat_id=-NNNNNNNNN
  

  Copy

⁠Outgoing Webhook Tests

Configured web notifications are tested on initial start by sending a test message. Test results are logged in the start log.

• Successful test

  [ATSD] Configure Slack Web Notifications.
  [ATSD]   Slack Web Notification test OK.
  

  Copy

• Sample test messages

  

  

• Failed test

  [ATSD] Configure Slack Web Notifications.
  [ATSD]   Slack Web Notification test failed.
  [ATSD]   Error: {"ok":false,"error":"invalid_auth"}
  

  Copy

⁠Mail Client Configuration

EMAIL_CONFIG variable specifies the file to read Mail Client configuration from. See path formats⁠.

The file contains configuration entries in property_name=value format.

Supported configuration parameters.

⁠Usage example

• Contents of the file /home/user/import/mail.properties

  server=mail.example.org
  [email protected]
  password=********
  

  Copy

• Container run command

  docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
    --name=atsd-sandbox \
    --volume /home/user/import:/import \
    --env EMAIL_CONFIG=mail.properties \
    axibase/atsd-sandbox:latest
  

  Copy

• Delivered test message

  

• Start log message if no errors occur and test is successful

  [ATSD] Mail Client test successful.
  

  Copy

  Start log message on incomplete configuration

  [ATSD] Error: empty user mail configuration field. Mail client won't be enabled.
  

  Copy

  Start log message when test fails

  [ATSD] Mail Client test failed: Invalid email address
  

  Copy

⁠Job Configuration Parameters

COLLECTOR_CONFIG is the semicolon-separated sequence of instructions to edit configuration files imported into Collector.

Specify each instruction in the format file_name.xml:/path/to/properties_file or file_name.xml:key1=value1,key2=value2. This updates the attributes in the XML file with new values prior to importing the file into Collector.

Instructions can be specified as follows:

1. A path to a file on the container file system (see path formats⁠), that contains key=value lines:

   docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
     --name=atsd-sandbox \
     --volume /home/user/aws.conf:/aws.conf \
     --env ATSD_IMPORT_PATH='https://github.com/axibase/atsd-use-cases/raw/master/how-to/aws/route53-health-checks/resources/aws-route53-xml.zip' \
     --env COLLECTOR_IMPORT_PATH='https://raw.githubusercontent.com/axibase/atsd-use-cases/master/how-to/aws/route53-health-checks/resources/job_aws_aws-route53.xml' \
     --env COLLECTOR_CONFIG="job_aws_aws-route53.xml:/aws.conf" \
   axibase/atsd-sandbox:latest
   

   Copy

   cat /home/user/aws.conf
   

   Copy

   accessKeyId=key
   secretAccessKey=secret
   

   Copy

2. A key-value pair in key=value format:

   docker run -d -p 8443:8443 -p 9443:9443 -p 8081:8081 \
     --name=atsd-sandbox \
     --volume /var/run/docker.sock:/var/run/docker.sock \
     --env ATSD_IMPORT_PATH='https://raw.githubusercontent.com/axibase/atsd-use-cases/master/how-to/marathon/capacity-and-usage/resources/atsd-marathon-xml.zip' \
     --env COLLECTOR_IMPORT_PATH='https://raw.githubusercontent.com/axibase/atsd-use-cases/master/how-to/marathon/capacity-and-usage/resources/marathon-jobs.xml' \
     --env COLLECTOR_CONFIG='marathon-jobs.xml:server=marathon_hostname,port=8080,userName=my-user,password=my-password' \
     axibase/atsd-sandbox:latest
   

   Copy

XML file update replaces of XML tag values, identified as key, with new values:

--env COLLECTOR_CONFIG='marathon-jobs.xml:server=mar1.example.com,userName=netops,password=1234456'

Before:

<server>marathon_hostname</server>
<port>8080</port>
<userName>axibase</userName>
<password></password>

After:

<server>mar1.example.com</server>
<port>8080</port>
<userName>netops</userName>
<password>1234456</password>  

⁠Collector Start Control

START_COLLECTOR variable enables or disables Collector start.

• on and true values enable start, and are set by default.
• off and false disable start.

--env COLLECTOR_CONFIG=off

This setting applies to the first and all subsequent container starts.

⁠Parameters Syntax

Variables cannot contain whitespace characters.

Escape semicolons and commas in file names, URLs, key and values via backslash \ per specifications below:

Variables WEBHOOK, SERVER_URL, EMAIL_CONFIG do not require special escaping for , and ;.

--env COLLECTOR_CONFIG='config.xml:password=password\,with\;separators'

Some consoles require additional character escape based on type and version.

⁠Build Sandbox Manually

git clone https://github.com/axibase/dockers.git

cd dockers

git branch -a

git checkout atsd-sandbox

docker build --rm -t axibase/atsd-sandbox .

docker save -o atsd-sandbox.tar axibase/atsd-sandbox:latest

gzip atsd-sandbox.tar

Check that tar.gz file is created.

ls -lah *gz

-rw-------  1 axibase  axibase   742M Jul  1 13:52 atsd-sandbox.tar.gz