Propelo Ingestion Satellite

This documentation will give step-by-step guidelines for installing the Propelo Ingestion Satellite.
In the Propelo UI, go to Settings​ > Integrations. Choose the application (e.g. JIRA) you are interested in integrating with and click Install. ​If you don’t want to integrate with any particular applications but still want to deploy the satellite to automate processes within your infrastructure, you can select “Custom Application”. 
​Note: Satellites are only needed for integrating on-premise systems and applications.
On the Credentials page
Enable “Satellite Integration”.
Enter your credentials. They will be used to generate the Satellite Configuration file. Propelo ​does not​ store satellite credentials.
Note: For on-prem JIRA, you may need to enter the Username and Password. For cloud JIRA, you may need to use email and JIRA APIKey.
After the new integration is added successfully, you will be prompted to download the satellite’s configuration file: ​satellite.yml.
The recommended memory for one container is between 4GB to 6GB.
Multiple containers can be deployed for the same integration without any issue, using the same configuration file.
Run the Satellite container
Run with Docker
Download and install Docker Desktop
Mac: https://download.docker.com/mac/stable/Docker.dmg​
Windows: https://download.docker.com/win/stable/Docker%20Desktop%20Installer.exe​
Linux (Ubuntu): https://docs.docker.com/install/linux/docker-ce/ubuntu/​
Note: On Linux, make sure to add your user to the Docker group. This will let you run Docker commands without having to use root or sudo.
sudo usermod -aG docker $(whoami)
You will need to log out and log back in for the change to take effect.
Pull the latest version of the container.
docker pull levelops/ingestion-satellite
Note: If you are unable to execute the docker pull command, you can download the image from here and install it manually. 
Run the satellite container in the foreground.
docker run \
  -v /absolute/path/to/satellite.yml:/levelops/config.yml \
  levelops/ingestion-satellite
If everything is working, run the container in the background by adding -d --restart unless-stopped.
docker run -d --restart unless-stopped \
  -v /absolute/path/to/satellite.yml:/levelops/config.yml \
  levelops/ingestion-satellite
Optionally, add --name levelops to give the container a name. Use a unique name per container if you are running more than one.
If the satellite is not able to find the config file (errors mentioning ${levelops.api_key} for example):

Older versions of Docker may run into issues when mounting single files. To work around that, rename the config file to config.yml, then move the file into an empty folder. On Linux, check that your user owns both the folder and file, not root, and that permissions are not too restrictive (chmod 755).

Run the command again, but this time the parent folder will be used instead:
docker run \
  -v /absolute/path/to/parent/folder:/levelops/config \
  -e CONFIG_FILE=/levelops/config/config.yml \
  levelops/ingestion-satellite
If this fixes the issue, add -d --restart unless-stopped to run the container in the background.
Manage background containers.
# list containers
docker container ls
​
# stop or restart container with ID
docker container stop ​<ID>
docker container restart ​<ID>
​
# delete container by id
docker rm ​<id>
Run with Kubernetes
Prepare the Kubernetes deployment using the following template.
You may want to change the namespace to something else.
The last section, ​data.config.yml,​ comes from the ​satellite.yml ​file downloaded after installing the custom integration. If copy-pasting, make sure to fix the indentation.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: levelops-satellite
subjects:
  - kind: ServiceAccount
    name: levelops-satellite
    namespace: default
    apiGroup: ""
roleRef:
  kind: ClusterRole
  name: edit
  apiGroup: rbac.authorization.k8s.io
apiVersion: v1
kind: ServiceAccount
metadata:
  name: levelops-satellite
  namespace: default
  labels:
    app: levelops
apiVersion: apps/v1
kind: Deployment
metadata:
  name: levelops-satellite
  namespace: default
spec:
  selector:
    matchLabels:
    app: levelops-satellite
template:
  metadata:
    labels:
      app: levelops-satellite
    spec:
      serviceAccountName: levelops-satellite
      containers:
        - name: levelops-satellite
        image: levelops/ingestion-satellite
        imagePullPolicy: Always
        resources:
          limits:
          memory: "5Gi"
          cpu: "1000m"
        volumeMounts:
          - name: config-volume
            mountPath: /levelops/config.yml
            subPath: config.yml
      volumes:
      - name: config-volume
      configMap:name: levelops-satellite-config
apiVersion: v1
kind: ConfigMap
metadata:
  name: levelops-satellite-config
  namespace: default
data:
  config.yml: |
    levelops:
      tenant: <TENANT>
      api_key: <API_KEY>
    integrations:
      - id: '<ID>'
      application: custom
Apply the deployment to your cluster.
kubectl apply --namespace=your-name-space -f deployment.yml
Make sure the satellite is running correctly. You should see heartbeats being sent.
kubectl get pods --namespace=your-name-space # to find pod
kubectl logs <satellite pod> -f --namespace=your-name-space
Since the deployment is configured with: ​imagePullPolicy: Always, you will get the latest updates when the pod is restarted.
kubectl rollout restart deployment levelops-satellite
Steps to Update the Satellite
If you are using Docker
# pull latest version
docker pull levelops/ingestion-satellite
​
# list containers and find the current satellite id
docker container ls
​
# stop and remove the old container (replace <id> accordingly)
docker container stop ​<id>
docker rm ​<id>
​
# ​run the container again - it will use the latest version
docker run -v   /local/path/to/satellite.yml​:/levelops/config.yml   -d levelops/ingestion-satellite
Note: If you are unable to execute the docker pull command, you can download the image from here and install it manually. 
If you are using Kubernetes
# find the name of the satellite deployment
kubectl get deployments
​
# restart the satellite deployment
kubectl rollout restart deployment ​<deployment-name>
​
# ​if you didn’t change the default values
kubectl rollout restart deployment levelops-satellite
How to encrypt the YAML configuration file
The satellite supports reading configuration files encrypted with AES-256. This way, the data at rest is not stored in clear. You will need to provide an environment variable with the encryption password.
Encrypt the configuration using the following command. There will not be any prompt, but you will need to enter your password in the terminal, then press enter. Adjust the following 2 parameters to your environment: the input path (/absolute/path/to/input/satellite.yml) and the output path (/path/to/output/satellite.yml.enc).
docker run -i --rm -v /absolute/path/to/input/satellite.yml:/levelops/input \
  --entrypoint /bin/bash levelops/ingestion-satellite \
  -c 'java -cp /levelops/satellite-agent.jar -Dloader.main=io.levelops.ingestion.agent.Encrypt org.springframework.boot.loader.PropertiesLauncher input' \
  > /path/to/output/satellite.yml.enc
Note: When entering your password, it will appear in clear, but it will not be stored.
If you need to make changes later on, you can decrypt the configuration file using the following command. Similarly, you will need to enter your password then press enter. Remember to adjust the following 2 parameters to your environment: the input path (/absolute/path/to/input/satellite.yml.enc) and the output path (/path/to/output/satellite.yml).
docker run -i --rm -v /absolute/path/to/input/satellite.yml.enc:/levelops/input \
  --entrypoint /bin/bash levelops/ingestion-satellite \
  -c 'java -cp /levelops/satellite-agent.jar -Dloader.main=io.levelops.ingestion.agent.Decrypt org.springframework.boot.loader.PropertiesLauncher input' \
  > /path/to/output/satellite.yml
Run the satellite using the encrypted configuration file:
export ENCRYPTION_PASSWORD="<YOUR PASSWORD>"; docker run -d --restart unless-stopped \
  -v /absolute/path/to/satellite.yml.enc:/levelops/config.yml \
  --env ENCRYPTION_PASSWORD \
  levelops/ingestion-satellite
How to connect with a Proxy
The satellite can send its traffic to a proxy, with or without authentication. The configuration is done in the satellite.yml config file, inside the levelops block:
levelops:
  ...
  proxy:
    host: <YOUR PROXY IP OR DOMAIN>
    port: <YOUR PROXY PORT>
To authenticate with the proxy, specify the username and password:
levelops:
  ...
  proxy:
    host: <YOUR PROXY IP OR DOMAIN>
    port: <YOUR PROXY PORT>
    username: <YOUR USERNAME>
    password: <YOUR PASSWORD>
Note 1: Do not include any scheme in the proxy host, just the domain. (e.g. https:// should not be included)
Note 2: By default, only the traffic to Propelo is sent to the proxy, meaning that the connections to your internal integrations are not. If you wish to proxy all traffic, you can set this option inside the proxy block: all_traffic: true
Note 3: If the proxy uses https, then the connection will only work with a valid SSL certificate. If you are using a self-signed certificate, it will have to be supplied to the satellite.
Last modified 15d ago