Create and Install an Analysis Data Server on Linux with Docker

This page describes how to set up an analysis data server to manage your CodeSonar analysis files after the analysis has completed. The analysis data server will store these files and use them to service hub requests for information such as source file listings and procedure-granularity metrics.

If you are using CodeSonar SaaS, or if your hub already has suitable associated remote analysis launch daemons, you do not need a separate analysis data server.

Note that you do not necessarily need to use a Docker container for your analysis data server, even if your pipeline is running in a Docker container. The most important aim is to have a "persistent" analysis data server host that will not be destroyed when your pipeline finishes.

We also provide instructions to create and install an analysis data server on other systems:

• Linux (without Docker)
• Microsoft Windows

Prerequisites

• A CodeSonar hub.
• A suitable host machine for the Docker container.

Overview

There are three stages:

• A. Prepare the hub
• B. Prepare the host machine
• C. Create Docker images

A. Prepare the hub

1. Log in to your hub as Administrator, or as another user with administrative privileges.

2. Create a new launchd group to contain the launch daemons that you will create for your analysis data server.

   For details, see the manual: Settings > Other Links > Analysis Cloud > Create New Launchd Group.

   The steps below will assume that the launchd group path name is '/analysis-data-server'.

3. Create a hub user account that you can use to run the launch daemons and associated pipeline jobs. Make sure the user has sufficient permissions: with factory settings, it is sufficient to assign the User and Enabled roles

   These instructions will assume that the hub user name is cshub_ci.

B. Prepare the host machine

1. If you have not already done so, identify a suitable host machine for the analysis data server container.

   • This can be a physical machine or a virtual machine.
   • It must have Docker installed.
   • It will not need to listen on any ports, and does not need a stable host name.
   • It will make outgoing network connections to your CodeSonar hub.
   • It will need plenty of disk space where you can store analysis data files.

2. Log in to the host machine as root.

3. If Docker Engine is not already installed, use your package manager to install it.

   See: https://docs.docker.com/engine/install/#server for more information

4. Create a new analysis data service user account. This account will run the CodeSonar launch daemon process (don't run the launch daemon as root).

   These instructions will assume that the CI user name is csonar_ci.

5. Ensure you have a user that can build and run Docker containers (don't run the Docker containers as root).

   If necessary, create another new user that has permission to execute Docker commands.

   These instructions will assume that the Docker user name is docker_ci.

6. Create a base directory for saving analysis data.

   Analysis data can take plenty of space depending on the size of your code. When you create the CodeSonar launch daemon you will provide a "quota" to help manage disk space use.

   The directory should be owned by the analysis data user account you previously created (csonar_ci).

   CI_USER=csonar_ci
   CI_USER_GROUP=csonar_ci
   HOST_ANALYSIS_DATA=/srv/codesonar/analysis_data
   
   mkdir -p "$HOST_ANALYSIS_DATA"
   chown $CI_USER:$CI_USER_GROUP "$HOST_ANALYSIS_DATA"
   

7. Log in to the host machine as the Docker user (docker_ci).

8. Install the CodeSonar-GitLab Integration tools.

   We will use these tools to create the Docker container. They are not used directly by service processes.

   1. Download the CodeSonar-GitLab integration tools package (e.g. codesonar-gitlab-integration-1.6p0.tar.gz).

   2. Extract the package.

      tar -xzf /path/to/codesonar-gitlab-integration-1.6p0.tar.gz
      

      This will create a new subdirectory named like codesonar-gitlab-integration-1.6p0. The remainder of these instructions will refer to this directory as <extract-dir>

9. [HTTPS hubs only] Download a copy of your hub's hub server certificate. We will use this copy to ensure that the CodeSonar command line tools in your Docker image will trust your hub.

   • If your CodeSonar hub uses plain HTTP (and not secure HTTPS), skip this step.

   • Otherwise:

     1. Download the hub server certificate from the hub Configure HTTPS page.

        See the Troubleshooting document for additional information about how to download this file.

     2. Save the certificate to your local machine in Base-64 ASCII text format (often called "PEM" format). A typical name for saving the certificate file is "cacert.pem".

C. Create Docker images

1. Log in to the host machine as the Docker user (docker_ci).

2. Create a base Docker image containing CodeSonar.

   1. Change directory to <extract-dir>/distro-image, where <extract-dir> is the directory where you extracted the CodeSonar-GitLab integration tools package.

   2. Save a copy of the CodeSonar installer archive into <extract-dir>/distro-image. The name of the installer archive be something like codesonar-7.3p0.20230330-x86_64-pc-linux.tar.gz (version number and datestamp will vary).

   3. [HTTPS hubs only] Copy your downloaded hub server certificate to this directory.

      cp <path-to>/cacert.pem .
      

   4. Execute the following. Make any necessary changes to variable settings before executing.

      Variable	Setting
      CODESONAR_PACKAGE	The name of the CodeSonar installer archive file located in <extract-dir>/distro-image.
      CODESONAR_HUB_CACERT	For HTTPS hubs, the path to the hub server certificate that you downloaded. For HTTP hubs, leave the setting unchanged.

      CODESONAR_PACKAGE=codesonar-7.3p0.20230330-x86_64-pc-linux.tar.gz
      CODESONAR_IMAGE=codesonar_launchd
      CODESONAR_IMAGE_VERSION=latest
      CODESONAR_HUB_CACERT=cacert.pem
      BASE_IMAGE=ubuntu:18.04
      
      docker build --tag $CODESONAR_IMAGE:$CODESONAR_IMAGE_VERSION \
             --build-arg BASE_IMAGE=$BASE_IMAGE \
             --build-arg CODESONAR_PACKAGE=$CODESONAR_PACKAGE \
             --build-arg CODESONAR_HUB_CACERT=$CODESONAR_HUB_CACERT \
             --build-arg TELEMETRY=1 \
             --build-arg HUB=0 \
             --build-arg JAVA_ANALYSIS=0 \
             --build-arg PYTHON_ANALYSIS=0 \
             --build-arg ECLIPSE=0 \
             .
      

3. Create a Docker image configured for a CodeSonar launch daemon.

   We will use the CodeSonar base image and add launch daemon configuration.

   1. Change directory to <extract-dir>/examples/analysis-server.remote.docker, where <extract-dir> is the directory where you installed the CodeSonar-GitLab integration tools package.

   2. Check the directory contents. The following files should be present.

      • Dockerfile: an image built from this Dockerfile will execute the run-cslaunchd.sh script.
      • run-cslaunchd.sh

   3. Execute the following. Make any necessary changes to variable settings before executing.

      Variable	Setting
      CI_USER	The username of the analysis data service user account that you created in part B.

      ANALYSIS_SERVER_IMAGE=codesonar_remote_analysis_server
      ANALYSIS_SERVER_IMAGE_VERSION=latest
      CI_USER=csonar_ci
      CI_USER_HOME=/home/$CI_USER
      CI_USER_UID=$(id -u $CI_USER)
      CI_USER_GID=$(id -g $CI_USER)
      CI_USER_GROUP=$(id -ng $CI_USER)
      CODESONAR_IMAGE=codesonar_launchd
      CODESONAR_IMAGE_VERSION=latest
      
      docker image build \
          --build-arg BASE_IMAGE=$CODESONAR_IMAGE:$CODESONAR_IMAGE_VERSION \
          --build-arg CI_USER=$CI_USER \
          --build-arg CI_USER_HOME=$CI_USER_HOME \
          --build-arg CI_USER_UID=$CI_USER_UID \
          --build-arg CI_USER_GID=$CI_USER_GID \
          --build-arg CI_USER_GROUP=$CI_USER_GROUP \
          -t $ANALYSIS_SERVER_IMAGE:$ANALYSIS_SERVER_IMAGE_VERSION \
          .
      

4. Set up hub credentials.

   If your hub has HTTPS enabled, we recommend using certificate-based authentication as described here. If your hub is HTTP-only, or does not permit certificate-based authentication, you will need to modify this step to set up password-based authentication instead.

   Execute the following to start a temporary Docker container that generates a hub certificate and saves it to a Docker volume. This volume will be used when starting the launch daemon container later.

   1. Set up variables. Make any necessary changes to variable settings before executing.

      Variable	Setting
      CODESONAR_HUB	The location of your CodeSonar hub (protocol://host:port).
      CODESONAR_HUBUSER	The username of the analysis data service user account that you created in part A.

      ANALYSIS_SERVER_IMAGE=codesonar_remote_analysis_server
      ANALYSIS_SERVER_IMAGE_VERSION=latest
      ANALYSIS_SERVER_USER_VOL=$ANALYSIS_SERVER_IMAGE.config
      CODESONAR_HUB=https://codesonar.example.com:7340
      CODESONAR_HUBUSER=cshub_ci
      CI_USER_HOME=/home/csonar_ci
      CS_USER_HOME=$CI_USER_HOME/cs_home
      CSONAR=/opt/codesonar
      

   2. Create the volume, if it does not exist.

      docker volume inspect $ANALYSIS_SERVER_USER_VOL  2> /dev/null || \
        docker volume create $ANALYSIS_SERVER_USER_VOL
      

   3. Start the temporary container.

      docker container run \
        -v $ANALYSIS_SERVER_USER_VOL:$CS_USER_HOME \
        --env CS_USER_HOME=$CS_USER_HOME \
        --rm \
        -it \
        $ANALYSIS_SERVER_IMAGE:$ANALYSIS_SERVER_IMAGE_VERSION \
        $CSONAR/codesonar/bin/codesonar generate-hub-cert -foruser "$CODESONAR_HUBUSER" "$CODESONAR_HUB"
      

      You will be prompted to enter a hub username and password to authorize certificate creation. This does not have to be the same as the CODESONAR_HUBUSER hub user account.

5. Start the analysis data server container.

   Execute the following. Make any necessary changes to variable settings before executing.

   Variable	Setting
   HOST_ANALYSIS_DATA	A path to a directory on your Docker host machine. This directory will be "bind mounted" into the container.
   CODESONAR_ANALYSIS_DATA	The home directory for your launch daemon: should be a subdirectory of CODESONAR_DATA. Data for each analysis managed by the launch daemon will be stored under a subdirectory of this directory. The launch daemon will be responsible for creating this directory.
   CODESONAR_ANALYSIS_DATA_MAX_MB	The maximum permitted size of CODESONAR_ANALYSIS_DATA.
   CODESONAR_HUB	The location of your CodeSonar hub (protocol://host:port).
   CODESONAR_HUBUSER	The username of the hub user account that you created in part A.
   CODESONAR_LAUNCHD_GROUP	The launchd group you created in part A.
   CODESONAR_LAUNCHD_KEY	A unique identifier to distinguish this launch daemon from any others you may wish to start on the analysis data server.
   CS_USER_HOME	This variable is recognized by CodeSonar and influences hub authentication commands. Do not use the same directory as CI_USER_HOME.
   ANALYSIS_SERVER_USER_VOL	The Docker volume where your credentials are saved.

   ANALYSIS_SERVER_IMAGE=codesonar_remote_analysis_server
   ANALYSIS_SERVER_IMAGE_VERSION=latest
   ANALYSIS_SERVER_CONTAINER=$ANALYSIS_SERVER_IMAGE
   ANALYSIS_SERVER_USER_VOL=$ANALYSIS_SERVER_IMAGE.config
   ANALYSIS_SERVER_HOSTNAME=csonar-analysis
   HOST_ANALYSIS_DATA=/srv/codesonar/analysis_data
   CODESONAR_ANALYSIS_DATA=/srv/codesonar/analysis_data
   CODESONAR_ANALYSIS_DATA_MAX_MB=100000
   CODESONAR_HUB=https://codesonar.example.com:7340
   CODESONAR_HUBUSER=cshub_ci
   CODESONAR_LAUNCHD_GROUP=/analysis-data-server
   CODESONAR_LAUNCHD_KEY=remote-archive
   CI_USER=csonar_ci
   CI_USER_HOME=/home/$CI_USER
   CS_USER_HOME=$CI_USER_HOME/cs_home
   
   docker container run \
       --env "CODESONAR_HUB=$CODESONAR_HUB" \
       --env "CODESONAR_HUBUSER=$CODESONAR_HUBUSER" \
       --env "CODESONAR_ANALYSIS_DATA=$CODESONAR_ANALYSIS_DATA" \
       --env "CODESONAR_ANALYSIS_DATA_MAX_MB=$CODESONAR_ANALYSIS_DATA_MAX_MB" \
       --env "CODESONAR_LAUNCHD_GROUP=$CODESONAR_LAUNCHD_GROUP" \
       --env "CODESONAR_LAUNCHD_KEY=$CODESONAR_LAUNCHD_KEY" \
       --env "CS_USER_HOME=$CS_USER_HOME" \
       -v "$ANALYSIS_SERVER_USER_VOL:$CS_USER_HOME" \
       -v "$HOST_ANALYSIS_DATA:$CODESONAR_ANALYSIS_DATA" \
       --hostname $ANALYSIS_SERVER_HOSTNAME \
       --name $ANALYSIS_SERVER_CONTAINER \
       --init \
       --detach \
       --restart unless-stopped \
       $ANALYSIS_SERVER_IMAGE:$ANALYSIS_SERVER_IMAGE_VERSION
   

6. Verify that your analysis data server container is running.

   Use docker container logs to show the console output from the docker container. You should see that the cslaunchd process was started successfully, and you should not see errors.

   For example:

   $ docker container logs $ANALYSIS_SERVER_CONTAINER
   codesonar: Authenticating with https://codesonar.example.com:443 (password)...ok
   codesonar: Running cslaunchd in foreground...
   

Next Steps

If you are setting up a GitLab pipeline to perform CodeSonar analysis, go on to step C to create and install a pipeline build runner.

Notes

Disk management

The data stored on your analysis data server may reach the maximum permitted size you specified when you started the server.

You can delete unwanted analysis data manually from the CodeSonar GUI. You can also configure your hub to automatically delete data from older analyses, using custom criteria to define "older".

See the CodeSonar manual for more information:

• Using CodeSonar > GUI Reference > Analysis Auto-Deletion and Analysis Log Auto-Deletion
• Using CodeSonar > GUI Reference > Analysis

Modifications for password authentication

If you need to authenticate your launch daemon with a password instead of with a certificate, make the following changes.

1. When you set up the temporary container to produce and store credentials, replace the line that begins

   $CSONAR/codesonar/bin/codesonar generate-hub-cert [...]
   

   with the following command:

   $CSONAR/codesonar/bin/codesonar generate_hubpwfile.py "$CS_USER_HOME/hubpwfile"
   

2. When you start the temporary container provide the hub user account password when prompted. The password will be saved in file $CS_USER_HOME/hubpwfile in the Docker volume. Make sure only your Docker user (docker_ci) has access to this file.

---