CodeSonar GitLab CI/CD Integration

CodeSonar can be used in a GitLab CI/CD pipeline to perform static analysis of project source code. GitLab provides this capability through its Static Application Security Testing (SAST) feature. GitLab can display the results of static analysis associated with your pipeline jobs. This feature is made possible by providing GitLab with a specially formatted SAST report artifact after a static analysis pipeline job completes.

The CodeSonar GitLab Integration package provides a utility for representing CodeSonar analysis results in GitLab's SAST report format. The package also provides some templates for setting-up your CI/CD pipelines to use CodeSonar, which are intended to make it easier for you to get started.

A typical way to use a GitLab CI/CD pipeline is to arrange for it to be executed whenever new Git commits are submitted to a Merge Request. When you add CodeSonar static analysis to your merge request pipeline, GitLab will display the new analysis warnings that are introduced by your merge request on the merge request page. The full set of warnings is always available on the pipeline page.

Contents

Prerequisites

  1. Access to a GitLab Ultimate edition website instance. GitLab provides a 30-day free trial of their Ultimate edition license. Instructions for setting up a GitLab instance can be found at https://about.gitlab.com/install.

  2. A source code project in your GitLab instance that you wish to analyze. You must be able to setup a GitLab CI/CD pipeline for your project that can build your source code successfully. This will include configuration of one or more GitLab pipeline job Runners.

  3. If using Docker, Docker Engine version 19.03.12 or later.

  4. CodeSonar version 5.4 or later. If using Docker, then you must have a copy of the CodeSonar software install package for Linux. CodeSonar supports only glibc-based Linux containers (i.e. not Alpine Linux). Windows containers are not supported by CodeSonar. If not using Docker, then you will need to install CodeSonar directly on your GitLab Runner machine.

  5. A dedicated, "persistent" CodeSonar hub to coordinate and receive the results of your analysis. See your CodeSonar manual for how to setup and license a hub.

  6. A valid CodeSonar hub license appropriate to your configuration.

  7. A copy of the CodeSonar GitLab Integration software package.

Walkthrough using Linux with Docker

The following section provides a step-by-step walkthrough for configuring a pipeline to use CodeSonar on Linux with Docker. Some tips for executing pipeline jobs on Microsoft Windows are provided later in this document.

I: Basic Docker pipeline setup before adding CodeSonar

The following steps will configure a pipeline that will build your code.

1. Create a GitLab project and push your source code to the projects repository on your GitLab server.

For more information see: https://docs.gitlab.com/ee/user/project/

2. Configure one or more GitLab pipeline job runners to run pipeline jobs using Docker.

It is recommended that you add tags to your runner named "Linux" and "Docker". These tags help distinguish Linux runners that can create Docker containers from other runners. The example pipeline configurations used later in this document will assume that runners exist with these tags.

For information about GitLab CI/CD see: https://docs.gitlab.com/ee/ci/

For information about GitLab pipeline job runners see: https://docs.gitlab.com/ee/ci/runners/

3. Create a base Docker image that has all the necessary requirements to build your source code.

For information about using Docker in your GitLab pipeline see: https://docs.gitlab.com/ee/ci/docker/

An example Dockerfile for building a simple C/C++ project looks like:

FROM ubuntu:18.04
RUN apt-get update && apt-get install -y gcc make

4. Build, Tag, and Push your Docker image to your GitLab project's container registry.

For information about GitLab container registries see: https://docs.gitlab.com/ee/user/packages/container_registry/

You'll need to know the URL for your GitLab project's container registry. You can find the container registry URL in the GitLab UI for your project under the "Packages & Registries" menu. Usually, the URL takes the form: HOST/GROUP/PROJECT. Example commands to build, tag, and push your image to the container registry look like:

CI_REGISTRY=gitlab.yourdomain.tld:5050
CI_REGISTRY_IMAGE=$CI_REGISTRY/group/project
CI_IMAGE_VERSION=latest
docker login $CI_REGISTRY
docker build -t $CI_REGISTRY_IMAGE:$CI_IMAGE_VERSION - < Dockerfile
docker push $CI_REGISTRY_IMAGE:$CI_IMAGE_VERSION

5. Configure a CI/CD pipeline that can build your code in a Docker pipeline job using your base Docker image.

This involves creating a GitLab .gitlab-ci.yml file with instructions to build your source code. This file must be saved in the root of your repository.

For information about the .gitlab-ci.yml file see: https://docs.gitlab.com/ee/ci/yaml/gitlab_ci_yaml.html

If you are not familiar with YAML format, you may want to read about it at: https://yaml.org/spec/1.2/spec.html#Preview

Be sure to configure the pipeline to execute for Merge Requests. You can use the workflow:rules keywords for this. See: https://docs.gitlab.com/ee/ci/merge_request_pipelines/ and see the additional comments in "Addendum A" further below.

An example .gitlab-ci.yml file file looks like:

workflow:
  rules:
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

stages:
  - build

build_job:
  stage: build
  tags:
    - Linux
    - Docker
  image: $CI_REGISTRY_IMAGE:latest
  script:
    - ./configure
    - make all

6. Commit and push your .gitlab-ci.yml file to your GitLab project's repository.

For example:

git checkout -b basic_ci
git add .gitlab-ci.yml
git commit -m "Add gitlab-ci configuration"
git push -u origin basic_ci

7. Create a Merge Request for the branch you just pushed.

The git push command may print a URL to help you create a merge request. Otherwise, you can use the GitLab GUI to create a new merge request associated with your new branch.

8. Verify that your pipeline executes and builds your source code successfully from your merge request.

II: Add CodeSonar to your base Docker pipeline configuration:

With your project and pipeline configuration setup and working, we can now add CodeSonar to the pipeline with the following steps:

1. Extract the CodeSonar GitLab Integration .tar file.

2. Find the distro-image subdirectory in the extracted files.

This directory contains a Dockerfile that can be used to create a new Docker image from your project's base image with CodeSonar installed added to it.

3. Copy your CodeSonar software install package .tar file into the distro-image directory.

The Dockerfile contains instructions to extract the CodeSonar programs from the CodeSonar install package and copy them into the "/opt" directory in the Docker image. It will also set the default system $PATH environment variable in the image to include the CodeSonar commandline tools.

4. Download a copy of your CodeSonar hub HTTPS certificate.

The CodeSonar client instance running in your pipeline job may not trust your CodeSonar hub server HTTPS certificate. This may happen if your hub has a self-signed HTTPS certificate or if your HTTPS certificate is approved by a recently issued root certificate authority.

To avoid this problem, you can add a copy of your CodeSonar hub's HTTPS certificate to your CodeSonar Docker image.

One way to do this is to replace the distro-image/cacert.pem file with a copy of your HTTPS certificate in .pem format.

The .pem certificate can be downloaded from the CodeSonar hub. See the "General Tips" section later in this document for information about how to download an HTTPS certificate in PEM format.

5. Build a new CodeSonar Docker image using the Dockerfile found in the distro-image subdirectory:

This new CodeSonar Docker image will be based on your project's base Docker image.

Consult the comments in distro-image/Dockerfile for descriptions of the Docker build arguments used in this section. Many of the build arguments used here are optional and are intended to reduce the size of the Docker image.

Note also that some of the variables used here were initialized in a previous step (see above).

To build the image, modify the following commands as needed and execute:

CODESONAR_PACKAGE=codesonar-5.4p0.20200911-x86_64-pc-linux.tar.gz
CODESONAR_IMAGE=$CI_REGISTRY_IMAGE/codesonar
CODESONAR_IMAGE_VERSION=$CI_IMAGE_VERSION
docker build --tag $CODESONAR_IMAGE:$CODESONAR_IMAGE_VERSION \
    --build-arg BASE_IMAGE=$CI_REGISTRY_IMAGE:$CI_IMAGE_VERSION \
    --build-arg CODESONAR_PACKAGE=$CODESONAR_PACKAGE \
    --build-arg CODESONAR_HUB_CACERT=cacert.pem \
    --build-arg TELEMETRY=1 \
    --build-arg HUB=0 \
    --build-arg JAVA_ANALYSIS=0 \
    --build-arg PYTHON_ANALYSIS=0 \
    --build-arg JAVA_API=0 \
    --build-arg CSHARP_API=0 \
    --build-arg ECLIPSE=0 \
    --build-arg CMDLINE_UTILITIES=0 \
    --build-arg CONFIG_TOOL=0 \
    --build-arg GENERATE_CERT=0 \
    .

6. Push the new CodeSonar Docker image to your container registry:

docker login $CI_REGISTRY
docker push $CODESONAR_IMAGE:$CODESONAR_IMAGE_VERSION

7. Download a copy of your GitLab server's HTTPS root certificate.

In some cases, CodeSonar's Python interpreter will refuse to communicate with your GitLab server since it does not trust your GitLab server's HTTPS certificate. This may happen if your GitLab server uses a recently-issued certificate. In order to avoid a potential problem later, you can provide a copy of the certificate to CodeSonar in your pipeline configuration. To do this, you must download a copy of the root authority certificate for your GitLab Server's HTTPS certificate. You should save this root certificate to your repository working directory in Base-64 ASCII text format (often called "PEM" format). See the "General Tips" at the end of this document for additional information about how to download this file. We will assume you have saved the root certificate file with the name "gitlab.root.cacert".

8. Setup your CI/CD pipeline to use CodeSonar with your new Docker image.

Look in the templates directory of the CodeSonar GitLab integration directory. Find the file named codesonar.linux.docker.yml.

This file is a template which defines a GitLab pipeline job which will execute CodeSonar to analyze your code. Copy this file into your repository directory.

9. Open the .yml file that you copied to your repository in a text editor.

Note that this .yml file will make use of several custom environment variables which you must configure on your GitLab project. You may need to make several changes, including:

An example, modified codesonar.linux.docker.yml looks like:

.codesonar_sast:
  stage: test
  tags:
    - Linux
    - Docker
  image: $CI_REGISTRY_IMAGE/codesonar:latest
  variables:
    SARIF2SAST: "/opt/cso-sarif2sast"
    CODESONAR: "/opt/codesonar/codesonar/bin/codesonar"
    CSPYTHON: "/opt/codesonar/codesonar/bin/cspython"
    CODESONAR_PROJECT_NAME: ${CI_PROJECT_NAME}
    CI_SERVER_CAFILE: "gitlab.root.cacert"
  script:
    - ./configure
    - >
        $CODESONAR analyze
        -foreground
        -auth password
        -hubuser ${CODESONAR_HUBUSER}
        -hubpwfile ${CODESONAR_HUBPWFILE}
        ${CODESONAR_PROJECT_NAME}
        ${CODESONAR_HUB_URL}
        make all
        |& tee analysis.log
    - CODESONAR_ANALYSIS_ID=$(cat ${CODESONAR_PROJECT_NAME}.prj_files/aid.txt)
    - >
        $CODESONAR get
        -auth password
        -hubuser ${CODESONAR_HUBUSER}
        -hubpwfile ${CODESONAR_HUBPWFILE}
        -o allwarnings.sarif
        ${CODESONAR_HUB_URL}/analysis/${CODESONAR_ANALYSIS_ID}-allwarnings.sarif?filter=${CODESONAR_VISIBILITY_FILTER}
    - >
        $CSPYTHON $SARIF2SAST/sarif2sast.py
        --sarif allwarnings.sarif
        --output gl-sast-report.json
        --summary-report sast-summary-report.md
        --codesonar-url ${CODESONAR_HUB_URL}
        --analysis-id ${CODESONAR_ANALYSIS_ID}
        --max ${CODESONAR_MAX_WARNINGS}
        --threshold ${CODESONAR_WARNING_THRESHOLD}
  after_script:
    - >
        $CSPYTHON $SARIF2SAST/upload_gitlab_mr_notes.py
        --api-token-variable GITLAB_TOKEN
        --report sast-summary-report.md
        --cafile "${CI_SERVER_CAFILE}"
  artifacts:
    reports:
      sast: gl-sast-report.json

10. Update your .gitlab-ci.yml file to include the codesonar.linux.docker.yml file that you just added to your repository.

You will need to add instructions to invoke your new CodeSonar job definition. For example, adding the CodeSonar job to the example .gitlab-ci.yml file shown further above might look like:

workflow:
  rules:
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

include:
  - local: 'codesonar.linux.docker.yml'

stages:
  - build
  - test

build_job:
  stage: build
  tags:
    - Linux
    - Docker
  image: $CI_REGISTRY_IMAGE:latest
  script:
    - ./configure
    - make all

analysis_job:
  stage: test
  extends: .codesonar_sast

11. Before you commit and push your changes to these .yml files, you should add the custom variables used by the .yml files to your GitLab project via the GitLab UI.

See the comments in the template .yml file for a description of what these variables represent.

Note that the CODESONAR_HUBPWFILE must be configured as a File-type variable.

Type Key Example value
Variable CODESONAR_HUB_URL https://codesonar.example.com:7443
Variable CODESONAR_HUBUSER gitlab_ci
File CODESONAR_HUBPWFILE p@55W0Rd
Variable CODESONAR_MAX_WARNINGS 5
Variable CODESONAR_VISIBILITY_FILTER 2
Variable CODESONAR_WARNING_THRESHOLD 60
Variable GITLAB_TOKEN BSDrHhFPWJkq3F9w5kfy

Some of these variables contain sensitive information that must be secured.

The GITLAB_TOKEN variable is used to automatically generate a Merge Request comment containing a summary of the CodeSonar analysis. To use this feature, you will need to generate a GitLab API Access Token. GitLab provides several ways to do this. A Project Access Token is likely the best option, see https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html. The Access Token will need full "api" (read and write) access.

Consider using a restricted user account for the the hub user and for any GitLab user associated with the GITLAB_TOKEN. Also consider using GitLab's Masked and Protected options for sensitive variable values when possible. However, note that "protected" variables can only be used on "protected" branches, and merge request branches will not typically be protected.

12. Before commiting your changes, create a new branch for your .yml files:

For example, if you decide to name your branch "codesonar_ci":

git checkout -b codesonar_ci

13. Add and commit your updated files to your Git repository and push the changes to GitLab.

If you need to add a server certificate file to your repository, you should do that here too.

For example:

git add .gitlab-ci.yml  codesonar.linux.docker.yml  gitlab.root.cacert
git commit -m "Add CodeSonar pipeline job"
git push -u origin codesonar_ci

14. Follow GitLab instructions to create a Merge Request for your new branch.

15. When the pipeline completes, you should see "security scanning" results on the Merge Request Overview page.

Click the "View Full Report" button to see the analysis results. Notice that the results shown in the "Full Report" are associated with a pipeline.

The analysis report will include hyperlinks to your CodeSonar hub. Note that the CodeSonar hub will be unable to provide code navigation and other features for your analysis. This is because the pipeline container gets deleted after the analysis job executes and analysis data is lost. See the "Relocating Analysis Files" section later in this document for information about how to preserve the analysis data.

Addendum A: Merge Request Pipeline and workflow:rules

By default, GitLab will trigger a pipeline whenever a branch is updated. However, a simple "Branch Pipeline" is not quite the same as a "Merge Request Pipeline". The CodeSonar GitLab Integration tools can post a summary report of analysis results from your pipeline to your Merge Request, but they can only do this for Merge Request pipelines.

For more information about configuring Merge Request pipelines, see: https://docs.gitlab.com/ee/ci/merge_request_pipelines/

There are several ways to configure Merge Request piplines. The example above showed how to use the workflow:rules keywords to restrict your pipelines to either of two conditions:

  1. Merge Request Pipelines (which contain special Merge Request environment variables)
  2. Or any pipeline executed on the default branch.

This configuration is reasonable for teams that work heavily through Merge Requests, but it may not be sufficient if you rely on simple branch updates to trigger pipelines outside of a Merge Request.


Relocating Analysis Files

Shortly after executing an analysis in a GitLab pipeline job you will no longer be able to explore your analysis with your CodeSonar hub. Instead, the hub information window will show you an error like the following:

Please execute
/opt/codesonar/codesonar/bin/codesonar install-launchd codesonar.example.com:7341 -hubuser "gitlab_ci"
on runner-dymnpbks-project-42-concurrent-0 as the OS user root and try again.

The requested information is stored on the computer that ran the analysis (runner-dymnpbks-project-42-concurrent-0).
...

This will happen because the pipeline mechanism will clean up all data from the pipeline job, including the analysis results. In the case of a Docker job, the job container is destroyed and all results are lost. For a GitLab runner not using Docker (e.g. Windows), the working directory will be cleaned and analysis results will be erased.

In order to preserve analysis results, your pipeline must "relocate" the analysis data to a dedicated server with persistent storage. This involves copying analysis data from the analysis runner working directory to an analysis server host and then executing codesonar relocate on the analysis server host. For a detailed discussion of analysis data and how to use codesonar relocate, see the CodeSonar manual, Using CodeSonar -> Typical CodeSonar Tasks -> Analysis Tasks -> Relocate Analysis Files.

A convenient way to relocate the analysis data is to use the GitLab pipeline mechanism itself. With this method, you execute the analysis on an "analysis runner" in an ephemeral Docker container, and you relocate the analysis on an "analysis server runner" in a non-ephemeral environment (with or without Docker). To do this, you install the GitLab runner software on your analysis server host, configure it as a "shell" executor (not Docker), give it a distinctive tag such as "codesonar_analysis_server", and add a pipeline job which will execute codesonar relocate on the analysis server runner.

There are several options for copying the analysis data files from the analysis runner working directory, including docker volume bind mounts, scp/rsync, or the pipeline's artifact mechanism. To use pipeline artifacts, you can upload an archive containing the analysis files using job keyword artifacts:paths, and you can download the artifacts in a subsequent job using keyword needs:artifacts. You will need to ensure artifacts expire quickly so that they do not take up too much space on your GitLab server. The analysis files should be copied into a unique directory, perhaps named using your pipeline's ID number, available in the CI_PIPELINE_ID predefined variable.

Examples are included which show how to configure the runner and the pipeline to relocate the analysis. See the fibonacci and analysis-server-runner.docker examples in the examples directory.

Merge requests and security dashboard

GitLab uses merge requests to make requests for changes, as well as running the pipeline. When you create a merge request in GitLab, the pipeline will automatically be triggered. When the job has finished, you'll see a section with security warnings. The integration does not offer configuration to mark a build as successful or failed, hence every analysis will have a green checkmark unless there were errors executing any of the commands in the pipeline.

Under a merge request, you can see the warnings that were found by CodeSonar. There are two identifiers for every warning, however, GitLab only shows the first in the overview, which is also points to a warning in CodeSonar. The other identifier refers not to a single warning, but to the general overview of the analysis page to which a warning belongs.

Warnings in a MR

When you click one of them, a pop-up is shown that contains more details, as well as links to both the specific line in the file, and a link to the warning in CodeSonar and the general analysis page.

Popup of a Warning

Note that the link in the warning points to a specific line of a file in GitLab.

In case of CodeSonar problems, e.g., the launch daemon not running correctly, the pipeline may be marked as failed. Pipelines can be started manually under a CI/CD -> Pipeline, and then Run Pipeline. Merge requests and pipelines are often linked together, so the overall status of a pipeline influences if a merge request is marked as either successful or failed. Note also that a failed pipeline can be restarted manually, which re-runs only the failed jobs.

Security dashboard

GitLab provides a dashboard that has quite a few visualization options. The dashboard can be used to visualize the number of security issues across many different projects, e.g., under an organization in GitLab. In addition, you can see the number of security issues in different time intervals, for instance every 30 days.

The identifier for a warning is equivalent to the warning class in CodeSonar, e.g., Missing Return Statement. The link points to a detailed overview of the warning in CodeSonar.

In the Security Dashboard under a project, it is possible to select other types of scanners. CodeSonar warnings are in the SAST category. In case that you enabled other security scanners in GitLab, it may be convenient to select only SAST to getting a better overview. You can sort the list by any of the columns.

Security Dashboard.

Three icons are present for each warning in the overview. These icons can be used to either create an issue, delete the warning

Icons for an issue

The details about the warnings are available in the dashboard. This will refer to the most recent report of the project. GitLab groups warnings into six categories. Note that Critical, Info and Unknown are not used in the warnings that come from CodeSonar. The warnings are translated based on the rank in CodeSonar::

CodeSonar GitLab
0 - 20 Low
21 - 55 Medium
56 - 100 High

The Security Dashboard is accessible using the left-side menu.

Accessing the Security Dashboard

There are a few key differences to the dashboard compared to the interface in CodeSonar. The interface in GitLab does not allow to filter on function names. Instead, you'll be presented with a list of warnings, which just presents the warnings with an identifier.


Tips for Using Microsoft Windows

It is recommended that you read the "Walkthrough using Linux with Docker" section above. Many of the recommendations and tips included in the walkthrough apply to Windows as well.

Be sure to install Git on your Windows pipeline job runner machine.

You must also install CodeSonar. Installation of CodeSonar is described in the CodeSonar manual under Getting Started With CodeSonar -> Setting Up -> Setting Up The Hub -> Starting a New Hub and it may be convenient to read How Codesonar Works -> CodeSonar As A Windows Service.

It may be necessary to modify settings in Windows Defender to enable access to your CodeSonar hub (default port 7340).

Under Windows Defender, you may also need to configure network access for the following CodeSonar programs:

GitLab runners on Microsoft Windows

You will need to setup a GitLab runner for Windows. This will involve several steps.

For installing the GitLab runner software, see: https://docs.gitlab.com/runner/install/windows.html

It is recommended that you add tags to your runner named "Windows" and "CodeSonar". These tags help to ensure your pipeline jobs get executed on a Windows runner host with CodeSonar installed, and not any other runner host that you may have configured.

Installing the CodeSonar-GitLab integration tools on Microsoft Windows

The CodeSonar GitLab Integration package includes a set of Python scripts which help automate tasks in your pipeline. These scripts are embedded in the 'distro-image\cso-sarif2sast.tar' archive file. You may need a suitable "archiver" program to extract the files from the .tar file on Windows. The extracted files may be placed anywhere in the system, as long as they can be accessed by the GitLab runner. For example, you might extract these files to 'C:\Program Files\GrammaTech\codesonar-gitlab-integration\distro-image\cso-sarif2sast'.

CodeSonar Hub HTTPS Certificate (Microsoft Windows)

If the CodeSonar hub is configured to use HTTPS, and if the certificate is not trusted by Codesonar on your Windows runner, then the certificate must be added to the CodeSonar trust store. This is necessary if your hub uses a self-signed HTTPS certificate.

You can append the PEM-formatted HTTPS certificate to the 'gtr\cacerts.pem' trusted certificate store file in the CodeSonar installation directory. In the trust-store, the certificate can be given a title to distinguish it from the other certificates in that file. For example:

CodeSonar TLS certificate
=========================
-----BEGIN CERTIFICATE-----
MIIFSDCCBDCg........................................
-----END CERTIFICATE-----

See the "General Tips" section later in this document for information about how to download an HTTPS certificate in PEM format.

Pipeline Configuration for Microsoft Windows

CodeSonar pipeline job definition (Windows)

An example GitLab pipeline configuration YAML file is shown below. This "codesonar.windows.yml" file is intended to be "included" in your main ".gitlab-ci.yml" file. It is a simplified version of the "templates\codesonar.windows.yml" file included with the CodeSonar GitLab integration tools. It demonstrates how to execute CodeSonar to analyze your code and to convert the analysis results to SAST format for GitLab.

# codesonar.windows.yml
.codesonar_windows_sast:
  stage: test
  tags:
    - Windows
    - CodeSonar
  variables:
    # ** Edit these variable values to match your Windows runner ***
    SARIF2SAST: 'C:\Program Files\GrammaTech\codesonar-gitlab-integration\distro-image\cso-sarif2sast'
    CODESONAR: 'C:\Program Files\GrammaTech\CodeSonar\codesonar\bin\codesonar.exe'
    CSPYTHON: 'C:\Program Files\GrammaTech\CodeSonar\codesonar\bin\cspython.exe'
    CI_SERVER_CAFILE: 'gitlab.root.cacert'
    CODESONAR_PROJECT_NAME: ${CI_PROJECT_NAME}
  script:
    # ** Replace the 'buildenv' example below to initialize your build environment **
    - $Env:Path = $Env:Path + ';' + 'C:\buildenv\bin'
    # ** Replace "make" command with your build command below **
    - >
        & "${CODESONAR}" analyze
        -foreground
        -auth password
        -hubuser ${CODESONAR_HUBUSER}
        -hubpwfile ${CODESONAR_HUBPWFILE}
        ${CODESONAR_PROJECT_NAME}
        "${CODESONAR_HUB_URL}"
        make
    -  $CODESONAR_ANALYSIS_ID = Get-Content -Path ./${CODESONAR_PROJECT_NAME}.prj_files/aid.txt -RAW
    - >
        & "${CODESONAR}" get
        -auth password
        -hubuser ${CODESONAR_HUBUSER}
        -hubpwfile ${CODESONAR_HUBPWFILE}
        -O allwarnings.sarif
        "${CODESONAR_HUB_URL}/analysis/${CODESONAR_ANALYSIS_ID}-allwarnings.sarif?filter=${CODESONAR_VISIBILITY_FILTER}"
    - >
        & "${CSPYTHON}" "${SARIF2SAST}\sarif2sast.py"
        --sarif allwarnings.sarif
        --output gl-sast-report.json
        --summary-report sast-summary-report.md
        --codesonar-url "${CODESONAR_HUB_URL}"
        --analysis-id ${CODESONAR_ANALYSIS_ID}
        --max ${CODESONAR_MAX_WARNINGS}
        --threshold ${CODESONAR_WARNING_THRESHOLD}
  after_script:
    - >
        & "${CSPYTHON}" "${SARIF2SAST}\upload_gitlab_mr_notes.py"
        --api-token-variable GITLAB_TOKEN
        --report sast-summary-report.md
        --cafile "${CI_SERVER_CAFILE}"
  artifacts:
    reports:
      sast: gl-sast-report.json

Integrating the CodeSonar pipeline job (Windows)

The ".gitlab-ci.yml" example file used in the Docker walkthrough (in a previous section of this document) can be adapted for use with a Windows runner. A modified version of that example file is shown below.

First, note that the that the pipeline configuration below includes the "codesonar.windows.yml" file which we defined previously.

Also, note that the example pipeline definition below includes two jobs: a simple "build" job and a CodeSonar "analysis" job. The "build" job builds the code, but does not analyze it. Notice that the "build" job uses the "Windows" runner tag, but it does not use the "CodeSonar" tag. This is because the "build" job does not require CodeSonar simply to build the code. In contrast, the "analysis" job requires "CodeSonar", so it must be executed on a runner where CodeSonar is installed.

# .gitlab-ci.yml (Windows)
workflow:
  rules:
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

include:
  - local: 'codesonar.windows.yml'

stages:
  - build
  - test

build_job:
  stage: build
  tags:
    - Windows
  script:
    # ** Replace the 'buildenv' example below to initialize your build environment **
    - $Env:Path = $Env:Path + ';' + 'C:\buildenv\bin'
    # ** Replace "make" command with your build command below **
    - make

analysis_job:
  stage: test
  extends: .codesonar_windows_sast

[]


General Tips

Trusted HTTPS GitLab Server Certificate

The CodeSonar GitLab Integration tools will attempt to post a summary report of the analysis to your merge request. If the CodeSonar cspython does not trust your GitLab server's HTTPS certificate you will see an "SSL" error in your pipeline job log. You can optionally provide a custom ASCII-encoded certificate authority (CA) file to the upload_gitlab_mr_notes.py script using the --cafile option. The CA file should include the root certificate of the certificate authority signing chain for your GitLab HTTPS certificate.

Additional tips for how to download the certificate file are provided further below.

The path to the certificate file must be specified in the YAML configuration file. The path must exist on the runner (i.e. in the docker container). An easy way to ensure that the path exists is to add the certificate to your Git repository and refer to it using a relative path.

Example YAML configuration:

  ...
  after_script:
    - >
        cspython $SARIF2SAST/upload_gitlab_mr_notes.py
        --api-token-variable GITLAB_TOKEN
        --report sast-summary-report.md
        --cafile path/to/your/gitlab_certificate.pem

Download an HTTPS certificate in Base-64 ASCII (PEM) format

You may need a Base-64 ASCII ("PEM") file containing an HTTPS certificate associated with your CodeSonar hub or your GitLab server.

For the CodeSonar hub, an administrator can download the certificate directly from within the hub. Go to Settings -> HTTP -> Configure HTTPS. Read more about CodeSonar hub server certificates in the CodeSonar manual under How CodeSonar works -> CodeSonar Structure -> Hub -> TLS Certificates.

If you cannot copy the certificate directly from the server, then you can download it over the network. A certificate does not need to be verified or trusted in order to download it over an SSL/TLS channel, however you should verify it if you have the means to do so.

For the purpose of HTTPS verification by a client (e.g. codesonar or cspython), it is often preferred to provide only the top (root) certificate in the certificate chain, but the entire chain will usually be sufficient (if you have the means to download it). In general, the bottom ("leaf") certificate in the certificate chain will not work. For self-signed certificates, there is only one certificate in the chain, but in other cases there may be more.

Most browsers provide a way to download an HTTPS certificate in ASCII format. The instructions for downloading the certificate are dependent on both the type of browser and the operating system.

Alternatively, you can use the OpenSSL commandline to download the bottom (leaf) certificate. This should be sufficient for self-signed certificates and may work in other cases too depending on the client. For example:

openssl s_client -connect $HOST:443 -showcerts | openssl x509 -outform PEM -out $CAFILE

where $HOST is the name of your HTTPS server host (e.g. codesonar.example.com) and $CAFILE is the name of the file where you want to save the certificate.

OpenSSL may already be installed on your operating system, however CodeSonar includes a copy in its third-party/openssl/inst/bin subdirectory.

Troubleshooting

Warnings for files outside your repository

If you see warnings for files that are not in the repository, you can configure the CodeSonar configuration variable SYSTEM_INCLUDE_PATHS. See the CodeSonar manual, section Using CodeSonar -> Building and Analyzing a CodeSonar Project -> Options, Preferences, and Configuration Files -> Compiler-Independent Configuration File Parameter Index for CodeSonar. CodeSonar's default configuration includes a set of "factory" default system include paths for common compiler tool chains. You may want to review the list (provided in the manual and the default generated .conf file) and add paths as necessary.