In order to support new features, such as CUDA, SonarCloud, Factory Generator, CLion and unit tests, some changes needed to be done on the continuous integration or Docker images.
Two jobs (build-ros-perception-cuda-x86-64 and build-ros-perception-cuda-arm64) have been added to the workflows defined by these files (on proc_image_processing's repository and sonia_common's repository):
docker-image-perception-feature.ymldocker-image-perception-develop.ymldocker-image-perception-master.ymlBoth jobs are identical to the other ones, except that they use the CUDA based image as base.
Since this tool is supposed to run on every commit in order to provide feedback to developpers, it has it's own workflow defined by the file sonarcloud.yml. This workflow has been optimized by caching Docker image layers and using Docker's multi-stage build feature. Overall this saves about 2 to 3 minutes of runtime and the single job defined takes about 6 minutes to complete, where 5 minutes are dedicated to building and scanning the application.
We strongly suggest SONIA to optimize other workflows by using the same strategy used for this workflow. Here's an example with the X86 build of proc_image_processing where 1 to 2 minutes can be saved by using this workflow (and by removing BUILD_DATE and VERSION arguments from the Dockerfile:
build-ros-perception-x86-64:
name: "Build ROS perception X86/64"
runs-on: ubuntu-latest
env:
ARCH: x86
TARGET_TYPE: perception
TARGET_VERSION: feature
IMAGE_REPOSITORY: ghcr.io/sonia-auv/proc_image_processing/proc_image_processing
BASE_IMAGE: ghcr.io/sonia-auv/sonia_common/sonia_common:x86-perception-latest
NODE_NAME: proc_image_processing
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Login to GitHub Container Registry
uses: docker/login-action@v1
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set environment variables
id: vars
run: |
echo "::set-output name=branch-name::${GITHUB_REF##*/}"
echo "::set-output name=branch-fullname::${GITHUB_REF#refs/heads/}"
echo "::set-output name=tag::${ARCH}-${TARGET_TYPE}-${TARGET_VERSION}-${GITHUB_REF##*/}"
echo "::set-output name=build-tag::build-${TARGET_VERSION}-${GITHUB_REF##*/}-${GITHUB_RUN_NUMBER}"
echo "::set-output name=date::$(date '+%Y-%m-%d_%H:%M:%S')"
- name: Build, Analyze and Push
uses: docker/build-push-action@v2
with:
context: .
file: Dockerfile
push: true
cache-from: type=registry,ref=${{env.IMAGE_REPOSITORY}}:${{ steps.vars.outputs.tag }}
cache-to: type=inline
tags: |
${{env.IMAGE_REPOSITORY}}:${{ steps.vars.outputs.tag }}
${{env.IMAGE_REPOSITORY}}:${{ steps.vars.outputs.build-tag }}
build-args: |
BASE_IMAGE=${{env.BASE_IMAGE}}
labels: |
net.etsmtl.sonia-auv.node.build-date=${{ steps.vars.outputs.date }}
net.etsmtl.sonia-auv.node.version=${{ steps.vars.outputs.branch-name }}-${{ steps.vars.outputs.date }}
net.etsmtl.sonia-auv.node.name=${{env.NODE_NAME}}
- name: Run Unit Tests
run: |
docker run ${{env.IMAGE_REPOSITORY}}:${{ steps.vars.outputs.tag }} catkin_make --use-ninja run_tests
docker run ${{env.IMAGE_REPOSITORY}}:${{ steps.vars.outputs.tag }} catkin_make --use-ninja tests
See the docker's GitHub Actions Build-and-Push, Buildx and QEMU for more details.
In order to make sure a commit doesn't break the Factory Generator, a new workflow, defined by the file factory_generator.yml, has been added. This workflow sets up a python environment, installs dependencies and then runs unit tests. Results (coverage and actual test results) are uploaded and can be downloaded from GitHub Actions' UI.
In order to use CLion with the Docker architecture, a new Dockerfile named Dockerfile.clion has been added. This image is simply an extension of proc_image_processing's image, but with the additionnal requirements for CLion. To use it, see the project's README.md
Since we added unit tests to the project, workflows have been modified in order to run these tests after the build step of the Docker image.