9:42 a.m.
Hi,
Add a Continuous Integration builds under Github (with its Actions) to
build and unit test on several configurations.
Changes since v1 [1]:
1. Rebase on latest master.
2. Remove CI-unrelated patches from this set.
3. Add CodeQL analysis.
4. Add builds and tests with GCC sanitizers.
5. Use matrix to extend the build configuration.
[1]
https://lore.kernel.org/linux-nfc/20210710033859.3989-1-krzysztof.kozlows...
Best regards,
Krzysztof
Krzysztof Kozlowski (11):
ci: add GitHub actions for building
bootstrap: parse CROSS_COMPILE and set proper configure option
ci: add clang builds
ci: add building without maintainer options
ci: be verbose when building
ci: add more build configurations (Fedora, Alpine, Debian,
cross-compile, i386)
ci: run unit tests
ci: add build with sanitizers (asan, lsan and ubsan)
ci: add CodeQL static analysis
ci: print configure logs on failures
ci: use matrix instead of duplicating each build configuration
.github/workflows/ci.yml | 255 ++++++++++++++++++++++++++
.github/workflows/codeql-analysis.yml | 45 +++++
bootstrap-configure | 6 +
ci/alpine.sh | 42 +++++
ci/debian.cross-compile.sh | 41 +++++
ci/debian.i386.sh | 32 ++++
ci/debian.sanitizers.sh | 18 ++
ci/debian.sh | 41 +++++
ci/fedora.sh | 33 ++++
ci/ubuntu.cross-compile.sh | 1 +
ci/ubuntu.i386.sh | 1 +
ci/ubuntu.sanitizers.sh | 1 +
ci/ubuntu.sh | 1 +
13 files changed, 517 insertions(+)
create mode 100644 .github/workflows/ci.yml
create mode 100644 .github/workflows/codeql-analysis.yml
create mode 100755 ci/alpine.sh
create mode 100755 ci/debian.cross-compile.sh
create mode 100755 ci/debian.i386.sh
create mode 100755 ci/debian.sanitizers.sh
create mode 100755 ci/debian.sh
create mode 100755 ci/fedora.sh
create mode 120000 ci/ubuntu.cross-compile.sh
create mode 120000 ci/ubuntu.i386.sh
create mode 120000 ci/ubuntu.sanitizers.sh
create mode 120000 ci/ubuntu.sh
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 01/11] ci: add GitHub actions for building
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/ci.yml | 79 ++++++++++++++++++++++++++++++++++++++++
ci/debian.sh | 32 ++++++++++++++++
ci/ubuntu.sh | 1 +
3 files changed, 112 insertions(+)
create mode 100644 .github/workflows/ci.yml
create mode 100755 ci/debian.sh
create mode 120000 ci/ubuntu.sh
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
new file mode 100644
index 000000000000..e48ee0eb6815
--- /dev/null
+++ b/.github/workflows/ci.yml
@@ -0,0 +1,79 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2021 Canonical Ltd.
+# Author: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
+# <krzk(a)kernel.org>
+# Loosely based on https://github.com/linux-test-project/ltp
+#
+name: "Builds"
+on: [push, pull_request]
+
+jobs:
+ job:
+ name: Build
+ runs-on: ubuntu-latest
+ permissions:
+ actions: read
+ contents: read
+
+ strategy:
+ fail-fast: false
+ matrix:
+ include:
+ - container: "ubuntu:hirsute"
+ env:
+ CC: gcc
+
+ - container: "ubuntu:focal"
+ env:
+ CC: gcc
+
+ - container: "ubuntu:bionic"
+ env:
+ CC: gcc
+
+ - container: "ubuntu:xenial"
+ env:
+ CC: gcc
+
+ container:
+ image: ${{ matrix.container }}
+ env: ${{ matrix.env }}
+
+ steps:
+ - name: Show OS
+ run: cat /etc/os-release
+
+ - name: Git checkout
+ uses: actions/checkout@v2
+
+ - name: Install additional packages
+ run: |
+ INSTALL=${{ matrix.container }}
+ INSTALL="${INSTALL%%:*}"
+ INSTALL="${INSTALL%%/*}"
+ ./ci/$INSTALL.sh
+ if [ "$VARIANT" ]; then ./ci/$INSTALL.$VARIANT.sh; fi
+
+ - name: Compiler version
+ run: $CC --version
+
+ - name: Display environment and Linux version
+ run: |
+ test -f /etc/issue && cat /etc/issue
+ lsb_release -a || true
+ uname -a
+ cat /proc/cmdline
+ printenv
+
+ - name: Configure
+ run: ./bootstrap-configure
+
+ - name: Compile
+ run: make -j$(nproc)
+
+ - name: Install
+ run: make install
+
+ - name: Distribution check
+ run: make distcheck
diff --git a/ci/debian.sh b/ci/debian.sh
new file mode 100755
index 000000000000..07b7c9305cdb
--- /dev/null
+++ b/ci/debian.sh
@@ -0,0 +1,32 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2021 Canonical Ltd.
+# Author: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
+# <krzk(a)kernel.org>
+#
+
+set -eEx
+
+apt update
+
+# Some distros (e.g. Ubuntu Hirsute) might pull tzdata which asks questions
+export DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true
+
+# Choose some random place in Europe
+echo "tzdata tzdata/Areas select Europe
+tzdata tzdata/Zones/Europe select Berlin
+" > /tmp/tzdata-preseed.txt
+debconf-set-selections /tmp/tzdata-preseed.txt
+
+apt install -y --no-install-recommends \
+ autoconf \
+ autoconf-archive \
+ automake \
+ build-essential \
+ libdbus-1-dev \
+ libglib2.0-dev \
+ libnl-3-dev \
+ libnl-genl-3-dev \
+ libtool \
+
diff --git a/ci/ubuntu.sh b/ci/ubuntu.sh
new file mode 120000
index 000000000000..0edcb8b838ca
--- /dev/null
+++ b/ci/ubuntu.sh
@@ -0,0 +1 @@
+debian.sh
\ No newline at end of file
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 02/11] bootstrap: parse CROSS_COMPILE and set proper configure option
Makes cross compiling in CI easier.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
bootstrap-configure | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/bootstrap-configure b/bootstrap-configure
index 7cb14588f816..0652e3b223b6 100755
--- a/bootstrap-configure
+++ b/bootstrap-configure
@@ -4,6 +4,11 @@ if [ -f config.status ]; then
make maintainer-clean
fi
+CONFIGURE_CROSS=""
+if [ "$CROSS_COMPILE" ]; then
+ CONFIGURE_CROSS="--host=${CROSS_COMPILE}"
+fi
+
./bootstrap && \
./configure --enable-maintainer-mode \
--enable-debug \
@@ -11,4 +16,5 @@ fi
--prefix=/usr \
--enable-ese \
--sysconfdir=/etc \
+ "$CONFIGURE_CROSS" \
--enable-tools "$@"
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 03/11] ci: add clang builds
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/ci.yml | 10 ++++++++++
ci/debian.sh | 10 ++++++++--
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index e48ee0eb6815..3d9b50251602 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -20,6 +20,7 @@ jobs:
fail-fast: false
matrix:
include:
+ # Ubuntu gcc
- container: "ubuntu:hirsute"
env:
CC: gcc
@@ -36,6 +37,15 @@ jobs:
env:
CC: gcc
+ # Ubuntu clang
+ - container: "ubuntu:hirsute"
+ env:
+ CC: clang
+
+ - container: "ubuntu:focal"
+ env:
+ CC: clang
+
container:
image: ${{ matrix.container }}
env: ${{ matrix.env }}
diff --git a/ci/debian.sh b/ci/debian.sh
index 07b7c9305cdb..8cc4d307cccf 100755
--- a/ci/debian.sh
+++ b/ci/debian.sh
@@ -19,14 +19,20 @@ tzdata tzdata/Zones/Europe select Berlin
" > /tmp/tzdata-preseed.txt
debconf-set-selections /tmp/tzdata-preseed.txt
+PKGS_CC="build-essential"
+if [ "$CC" = "clang" ]; then
+ PKGS_CC="clang"
+fi
+
+
apt install -y --no-install-recommends \
autoconf \
autoconf-archive \
automake \
- build-essential \
libdbus-1-dev \
libglib2.0-dev \
libnl-3-dev \
libnl-genl-3-dev \
libtool \
-
+ make \
+ $PKGS_CC
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 04/11] ci: add building without maintainer options
Check also silent builds.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/ci.yml | 36 +++++++++++++++++++++++++++++++++++-
1 file changed, 35 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 3d9b50251602..589ad66280f5 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -24,27 +24,49 @@ jobs:
- container: "ubuntu:hirsute"
env:
CC: gcc
+ MODE: maintainer
+
+ - container: "ubuntu:hirsute"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ MODE: no-maintainer
- container: "ubuntu:focal"
env:
CC: gcc
+ MODE: maintainer
+
+ - container: "ubuntu:focal"
+ env:
+ CC: gcc
+ MODE: no-maintainer
- container: "ubuntu:bionic"
env:
CC: gcc
+ MODE: maintainer
- container: "ubuntu:xenial"
env:
CC: gcc
+ MODE: maintainer
# Ubuntu clang
- container: "ubuntu:hirsute"
env:
CC: clang
+ MODE: maintainer
+
+ - container: "ubuntu:focal"
+ env:
+ CC: clang
+ MODE: maintainer
- container: "ubuntu:focal"
env:
CC: clang
+ MODE: no-maintainer
container:
image: ${{ matrix.container }}
@@ -77,7 +99,19 @@ jobs:
printenv
- name: Configure
- run: ./bootstrap-configure
+ run: |
+ echo "Bootstraping in mode: $MODE"
+ if [ "$MODE" = "maintainer" ]; then
+ ./bootstrap-configure
+ else
+ ./bootstrap && \
+ ./configure \
+ --disable-systemd \
+ --prefix=/usr \
+ --enable-ese \
+ --sysconfdir=/etc \
+ --enable-tools
+ fi
- name: Compile
run: make -j$(nproc)
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 05/11] ci: be verbose when building
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/ci.yml | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 589ad66280f5..389c47dcd781 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -102,11 +102,13 @@ jobs:
run: |
echo "Bootstraping in mode: $MODE"
if [ "$MODE" = "maintainer" ]; then
- ./bootstrap-configure
+ ./bootstrap-configure \
+ --disable-silent-rules
else
./bootstrap && \
./configure \
--disable-systemd \
+ --disable-silent-rules \
--prefix=/usr \
--enable-ese \
--sysconfdir=/etc \
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 06/11] ci: add more build configurations (Fedora, Alpine, Debian, cross-compile, i386)
The installation scripts were based on LTP project.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/ci.yml | 160 ++++++++++++++++++++++++++++++++++++-
ci/alpine.sh | 35 ++++++++
ci/debian.cross-compile.sh | 41 ++++++++++
ci/debian.i386.sh | 32 ++++++++
ci/debian.sh | 13 +--
ci/fedora.sh | 33 ++++++++
ci/ubuntu.cross-compile.sh | 1 +
ci/ubuntu.i386.sh | 1 +
8 files changed, 309 insertions(+), 7 deletions(-)
create mode 100755 ci/alpine.sh
create mode 100755 ci/debian.cross-compile.sh
create mode 100755 ci/debian.i386.sh
create mode 100755 ci/fedora.sh
create mode 120000 ci/ubuntu.cross-compile.sh
create mode 120000 ci/ubuntu.i386.sh
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 389c47dcd781..c8e3fb8fb10b 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -20,9 +20,140 @@ jobs:
fail-fast: false
matrix:
include:
- # Ubuntu gcc
+ # Debian builds
+ # 32bit build
+ - container: "debian:stable"
+ env:
+ ARCH: i386
+ CC: gcc -m32
+ CROSS_COMPILE: i686-linux-gnu
+ VARIANT: i386
+ MODE: maintainer
+
+ # Debian cross compilation builds
+ - container: "debian:stable"
+ env:
+ ARCH: armel
+ CC: arm-linux-gnueabi-gcc
+ CROSS_COMPILE: arm-linux-gnueabi
+ VARIANT: cross-compile
+ MODE: maintainer
+
+ - container: "debian:stable"
+ env:
+ ARCH: arm64
+ CC: aarch64-linux-gnu-gcc
+ CROSS_COMPILE: aarch64-linux-gnu
+ VARIANT: cross-compile
+ MODE: maintainer
+
+ - container: "debian:stable"
+ env:
+ ARCH: ppc64el
+ CC: powerpc64le-linux-gnu-gcc
+ CROSS_COMPILE: powerpc64le-linux-gnu
+ VARIANT: cross-compile
+ MODE: maintainer
+
+ - container: "debian:stable"
+ env:
+ ARCH: s390x
+ CC: s390x-linux-gnu-gcc
+ CROSS_COMPILE: s390x-linux-gnu
+ VARIANT: cross-compile
+ MODE: maintainer
+
+ # Debian gcc
+ - container: "debian:stable"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ MODE: maintainer
+
+ - container: "debian:stable"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ MODE: no-maintainer
+
+ - container: "debian:testing"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ MODE: maintainer
+
+ - container: "debian:testing"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ MODE: no-maintainer
+
+ # Debian clang
+ - container: "debian:stable"
+ env:
+ ARCH: x86-64
+ CC: clang
+ MODE: maintainer
+
+ - container: "debian:stable"
+ env:
+ ARCH: x86-64
+ CC: clang
+ MODE: no-maintainer
+
+ - container: "debian:testing"
+ env:
+ ARCH: x86-64
+ CC: clang
+ MODE: maintainer
+
+ # musl (native)
+ - container: "alpine:latest"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ MODE: maintainer
+
+ # Fedora
+ - container: "fedora:latest"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ MODE: maintainer
+
+ - container: "fedora:latest"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ MODE: no-maintainer
+
+ - container: "fedora:latest"
+ env:
+ ARCH: x86-64
+ CC: clang
+ MODE: maintainer
+
+ # Ubuntu 32bit builds: gcc and clang
+ - container: "ubuntu:hirsute"
+ env:
+ ARCH: i386
+ CC: gcc -m32
+ CROSS_COMPILE: i686-linux-gnu
+ VARIANT: i386
+ MODE: maintainer
+
+ - container: "ubuntu:focal"
+ env:
+ ARCH: i386
+ CC: gcc -m32
+ CROSS_COMPILE: i686-linux-gnu
+ VARIANT: i386
+ MODE: maintainer
+
+ # Ubuntu x86-64 gcc
- container: "ubuntu:hirsute"
env:
+ ARCH: x86-64
CC: gcc
MODE: maintainer
@@ -34,37 +165,44 @@ jobs:
- container: "ubuntu:focal"
env:
+ ARCH: x86-64
CC: gcc
MODE: maintainer
- container: "ubuntu:focal"
env:
+ ARCH: x86-64
CC: gcc
MODE: no-maintainer
- container: "ubuntu:bionic"
env:
+ ARCH: x86-64
CC: gcc
MODE: maintainer
- container: "ubuntu:xenial"
env:
+ ARCH: x86-64
CC: gcc
MODE: maintainer
- # Ubuntu clang
+ # Ubuntu x86-64 clang
- container: "ubuntu:hirsute"
env:
+ ARCH: x86-64
CC: clang
MODE: maintainer
- container: "ubuntu:focal"
env:
+ ARCH: x86-64
CC: clang
MODE: maintainer
- container: "ubuntu:focal"
env:
+ ARCH: x86-64
CC: clang
MODE: no-maintainer
@@ -118,8 +256,26 @@ jobs:
- name: Compile
run: make -j$(nproc)
+ - name: Check final binary
+ run: |
+ file src/neard
+ ARCH_CHECK="$ARCH"
+ case "$ARCH" in
+ armel) ARCH_CHECK="ARM, EABI";;
+ arm64) ARCH_CHECK="aarch64";;
+ i386) ARCH_CHECK="Intel 80386";;
+ ppc64el) ARCH_CHECK="64-bit PowerPC";;
+ s390x) ARCH_CHECK="IBM S/390";;
+ *) ARCH_CHECK="x86-64";;
+ esac
+ echo "Checking for built matching architecture: $ARCH_CHECK"
+ file src/neard | grep "$ARCH_CHECK"
+
- name: Install
run: make install
- name: Distribution check
run: make distcheck
+ # distcheck runs unit tests so no point for cross compile
+ # TODO: figure out how to re-configure during distcheck with --host for i386
(DISTCHECK_CONFIGURE_FLAGS)
+ if: ${{ (matrix.env['ARCH'] == 'x86-64')}}
diff --git a/ci/alpine.sh b/ci/alpine.sh
new file mode 100755
index 000000000000..81b5c72a392e
--- /dev/null
+++ b/ci/alpine.sh
@@ -0,0 +1,35 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2019-2021 Petr Vorel <petr.vorel(a)gmail.com>
+# Copyright (c) 2021 Canonical Ltd.
+# Author: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
+# <krzk(a)kernel.org>
+#
+
+set -ex
+
+apk update
+
+PKGS_CC="gcc"
+
+# gzip: for distcheck
+apk add \
+ autoconf \
+ autoconf-archive \
+ automake \
+ gcc \
+ gzip \
+ dbus-dev \
+ glib-dev \
+ libnl3-dev \
+ libtool \
+ make \
+ musl-dev \
+ $PKGS_CC
+
+# Packages needed by CI
+apk add \
+ file
+
+echo "Install finished: $0"
diff --git a/ci/debian.cross-compile.sh b/ci/debian.cross-compile.sh
new file mode 100755
index 000000000000..8d4c5bb38cfd
--- /dev/null
+++ b/ci/debian.cross-compile.sh
@@ -0,0 +1,41 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2018-2020 Petr Vorel <pvorel(a)suse.cz>
+# Copyright (c) 2021 Canonical Ltd.
+# Author: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
+# <krzk(a)kernel.org>
+#
+
+set -ex
+
+if [ -z "$ARCH" ]; then
+ echo "missing \$ARCH!" >&2
+ exit 1
+fi
+
+case "$ARCH" in
+ armel) PKGS_CC="gcc-arm-linux-gnueabi libc6-dev-${ARCH}-cross";;
+ arm64) PKGS_CC="gcc-aarch64-linux-gnu libc6-dev-${ARCH}-cross";;
+ ppc64el) PKGS_CC="gcc-powerpc64le-linux-gnu libc6-dev-${ARCH}-cross";;
+ # TODO: libraries for riscv?
+ #riscv64) PKGS_CC="gcc-riscv64-linux-gnu libc6-dev-${ARCH}-cross";;
+ s390x) PKGS_CC="gcc-${ARCH}-linux-gnu libc6-dev-${ARCH}-cross";;
+ *) echo "unsupported arch: '$ARCH'!" >&2; exit 1;;
+esac
+
+dpkg --add-architecture $ARCH
+apt update
+
+apt install -y --no-install-recommends \
+ autoconf:${ARCH} \
+ autoconf-archive \
+ automake:${ARCH} \
+ libdbus-1-dev:${ARCH} \
+ libglib2.0-dev:${ARCH} \
+ libnl-3-dev:${ARCH} \
+ libnl-genl-3-dev:${ARCH} \
+ libtool:${ARCH} \
+ $PKGS_CC
+
+echo "Install finished: $0"
diff --git a/ci/debian.i386.sh b/ci/debian.i386.sh
new file mode 100755
index 000000000000..8d154412e3b7
--- /dev/null
+++ b/ci/debian.i386.sh
@@ -0,0 +1,32 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2018-2020 Petr Vorel <pvorel(a)suse.cz>
+# Copyright (c) 2021 Canonical Ltd.
+# Author: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
+# <krzk(a)kernel.org>
+#
+
+set -ex
+
+dpkg --add-architecture i386
+apt update
+
+# gcc-multilib are also needed for clang 32-bit builds
+PKGS_CC="gcc-multilib"
+
+apt install -y --no-install-recommends \
+ linux-libc-dev:i386
+
+apt install -y --no-install-recommends \
+ autoconf:i386 \
+ autoconf-archive \
+ automake:i386 \
+ libdbus-1-dev:i386 \
+ libglib2.0-dev:i386 \
+ libnl-3-dev:i386 \
+ libnl-genl-3-dev:i386 \
+ libtool:i386 \
+ $PKGS_CC
+
+echo "Install finished: $0"
diff --git a/ci/debian.sh b/ci/debian.sh
index 8cc4d307cccf..962e554304ee 100755
--- a/ci/debian.sh
+++ b/ci/debian.sh
@@ -6,7 +6,7 @@
# <krzk(a)kernel.org>
#
-set -eEx
+set -ex
apt update
@@ -20,10 +20,11 @@ tzdata tzdata/Zones/Europe select Berlin
debconf-set-selections /tmp/tzdata-preseed.txt
PKGS_CC="build-essential"
-if [ "$CC" = "clang" ]; then
- PKGS_CC="clang"
-fi
-
+case $CC in
+ clang*)
+ PKGS_CC="clang"
+ ;;
+esac
apt install -y --no-install-recommends \
autoconf \
@@ -36,3 +37,5 @@ apt install -y --no-install-recommends \
libtool \
make \
$PKGS_CC
+
+echo "Install finished: $0"
diff --git a/ci/fedora.sh b/ci/fedora.sh
new file mode 100755
index 000000000000..c5a67d237e91
--- /dev/null
+++ b/ci/fedora.sh
@@ -0,0 +1,33 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2021 Canonical Ltd.
+# Author: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
+# <krzk(a)kernel.org>
+#
+
+set -ex
+
+PKGS_CC="gcc"
+case $CC in
+ clang*)
+ PKGS_CC="clang"
+ ;;
+esac
+
+yum -y install \
+ autoconf \
+ autoconf-archive \
+ automake \
+ dbus-devel \
+ glib2-devel \
+ libnl3-devel \
+ libtool \
+ make \
+ $PKGS_CC
+
+# Packages needed by CI
+yum -y install \
+ file
+
+echo "Install finished: $0"
diff --git a/ci/ubuntu.cross-compile.sh b/ci/ubuntu.cross-compile.sh
new file mode 120000
index 000000000000..18f77ad009bb
--- /dev/null
+++ b/ci/ubuntu.cross-compile.sh
@@ -0,0 +1 @@
+debian.cross-compile.sh
\ No newline at end of file
diff --git a/ci/ubuntu.i386.sh b/ci/ubuntu.i386.sh
new file mode 120000
index 000000000000..813bdf90e6ec
--- /dev/null
+++ b/ci/ubuntu.i386.sh
@@ -0,0 +1 @@
+debian.i386.sh
\ No newline at end of file
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 07/11] ci: run unit tests
Although unit tests are executed as part of distcheck, but add it as a
dedicated step to make it explicit with getting the logs afterwards.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/ci.yml | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index c8e3fb8fb10b..e8f507a128e0 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -256,6 +256,17 @@ jobs:
- name: Compile
run: make -j$(nproc)
+ - name: Run unit tests
+ id: unit_tests
+ run: |
+ make check
+ if: ${{ (matrix.env['ARCH'] == 'x86-64') ||
(matrix.env['ARCH'] == 'i386') }}
+
+ - name: Get unit tests logs
+ run: cat test-suite.log
+ if: ${{ always() && ((steps.unit_tests.outcome == 'failure') ||
+ (steps.unit_tests.outcome == 'success')) }}
+
- name: Check final binary
run: |
file src/neard
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 08/11] ci: add build with sanitizers (asan, lsan and ubsan)
Check the results if sanitizers were correctly linked.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/ci.yml | 44 ++++++++++++++++++++++++++++++++++++++--
ci/debian.sanitizers.sh | 18 ++++++++++++++++
ci/ubuntu.sanitizers.sh | 1 +
3 files changed, 61 insertions(+), 2 deletions(-)
create mode 100755 ci/debian.sanitizers.sh
create mode 120000 ci/ubuntu.sanitizers.sh
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index e8f507a128e0..b6f29cf9ddfb 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -70,6 +70,13 @@ jobs:
CC: gcc
MODE: maintainer
+ - container: "debian:stable"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ VARIANT: sanitizers
+ MODE: maintainer
+
- container: "debian:stable"
env:
ARCH: x86-64
@@ -82,6 +89,13 @@ jobs:
CC: gcc
MODE: maintainer
+ - container: "debian:testing"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ VARIANT: sanitizers
+ MODE: maintainer
+
- container: "debian:testing"
env:
ARCH: x86-64
@@ -157,6 +171,13 @@ jobs:
CC: gcc
MODE: maintainer
+ - container: "ubuntu:hirsute"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ VARIANT: sanitizers
+ MODE: maintainer
+
- container: "ubuntu:hirsute"
env:
ARCH: x86-64
@@ -169,6 +190,13 @@ jobs:
CC: gcc
MODE: maintainer
+ - container: "ubuntu:focal"
+ env:
+ ARCH: x86-64
+ CC: gcc
+ VARIANT: sanitizers
+ MODE: maintainer
+
- container: "ubuntu:focal"
env:
ARCH: x86-64
@@ -239,9 +267,14 @@ jobs:
- name: Configure
run: |
echo "Bootstraping in mode: $MODE"
+ CONFIGURE_ARGS=""
+ if [ "$VARIANT" = "sanitizers" ]; then
+ CONFIGURE_ARGS="--enable-asan --enable-lsan --enable-ubsan
--enable-pie"
+ fi
if [ "$MODE" = "maintainer" ]; then
./bootstrap-configure \
- --disable-silent-rules
+ --disable-silent-rules \
+ $CONFIGURE_ARGS
else
./bootstrap && \
./configure \
@@ -250,7 +283,8 @@ jobs:
--prefix=/usr \
--enable-ese \
--sysconfdir=/etc \
- --enable-tools
+ --enable-tools \
+ $CONFIGURE_ARGS
fi
- name: Compile
@@ -281,6 +315,12 @@ jobs:
esac
echo "Checking for built matching architecture: $ARCH_CHECK"
file src/neard | grep "$ARCH_CHECK"
+ if [ "$VARIANT" = "sanitizers" ]; then
+ echo "Checking for linking against sanitizer libraries"
+ ldd src/neard | grep libasan.so
+ # liblsan won't appear if asan is used
+ ldd src/neard | grep libubsan.so
+ fi
- name: Install
run: make install
diff --git a/ci/debian.sanitizers.sh b/ci/debian.sanitizers.sh
new file mode 100755
index 000000000000..40dc1f44b55a
--- /dev/null
+++ b/ci/debian.sanitizers.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2021 Canonical Ltd.
+# Author: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
+# <krzk(a)kernel.org>
+#
+
+set -ex
+
+apt install -y --no-install-recommends \
+ liblsan0 \
+ libubsan1
+
+apt install -y --no-install-recommends libasan6 || \
+ apt install -y --no-install-recommends libasan5
+
+echo "Install finished: $0"
diff --git a/ci/ubuntu.sanitizers.sh b/ci/ubuntu.sanitizers.sh
new file mode 120000
index 000000000000..53cd9418ed35
--- /dev/null
+++ b/ci/ubuntu.sanitizers.sh
@@ -0,0 +1 @@
+debian.sanitizers.sh
\ No newline at end of file
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 09/11] ci: add CodeQL static analysis
Add Github CodeQL static analysis workflow.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/codeql-analysis.yml | 45 +++++++++++++++++++++++++++
1 file changed, 45 insertions(+)
create mode 100644 .github/workflows/codeql-analysis.yml
diff --git a/.github/workflows/codeql-analysis.yml
b/.github/workflows/codeql-analysis.yml
new file mode 100644
index 000000000000..66ad27d5eca0
--- /dev/null
+++ b/.github/workflows/codeql-analysis.yml
@@ -0,0 +1,45 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2021 Canonical Ltd.
+# Author: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
+# <krzk(a)kernel.org>
+#
+name: "CodeQL"
+on: [push, pull_request]
+
+jobs:
+ analyze:
+ name: Analyze
+ runs-on: ubuntu-latest
+ permissions:
+ actions: read
+ contents: read
+ security-events: write
+
+ strategy:
+ fail-fast: false
+ matrix:
+ language: [ 'cpp', 'python' ]
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v2
+
+ - name: Initialize CodeQL
+ uses: github/codeql-action/init@v1
+ with:
+ languages: ${{ matrix.language }}
+
+ - name: Install additional packages
+ run: sudo ./ci/ubuntu.sh
+
+ - name: Configure
+ run: ./bootstrap-configure
+
+ - name: Compile
+ run: |
+ make -j$(nproc)
+ make check
+
+ - name: Perform CodeQL Analysis
+ uses: github/codeql-action/analyze@v1
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 10/11] ci: print configure logs on failures
Helps debugging of container-specific configure failures.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/ci.yml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index b6f29cf9ddfb..d4ddcdd3f8f4 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -265,6 +265,7 @@ jobs:
printenv
- name: Configure
+ id: configure
run: |
echo "Bootstraping in mode: $MODE"
CONFIGURE_ARGS=""
@@ -287,6 +288,11 @@ jobs:
$CONFIGURE_ARGS
fi
+ - name: Get configure logs
+ run: cat config.log
+ if: ${{ always() && ((steps.configure.outcome == 'failure') ||
+ (steps.configure.outcome == 'success')) }}
+
- name: Compile
run: make -j$(nproc)
--
2.30.2
9:42 a.m.
New subject: [neard][PATCH v2 11/11] ci: use matrix instead of duplicating each build configuration
Use matrix to list all typical configurations which allows easily to
build different distros with clang and maintainer mode. This extends
the number of jobs considerably but makes the entire CI YAML file
smaller and easier to understand.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski(a)canonical.com>
---
.github/workflows/ci.yml | 295 ++++++++++++++-------------------------
ci/alpine.sh | 9 +-
2 files changed, 114 insertions(+), 190 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index d4ddcdd3f8f4..6a4eeedf3d66 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -19,229 +19,141 @@ jobs:
strategy:
fail-fast: false
matrix:
+ arch: [x86-64]
+ compiler: [gcc, clang]
+ container:
+ - alpine:latest
+ - debian:testing
+ - debian:stable
+ - fedora:latest
+ - ubuntu:hirsute
+ - ubuntu:focal
+ - ubuntu:bionic
+ - ubuntu:xenial
+ cross_compile: [""]
+ mode: [maintainer, no-maintainer]
+ variant: [""]
include:
- # Debian builds
- # 32bit build
+ # Debian 32-bit builds
- container: "debian:stable"
- env:
- ARCH: i386
- CC: gcc -m32
- CROSS_COMPILE: i686-linux-gnu
- VARIANT: i386
- MODE: maintainer
-
- # Debian cross compilation builds
- - container: "debian:stable"
- env:
- ARCH: armel
- CC: arm-linux-gnueabi-gcc
- CROSS_COMPILE: arm-linux-gnueabi
- VARIANT: cross-compile
- MODE: maintainer
+ arch: i386
+ compiler: gcc -m32
+ cross_compile: i686-linux-gnu
+ mode: maintainer
+ variant: i386
- container: "debian:stable"
- env:
- ARCH: arm64
- CC: aarch64-linux-gnu-gcc
- CROSS_COMPILE: aarch64-linux-gnu
- VARIANT: cross-compile
- MODE: maintainer
+ arch: i386
+ compiler: gcc -m32
+ cross_compile: i686-linux-gnu
+ mode: no-maintainer
+ variant: i386
+ # Debian cross compilation builds
- container: "debian:stable"
- env:
- ARCH: ppc64el
- CC: powerpc64le-linux-gnu-gcc
- CROSS_COMPILE: powerpc64le-linux-gnu
- VARIANT: cross-compile
- MODE: maintainer
+ arch: armel
+ compiler: arm-linux-gnueabi-gcc
+ cross_compile: arm-linux-gnueabi
+ mode: maintainer
+ variant: cross-compile
- container: "debian:stable"
- env:
- ARCH: s390x
- CC: s390x-linux-gnu-gcc
- CROSS_COMPILE: s390x-linux-gnu
- VARIANT: cross-compile
- MODE: maintainer
+ arch: arm64
+ compiler: aarch64-linux-gnu-gcc
+ cross_compile: aarch64-linux-gnu
+ mode: maintainer
+ variant: cross-compile
- # Debian gcc
- container: "debian:stable"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: maintainer
+ arch: ppc64el
+ compiler: powerpc64le-linux-gnu-gcc
+ cross_compile: powerpc64le-linux-gnu
+ mode: maintainer
+ variant: cross-compile
- container: "debian:stable"
- env:
- ARCH: x86-64
- CC: gcc
- VARIANT: sanitizers
- MODE: maintainer
+ arch: s390x
+ compiler: s390x-linux-gnu-gcc
+ cross_compile: s390x-linux-gnu
+ mode: maintainer
+ variant: cross-compile
+ # Debian GCC sanitizer builds
- container: "debian:stable"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: no-maintainer
+ arch: x86-64
+ compiler: gcc
+ mode: maintainer
+ variant: sanitizers
- container: "debian:testing"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: maintainer
-
- - container: "debian:testing"
- env:
- ARCH: x86-64
- CC: gcc
- VARIANT: sanitizers
- MODE: maintainer
-
- - container: "debian:testing"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: no-maintainer
-
- # Debian clang
- - container: "debian:stable"
- env:
- ARCH: x86-64
- CC: clang
- MODE: maintainer
-
- - container: "debian:stable"
- env:
- ARCH: x86-64
- CC: clang
- MODE: no-maintainer
-
- - container: "debian:testing"
- env:
- ARCH: x86-64
- CC: clang
- MODE: maintainer
-
- # musl (native)
- - container: "alpine:latest"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: maintainer
-
- # Fedora
- - container: "fedora:latest"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: maintainer
-
- - container: "fedora:latest"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: no-maintainer
-
- - container: "fedora:latest"
- env:
- ARCH: x86-64
- CC: clang
- MODE: maintainer
-
- # Ubuntu 32bit builds: gcc and clang
- - container: "ubuntu:hirsute"
- env:
- ARCH: i386
- CC: gcc -m32
- CROSS_COMPILE: i686-linux-gnu
- VARIANT: i386
- MODE: maintainer
-
- - container: "ubuntu:focal"
- env:
- ARCH: i386
- CC: gcc -m32
- CROSS_COMPILE: i686-linux-gnu
- VARIANT: i386
- MODE: maintainer
-
- # Ubuntu x86-64 gcc
- - container: "ubuntu:hirsute"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: maintainer
+ arch: x86-64
+ compiler: gcc
+ mode: maintainer
+ variant: sanitizers
+ # Ubuntu 32-bit builds
- container: "ubuntu:hirsute"
- env:
- ARCH: x86-64
- CC: gcc
- VARIANT: sanitizers
- MODE: maintainer
+ arch: i386
+ compiler: gcc -m32
+ cross_compile: i686-linux-gnu
+ mode: maintainer
+ variant: i386
- container: "ubuntu:hirsute"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: no-maintainer
-
- - container: "ubuntu:focal"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: maintainer
+ arch: i386
+ compiler: gcc -m32
+ cross_compile: i686-linux-gnu
+ mode: no-maintainer
+ variant: i386
- container: "ubuntu:focal"
- env:
- ARCH: x86-64
- CC: gcc
- VARIANT: sanitizers
- MODE: maintainer
+ arch: i386
+ compiler: gcc -m32
+ cross_compile: i686-linux-gnu
+ mode: maintainer
+ variant: i386
- container: "ubuntu:focal"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: no-maintainer
-
- - container: "ubuntu:bionic"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: maintainer
+ arch: i386
+ compiler: gcc -m32
+ cross_compile: i686-linux-gnu
+ mode: no-maintainer
+ variant: i386
- - container: "ubuntu:xenial"
- env:
- ARCH: x86-64
- CC: gcc
- MODE: maintainer
-
- # Ubuntu x86-64 clang
+ # Ubuntu GCC sanitizer builds
- container: "ubuntu:hirsute"
- env:
- ARCH: x86-64
- CC: clang
- MODE: maintainer
-
- - container: "ubuntu:focal"
- env:
- ARCH: x86-64
- CC: clang
- MODE: maintainer
+ arch: x86-64
+ compiler: gcc
+ mode: maintainer
+ variant: sanitizers
- container: "ubuntu:focal"
- env:
- ARCH: x86-64
- CC: clang
- MODE: no-maintainer
+ arch: x86-64
+ compiler: gcc
+ mode: maintainer
+ variant: sanitizers
container:
image: ${{ matrix.container }}
- env: ${{ matrix.env }}
+ env:
+ ARCH: ${{ matrix.arch }}
+ CC: ${{ matrix.compiler }}
+ CROSS_COMPILE: ${{ matrix.cross_compile }}
+ MODE: ${{ matrix.mode }}
+ VARIANT: ${{ matrix.variant }}
steps:
- name: Show OS
run: cat /etc/os-release
+ - name: Show env (matrix settings)
+ run: |
+ echo "ARCH: $ARCH"
+ echo "CC: $CC"
+ echo "CROSS_COMPILE: $CROSS_COMPILE"
+ echo "MODE: $MODE"
+ echo "VARIANT: $VARIANT"
+
- name: Git checkout
uses: actions/checkout@v2
@@ -272,11 +184,16 @@ jobs:
if [ "$VARIANT" = "sanitizers" ]; then
CONFIGURE_ARGS="--enable-asan --enable-lsan --enable-ubsan
--enable-pie"
fi
+ CONFIGURE_CROSS=""
+ if [ "$CROSS_COMPILE" ]; then
+ CONFIGURE_CROSS="--host=${CROSS_COMPILE}"
+ fi
if [ "$MODE" = "maintainer" ]; then
./bootstrap-configure \
--disable-silent-rules \
$CONFIGURE_ARGS
else
+ CONFIGURE_ARGS="$CONFIGURE_ARGS $CONFIGURE_CROSS"
./bootstrap && \
./configure \
--disable-systemd \
@@ -300,7 +217,7 @@ jobs:
id: unit_tests
run: |
make check
- if: ${{ (matrix.env['ARCH'] == 'x86-64') ||
(matrix.env['ARCH'] == 'i386') }}
+ if: ${{ (matrix.arch == 'x86-64') || (matrix.arch == 'i386') }}
- name: Get unit tests logs
run: cat test-suite.log
@@ -335,4 +252,4 @@ jobs:
run: make distcheck
# distcheck runs unit tests so no point for cross compile
# TODO: figure out how to re-configure during distcheck with --host for i386
(DISTCHECK_CONFIGURE_FLAGS)
- if: ${{ (matrix.env['ARCH'] == 'x86-64')}}
+ if: ${{ (matrix.arch == 'x86-64')}}
diff --git a/ci/alpine.sh b/ci/alpine.sh
index 81b5c72a392e..56449dd7c969 100755
--- a/ci/alpine.sh
+++ b/ci/alpine.sh
@@ -12,13 +12,20 @@ set -ex
apk update
PKGS_CC="gcc"
+case $CC in
+ clang*)
+ # On Alpine v3.14 clang fails without gcc:
+ # cannot find crtbeginS.o: No such file or directory
+ PKGS_CC="clang gcc"
+ ;;
+esac
# gzip: for distcheck
apk add \
autoconf \
autoconf-archive \
automake \
- gcc \
+ binutils \
gzip \
dbus-dev \
glib-dev \
--
2.30.2
9:08 p.m.
On Wed, Aug 04, 2021 at 10:42:07AM +0200, Krzysztof Kozlowski wrote:
Hi,
Add a Continuous Integration builds under Github (with its Actions) to
build and unit test on several configurations.
Changes since v1 [1]:
1. Rebase on latest master.
2. Remove CI-unrelated patches from this set.
3. Add CodeQL analysis.
4. Add builds and tests with GCC sanitizers.
5. Use matrix to extend the build configuration.
[1]
https://lore.kernel.org/linux-nfc/20210710033859.3989-1-krzysztof.kozlows...
Best regards,
Krzysztof
Applied.
Thank you.
Mark
--
167
days inactive
170
days old
12 comments
2 participants
participants (2)
-
Krzysztof Kozlowski -
Mark Greer