diff options
-rw-r--r-- | .gitignore | 1 | ||||
-rw-r--r-- | Jenkinsfile | 11 | ||||
-rw-r--r-- | README.md | 149 | ||||
-rw-r--r-- | docker/Dockerfile (renamed from Dockerfile) | 15 | ||||
-rwxr-xr-x | docker/entrypoint.sh | 32 | ||||
-rwxr-xr-x | scripts/build-submodules | 17 | ||||
-rwxr-xr-x | scripts/docker-entrypoint.sh | 17 |
7 files changed, 149 insertions, 93 deletions
@@ -3,3 +3,4 @@ build/* packer_build/* packer_cache/* key/* +packages/*.buildlog diff --git a/Jenkinsfile b/Jenkinsfile index 611777cc..5dda60ef 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -21,7 +21,8 @@ pipeline { dockerfile { filename 'Dockerfile' label 'jessie-amd64' - args '--privileged --sysctl net.ipv6.conf.lo.disable_ipv6=0' + dir 'docker' + args '--privileged --sysctl net.ipv6.conf.lo.disable_ipv6=0 -e GOSU_UID=1006 -e GOSU_GID=1006' } } @@ -44,12 +45,16 @@ pipeline { sh ''' #!/bin/sh ./configure --build-by="autobuild@vyos.net" --debian-mirror="http://ftp.us.debian.org/debian/" - ls -al - ls -al packages + ls -al packages/*.deb sudo make iso ''' } } + stage('Deploy') { + steps { + archiveArtifacts artifacts: 'build/vyos-*-rolling*.iso', fingerprint: true + } + } } post { @@ -3,9 +3,9 @@ VyOS toplevel build # Important! -This repository is for building the VyOS version 1.2.0 and above. -For VyOS 1.1.x, use the build-iso repository. +This repository is for building the VyOS versions 1.2.x and above. +For VyOS 1.1, use the [build-iso](https://github.com/vyos/build-iso) repository. # What is VyOS @@ -48,12 +48,14 @@ There are several directories with their own purpose: ## Prerequisites -To build a VyOS 1.2.0 image, you need Debian 8 "Jessie" environment (with jessie-backports repository). +To build a VyOS 1.2.0 image, you need Debian 8 "Jessie" environment (with +jessie-backports repository). -If you do not have a Debian Jessie machine, you may create a chroot environment with the -[debootstrap](https://wiki.debian.org/Debootstrap) tool. +If you do not have a Debian Jessie machine, you may create a chroot environment +with the [debootstrap](https://wiki.debian.org/Debootstrap) tool. -For example, on another version of Debian or another Debian-based distro, these commands will work: +For example, on another version of Debian or another Debian-based distro, these +commands will work: ```bash $ sudo apt-get install debootstrap @@ -64,8 +66,9 @@ $ echo "deb http://deb.debian.org/debian jessie-backports main" >> /etc/apt/sour $ apt-get update ``` -If you are working on a Debian Jessie machine, no special preparation is needed, you only -need to enable jessie-backports and install build dependencies. +If you are working on a Debian Jessie machine, no special preparation is needed, +you only need to enable jessie-backports and install build dependencies. An +up-to-date depnedency list can be found in out [Dockerfile](docker/Dockerfile). Several packages are required for building the ISO: * `python3` @@ -79,9 +82,11 @@ package build scripts will tell you. ## Building the ISO image inside a docker container -Using our `Dockerfile` you can create your own Docker container that can be used -to build a VyOS ISO image. The `Dockerfile` contains some of the most used -packages needed to build a VyOS ISO, a qemu image, and several of the submodules. +Using our [Dockerfile](docker/Dockerfile) you create your own Docker container +that is used to build a VyOS ISO image or other required VyOS packages. The +[Dockerfile](docker/Dockerfile) contains some of the most used packages needed +to build a VyOS ISO, a qemu image, and several of the submodules. Please note +that this is not complete and only gives you a brief overview! ``` squashfs-tools # Required for squashfs file system @@ -114,70 +119,73 @@ python3-lxml # Optional, for building vyos-1x python3-setuptools # Optional, for building vyos-1x python3-nose # Optional, for building vyos-1x python3-coverage # Optional, for building vyos-1x +... ``` -To build the docker image: -``` -docker build -t vyos-builder $PATH_TO_Dockerfile -``` +To build the docker image ensure you have a working [Docker](https://www.docker.com) +environment and then run the following commands: -### Linux - -To run the docker image: -``` -docker run -it --privileged -v /HOST_PATH_OF_VYOS_BUILD_REPO:/vyos -w="/vyos" vyos-builder bash +```bash +$ docker build -t vyos-builder docker ``` -This will drop you into a bash shell with this vyos-build repo mounted at /vyos. -Then follow the instructions bellow to build the VyOS iso and qemu image. +Run the newly built container: +```bash +$ docker run --rm -it --privileged -v $(pwd):/vyos -w /vyos vyos-builder bash +``` -### MacOS and Windows +This will drop you into a bash shell with this vyos-build repo mounted at +`/vyos`. Then follow the instructions bellow to build the VyOS ISO and QEMU +image. -To run the docker image: -``` -docker run -dt --privileged -v /HOST_PATH/images:/vyos --name=vyos_node_builder vyos-builder bash +```bash +vyos_bld@948a2be7c52c:/vyos$ uname -a +Linux 948a2be7c52c 3.16.0-7-amd64 #1 SMP Debian 3.16.59-1 (2018-10-03) x86_64 GNU/Linux ``` -NOTE: +**NOTE:** * Docker container must be run with `--privileged` flag * We recommended to run the container with a volume mapped in order to easy export built VyOS ISO images to the "external" world +* UNIX ownership is automatically inherited from your host directory but can be + altered by specifying the following environment variables when running the + container: `-e GOSU_UID=$(id -u)` and/or `-e GOSU_GID=$(id -g)` -To connect to the docker image once is running: -``` -docker exec -it vyos_node_builder bash -``` - -After the docker container is running you can git clone the vyos-build repository -inside the container and follow up the bellow instructions in order to build the -VyOS ISO image +After the Docker container is running you can follow up the instructions below in +order to build the VyOS ISO image. ## Building subpackages inside Docker + Prior to building packages you need to checkout and update the submodules you want to compile + ```bash -git submodule update --init packages/PACKAGENAME -cd packages/PACKAGENAME -git checkout BRANCH +$ git submodule update --init packages/PACKAGENAME +$ cd packages/PACKAGENAME +$ git checkout BRANCH ``` + `PACKAGENAME` is the name of the package you want to compile -`BRANCH` is for Crux(1.2) `crux`, for latest rolling use `current` +`BRANCH` is `crux` for VyOS 1.2.x, latest rolling releases use `current` +Fetching all submodules at once and update them to the recent remote branches +`HEAD` is done by calling: -### Pulling all packages -Use this with caution, only run this on a unmodified newly cloned repository ```bash -for dir in packages/*; do - git submodule update --init $dir - pushd $dir - git checkout current - popd -done +$ git submodule update --init --recursive +$ git submodule update --remotes ``` + ### Building packages -The script `./scripts/build-submodules` is created to automate the process of building packages, execute it in the root of `vyos-build` to start compilation on all supported packages that are checked out. -The easiest way to compile is with the `vyos-builder` docker container, it includes all dependencies for compiling supported packages. +The [scripts/build-submodules](scripts/build-submodules) script is used to +automate the process of building (in the future) all VyOS related packages that +are not part of the upstream Debian version. Execute it in the root of the +`vyos-build` directory to start compilation on all supported packages that are +checked out. + +The easiest way to compile is with the above mentioned [Docker](docker/Dockerfile) +container, it includes all dependencies for compiling supported packages. ```bash $ docker run --rm -it -v $(pwd):/vyos -w /vyos \ @@ -186,37 +194,52 @@ $ docker run --rm -it -v $(pwd):/vyos -w /vyos \ ./scripts/build-submodules ``` -NOTE: Prior to executing this script you need to create/build the `vyos-builder` container and checkout all packages you want to compile. +**NOTE:** `--sysctl net.ipv6.conf.lo.disable_ipv6=0` is required to build the +`vyos-strongswan` package + +**NOTE:** Prior to executing this script you need to create or build the Docker +container and checkout all packages you want to compile. + +### Building a single package + +The script above runs all package build inside the Docker container, this is also +possible to do by hand using: + +Executed from the root of `vyos-build` -### Building one package -the script above runs all package build inside a docker container, this is also possible to do by hand using: -Executed from the root directory of vyos-build ```bash $ docker run --rm -it -v $(pwd):/vyos -w /vyos/packages/PACKAGENAME \ --sysctl net.ipv6.conf.lo.disable_ipv6=0 \ vyos-builder \ dpkg-buildpackage -uc -us -tc -b ``` -NOTE: `--sysctl net.ipv6.conf.lo.disable_ipv6=0` is only needed when building vyos-strongswan and can be ignored on other packages -NOTE: Prior to executing this you need to checkout and update the submodules you want to recompile -NOTE: vyos-strongswan will only compile on a linux system, running on osx or windows migth result in a unittest lockup. (it never exits) -Packages that are known to not build using this procedure: +**NOTE:** `--sysctl net.ipv6.conf.lo.disable_ipv6=0` is only needed when +building `vyos-strongswan` and can be ignored on other packages. + +**NOTE:** Prior to executing this you need to checkout and update the submodules +you want to recompile! + +**NOTE:** `vyos-strongswan` will only compile on a Linux system, running on macOS +or Windows migth result in a unittest deadlock (it never exits). + +Packages that are known to not build using this procedure (as of now): + ``` vyatta-util - Not needed anymore -vyatta-quagga - Not needed anymore +vyatta-quagga - Not needed anymore vyos-1x - Unmet build dependencies: whois libvyosconfig0 vyos-frr - Alott of requirements, scary stuff... vyos-kernel - Need special build instructions vyos-wireguard - Needs special build instructions ``` - ## Building the ISO image Before you can build an image, you need to configure your build. To build an image, use the following commands: + ```bash $ ./configure $ make iso @@ -230,6 +253,7 @@ with `--help` ### QEMU Run following command after building the ISO image. + ```bash $ make qemu ``` @@ -237,6 +261,7 @@ $ make qemu ### VMware Run following command after building the QEMU image. + ```bash $ make vmware ``` @@ -262,8 +287,6 @@ In packages that originate from VyOS the master branch is kept in sync with last legacy package is gone, we will switch to using the `master` branch and retire `current`. -For branch naming we use chemical elements: -* hydrogen -* helium -* lithium +For branch naming we switched to use constellations: +* `crux` * ... diff --git a/Dockerfile b/docker/Dockerfile index 4c66a15a..505660ab 100644 --- a/Dockerfile +++ b/docker/Dockerfile @@ -123,7 +123,7 @@ RUN apt-get update && apt-get install -y \ liblua5.1-dev # Packages needed for vyos-frr -RUN sudo apt-get update && sudo apt-get install -y \ +RUN apt-get update && apt-get install -y \ texinfo \ imagemagick \ groff \ @@ -156,12 +156,9 @@ RUN export LATEST="$(curl -s https://checkpoint-api.hashicorp.com/v1/check/packe curl -K- | gzip -d > /usr/bin/packer && \ chmod +x /usr/bin/packer -COPY scripts/docker-entrypoint.sh /usr/local/bin/ -# Create vyos_bld user account and enable sudo -#RUN useradd -ms /bin/bash -u 1006 --gid users vyos_bld && \ -# usermod -aG sudo vyos_bld && \ -# echo "%sudo ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers +# Allow password-less 'sudo' for all users in group 'sudo' +RUN sed "s/^%sudo.*/%sudo\tALL=(ALL) NOPASSWD:ALL/g" -i /etc/sudoers && \ + chmod a+s /usr/sbin/useradd /usr/sbin/groupadd /usr/sbin/gosu /usr/sbin/usermod -#USER vyos_bld -#WORKDIR /home/vyos_bld -ENTRYPOINT ["docker-entrypoint.sh"] +COPY entrypoint.sh /usr/local/bin/entrypoint.sh +ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh new file mode 100755 index 00000000..8db41103 --- /dev/null +++ b/docker/entrypoint.sh @@ -0,0 +1,32 @@ +#!/bin/bash +set -e + +USER_NAME="vyos_bld" +NEW_UID=$(stat -c "%u" .) +NEW_GID=$(stat -c "%g" .) + +# Change effective UID to the one specified via "-e GOSU_UID=`id -u $USER`" +if [ -n "$GOSU_UID" ]; then + NEW_UID=$GOSU_UID +fi + +# Change effective UID to the one specified via "-e GOSU_GID=`id -g $USER`" +if [ -n "$GOSU_GID" ]; then + NEW_GID=$GOSU_GID +fi + +# Notify user about selected UID/GID +echo "Current UID/GID: $NEW_UID/$NEW_GID" + +# Create UNIX group on the fly if it does not exist +if ! grep -q $NEW_GID /etc/group; then + groupadd --gid $NEW_GID $USER_NAME +fi + +useradd --shell /bin/bash --uid $NEW_UID --gid $NEW_GID --non-unique --create-home $USER_NAME +usermod --append --groups sudo $USER_NAME +sudo chown $NEW_UID:$NEW_GID /home/$USER_NAME +export HOME=/home/$USER_NAME + +# Execute process +exec /usr/sbin/gosu $USER_NAME "$@" diff --git a/scripts/build-submodules b/scripts/build-submodules index 86a7d8ae..bb5f55ce 100755 --- a/scripts/build-submodules +++ b/scripts/build-submodules @@ -160,6 +160,10 @@ for PKG in mdns-repeater \ vyos-world \ ; do build_package "$PKG" + ERRCODE=$? + if [ "$ERRCODE" -ne "0" ]; then + exit $ERRCODE + fi done @@ -192,6 +196,10 @@ build_kernel() { status_ok } build_kernel +ERRCODE=$? +if [ "$ERRCODE" -ne "0" ]; then + exit $ERRCODE +fi # WIREGUARD @@ -236,7 +244,10 @@ build_wireguard() { status_ok } build_wireguard - +ERRCODE=$? +if [ "$ERRCODE" -ne "0" ]; then + exit $ERRCODE +fi # ACCEL-PPP build_accel-ppp() { @@ -279,3 +290,7 @@ build_accel-ppp() { status_ok } build_accel-ppp +ERRCODE=$? +if [ "$ERRCODE" -ne "0" ]; then + exit $ERRCODE +fi diff --git a/scripts/docker-entrypoint.sh b/scripts/docker-entrypoint.sh deleted file mode 100755 index 7520a8d9..00000000 --- a/scripts/docker-entrypoint.sh +++ /dev/null @@ -1,17 +0,0 @@ -#!/bin/bash - -set -e - -# Use GOSU_USER if its specified, else wirking dir user -if [ -n "$GOSU_USER" ]; then - ID=$GOSU_USER -else - ID=$(stat -c "%u:%g" .) -fi - -# Don't use GOSU if we are root -if [ ! "$ID" = "0:0" ]; then - exec gosu $ID "$@" -else - exec "$@" -fi |