Update tooling for GitHub-based new releases (#223)

* tools/read-version: don't enforce version parity in release branch CI

We have a bootstrapping problem with new releases, currently.  To take
the example of 20.1: the branch that bumps the version fails CI because
there is no 20.1 tag for it to use in read-version.  Previously, this
was solved by creating a tag and pushing it to the cloud-init repo
before the commit landed.  However, we have GitHub branch protection
enabled, so the commit that needs to be tagged is not created until the
pull request lands in master.

This works around this problem by introducing a very specific check: if
we are performing CI for an upstream release branch, we skip the
read-version checking that we know will fail.

* tools/make-tarball: add --version parameter

When using make-tarball as part of a CI build of a new upstream release,
the version it determines is inconsistent with the version that other
tools determine.  Instead of encoding the logic here (as well as in
Python elsewhere), we add a parameter to allow us to set it from outside
the script.

* packages/bddeb: handle missing version_long in new version CI

If we're running in CI for a new upstream release, we have to use
`version` instead of `version_long` (because we don't yet have the tag
required to generate `version_long`).

3 files changed, 26 insertions, 4 deletions

diff --git a/packages/bddeb b/packages/bddeb
index 95602a02..209765a5 100755
--- a/packages/bddeb
+++ b/packages/bddeb
@@ -177,6 +177,11 @@ def main():
 
         # output like 0.7.6-1022-g36e92d3
         ver_data = read_version()
+        if ver_data['is_release_branch_ci']:
+            # If we're performing CI for a new release branch, we don't yet
+            # have the tag required to generate version_long; use version
+            # instead.
+            ver_data['version_long'] = ver_data['version']
 
         # This is really only a temporary archive
         # since we will extract it then add in the debian
@@ -192,7 +197,9 @@ def main():
                 break
         if path is None:
             print("Creating a temp tarball using the 'make-tarball' helper")
-            run_helper('make-tarball', ['--long', '--output=' + tarball_fp])
+            run_helper('make-tarball',
+                       ['--version', ver_data['version_long'],
+                        '--output=' + tarball_fp])
 
         print("Extracting temporary tarball %r" % (tarball))
         cmd = ['tar', '-xvzf', tarball_fp, '-C', tdir]
diff --git a/tools/make-tarball b/tools/make-tarball
index 8d540139..462e7d04 100755
--- a/tools/make-tarball
+++ b/tools/make-tarball
@@ -15,24 +15,27 @@ Usage: ${0##*/} [revision]
     options:
       -h | --help		print usage
       -o | --output FILE	write to file
+           --version VERSION	Set the version used in the tarball. Default value is determined with 'git describe'.
            --orig-tarball	Write file cloud-init_<version>.orig.tar.gz
            --long		Use git describe --long for versioning
 EOF
 }
 
 short_opts="ho:v"
-long_opts="help,output:,orig-tarball,long"
+long_opts="help,output:,version:,orig-tarball,long"
 getopt_out=$(getopt --name "${0##*/}" \
     --options "${short_opts}" --long "${long_opts}" -- "$@") &&
     eval set -- "${getopt_out}" || { Usage 1>&2; exit 1; }
 
 long_opt=""
 orig_opt=""
+version=""
 while [ $# -ne 0 ]; do
     cur=$1; next=$2
     case "$cur" in
         -h|--help) Usage; exit 0;;
         -o|--output) output=$next; shift;;
+           --version) version=$next; shift;;
            --long) long_opt="--long";;
            --orig-tarball) orig_opt=".orig";;
         --) shift; break;;
@@ -41,7 +44,12 @@ while [ $# -ne 0 ]; do
 done
 
 rev=${1:-HEAD}
-version=$(git describe --abbrev=8 "--match=[0-9]*" ${long_opt} $rev)
+if [ -z "$version" ]; then
+    version=$(git describe --abbrev=8 "--match=[0-9]*" ${long_opt} $rev)
+elif [ ! -z "$long_opt" ]; then
+    echo "WARNING: --long has no effect when --version is passed" >&2
+    exit 1
+fi
 
 archive_base="cloud-init-$version"
 if [ -z "$output" ]; then
diff --git a/tools/read-version b/tools/read-version
index 6dca659e..92e9fc96 100755
--- a/tools/read-version
+++ b/tools/read-version
@@ -65,7 +65,13 @@ output_json = '--json' in sys.argv
 src_version = ci_version.version_string()
 version_long = None
 
-if is_gitdir(_tdir) and which("git"):
+# If we're performing CI for a new release branch (which our tooling creates
+# with an "upstream/" prefix), then we don't want to enforce strict version
+# matching because we know it will fail.
+is_release_branch_ci = (
+    os.environ.get("TRAVIS_PULL_REQUEST_BRANCH", "").startswith("upstream/")
+)
+if is_gitdir(_tdir) and which("git") and not is_release_branch_ci:
     flags = []
     if use_tags:
         flags = ['--tags']
@@ -113,6 +119,7 @@ data = {
     'extra': extra,
     'commit': commit,
     'distance': distance,
+    'is_release_branch_ci': is_release_branch_ci,
 }
 
 if output_json: