summaryrefslogtreecommitdiff
path: root/functions
AgeCommit message (Collapse)Author
2020-03-11remove obsolete fdisk hackLyndon Brown
8321653cb36511324d576e65cb13b5c9b0c5f438 (from 2007) introduced a hack to work around bug #445304 in gnu-fdisk for users who may have replaced fdisk with the classic gnu version. the hack allowed users to select an alternate fdisk binary to use to work around the buggy binary. bug #445304 is marked as found in v1.0-1 and fixed in v1.2-1, though may have been fixe din v1.1. it was marked fixed in 2009. checking the package archive, gnu-fdisk does not actually exist anymore in debian, with one exception - it is available for arm64 on sid via debports, and that version is 1.3 so thus includes the necessary fix anyway. it is thus pointless now that we still carry this hack. Gbp-Dch: Short
2020-03-11"Live Systems" -> "Debian Live"Lyndon Brown
2020-03-11amend copyright & licensing blocksLyndon Brown
Current versions of the project files are built upon versions published and licensed by Daniel Baumann, but are modified copies of those files and thus need to be marked as such per licensing requirements (afaik he did not pass along ownership / licensing rights to anyone when he left the project). We should also be careful to not be misrepresenting such modified copies as being attributed to Daniel. Adding a new copyright line referring to "The Debian Live team" should suffice for this. The authorship block in man pages has also similarly been updated. Notes: - tweaked a copy of daniel copyright lines stating 2014 instead of 2015. both of these cases were in files that i had personally introduced in some of my past merged commits that moved some code around. i don't know why they stated 2014. - binary_onie was introduced in 2018, so that has a 2018 date instead of 2016 unlike the rest. - 'efi-image' is a 3rd-party (Canonical Ltd) work that we bundle, but it has been modified by 674794a8f4d61a729d2dbd6d99385d2826138694 and 36a3ba76347ef72df1c316312ed3a26aa4b0c816 so I similarly added a debian live copyright line. - 'grub-cpmodules' is similar. it was only changed by the indentation fix of 36a3ba76347ef72df1c316312ed3a26aa4b0c816 but modification is modification, and this does help cover any possible future changes that might be made.
2020-03-11functions: consistency fixLyndon Brown
missed in 7ee59d408ed7681908966a5b2fb28e8f98116d31 Gbp-Dch: Ignore
2020-03-11exit: fix no /usr/bin/env errorLyndon Brown
if you execute the bootstrap stage with no internet connection, you get the following output: ``` [2020-03-10 19:18:46] lb bootstrap P: Setting up clean exit handler [2020-03-10 19:18:46] lb bootstrap_cache restore [2020-03-10 19:18:46] lb bootstrap_debootstrap P: Begin bootstrapping system... P: If the following stage fails, the most likely cause of the problem is with your mirror configuration or a caching proxy. P: Running debootstrap (download-only)... I: Retrieving InRelease I: Retrieving Release E: Failed getting release file http://deb.debian.org/debian/dists/buster/Release P: Begin unmounting filesystems... P: Saving caches... chroot: failed to run command ‘/usr/bin/env’: No such file or directory ``` the last line looked suspicious. investigating it turns out that there was a deficiency in the exit handler. when debootstrap fails to download what it needs due to lack of a connection, that failure due to `set -e` causes the Exit() handler to kick in. Part of this includes outputting the "Saving caches..." line, before then making a call to Save_package_cache(). That in turn runs the following command: ``` Chroot chroot "apt-get autoclean" || true ``` The Chroot() function includes a line starting with: ``` ${_LINUX32} chroot "${CHROOT}" /usr/bin/env ``` which is the source of the last output line. the reason we see this unexpected output is that with bootstrapping having failed, there is no /usr/bin/env within the chroot so it is bound to fail. the fact is, the exit handler has no business trying to pretty much anything that it does if the bootstrap_debootstrap stage has not completed. this implements such a restriction and thus resolves the problem of this unexpected and confusing output in the described situation. we will now see: ``` [2020-03-10 19:18:46] lb bootstrap P: Setting up clean exit handler [2020-03-10 19:18:46] lb bootstrap_cache restore [2020-03-10 19:18:46] lb bootstrap_debootstrap P: Begin bootstrapping system... P: If the following stage fails, the most likely cause of the problem is with your mirror configuration or a caching proxy. P: Running debootstrap (download-only)... I: Retrieving InRelease I: Retrieving Release E: Failed getting release file http://deb.debian.org/debian/dists/buster/Release ```
2020-03-10config: improve documentationLyndon Brown
2020-03-10archives: always include enabled/disabled deb-src apt entriesLyndon Brown
LB_APT_SOURCE_ARCHIVES determines whether or not deb-src entries are desired to be included in apt's sources.list. here, instead of excuding them we always include them but commented out where they would previously have been excluded. this means that if a user later changes their mind and wants to make use of them all they have to do is uncomment them rather than add the necessary lines. Gbp-Dch: Short Closes: #952929
2020-03-10arguments: fix unreachable and poor argument error handlingLyndon Brown
all scripts use `set -e` which means that if getop fails, the subsequent error check that would print an error in addition to any printed by getopt itself would never actually be reached. the first though here would be to remove the pointless error check, but getopt does not include the word "error" with an unrecognised option failure, nor does it use colour to highlight problems, both of which mean that it is a little lacking in terms of highlighting problems to users. thus we properly capture and use the exit code here and output an appropriate message per invalid argument vs getopt internal error. also, removed the redundant stderr redirection which is already done by Echo_error(). Gbp-Dch: Short
2020-03-10help/usage: simplifyLyndon Brown
Gbp-Dch: Ignore
2020-03-10help/usage: fix output of `lb config --usage`Lyndon Brown
broken by d0eb72a5efd0b1cdbd22f3b5afa3cf16a731b482 also, needed spaces before per-script output Gbp-Dch: Ignore
2020-03-10help/usage: fix too many empty linesLyndon Brown
Gbp-Dch: Ignore
2020-03-10config: rename the config set/check functions for clarityjnqnfe
Gbp-Dch: Short Closes: #952920
2020-03-10tidy script init (4/4) - top level cmd "auto redirect" handlingLyndon Brown
Partial fix for #952919 Gbp-Dch: Short Closes: #952919
2020-03-10tidy script init (2/4) - build stage scriptsjnqnfe
Partial fix for #952919 Gbp-Dch: Short
2020-03-10tidy script init (1/4) - arg and config processingjnqnfe
Partial fix for #952919 Gbp-Dch: Short
2020-03-09echo: fix bad colour choiceLyndon Brown
white is not going to work well on a terminal with a white background, obviously. We should keep the standard colour and just try applying the bold. or do we want to consider a non black/white colour? like blue...? Gbp-Dch: Short
2020-03-09echo: don't mangle backslashes for file printingLyndon Brown
2020-03-09use actual bool state where applicable within functionsLyndon Brown
2020-03-09archives: tidy deb-src exclusionLyndon Brown
instead of conditionally writing deb-src lines, it is **much** neater if we use sed to optionally removed them at the end. Gbp-Dch: Short Closes: #952928
2020-03-09fix consistency in binary execution and existance checkingLyndon Brown
- prefer using `which` over hard coded paths - it is redundant to check that the bin pointed to the return of `which` exists and is executable, `which` already gives us assurance of that if it returns true! - the redirection of output (`2>/dev/null`) seems to be unnecessary from my testing. the instances relatnig to fdisk and losetup in functions/defaults.sh have been left as they are since they get executed by `lb config` which can run without sudo elevation unlike `lb build` and in that case `which` would fail to find these binaries resulting in error. this also fixes a bug showing an error for missing debootstrap - this tool requires sudo privileges to run and thus is not found via a none elevated which search. Gbp-Dch: Short Closes: #952927
2020-03-09defaults: ensure global caching param overrides specific caching params if ↵jnqnfe
disabled Closes: #952923
2020-03-05firmware: enable caching for archive content file with firmware listsLyndon Brown
the existing logic for obtaining a list of firmware packages always downloaded a fresh copy of the archive content file, deleting the file already in the cache. here we move to actually making use of the cache. this helps when building multiple times, at least for the same distro. the package list obtained is rarely going to change after all. it could of course differ between distros, but the cache is per-distro, as it has always been. we of course here switch to caching each of the archive-area files individually rather than having one file that gets overwritten (or appended to in the case of when we kept the decompressed file). Gbp-Dch: Short Closes: #952911
2020-03-05firmware: save the compressed contents file to disk instead of decompressedLyndon Brown
the existing logic was to decompress the contents file from the downloaded archive to disk, then process it to obtain a package list. the largest one by far is for 'main'; 'non-free' and 'contrib' are tiny in comparison. for sid-amd64 currently, the archive file is 37 MB, while the decompressed file it contains is 592.3 MB. we always delete the files and download afresh (currently), and a previous commit optimised by deleting the files once we're done with them to avoid wasting disk space leaving them behind. here we switch to storing the downloaded compressed file to disk instead, reducing disk space usage (and IO) by hundreds of megabytes; piping the decompression directly into awk instead of having awk read from the stored file. this moves the appending of new items into the list back within the archive area loop, which is fine since we're replacing the file for each loop now so the previous issue relating to appending is of no concern. Gbp-Dch: Short Closes: #952910
2020-03-05firmware: de-dup firmware list parsingLyndon Brown
Edit: There were four copies of the same logic to keep in sync; Originally this patch deduplicated each file, but leaving a copy of the new function in each, thus reducing the duplication but not eliminating it. A later patch moved it into a shared function file following further enhancements to the code in question. This has since been revised to have the function moved to a shared file here, which simplifies and gives a cleaner diff. Gbp-Dch: Short Closes: #952908
2020-03-05cache: clarify and simplify package cache save/restorejnqnfe
These functions are specific to handling packages stored in the cache, not other files. They are also always used with the same `cache/packages.` prefix to the path. Gbp-Dch: Short Closes: #952916
2020-03-05aliases: simplify excessively complex In_list functionjnqnfe
This function is never used to find multiple needles at one time. Gbp-Dch: Short Closes: #952915
2020-03-05archives: deduplicate apt sourcelist file constructionjnqnfe
Geez :O Gbp-Dch: Short Closes: #952889
2020-03-05echo: tidyLyndon Brown
Gbp-Dch: Ignore
2020-03-05cursor: purge unused cursor functionsLyndon Brown
only a couple were in use and only by unused echo helpers which have now themselves been removed, so nothing in this file is needed. Partial fix for #952880 Gbp-Dch: Short Closes: #952880
2020-03-05echo: purge unused *_running echo helpersLyndon Brown
Partial fix for #952880 Gbp-Dch: Short
2020-03-05echo: purge some unused echo helpersLyndon Brown
Partial fix for #952880 Gbp-Dch: Short
2020-03-05echo: direct warnings via echo helper to stderrLyndon Brown
2020-03-05echo: ensure output goes to stdout/stderrLyndon Brown
all of these echo helpers are essentially 'logging' functions with output always intended for stdout/stderr. lack of explicit stdout/stderr direction means that their output could be captured unintentionally should they be used within a function designed to construct a string. Gbp-Dch: Short Closes: #952879
2020-03-05echo: fix problem with error printingLyndon Brown
lack of stderr directed output for the `E:` prefix meant that it would not appear alongside the message in some use cases Gbp-Dch: Short Closes: #952878
2020-03-05fix missing use of echo helpersjnqnfe
Closes: #952876
2020-03-05help/usage: fix overly complex script description handlingjnqnfe
Closes: #952887
2020-03-05help/usage: fix incorrect program commandLyndon Brown
Closes: #952884
2020-03-05help/usage: avoid unnecessary use of echo helpersjnqnfe
Closes: #952877
2020-03-05fix capitalisation of some output messagesjnqnfe
Closes: #952875
2020-03-05defaults: fix pointlessly split line errorjnqnfe
inconsistent with all other output Gbp-Dch: Ignore Closes: #952874
2020-03-05memtest: better handle obsolete LB_MEMTEST valueLyndon Brown
available values currently are memtest86+|memtest86|none; "false" is presumably handled for backwards compatibility there is no need to handle this in individual scripts. the right place to handle it is in Set_Defaults as now done Gbp-Dch: Short Closes: #952866
2020-03-05simplify --debian-installer choicesjnqnfe
`false` and `none` make no sense as choices for this option. Here we replace `false` with `none`, and remove `true`. Note that `true` was treated as an alias for netinst (see the changes to source_disk and and binary_disk). For backwards compatibility we still allow `true` and `false` by converting them to `netinst` and `none` respectively, whilst printing a warning to encourage users to move to `netinst`/`none`. Gbp-Dch: Short Closes: #952864
2020-03-05don't unnecessarily convert exit codes to stringsLyndon Brown
Closes: #952861
2020-03-05fix indentationLyndon Brown
including: - spaces replaced with tabs for consistency - alignment of `;;` in some case statements changed for consistency Gbp-Dch: Short Closes: #952857
2020-01-22Get rid of extraneous whitespace showing up in syslinux menuRaphaël Hertzog
The leading space ends up in the FLAVOUR substitution in live.cfg.
2019-12-19Support --distribution-binary and --distribution-chrootRaphaël Hertzog
This makes it possible to build an image against a first distribution (--distribution-chroot) and have the resulting image point to another distribution (--distribution-binary). We can use this to build against a snapshot and have the result use the original distribution that was snapshotted. Closes: #888507
2019-12-19installer_debian-installer: don't mess with /var/lib/dpkg/statusRaphaël Hertzog
Use a similar trick as for chroot_live-packages. Relates to #944983.
2019-12-19Set default distribution to busterRaphaël Hertzog
Closes: #934495 Thanks: Алексей Шилин
2019-12-19chroot_live-packages: use dpkg-query to see if a package is installedRaphaël Hertzog
Closes: #944983
2019-06-03Reenabled the command line option for colored messages for all scripts. The ↵Roland Clobus
functionality was added 2008-11-01, but was not really active.