summaryrefslogtreecommitdiff
path: root/scripts/build/chroot_firmware
AgeCommit message (Collapse)Author
2020-04-23remove some unwanted quoting of list parametersLyndon Brown
there are additional instances in binary_* scripts that are left here because they are covered by changes in MR #157 Gbp-Dch: Short
2020-03-20strip progress-linux distro hacksLyndon Brown
progress-linux, as discussed in MR #142 ([1]) is a little known distro, which appears to be little more than a personal project of the original author of live-build. given that, the expense of maintaining all of these old hacks for it cannot be justified. it is not known whether or not live-build is even used with respect to it since the author abandoned live-build some years ago. also, at least one past change in live-build possibly broke progress-linux compatibility anyway, which would have required progress-linux users of live-build to use a custom progress-linux config, or a progress-linux fork of live-build, and there is no knowing how much of the hacks in this "upstream" codebase any user of progress-linux currently relies upon. and again, progress-linux appears to just be a personal project of Daniel's, with afaik very little userbase. (Daniel seems to be the only developer working on the project which speaks to how small it is). [1]: https://salsa.debian.org/live-team/live-build/-/merge_requests/142 Gbp-Dch: Short
2020-03-17stagefiles: s/Require_stagefile/Require_stagefiles/Lyndon Brown
this function takes one or more required stage fileS _plural_, and exits if any are missing (or at least it does now after the refactor). let's rename it to make things more clear Gbp-Dch: Short
2020-03-17stagefiles: further robustify with auto filenamesLyndon Brown
as suggested by Raphaël rather than have fixed stagefile filename strings at all in the scripts, use `$(basename $0)` to use the name of the script (which is the same for almost all cases anyway, and the stage files are supposed to be almost exclusively unique per-script). we can thus simplify things by determining the filename for most use cases within the functions themselves. this does change the file used by a couple of scripts, affecting backwards compatibility of executing live-build upon an existing partially or fully completed build: - binary_grub-pc used "binary_grub" - chroot_includes used "includes.chroot" care had to be taken for the following cases: - there are some cases like bootstrap_cache, source_debian and bootstrap_debootstrap which are dealing with more than one file, and/or otherwise a filename that is not specific to the script itself exactly, or should not be based upon its name. - some cases like chroot_cache, bootstrap_cache and chroot_install-packages need to append something to the end of the name depending upon which pass/action mode the script is being executed with. - furthermore in the bootstrap_cache case one of the filenames is used within the bootstrap_debootstrap and thus needs very careful handling to be certain that a change in filename of bootstrap_cache does not break bootstrap_debootstrap. Gbp-Dch: Short
2020-03-17stagefiles: simplify & robustifyLyndon Brown
- avoid all need to pass ".build/" path in stage file names into the functions - add a helper to remove a stage file (required to complete the above properly) - avoid duplicating filenames within scripts which makes them prone to mistakes (some instances of which I've actually encountered and had to fix) Gbp-Dch: Short
2020-03-15fix wrong descriptionLyndon Brown
Gbp-Dch: Ignore
2020-03-15fix missing stage descriptionsLyndon Brown
Gbp-Dch: Ignore
2020-03-13fix comment typoLyndon Brown
Gbp-Dch: Ignore
2020-03-13locks: tidy lock acquisitionjnqnfe
Combine the check+create done in each script. (The original functions are still callable as before, but a new combined `Aquire_lockfile` function can be called instead, as now used). Note, a further simplification could be done in removing the passing of the lock filename in as a parameter since every use of the functions is with ".lock". The lock functions already have a fallback to ".build/lock" though. Checking the history, the fallback used to be for a system wide lock, which was then replaced with this config-tree specific one. As long as that is not used implicitly by 3rd-party hooks then surely we are free to change the fallback to ".lock" and further remove passing in a name as a param...? history: db5d2b0dcdae96e712661605e17bc9875e224f9f 0aa8289a3773fd8a3885090b72622c2f95ab099c Gbp-Dch: Short Closes: #952918
2020-03-11amend copyright & licensing blocksLyndon Brown
Current versions of the project files are built upon versions published and licensed by Daniel Baumann, but are modified copies of those files and thus need to be marked as such per licensing requirements (afaik he did not pass along ownership / licensing rights to anyone when he left the project). We should also be careful to not be misrepresenting such modified copies as being attributed to Daniel. Adding a new copyright line referring to "The Debian Live team" should suffice for this. The authorship block in man pages has also similarly been updated. Notes: - tweaked a copy of daniel copyright lines stating 2014 instead of 2015. both of these cases were in files that i had personally introduced in some of my past merged commits that moved some code around. i don't know why they stated 2014. - binary_onie was introduced in 2018, so that has a 2018 date instead of 2016 unlike the rest. - 'efi-image' is a 3rd-party (Canonical Ltd) work that we bundle, but it has been modified by 674794a8f4d61a729d2dbd6d99385d2826138694 and 36a3ba76347ef72df1c316312ed3a26aa4b0c816 so I similarly added a debian live copyright line. - 'grub-cpmodules' is similar. it was only changed by the indentation fix of 36a3ba76347ef72df1c316312ed3a26aa4b0c816 but modification is modification, and this does help cover any possible future changes that might be made.
2020-03-10help/usage: remove pointless varsLyndon Brown
build scripts never call Help() and so the empty HELP strings are pointless. (when called with --help they call Man()). Closes: #952859 Gbp-Dch: Short
2020-03-10tidy script init (1/4) - arg and config processingjnqnfe
Partial fix for #952919 Gbp-Dch: Short
2020-03-05firmware: enable caching for archive content file with firmware listsLyndon Brown
the existing logic for obtaining a list of firmware packages always downloaded a fresh copy of the archive content file, deleting the file already in the cache. here we move to actually making use of the cache. this helps when building multiple times, at least for the same distro. the package list obtained is rarely going to change after all. it could of course differ between distros, but the cache is per-distro, as it has always been. we of course here switch to caching each of the archive-area files individually rather than having one file that gets overwritten (or appended to in the case of when we kept the decompressed file). Gbp-Dch: Short Closes: #952911
2020-03-05firmware: avoid potentially duplicate workLyndon Brown
addressing an old fixme, should distro and parent-distro be identical (in a derivative build) we want to avoid wasting effort downloading and processing the same content files twice. since parent and non-parent have separate archive-area lists though we should perhaps not just assume that they are identical or ignore any differences; thus here in such a situation we ensure that we handle any archive areas not already done for parent-distro handling in such a case, while skipping those already done. i notice that the fixme actually also refers to avoiding actual overlapping of the cached files, however the cached files are (currently) always ignored anyway, so this is of no concern. reusing the cached files is an entirely separate issue. Gbp-Dch: Short Closes: #952909
2020-03-05firmware: reorder firmware list constructionLyndon Brown
the 'manually add firmware-linux package' bit was stuck inbetween the parent and non-parent logic, which was especially unhelpful before we de-duplicated the logic into a common function. Gbp-Dch: Short
2020-03-05firmware: de-dup firmware list parsingLyndon Brown
Edit: There were four copies of the same logic to keep in sync; Originally this patch deduplicated each file, but leaving a copy of the new function in each, thus reducing the duplication but not eliminating it. A later patch moved it into a shared function file following further enhancements to the code in question. This has since been revised to have the function moved to a shared file here, which simplifies and gives a cleaner diff. Gbp-Dch: Short Closes: #952908
2020-03-05firmware: delete pointlessly cached (large) file after useLyndon Brown
the archive content file downloaded to obtain a list of firmware packages is always deleted and downloaded afresh currently. it may not be ideal that we do not make use of the cache here, however while that remains unaddressed, we might as well delete the file after we've used it in order to not pointlessly waste disk space. note that this file is ~613 MB for sid-amd64 currently. Gbp-Dch: Short Closes: #952907
2020-03-05firmware: fix possible duplication in firmware package lists (inefficiency)Lyndon Brown
where multiple archive areas are used, the code here on each loop is: 1) fetching the archive area contents file (compressed) 2) **appending** the output to that of the previous loop 3) searching the file for firmware references, adding to the list since it appends rather than replaces, entries found in each loop get re-added on each subsequent loop, resulting in duplication in the resulting list below I evaluate the possible solutions to explain why I chose the one I chose, however the reader should not waste too much time worrying about whether one of the other solution would have actually been better because things are changed significantly in further commits shortly! possible solutions: a) switching to output (>) rather than append (>>), but this might fail against an existing file b) removing the file on each loop, but this will complicate any future caching improvements that might be made here (currently the files are always deleted and thus downloaded fresh) c) allow the appending, evaluating the complete file after the loop solution C warrants consideration of disk space consumption; currently the compressed 'main' archive (for sid on amd64) expands to 592.3 MB (feb-2020), 'contrib' is 3.1 MB, and 'non-free' is 18.5 MB. solution C was chosen here; the difference of accumulated file size vs. max-single was minor enough to not be of particular concern (~613 vs. ~592 MB). Gbp-Dch: Short Closes: #952906
2020-03-05firmware: construct file location once and reuseLyndon Brown
2020-03-05firmware: avoid building unnecessary listsLyndon Brown
2020-03-05help/usage: fix overly complex script description handlingjnqnfe
Closes: #952887
2019-12-19Support --distribution-binary and --distribution-chrootRaphaël Hertzog
This makes it possible to build an image against a first distribution (--distribution-chroot) and have the resulting image point to another distribution (--distribution-binary). We can use this to build against a snapshot and have the result use the original distribution that was snapshotted. Closes: #888507
2015-04-27Removing temporary hack to exclude prism2-usb-firmware-installer, the ↵Daniel Baumann
download homepage seems to work again (Closes: #783433).
2015-01-04Updating year in copyright notices to 2015.Daniel Baumann
2015-01-04Adding wget dependency in chroot_firmware.Daniel Baumann
2014-12-10Cleaning up from python removal (LIVE_IMAGE_PARENT_ARCHIVE_AREAS).Daniel Baumann
2014-12-10Address "should check that we're building on debian through e.g. a ↵jnqnfe
'derivative-is-based-on' variable or somesuch" fixme.
2014-12-10Cleaning up from python removal (LIVE_IMAGE_ARCHIVE_AREAS).Daniel Baumann
2014-12-10Cleaning up from python removal (LIVE_IMAGE_ARCHITECTURE).Daniel Baumann
2014-11-28Dropping prism2-usb-firmware-installer which fetches firmware from (at ↵Daniel Baumann
least) temporarily broken website.
2014-02-08Updating copyright notices for 2014.Daniel Baumann
2013-05-06Removing all references to my old email address.Daniel Baumann
2013-05-06Updating year in all copyright notices.Daniel Baumann
2013-05-06Splitting off chroot firmware handling into an own script.Daniel Baumann