Merge tag '16.0' into vyos/current

shim-16.0

What's Changed
* Validate that a supplied vendor cert is not in PEM format by @steve-mcintyre in https://github.com/rhboot/shim/pull/646
* sbat: Add grub.peimage,2 to latest (CVE-2024-2312) by @julian-klode in https://github.com/rhboot/shim/pull/651
* sbat: Also bump latest for grub,4 (and to todays date) by @julian-klode in https://github.com/rhboot/shim/pull/653
* undo change that limits certificate files to a single file by @jsetje in https://github.com/rhboot/shim/pull/659
* shim: don't set second_stage to the empty string by @jjd27 in https://github.com/rhboot/shim/pull/640
* Fix SBAT.md for today's consensus about numbers by @aronowski in https://github.com/rhboot/shim/pull/672
* Update Code of Conduct contact address by @aronowski in https://github.com/rhboot/shim/pull/683
* make-certs: Handle missing OpenSSL installation by @aronowski in https://github.com/rhboot/shim/pull/595
* Update MokVars.txt by @mikebeaton in https://github.com/rhboot/shim/pull/598
* export DEFINES for sub makefile by @bryteise in https://github.com/rhboot/shim/pull/600
* Drop unused EFI_IMAGE_SECURITY_DATABASE_GUID definition by @vittyvk in https://github.com/rhboot/shim/pull/609
* Null-terminate 'arguments' in fallback by @vittyvk in https://github.com/rhboot/shim/pull/611
* Fix "Verifiying" typo in error message by @chrisbainbridge in https://github.com/rhboot/shim/pull/706
* Update Fedora CI targets by @vathpela in https://github.com/rhboot/shim/pull/708
* Force gcc to produce DWARF4 so that gdb can use it by @mikebeaton in https://github.com/rhboot/shim/pull/607
* Minor housekeeping 2024121700 by @vathpela in https://github.com/rhboot/shim/pull/709
* Discard load-options that start with WINDOWS by @Metabolix in https://github.com/rhboot/shim/pull/621
* Fix the issue that the gBS->LoadImage pointer was empty. by @15058718379 in https://github.com/rhboot/shim/pull/703
* shim: Allow data after the end of device path node in load options by @dbnicholson in https://github.com/rhboot/shim/pull/694
* Handle network file not found like disks by @dbnicholson in https://github.com/rhboot/shim/pull/695
* Update gnu-efi submodule for EFI_HTTP_ERROR by @vathpela in https://github.com/rhboot/shim/pull/674
* Increase EFI file alignment by @lumag in https://github.com/rhboot/shim/pull/673
* avoid EFIv2 runtime services on Apple x86 machines by @eduardacatrinei in https://github.com/rhboot/shim/pull/690
* Improve shortcut performance when comparing two boolean expressions by @dennis-tseng99 in https://github.com/rhboot/shim/pull/667
* Provide better error message when MokManager is not found by @rmetrich in https://github.com/rhboot/shim/pull/663
* tpm: Boot with a warning if the event log is full by @kukrimate in https://github.com/rhboot/shim/pull/657
* MokManager: remove redundant logical constraints by @xypron in https://github.com/rhboot/shim/pull/409
* Test import_mok_state() when MokListRT would be bigger than available size by @vathpela in https://github.com/rhboot/shim/pull/417
* test-mok-mirror: minor bug fix by @vathpela in https://github.com/rhboot/shim/pull/715
* Fix file system browser hang when enrolling MOK from disk by @miczyg1 in https://github.com/rhboot/shim/pull/622
* Ignore a minor clang-tidy nit by @vathpela in https://github.com/rhboot/shim/pull/716
* Allow fallback to default loader when encountering errors on network boot by @nathan-omeara in https://github.com/rhboot/shim/pull/666
* test.mk: don't use a temporary random.bin by @vathpela in https://github.com/rhboot/shim/pull/718
* pe: Enhance debug report for update_mem_attrs by @jongwu in https://github.com/rhboot/shim/pull/594
* Multiple certificate handling improvements by @rosslagerwall in https://github.com/rhboot/shim/pull/644
* Generate SbatLevel Metadata from SbatLevel_Variable.txt by @jsetje in https://github.com/rhboot/shim/pull/711
* Apply EKU check with compile option by @dennis-tseng99 in https://github.com/rhboot/shim/pull/664
* Add configuration option to boot an alternative 2nd stage by @esnowberg in https://github.com/rhboot/shim/pull/608
* Loader protocol (with Device Path resolution support) by @kukrimate in https://github.com/rhboot/shim/pull/656
* netboot cleanup for additional files by @jsetje in https://github.com/rhboot/shim/pull/686
* Document how revocations can be delivered by @jsetje in https://github.com/rhboot/shim/pull/722
* post-process-pe: add tests to validate NX compliance by @vathpela in https://github.com/rhboot/shim/pull/705
* regression: CopyMem() in ad8692e copies out of bounds by @jsetje in https://github.com/rhboot/shim/pull/725
* Save the debug and error logs in mok-variables by @vathpela in https://github.com/rhboot/shim/pull/726
* Add features for the Host Security ID program by @vathpela in https://github.com/rhboot/shim/pull/660
* Mirror some more efi variables to mok-variables by @vathpela in https://github.com/rhboot/shim/pull/723
* This adds DXE Services measurements to HSI and uses them for NX by @vathpela in https://github.com/rhboot/shim/pull/724
* Add shim's current NX_COMPAT status to HSIStatus by @vathpela in https://github.com/rhboot/shim/pull/727
* README.tpm: reflect that vendor_db is in fact logged as "vendor_db" by @jsetje in https://github.com/rhboot/shim/pull/728
* Reject HTTP message with duplicate Content-Length header fields by @dennis-tseng99 in https://github.com/rhboot/shim/pull/637
* Disable log saving by @vathpela in https://github.com/rhboot/shim/pull/729
* fallback: don't add new boot order entries backwards by @vathpela in https://github.com/rhboot/shim/pull/730
* Misc fixes... by @vathpela in https://github.com/rhboot/shim/pull/735
* README.tpm: Update MokList entry to MokListRT by @trungams in https://github.com/rhboot/shim/pull/732
* SBAT Level update for February 2025 GRUB CVEs by @jsetje in https://github.com/rhboot/shim/pull/736

New Contributors
* @jjd27 made their first contribution in https://github.com/rhboot/shim/pull/640
* @mikebeaton made their first contribution in https://github.com/rhboot/shim/pull/598
* @bryteise made their first contribution in https://github.com/rhboot/shim/pull/600
* @vittyvk made their first contribution in https://github.com/rhboot/shim/pull/609
* @chrisbainbridge made their first contribution in https://github.com/rhboot/shim/pull/706
* @Metabolix made their first contribution in https://github.com/rhboot/shim/pull/621
* @15058718379 made their first contribution in https://github.com/rhboot/shim/pull/703
* @dbnicholson made their first contribution in https://github.com/rhboot/shim/pull/694
* @lumag made their first contribution in https://github.com/rhboot/shim/pull/673
* @eduardacatrinei made their first contribution in https://github.com/rhboot/shim/pull/690
* @kukrimate made their first contribution in https://github.com/rhboot/shim/pull/657
* @miczyg1 made their first contribution in https://github.com/rhboot/shim/pull/622
* @nathan-omeara made their first contribution in https://github.com/rhboot/shim/pull/666
* @jongwu made their first contribution in https://github.com/rhboot/shim/pull/594
* @rosslagerwall made their first contribution in https://github.com/rhboot/shim/pull/644
* @trungams made their first contribution in https://github.com/rhboot/shim/pull/732

**Full Changelog**: https://github.com/rhboot/shim/compare/15.8...16.0

* tag '16.0': (451 commits)
  Update version to 16.0
  SBAT Level update for February 2025 GRUB CVEs
  README.tpm: Update MokList entry to MokListRT
  Make 'make fanalyzer' work again.
  simple_dir_filter(): test our 'next' pointer
  shim_load_image(): initialize the buffer fully
  mirror_mok_db(): Free our mok variable name correctly
  mirror_one_mok_variable(): fix a memory leak on TPM log error.
  mirror_mok_db(): get rid of an unused variable+allocation
  generate_sbat_var_defs: Ensure revlistentry->revocations is initialized.
  generate_sbat_var_defs: Fix memory leak on realloc failure and fd leak.
  generate_sbat_var_defs: run clang-format on readfile()
  SetSecureVariable(): free Cert on failure
  Update version to 16.0~rc1
  make-archive: some minor housekeeping
  makefiles: Make GITTAG swizzle tildes to dashes
  fallback: don't add new boot order entries backwards
  Disable log saving for now.
  Some save_logs() improvements.
  reject message with different values in multiple Content-Length header field
  ...

1 files changed, 83 insertions, 31 deletions

diff --git a/mock-variables.c b/mock-variables.c
index 03044549..723cdda2 100644
--- a/mock-variables.c
+++ b/mock-variables.c
@@ -163,7 +163,7 @@ variable_cmp(const struct mock_variable * const v0,
 
 	ret = CompareGuid(&v0->guid, &v1->guid);
 	ret <<= 8ul;
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if (defined(SHIM_DEBUG) && SHIM_DEBUG > 3)
 	printf("%s:%d:%s():  "GUID_FMT" %s "GUID_FMT" (0x%011"PRIx64" %"PRId64")\n",
 	       __FILE__, __LINE__-1, __func__,
 	       GUID_ARGS(v0->guid),
@@ -177,7 +177,7 @@ variable_cmp(const struct mock_variable * const v0,
 	}
 
 	ret = StrCmp(v0->name, v1->name);
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if (defined(SHIM_DEBUG) && SHIM_DEBUG > 3)
 	printf("%s:%d:%s():   \"%s\" %s \"%s\" (0x%02hhx (%d)\n",
 	       __FILE__, __LINE__-1, __func__,
 	       Str2str(v0->name),
@@ -284,7 +284,7 @@ mock_gnvn_set_result(UINTN *size, CHAR16 *name, EFI_GUID *guid,
 		*size = StrSize(result->name);
 		status = EFI_BUFFER_TOO_SMALL;
 		mock_gnvn_post_hook(size, name, guid, &status);
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 3
 		printf("%s:%d:%s():  returning %lx\n",
 		       __FILE__, __LINE__-1, __func__, status);
 #endif
@@ -297,7 +297,7 @@ mock_gnvn_set_result(UINTN *size, CHAR16 *name, EFI_GUID *guid,
 
 	status = EFI_SUCCESS;
 	mock_gnvn_post_hook(size, name, guid, &status);
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 3
 	printf("%s:%d:%s():  returning %lx\n",
 	       __FILE__, __LINE__-1, __func__, status);
 #endif
@@ -351,15 +351,20 @@ mock_get_next_variable_name(UINTN *size, CHAR16 *name, EFI_GUID *guid)
 		struct mock_variable *var;
 
 		var = list_entry(pos, struct mock_variable, list);
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if defined(SHIM_DEBUG)
+# if SHIM_DEBUG > 1
 		printf("%s:%d:%s():  candidate var:%p &var->guid:%p &var->list:%p\n",
 			__FILE__, __LINE__-1, __func__, var, &var->guid, &var->list);
+# elif SHIM_DEBUG > 0
+		printf("%s:%d:%s():  candidate var:%p var->guid:" GUID_FMT"\n",
+			__FILE__, __LINE__-1, __func__, var, GUID_ARGS(var->guid));
+# endif
 #endif
 		if (name[0] == 0) {
 			if (CompareGuid(&var->guid, guid) == 0) {
 #if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
-				printf("%s:%d:%s():  found\n",
-				       __FILE__, __LINE__-1, __func__);
+				printf("%s:%d:%s():  found guid in entry var:%p var->name:%p\n",
+				       __FILE__, __LINE__-1, __func__, var, var->name);
 #endif
 				result = var;
 				found = true;
@@ -374,14 +379,14 @@ mock_get_next_variable_name(UINTN *size, CHAR16 *name, EFI_GUID *guid)
 				continue;
 			}
 
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
 		printf("%s:%d:%s(): varcmp("GUID_FMT"-%s, "GUID_FMT"-%s)\n",
 		       __FILE__, __LINE__-1, __func__,
 		       GUID_ARGS(goal.guid), Str2str(goal.name),
 		       GUID_ARGS(var->guid), Str2str(var->name));
 #endif
 			if (variable_cmp(&goal, var) == 0) {
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
 				printf("%s:%d:%s():  found\n",
 				       __FILE__, __LINE__-1, __func__);
 #endif
@@ -391,15 +396,15 @@ mock_get_next_variable_name(UINTN *size, CHAR16 *name, EFI_GUID *guid)
 	}
 #if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
 	if (result) {
-		printf("%s:%d:%s():  found:%d result:%p &result->guid:%p &result->list:%p\n"
+		printf("%s:%d:%s():  found:%d result:%p &result->guid:%p &result->list:%p\n",
 		       __FILE__, __LINE__-1, __func__, found, result,
 		       &result->guid, &result->list);
 		printf("%s:%d:%s():  "GUID_FMT"-%s\n",
 		       __FILE__, __LINE__-1, __func__, GUID_ARGS(result->guid),
 		       Str2str(result->name));
 	} else {
-		printf("%s:%d:%s():  not found\n",
-		       __FILE__, __LINE__-1, __func__);
+		printf("%s:%d:%s():  not found (found:%d status:0x%016x)\n",
+		       __FILE__, __LINE__-1, __func__, found, status);
 	}
 #endif
 
@@ -408,13 +413,25 @@ mock_get_next_variable_name(UINTN *size, CHAR16 *name, EFI_GUID *guid)
 			status = EFI_NOT_FOUND;
 		else
 			status = EFI_INVALID_PARAMETER;
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
+		printf("%s:%d:%s():  not found (found:%d status:0x%016x)\n",
+		       __FILE__, __LINE__-1, __func__, found, status);
+#endif
 		mock_gnvn_post_hook(size, name, guid, &status);
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
+		printf("%s:%d:%s():  not found (found:%d status:0x%016x)\n",
+		       __FILE__, __LINE__-1, __func__, found, status);
+#endif
 		return status;
 	}
 
 	if (!result) {
 		status = EFI_NOT_FOUND;
 		mock_gnvn_post_hook(size, name, guid, &status);
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
+		printf("%s:%d:%s():  found (found:%d status:0x%016x)\n",
+		       __FILE__, __LINE__-1, __func__, found, status);
+#endif
 		return status;
 	}
 
@@ -678,7 +695,7 @@ mock_new_variable(CHAR16 *name, EFI_GUID *guid, UINT32 attrs, UINTN size,
 	}
 	var = (struct mock_variable *)buf;
 
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
 	printf("%s:%d:%s():  var:%p &var->guid:%p &var->list:%p\n",
 	       __FILE__, __LINE__-1, __func__, var, &var->guid, &var->list);
 #endif
@@ -695,7 +712,7 @@ mock_new_variable(CHAR16 *name, EFI_GUID *guid, UINT32 attrs, UINTN size,
 	var->attrs = attrs;
 	INIT_LIST_HEAD(&var->list);
 
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
 	printf("%s:%d:%s():  var: "GUID_FMT"-%s\n",
 	       __FILE__, __LINE__-1, __func__,
 	       GUID_ARGS(var->guid), Str2str(var->name));
@@ -772,10 +789,10 @@ mock_set_variable(CHAR16 *name, EFI_GUID *guid, UINT32 attrs, UINTN size,
 	}
 #endif
 
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
-	printf("%s:%d:%s():Setting "GUID_FMT"-%s\n",
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
+	printf("%s:%d:%s():Setting "GUID_FMT"-%s size:0x%"PRIx64"\n",
 	       __FILE__, __LINE__ - 1, __func__,
-	       GUID_ARGS(*guid), Str2str(name));
+	       GUID_ARGS(*guid), Str2str(name), size);
 #endif
 	switch (mock_variable_sort_policy) {
 	case MOCK_SORT_PREPEND:
@@ -800,7 +817,7 @@ mock_set_variable(CHAR16 *name, EFI_GUID *guid, UINT32 attrs, UINTN size,
 	list_for_each_safe(pos, tmp, &mock_variables) {
 		found = false;
 		var = list_entry(pos, struct mock_variable, list);
-#if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
 		printf("%s:%d:%s(): varcmp("GUID_FMT"-%s, "GUID_FMT"-%s)\n",
 		       __FILE__, __LINE__-1, __func__,
 		       GUID_ARGS(goal.guid), Str2str(goal.name),
@@ -832,32 +849,32 @@ mock_set_variable(CHAR16 *name, EFI_GUID *guid, UINT32 attrs, UINTN size,
 		if (found)
 			break;
 	}
-#if defined(SHIM_DEBUG) && SHIM_DEBUG != 0
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
 	printf("%s:%d:%s():var_list:%p &mock_variables:%p cmp:%ld\n",
 	       __FILE__, __LINE__ - 1, __func__,
 	       var_list, &mock_variables, cmp);
 #endif
 	if (cmp != 0 || (cmp == 0 && var_list == &mock_variables)) {
 		size_t totalsz = size + StrSize(name);
-#if defined(SHIM_DEBUG) && SHIM_DEBUG != 0
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
 		printf("%s:%d:%s():var:%p attrs:0x%lx\n",
 		       __FILE__, __LINE__ - 1, __func__, var, attrs);
 #endif
-		status = mock_new_variable(name, guid, attrs, size, data, &var);
+		status = mock_sv_adjust_usage_data(attrs, size, -totalsz);
 		if (EFI_ERROR(status)) {
 			mock_sv_post_hook(name, guid, attrs, size, data,
 					  &status, CREATE);
 			return status;
 		}
-		mock_sv_adjust_usage_data(attrs, size, totalsz);
+		status = mock_new_variable(name, guid, attrs, size, data, &var);
 		mock_sv_post_hook(name, guid, attrs, size, data,
 				  &status, CREATE);
 		if (EFI_ERROR(status)) {
-			mock_sv_adjust_usage_data(attrs, 0, -totalsz);
+			mock_sv_adjust_usage_data(attrs, 0, totalsz);
 			return status;
 		}
 
-#if defined(SHIM_DEBUG) && SHIM_DEBUG != 0
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 1
 		printf("%s:%d:%s(): Adding "GUID_FMT"-%s %s %s\n",
 		       __FILE__, __LINE__ - 1, __func__,
 		       GUID_ARGS(var->guid), Str2str(var->name),
@@ -1002,18 +1019,27 @@ static struct mock_variable_limits default_limits[] = {
 };
 
 void
+mock_set_usage_limits(list_t *limit_list,
+		      struct mock_variable_limits *limits)
+{
+	INIT_LIST_HEAD(limit_list);
+	for (size_t i = 0; limits[i].attrs != 0; i++) {
+		INIT_LIST_HEAD(&limits[i].list);
+		list_add_tail(&limits[i].list, limit_list);
+	}
+
+	mock_qvi_limits = limit_list;
+	mock_sv_limits = limit_list;
+}
+
+void
 mock_set_default_usage_limits(void)
 {
 	default_max_var_storage = 65536;
 	default_remaining_var_storage = 65536;
 	default_max_var_size = 32768;
 
-	INIT_LIST_HEAD(&mock_default_variable_limits);
-	for (size_t i = 0; default_limits[i].attrs != 0; i++) {
-		INIT_LIST_HEAD(&default_limits[i].list);
-		list_add_tail(&default_limits[i].list,
-			      &mock_default_variable_limits);
-	}
+	mock_set_usage_limits(&mock_default_variable_limits, &default_limits[0]);
 }
 
 void
@@ -1079,7 +1105,8 @@ mock_load_one_variable(int dfd, const char * const dirname, char * const name)
 
 	name[namelen-1] = 0;
 #if (defined(SHIM_DEBUG) && SHIM_DEBUG != 0)
-	printf("loading %s-%s\n", &name[namelen], name);
+	printf("%s:%d:%s(): loading %s-%s\n", __FILE__, __LINE__, __func__,
+	       &name[namelen], name);
 #endif
 	for (size_t i = 0; i < namelen; i++)
 		namebuf[i] = name[i];
@@ -1109,6 +1136,9 @@ mock_load_variables(const char *const dirname, const char *filters[],
 	DIR *d;
 	struct dirent *entry;
 
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 1
+	printf("Started loading variablles from \"%s\"\n", dirname);
+#endif
 	d = opendir(dirname);
 	if (!d)
 		err(1, "Could not open directory \"%s\"", dirname);
@@ -1121,6 +1151,11 @@ mock_load_variables(const char *const dirname, const char *filters[],
 	while ((entry = readdir(d)) != NULL) {
 		size_t len = strlen(entry->d_name);
 		bool found = false;
+		if (entry->d_type != DT_REG)
+			continue;
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 1
+		printf("%s:%d:%s(): maybe adding entry \"%s\"\n", __FILE__, __LINE__, __func__, entry->d_name);
+#endif
 		if (filters && len > guidstr_size + 1) {
 			char spacebuf[len];
 
@@ -1131,6 +1166,9 @@ mock_load_variables(const char *const dirname, const char *filters[],
 				if (strlen(filters[i]) > len)
 					continue;
 				if (!strncmp(entry->d_name, filters[i], len)) {
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 2
+					printf("%s:%d:%s(): filter matched for \"%s\" && \"%s\"\n", __FILE__, __LINE__, __func__, entry->d_name, filters[i]);
+#endif
 					found = true;
 					break;
 				}
@@ -1138,9 +1176,23 @@ mock_load_variables(const char *const dirname, const char *filters[],
 		}
 		if ((found == false && filter_out == true) ||
 		    (found == true && filter_out == false)) {
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 1
+			printf("%s:%d:%s(): Adding \"%s\" because filter %s\n",
+			       __FILE__, __LINE__-1, __func__, entry->d_name,
+			       found ? "matched" : "did not match");
+#endif
 			mock_load_one_variable(dfd, dirname, entry->d_name);
+		} else {
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 1
+			printf("%s:%d:%s(): Skipping \"%s\" because filter %s\n",
+			       __FILE__, __LINE__-1, __func__, entry->d_name,
+			       found ? "matched" : "did not match");
+#endif
 		}
 	}
+#if defined(SHIM_DEBUG) && SHIM_DEBUG >= 1
+	printf("Done loading variablles from \"%s\"\n", dirname);
+#endif
 
 	closedir(d);
 #if 0