hexdump.h: fix arithmetic error.

When I modified the hexdumper to help debug MokListRT mirroring not
working because of PcdMaxVolatileVariableSize being tiny, I
inadvertently added something that is effectively:

hexdump(..., char *buf, ..., int position)
{
	unsigned long begin = (position % 16);
	unsigned long i;
	...
	for (i = 0; i < begin; i++) {
		...
	}
	...
}

Unfortunately, in c if 0x8 is set in position, that means begin is
0xfffffffffffff8, because signed integer math is horrifying:

include/hexdump.h:99:vhexdumpf() &data[offset]:0x9E77E6BC size-offset:0x14
include/hexdump.h:15:prepare_hex() position:0x9E77E6BC
include/hexdump.h:17:prepare_hex() before:0xFFFFFFFFFFFFFFFC size:0x14
include/hexdump.h:19:prepare_hex() before:0xFFFFFFFFFFFFFFFC after:0x0
include/hexdump.h:21:prepare_hex() buf:0x000000009E77E2BC offset:0 &buf[offset]:0x000000009E77E2BC

Woops.

This could further have been prevented in /some/ cases by simply not
preparing the hexdump buffer when "verbose" is disabled.

This patch makes "pos" be unsigned in all cases, and also checks for
verbose in vhexdumpf() and simply returns if it is 0.

Signed-off-by: Peter Jones <pjones@redhat.com>

0 files changed, 0 insertions, 0 deletions