fix integer overflow in float printf needed-precision computation

if the requested precision is close to INT_MAX, adding
LDBL_MANT_DIG/3+8 overflows. in practice the resulting undefined
behavior manifests as a large negative result, which is then used to
compute the new end pointer (z) with a wildly out-of-bounds value
(more overflow, more undefined behavior). the end result is at least
incorrect output and character count (return value); worse things do
not seem to happen, but detailed analysis has not been done.

this patch fixes the overflow by performing the intermediate
computation as unsigned; after division by 9, the final result
necessarily fits in int.

1 files changed, 1 insertions, 1 deletions

diff --git a/src/stdio/vfprintf.c b/src/stdio/vfprintf.c
index e439a07a..cd17ad70 100644
--- a/src/stdio/vfprintf.c
+++ b/src/stdio/vfprintf.c
@@ -312,7 +312,7 @@ static int fmt_fp(FILE *f, long double y, int w, int p, int fl, int t)
 	}
 	while (e2<0) {
 		uint32_t carry=0, *b;
-		int sh=MIN(9,-e2), need=1+(p+LDBL_MANT_DIG/3+8)/9;
+		int sh=MIN(9,-e2), need=1+(p+LDBL_MANT_DIG/3U+8)/9;
 		for (d=a; d<z; d++) {
 			uint32_t rm = *d & (1<<sh)-1;
 			*d = (*d>>sh) + carry;