rework unpacking of packages and harden package file format requirements

A crafted .apk file could to trick apk writing unverified data to
an unexpected file during temporary file creation due to bugs in handling
long link target name and the way a regular file is extracted.

Several hardening steps are implemented to avoid this:
 - the temporary file is now always first unlinked (apk thus reserved
   all filenames .apk.* to be it's working files)
 - the temporary file is after that created with O_EXCL to avoid races
 - the temporary file is no longer directly the archive entry name
   and thus directly controlled by potentially untrusted data
 - long file names and link target names are now rejected
 - hard link targets are now more rigorously checked
 - various additional checks added for the extraction process to
   error out early in case of malformed (or old legacy) file

Reported-by: Max Justicz <max@justi.cz>

1 files changed, 4 insertions, 7 deletions

diff --git a/src/package.c b/src/package.c
index 3be8b84..e19250a 100644
--- a/src/package.c
+++ b/src/package.c
@@ -476,13 +476,7 @@ void apk_sign_ctx_init(struct apk_sign_ctx *ctx, int action,
 		ctx->md = EVP_md_null();
 		break;
 	case APK_SIGN_VERIFY_IDENTITY:
-		if (identity->type == APK_CHECKSUM_MD5) {
-			ctx->md = EVP_md5();
-			ctx->control_started = 1;
-			ctx->data_started = 1;
-		} else {
-			ctx->md = EVP_sha1();
-		}
+		ctx->md = EVP_sha1();
 		memcpy(&ctx->identity, identity, sizeof(ctx->identity));
 		break;
 	case APK_SIGN_GENERATE:
@@ -552,6 +546,9 @@ int apk_sign_ctx_process_file(struct apk_sign_ctx *ctx,
 		 * style .PKGINFO */
 		if (ctx->has_data_checksum)
 			return -ENOMSG;
+		/* Error out early if identity part is missing */
+		if (ctx->action == APK_SIGN_VERIFY_IDENTITY)
+			return -EKEYREJECTED;
 		ctx->data_started = 1;
 		ctx->control_started = 1;
 		r = check_signing_key_trust(ctx);