summaryrefslogtreecommitdiff
path: root/libfetch
AgeCommit message (Collapse)AuthorFilesLines
2023-05-02libfetch, apk: display warning for permanent redirects during initTimo Teräs3-0/+7
fixes #10776
2023-05-02fetch: enable --timeout to set network progress timeoutTimo Teräs2-14/+55
implement it also for connecting to hosts fixes #10869
2022-12-21apk, fetch: implement --no-check-certificateTimo Teräs2-2/+13
fixes #10650
2021-12-17libfetch: Allow special characters in http basic authJan Hendrik Farr1-14/+62
Currently, special characters in the username or password are not handled correctly (when set in $http_proxy and $https_proxy). They should be percent encoded in the environment variables then decoded by libfetch and reencoded using base64. This implementation is mainly taken from the current FreeBSD source and adapted to the apk-tools version of libfetch. fixes #10775
2021-12-14libfetch: use pkg-config to find openssl cflagsAriadne Conill1-0/+7
on mac, openssl is usually provided by Homebrew or some other third-party package management system, which means pkg-config is needed to find it. we already use pkg-config to find openssl when building apk itself. ref #10794
2021-08-03libfetch: fix http chunked mode handlingTimo Teräs1-1/+1
Unbreak handling of base 16 in fetch_parseuint(). It is used only in http chunked mode handling. Fixes: "libfetch: fix range checking for http/ftp protocol parsing"
2021-07-26libfetch: fix range checking for http/ftp protocol parsingTimo Teräs5-61/+70
Various parsing of numeric strings were not having adequate range checking causing information leak or potential crash. CVE-2021-36159 fixes #10749 Co-authored-by: Ariadne Conill <ariadne@dereferenced.org> Reported-by: Samanta Navarro <ferivoz@riseup.net>
2021-07-23libfetch: simplify code by merging protocol error handling branchesTimo Teräs2-36/+22
removes some code duplication
2021-03-16libfetch: send Proxy-Authorization also for https connectTimo Teräs1-8/+20
fixes #10738
2021-03-13Use correct port when redirectedMartin Vahlensieck1-1/+1
If server redirects from http to https, libfetch detects this, but wrongly uses the old url scheme to determine the port. This subsequently leads to the following OpenSSL error: 139741541575496:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:ssl/record/ssl3_record.c:331: Using the new scheme fixes this. This error message comes from trying to connect to port 80 with TLS, it can also be observed by issuing $ openssl s_client -connect alpinelinux.org:80 This bug was introduced in commit: 7158474 libfetch: keep http auth only if redirect is for the same host
2021-01-19libfetch: harden URL parsingTimo Teräs1-11/+28
Treat URLs with too long individual components as malformed instead of silently truncating that field. There might be unexpected results if hostname, username or password field gets truncated.
2021-01-19libfetch: fix connection pooling for proxied http/https requestsTimo Teräs4-15/+14
The connection pooling was broken in two ways: 1. The original URL was always used as the connection pool URL, resulting in duplicate connections to the proxy for http URLs (each http URL would get separate proxy connection) 2. The cache_url stored was always the socket level connect URL. In case of HTTPS, the lookup was done done with the real URL, but the proxy URL was stored as the "cache URL". Thus HTTPS CONNECT connections were never re-used. This fixes the code with following logic: 1. The cache key url is the real URL when no-proxy, or when HTTPS with proxy (the socket is connected to proxy, but logically it is connected to the real URL due to HTTP CONNECT request). And for HTTP with proxy, it's the proxy URL so same proxy connection can be reused for all requests going through it. 2. fetch_connect() now gets cache key URL separately, and it always gets the same value as the fetch_cache_get() calls.
2021-01-19libfetch: fix use-after-free in connection cache managementTimo Teräs1-3/+3
fixes #10734
2021-01-19libfetch: fix parsing of proxy response to CONNECT requestsConny Seifert1-6/+1
Instead of skipping just one line, properly parse the response headers. [TT: reworded commit message]
2021-01-17libfetch: support for CIDR subnets in no_proxy/NO_PROXYTimo Teräs1-1/+65
This enables matching numeric hosts against CIDR style subnet matches in no_proxy environment variable. Based on work by Thermi. Co-authored-by: Noel Kuntze <noel.kuntze@thermi.consulting>
2021-01-08libfetch: Increase URL password lengthMike Detwiler1-1/+1
Signed-off-by: Mike Detwiler <det@shift5.io>
2020-10-06libfetch: keep http auth only if redirect is for the same hostTimo Teräs1-1/+7
fixes #10688
2020-05-29libfetch: implement default port for HTTPS.Ariadne Conill2-0/+3
If getservbyname() fails, libfetch will attempt to connect to port 0.
2020-05-06add script to autogenerate help from man pagesTimo Teräs1-3/+3
This creates main help like: -- usage: apk [<OPTIONS>...] COMMAND [<ARGUMENTS>...] Package installation and removal: add Add packages to WORLD and commit changes del Remove packages from WORLD and commit changes System maintenance: fix Check WORLD against the system and ensure consistency update Update repository indexes upgrade Install upgrades available from repositories cache Commands related to the management of an offline package cache Querying package information: info Give detailed information about packages or repositories list List packages matching a pattern or other criteria dot Generate graphviz graphs policy Show repository policy for packages Repository maintenance: index Create repository index file from packages fetch Download packages from global repositories to a local directory manifest Show checksums of package contents verify Verify package integrity and signature Miscellaneous: audit Audit directories for changes stats Show statistics about repositories and installations version Compare package versions or perform tests on version strings This apk has coffee making abilities. -- And applet specific help like: -- usage: apk add [<OPTIONS>...] PACKAGES... Description: apk add adds the requested packages to WORLD and installs (or upgrades) them if not already present, ensuring all dependencies are met. Options: --initdb Initialize a new package database -l, --latest Disables normal heuristics for choosing which repository to install a -u, --upgrade When adding packages which are already installed, upgrade them rather -t, --virtual NAME Instead of adding the specified packages to WORLD, create a new --no-chown Do not change file owner or group --
2020-04-01libfetch: fix no_proxy domain name comparisionTimo Teräs1-1/+1
Fix comparing of the hostname portion that matches exactly. The no_proxy matching is pretty rudimentary though and probably could go through a bit of additional rework. Fixes #10681
2020-03-25libfetch: fixups to packetization socket option settingTimo Teräs1-26/+22
- split the code to a helper function - do not set sockets to corked state when putting back to cache so socket state is always deterministic - cork/uncork also when sending CONNECT to a proxy, this can reduce a little bit the latency how fast the packet gets sent out - also pair corking with uncorking in http_request to make it more obvious pairing
2020-03-25libfetch: minor HTTP handling improvementAlexander Wauck1-5/+2
The recent TCP_CORK change missed this bit of code. This change should improve performance a bit when making HTTP requests by calling http_cmd only once instead of three times.
2020-02-18libfetch: support TCP_CORKTimo Teräs1-23/+22
Unfortunately libfetch operates on raw sockets and is sending each HTTP request line using separate syscall which causes the HTTP request to be sent as multiple packets over the wire in most configurations. This is not good for performance, but can also cause subtle breakage if there's DPI firewall that does not get the Host header. Incidentally, it seems that on BSDs libfetch already sets TCP_NOPUSH optimize the packetization. This commit adds same logic for using TCP_CORK if available. When using TCP_CORK there is no requirement to set TCP_NODELAY as uncorking will also cause immediate send. Keep TCP_NODELAY in the fallback codepaths. Long term, it might make sense to replace or rewrite libfetch to use application level buffering.
2019-02-13fix strncpy bounds errorsTimo Teräs1-2/+2
error: 'strncpy' specified bound 4096 equals destination size [-Werror=stringop-truncation] Based on patch by Elan Ruusamäe <glen@delfi.ee>
2018-09-05libfetch: do not give out user/hostname as ftp anonymous passwordTimo Teräs1-15/+5
This is unwanted information disclosure. Reported-by: Max Justicz <max@justi.cz>
2018-01-31libfetch: support OpenSSLA. Wilcox1-1/+1
TLS_client_method is a LibreSSL extension. SSLv23_client_method is generic, and doesn't mean SSL v2/v3 only.
2018-01-03libfetch: add option to set "Cache-Control: no-cache"Timo Teräs1-1/+4
ref #8161
2018-01-03libfetch: honor https_proxy variable for httpsTimo Teräs1-11/+30
fixes #8160
2017-10-06libfetch: fix certificate host name checkTimo Teräs1-1/+1
OpenSSL allows passing zero-length to indicate "use strlen". LibreSSL requires using the real length always, so pass the length.
2017-10-06libfetch: improve openssl/libressl compatibilityTimo Teräs4-7/+436
X509_check_host() is introduced in libressl-2.5.0 and openssl-1.0.2 which are not yet universally available. Add support for building against the older versions.
2017-10-05libfetch: improve ssl connectionsTimo Teräs3-6/+111
loosely based on the freebsd implementation, implement https connection settings to override CA, and use client certificate. new files supported in /etc/apk/: ca.pem - if exists, it contains CAs acceptable for https (otherwise system wide CAs are used) crl.pem - if ca.pem is used, this is the (optional) CRL for it cert.pem - used as client authentication certificate (+ key) cert.key - used as client key (can be also inside cert.pem)
2017-10-05libfetch: remove unwanted code conditionalsTimo Teräs7-94/+7
2017-10-05libfetch: fix ssl context leakTimo Teräs1-0/+14
from freebsd
2017-10-05build and use bundled libfetch nativelyTimo Teräs6-100/+9
2017-10-05import libfetch-2.38 from NetBSDTimo Teräs13-0/+6630
ftp://ftp.fu-berlin.de/unix/NetBSD/packages/current-src/pkgsrc/net/libfetch/files libfetch comes (at least) in netbsd and freebsd flavors which differing functionality. Alpine and Arch package netbsd one, but it's not widely packaged across other distributions. We need NetBSD version as it does not use funopen(3) which is not supported in musl, and supports connection pooling. FreeBSD seems to be the orignal and better maintained version with support for SSL CAs, client certificate authentication, proxy authentication, and improved http redirect handling. So this imports NetBSD version, and future commits will pick up the needed improvements from FreeBSD tree. Incidentally, this also fixes #7857 and likes for good.
generated by cgit v1.2.3-60-g2f50 (git 2.42.0) at 2024-06-29 09:44:12 +0000