PaulHowarth/Blog/2023-12-06

Wednesday 6th December 2023

Local Packages

  • Updated curl to 8.5.0:

    • gnutls: Support CURLSSLOPT_NATIVE_CA

    • HTTP3: ngtcp2 builds are no longer experimental

    • appveyor: Make VS2008-built curl tool runnable

    • asyn-thread: Use pipe instead of socketpair for IPC when available

    • autotools: Accept linker flags via 'CURL_LDFLAGS_{LIB,BIN}'

    • autotools: Avoid passing 'LDFLAGS' twice to libcurl

    • autotools: Delete LCC compiler support bits

    • autotools: Fix/improve gcc and Apple clang version detection

    • autotools: Stop setting '-std=gnu89' with '--enable-warnings'

    • autotools: Update references to deleted 'crypt-auth' option

    • BINDINGS: Add V binding

    • build: Add 'src/.checksrc' to source tarball

    • build: Add more picky warnings and fix them

    • build: Always revert '#pragma GCC diagnostic' after use

    • build: Delete 'HAVE_STDINT_H' and 'HAVE_INTTYPES_H'

    • build: Delete support bits for obsolete Windows compilers

    • build: Fix 'threadsafe' feature detection for older gcc

    • build: Fix builds that disable protocols but not digest auth
    • build: Fix compiler warning with auths disabled

    • build: Fix libssh2 + 'CURL_DISABLE_DIGEST_AUTH' + 'CURL_DISABLE_AWS'

    • build: Picky warning updates
    • build: Require Windows XP or newer

    • cfilter: Provide call to tell connection to forget a socket

    • checksrc.pl: Support #line instructions

    • CI: Add autotools, out-of-tree, debug build to distro check job

    • CI: Ignore test 286 on Appveyor gcc 9 build

    • cmake: Add 'CURL_DISABLE_BINDLOCAL' option

    • cmake: Add test for 'DISABLE' options, add 'CURL_DISABLE_HEADERS_API'

    • cmake: Dedupe Windows system libs

    • cmake: Fix 'HAVE_H_ERRNO_ASSIGNABLE' detection

    • cmake: Fix CURL_DISABLE_GETOPTIONS

    • cmake: Fix multiple include of CURL package
    • cmake: Fix OpenSSL quic detection in quiche builds

    • cmake: Option to disable install and drop 'curlu' target when unused

    • cmake: Pre-fill rest of detection values for Windows

    • cmake: Replace 'check_library_exists_concat()'

    • cmake: Speed up threads setup for Windows

    • cmake: Speed up zstd detection

    • config-win32: Set 'HAVE_SNPRINTF' for mingw-w64

    • configure: Better --disable-http

    • configure: Check for the fseeko declaration too

    • conncache: Use the closure handle when disconnecting surplus connections

    • content_encoding: Make Curl_all_content_encodings allocless

    • cookie: Lowercase the domain names before PSL checks (CVE-2023-46218)

    • curl.h: Delete Symbian OS references

    • curl.h: On FreeBSD include sys/param.h instead of osreldate.h

    • curl.rc: Switch out the copyright symbol for plain ASCII

    • curl: Improved IPFS and IPNS URL support

    • curl_easy_duphandle.3: Clarify how HSTS and alt-svc are duped

    • Curl_http_body: Clean up properly when Curl_getformdata errors

    • curl_setup: Disallow Windows IPv6 builds missing getaddrinfo

    • curl_sspi: Support more revocation error names in error messages

    • CURLINFO_PRETRANSFER_TIME_T.3: Fix time explanation

    • CURLMOPT_MAX_CONCURRENT_STREAMS: Make sure the set value is within range

    • CURLOPT_CAINFO_BLOB.3: Explain what CURL_BLOB_COPY does

    • CURLOPT_WRITEFUNCTION.3: Clarify libcurl returns for CURL_WRITEFUNC_ERROR

    • CURPOST_POSTFIELDS.3: Add CURLOPT_COPYPOSTFIELDS in SEE ALSO

    • docs/example/keepalive.c: Show TCP keep-alive options

    • docs/example/localport.c: Show off CURLOPT_LOCALPORT

    • docs/examples/interface.c: Show CURLOPT_INTERFACE use

    • docs/libcurl: Fix three minor man page format mistakes

    • docs/libcurl: SYNOPSIS clean up

    • docs: Add supported version for the json write-out

    • docs: Clarify that curl passes on input unfiltered

    • docs: Fix function typo in curl_easy_option_next.3

    • docs: KNOWN_BUGS clean up

    • docs: Make all examples in all libcurl man pages compile

    • docs: Preserve the modification date when copying the prebuilt man page

    • docs: Remove bold from some man page SYNOPSIS sections

    • docs: Use SOURCE_DATE_EPOCH for generated manpages

    • doh: Provide better return code for responses w/o addresses

    • doh: Use PIPEWAIT when HTTP/2 is attempted

    • duphandle: Also free 'outcurl->cookies' in error path

    • duphandle: Make dupset() not return with pointers to old alloced data

    • duphandle: Use strdup to clone *COPYPOSTFIELDS if size is not set

    • easy: In duphandle, init the cookies for the new handle

    • easy: Remove duplicate wolfSSH init call

    • easy_lock: Add a pthread_mutex_t fallback

    • examples/rtsp-options.c: Add

    • fopen: Create new file using old file's mode

    • fopen: Create short(er) temporary file name (CVE-2023-46219)

    • getenv: PlayStation doesn't have getenv()

    • GHA: Move mod_h2 version in CI to v2.0.25

    • hostip: Show the list of IPs when resolving is done

    • hostip: Silence compiler warning '-Wparentheses-equality'

    • hsts: Skip single-dot hostname
    • HTTP/2, HTTP/3: Handle detach of ongoing transfers
    • http2: Header conversion tightening

    • http2: Provide an error callback and failf the message

    • http2: Safer invocation of populate_binsettings

    • http: Allow longer HTTP/2 request method names

    • http: Avoid Expect: 100-continue if Upgrade: is used

    • http: Consider resume with CURLOPT_FAILONERRROR and 416 to be fine

    • http: Fix '-Wunused-parameter' with no auth and no proxy

    • http: Fix '-Wunused-variable' compiler warning

    • http: Fix empty-body warning

    • http_aws_sigv4: Canonicalise valueless query params

    • hyper: Temporarily remove HTTP/2 support

    • INSTALL: Update list of ports and CPU archs

    • IPFS: Fix IPFS_PATH and file parsing

    • keylog: Disable if unused

    • lib: Add and use Curl_strndup()

    • lib: Apache style infof and trace macros/functions

    • lib: Fix gcc warning in printf call

    • libcurl-errors.3: Sync with current public headers

    • libcurl-thread.3: Simplify the TLS section

    • Makefile.am: Drop vc10, vc11 and vc12 projects from dist

    • Makefile.mk: Fix '-rtmp' option for non-Windows

    • mime: Store "form escape" as a single bit

    • misc: Fix -Walloc-size warnings

    • msh3: Error when built with CURL_DISABLE_SOCKETPAIR set

    • multi: During ratelimit multi_getsock should return no sockets

    • multi: Use pipe instead of socketpair to *wakeup()

    • ngtcp2: Fix races in stream handling
    • ngtcp2: Ignore errors on unknown streams

    • ntlm_wb: Use pipe instead of socketpair when possible

    • openldap: Move the alloc of ldapconninfo to *connect()

    • openldap: Set the callback argument in oldap_do

    • openssl: Avoid BN_num_bits() NULL pointer derefs

    • openssl: Fix building with v3 'no-deprecated' + add CI test

    • openssl: Fix infof() to avoid compiler warning for %s with null

    • openssl: Identify the "quictls" backend correctly

    • openssl: Include SIG and KEM algorithms in verbose

    • openssl: Make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs

    • openssl: Two multi pointer checks should probably rather be asserts

    • openssl: When a session-ID is reused, skip OCSP stapling
    • page-footer: Clarify exit code 25
    • projects: Add VC14.20 project files
    • pytest: Use lower count in repeat tests
    • quic: Make eyeballers connect retries stop at weird replies
    • quic: Manage connection idle timeouts

    • quiche: Use quiche_conn_peer_transport_params()

    • rand: Fix build error with autotools + LibreSSL

    • resolve.d: Drop a multi use-sentence

    • RTSP: Improved RTP parser

    • rustls: Implement connect_blocking

    • sasl: Fix '-Wunused-function' compiler warning

    • schannel: Add CA cache support for files and memory blobs

    • setopt: Check CURLOPT_TFTP_BLKSIZE range on set

    • setopt: Remove outdated cookie comment
    • setopt: Remove superfluous use of ternary expressions
    • socks: Better buffer size checks for socks4a user and hostname

    • socks: Make SOCKS5 use the CURLOPT_IPRESOLVE choice

    • symbols-in-versions: The CLOSEPOLICY options are deprecated

    • test1683: Remove commented-out check alternatives

    • test3103: Add missing quotes around a test tag attribute

    • test613: Stop showing an error on missing output file

    • tests/README: SOCKS tests are not using OpenSSH; it has its own server

    • tests/server: Add more SOCKS5 handshake error checking

    • tests: Fix Windows test helper tool search and use it for handle64

    • tidy-up: Casing typos, delete unused Windows version aliases

    • tool: Fix --capath when proxy support is disabled

    • tool: Support bold headers in Windows

    • tool_cb_hdr: Add an additional parsing check

    • tool_cb_prg: Make the carriage return fit for wide progress bars

    • tool_cb_wrt: Fix write output for very old Windows versions

    • tool_getparam: Limit --rate to be smaller than number of ms

    • tool_operate: Do not mix memory models

    • tool_operate: Fix links in IPFS errors

    • tool_parsecfg: Make warning output propose double-quoting

    • tool_urlglob: Fix build for old gcc versions

    • tool_urlglob: Make multiply() bail out on negative values

    • tool_writeout_json: Fix JSON encoding of non-ascii bytes

    • transfer: Abort pause send when connection is marked for closing
    • transfer: Avoid calling the read callback again after EOF

    • transfer: Only reset the FTP wildcard engine in CLEAR state

    • url: Don't touch the multi handle when closing internal handles
    • url: Find scheme with a "perfect hash"

    • url: Fix '-Wzero-length-array' with no protocols

    • url: Fix builds with 'CURL_DISABLE_HTTP'

    • url: Protocol handler lookup tidy-up
    • url: Proxy ssl connection reuse fix
    • urlapi: Avoid null deref if setting blank host to url encode
    • urlapi: Skip appending NULL pointer query
    • urlapi: When URL encoding the fragment, pass in the right length

    • urldata: Make maxconnects a 32-bit value

    • urldata: Move async resolver state from easy handle to connectdata

    • urldata: Move cookielist from UserDefined to UrlState

    • urldata: Move hstslist from 'set' to 'state'

    • urldata: Move the 'internal' boolean to the state struct

    • vssh: Remove the #ifdef for Curl_ssh_init, use empty macro

    • vtls: Clean up SSL config management

    • vtls: Consistently use typedef names for OpenSSL structs

    • vtls: Late clone of connection ssl config
    • vtls: Use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0

    • VULN-DISCLOSURE-POLICY: Escape sequences are not a security flaw

    • windows: Use built-in '_WIN32' macro to detect Windows

    • wolfssh: Remove redundant static prototypes

    • wolfssl: Add default case for wolfssl_connect_step1 switch

    • wolfssl: Require WOLFSSL_SYS_CA_CERTS for loading system CA

  • I had to locally include errorcodes.pl, missing from tarball (GH#12462), to get the test suite to pass

  • Updated curl (8.2.1) to fix cookie mixed case PSL bypass (CVE-2023-46218) and HSTS long file name clears contents (CVE-2023-46219)

  • Updated libxml2 to 2.12.2:

  • Regressions:

    • parser: Fix invalid free in xmlParseBalancedChunkMemoryRecover

    • globals: Disable TLS in static Windows builds
    • html: Re-enable buggy detection of XML declarations
    • tree: Fix regression when copying DTDs
    • parser: Make CRLF increment line number
  • Build fixes:
    • build: Disable compiler TLS by default

    • cmake: Update config.h.cmake.in

    • tests: Fix tests --with-valid --without-xinclude

  • I also enabled the W3C XML Conformance and Schema test suites, which required separate sources

Recent