PaulHowarth/Blog/2023-12-06

Wednesday 6th December 2023

Local Packages

• Updated curl to 8.5.0:

  • gnutls: Support CURLSSLOPT_NATIVE_CA

  • HTTP3: ngtcp2 builds are no longer experimental

  • appveyor: Make VS2008-built curl tool runnable

  • asyn-thread: Use pipe instead of socketpair for IPC when available

  • autotools: Accept linker flags via 'CURL_LDFLAGS_{LIB,BIN}'

  • autotools: Avoid passing 'LDFLAGS' twice to libcurl

  • autotools: Delete LCC compiler support bits

  • autotools: Fix/improve gcc and Apple clang version detection

  • autotools: Stop setting '-std=gnu89' with '--enable-warnings'

  • autotools: Update references to deleted 'crypt-auth' option

  • BINDINGS: Add V binding

  • build: Add 'src/.checksrc' to source tarball

  • build: Add more picky warnings and fix them

  • build: Always revert '#pragma GCC diagnostic' after use

  • build: Delete 'HAVE_STDINT_H' and 'HAVE_INTTYPES_H'

  • build: Delete support bits for obsolete Windows compilers

  • build: Fix 'threadsafe' feature detection for older gcc

  • build: Fix builds that disable protocols but not digest auth
  • build: Fix compiler warning with auths disabled

  • build: Fix libssh2 + 'CURL_DISABLE_DIGEST_AUTH' + 'CURL_DISABLE_AWS'

  • build: Picky warning updates
  • build: Require Windows XP or newer

  • cfilter: Provide call to tell connection to forget a socket

  • checksrc.pl: Support #line instructions

  • CI: Add autotools, out-of-tree, debug build to distro check job

  • CI: Ignore test 286 on Appveyor gcc 9 build

  • cmake: Add 'CURL_DISABLE_BINDLOCAL' option

  • cmake: Add test for 'DISABLE' options, add 'CURL_DISABLE_HEADERS_API'

  • cmake: Dedupe Windows system libs

  • cmake: Fix 'HAVE_H_ERRNO_ASSIGNABLE' detection

  • cmake: Fix CURL_DISABLE_GETOPTIONS

  • cmake: Fix multiple include of CURL package
  • cmake: Fix OpenSSL quic detection in quiche builds

  • cmake: Option to disable install and drop 'curlu' target when unused

  • cmake: Pre-fill rest of detection values for Windows

  • cmake: Replace 'check_library_exists_concat()'

  • cmake: Speed up threads setup for Windows

  • cmake: Speed up zstd detection

  • config-win32: Set 'HAVE_SNPRINTF' for mingw-w64

  • configure: Better --disable-http

  • configure: Check for the fseeko declaration too

  • conncache: Use the closure handle when disconnecting surplus connections

  • content_encoding: Make Curl_all_content_encodings allocless

  • cookie: Lowercase the domain names before PSL checks (CVE-2023-46218)

  • curl.h: Delete Symbian OS references

  • curl.h: On FreeBSD include sys/param.h instead of osreldate.h

  • curl.rc: Switch out the copyright symbol for plain ASCII

  • curl: Improved IPFS and IPNS URL support

  • curl_easy_duphandle.3: Clarify how HSTS and alt-svc are duped

  • Curl_http_body: Clean up properly when Curl_getformdata errors

  • curl_setup: Disallow Windows IPv6 builds missing getaddrinfo

  • curl_sspi: Support more revocation error names in error messages

  • CURLINFO_PRETRANSFER_TIME_T.3: Fix time explanation

  • CURLMOPT_MAX_CONCURRENT_STREAMS: Make sure the set value is within range

  • CURLOPT_CAINFO_BLOB.3: Explain what CURL_BLOB_COPY does

  • CURLOPT_WRITEFUNCTION.3: Clarify libcurl returns for CURL_WRITEFUNC_ERROR

  • CURPOST_POSTFIELDS.3: Add CURLOPT_COPYPOSTFIELDS in SEE ALSO

  • docs/example/keepalive.c: Show TCP keep-alive options

  • docs/example/localport.c: Show off CURLOPT_LOCALPORT

  • docs/examples/interface.c: Show CURLOPT_INTERFACE use

  • docs/libcurl: Fix three minor man page format mistakes

  • docs/libcurl: SYNOPSIS clean up

  • docs: Add supported version for the json write-out

  • docs: Clarify that curl passes on input unfiltered

  • docs: Fix function typo in curl_easy_option_next.3

  • docs: KNOWN_BUGS clean up

  • docs: Make all examples in all libcurl man pages compile

  • docs: Preserve the modification date when copying the prebuilt man page

  • docs: Remove bold from some man page SYNOPSIS sections

  • docs: Use SOURCE_DATE_EPOCH for generated manpages

  • doh: Provide better return code for responses w/o addresses

  • doh: Use PIPEWAIT when HTTP/2 is attempted

  • duphandle: Also free 'outcurl->cookies' in error path

  • duphandle: Make dupset() not return with pointers to old alloced data

  • duphandle: Use strdup to clone *COPYPOSTFIELDS if size is not set

  • easy: In duphandle, init the cookies for the new handle

  • easy: Remove duplicate wolfSSH init call

  • easy_lock: Add a pthread_mutex_t fallback

  • examples/rtsp-options.c: Add

  • fopen: Create new file using old file's mode

  • fopen: Create short(er) temporary file name (CVE-2023-46219)

  • getenv: PlayStation doesn't have getenv()

  • GHA: Move mod_h2 version in CI to v2.0.25

  • hostip: Show the list of IPs when resolving is done

  • hostip: Silence compiler warning '-Wparentheses-equality'

  • hsts: Skip single-dot hostname
  • HTTP/2, HTTP/3: Handle detach of ongoing transfers
  • http2: Header conversion tightening

  • http2: Provide an error callback and failf the message

  • http2: Safer invocation of populate_binsettings

  • http: Allow longer HTTP/2 request method names

  • http: Avoid Expect: 100-continue if Upgrade: is used

  • http: Consider resume with CURLOPT_FAILONERRROR and 416 to be fine

  • http: Fix '-Wunused-parameter' with no auth and no proxy

  • http: Fix '-Wunused-variable' compiler warning

  • http: Fix empty-body warning

  • http_aws_sigv4: Canonicalise valueless query params

  • hyper: Temporarily remove HTTP/2 support

  • INSTALL: Update list of ports and CPU archs

  • IPFS: Fix IPFS_PATH and file parsing

  • keylog: Disable if unused

  • lib: Add and use Curl_strndup()

  • lib: Apache style infof and trace macros/functions

  • lib: Fix gcc warning in printf call

  • libcurl-errors.3: Sync with current public headers

  • libcurl-thread.3: Simplify the TLS section

  • Makefile.am: Drop vc10, vc11 and vc12 projects from dist

  • Makefile.mk: Fix '-rtmp' option for non-Windows

  • mime: Store "form escape" as a single bit

  • misc: Fix -Walloc-size warnings

  • msh3: Error when built with CURL_DISABLE_SOCKETPAIR set

  • multi: During ratelimit multi_getsock should return no sockets

  • multi: Use pipe instead of socketpair to *wakeup()

  • ngtcp2: Fix races in stream handling
  • ngtcp2: Ignore errors on unknown streams

  • ntlm_wb: Use pipe instead of socketpair when possible

  • openldap: Move the alloc of ldapconninfo to *connect()

  • openldap: Set the callback argument in oldap_do

  • openssl: Avoid BN_num_bits() NULL pointer derefs

  • openssl: Fix building with v3 'no-deprecated' + add CI test

  • openssl: Fix infof() to avoid compiler warning for %s with null

  • openssl: Identify the "quictls" backend correctly

  • openssl: Include SIG and KEM algorithms in verbose

  • openssl: Make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs

  • openssl: Two multi pointer checks should probably rather be asserts

  • openssl: When a session-ID is reused, skip OCSP stapling
  • page-footer: Clarify exit code 25
  • projects: Add VC14.20 project files
  • pytest: Use lower count in repeat tests
  • quic: Make eyeballers connect retries stop at weird replies
  • quic: Manage connection idle timeouts

  • quiche: Use quiche_conn_peer_transport_params()

  • rand: Fix build error with autotools + LibreSSL

  • resolve.d: Drop a multi use-sentence

  • RTSP: Improved RTP parser

  • rustls: Implement connect_blocking

  • sasl: Fix '-Wunused-function' compiler warning

  • schannel: Add CA cache support for files and memory blobs

  • setopt: Check CURLOPT_TFTP_BLKSIZE range on set

  • setopt: Remove outdated cookie comment
  • setopt: Remove superfluous use of ternary expressions
  • socks: Better buffer size checks for socks4a user and hostname

  • socks: Make SOCKS5 use the CURLOPT_IPRESOLVE choice

  • symbols-in-versions: The CLOSEPOLICY options are deprecated

  • test1683: Remove commented-out check alternatives

  • test3103: Add missing quotes around a test tag attribute

  • test613: Stop showing an error on missing output file

  • tests/README: SOCKS tests are not using OpenSSH; it has its own server

  • tests/server: Add more SOCKS5 handshake error checking

  • tests: Fix Windows test helper tool search and use it for handle64

  • tidy-up: Casing typos, delete unused Windows version aliases

  • tool: Fix --capath when proxy support is disabled

  • tool: Support bold headers in Windows

  • tool_cb_hdr: Add an additional parsing check

  • tool_cb_prg: Make the carriage return fit for wide progress bars

  • tool_cb_wrt: Fix write output for very old Windows versions

  • tool_getparam: Limit --rate to be smaller than number of ms

  • tool_operate: Do not mix memory models

  • tool_operate: Fix links in IPFS errors

  • tool_parsecfg: Make warning output propose double-quoting

  • tool_urlglob: Fix build for old gcc versions

  • tool_urlglob: Make multiply() bail out on negative values

  • tool_writeout_json: Fix JSON encoding of non-ascii bytes

  • transfer: Abort pause send when connection is marked for closing
  • transfer: Avoid calling the read callback again after EOF

  • transfer: Only reset the FTP wildcard engine in CLEAR state

  • url: Don't touch the multi handle when closing internal handles
  • url: Find scheme with a "perfect hash"

  • url: Fix '-Wzero-length-array' with no protocols

  • url: Fix builds with 'CURL_DISABLE_HTTP'

  • url: Protocol handler lookup tidy-up
  • url: Proxy ssl connection reuse fix
  • urlapi: Avoid null deref if setting blank host to url encode
  • urlapi: Skip appending NULL pointer query
  • urlapi: When URL encoding the fragment, pass in the right length

  • urldata: Make maxconnects a 32-bit value

  • urldata: Move async resolver state from easy handle to connectdata

  • urldata: Move cookielist from UserDefined to UrlState

  • urldata: Move hstslist from 'set' to 'state'

  • urldata: Move the 'internal' boolean to the state struct

  • vssh: Remove the #ifdef for Curl_ssh_init, use empty macro

  • vtls: Clean up SSL config management

  • vtls: Consistently use typedef names for OpenSSL structs

  • vtls: Late clone of connection ssl config
  • vtls: Use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0

  • VULN-DISCLOSURE-POLICY: Escape sequences are not a security flaw

  • windows: Use built-in '_WIN32' macro to detect Windows

  • wolfssh: Remove redundant static prototypes

  • wolfssl: Add default case for wolfssl_connect_step1 switch

  • wolfssl: Require WOLFSSL_SYS_CA_CERTS for loading system CA

• I had to locally include errorcodes.pl, missing from tarball (GH#12462), to get the test suite to pass

• Updated curl (8.2.1) to fix cookie mixed case PSL bypass (CVE-2023-46218) and HSTS long file name clears contents (CVE-2023-46219)

• Updated libxml2 to 2.12.2:

• Regressions:

  • parser: Fix invalid free in xmlParseBalancedChunkMemoryRecover

  • globals: Disable TLS in static Windows builds
  • html: Re-enable buggy detection of XML declarations
  • tree: Fix regression when copying DTDs
  • parser: Make CRLF increment line number
• Build fixes:
  • build: Disable compiler TLS by default

  • cmake: Update config.h.cmake.in

  • tests: Fix tests --with-valid --without-xinclude

• I also enabled the W3C XML Conformance and Schema test suites, which required separate sources