HTTP client: remove unsafe from src/http (h1/h2/h3 client, proxy tunnel, HTTP thread, keep-alive pool)

JA
2mo ago
+6882-6787

What

Same programme as #40055 … #40383, applied to the fetch client core below the JS layer. Stacked on #40202 (base branch claude/fetch-zero-unsafe; also carries #40210's two bun_ptr commits). Retarget to main once #40202 lands.

src/http/ (excluding websocket_client/, which #40055 already did): every file → 0. lib.rs 64, HTTPContext.rs 25, HTTPThread.rs 22, session_cache.rs 18, AsyncHTTP.rs 14, ProxyTunnel.rs 12, lshpack.rs 11 (→ new bun_lshpack_sys crate), ssl_config.rs 9, ThreadSafeStreamBuffer.rs 7, SendFile.rs 3, h2_client/* 13, h3_client/* 28, and the singles — all → 0. No new unsafe in src/runtime/** or src/http_jsc/** (several removed there).

  • Request ownership: the HTTP thread no longer ptr::reads the caller's AsyncHTTP and copies state back. AsyncHTTP::schedule() moves its HTTPClient into a Box<RequestCell> (built in place) and parks it on the request; the HTTP thread takes the box; RequestRef = BackRef<RequestCell> is the socket ext tag / h2-h3 stream back-ref / tunnel owner / pending-connect waiter; a terminal result is stored on the cell and ThreadState::flush_completions() (after every socket event / drain step / h3 callback) resets the client to the owner's configuration, parks the cell back on the owner, sets handed_back, then runs the callback — re-scheduling (install retries) reuses the cell, and no callback runs under a live &mut HTTPClient.
  • Thread state: ThreadState (HTTP-thread only, Cell/RefCell) replaces the &'static mut globals; cross-thread queues live on static HTTP_THREAD: HttpThread (Guarded/atomics/OnceLock<LoopWaker>).
  • ProxyTunnel = SSLWrapper<BackRef<ProxyTunnel>> + a deferred TunnelEvent queue. h2/h3 sessions are &self + Cell/RefCell, Rc<Stream>, typed ref slots; h3 DNS pending is an index ticket. RequestUrl = pre-resolved components + optional owned backing; HTTPResponseMetadata owns its headers; SSLConfig strings are SecretCString; TLS session resumption via boringssl_sys::SslSession RAII + ssl_session_sink.
  • Primitives: bun_core::heap::new_with (in-place Box construction), FfiSliceMut, SecretCString; bun_uws_sys::{LoopWaker, SocketGroupCell, Socket::{ext_word, connect_group_tagged}, ssl_session_sink, quic::ClientHandler}; MiniEventLoop::run_uws_loop_forever; boringssl_sys::SSL::{peer_leaf_certificate, is_init_finished, configure_client, alpn_selected, set_session}, X509::to_der; bun_sys::{linux::sendfile_at, c::sendfile_plain}; bun_url::{ParsedURL, URL::from_parts}; bun_picohttp::decode_chunked; bun_lshpack_sys; HTTPClientResultCallback::{Raw, Handler(Arc), ThreadOwned}; Bun__defaultMaxHttpHeaderSize() HOST_EXPORT.

Parity deltas (intentional, small): terminal teardown happens at completion-flush right after the callback rather than just before; h3 same-thread DNS completion is observed one loop turn later; an h3 retry after a refused re-connect surfaces the original error; UnsupportedProxyProtocol also enforced on the custom-TLS cache-hit path; install/S3 retries start from pristine request config. Pre-existing issues fixed in passing: write_to_stream released the JS stream-buffer lock after a synchronous tunnel on_close could have reset the request; on_handshake expect-panicked on a missing SSL*; h2 encode_header wrote into uninitialised Vec capacity; h2 give_up_socket_ref could overwrite a parked ref; h2 attach compress-failure didn't flush the preface; h3 connect-error path detached a freed stream.

Perf (release, taskset 8 cores, perf stat -e instructions:u on the client process, interleaved ×8): 20k keep-alive fetch() against a local server — after the rebase onto #40478/#40391/#35886, −0.37 % vs the parent branch (7/8 rounds lower; an earlier base measured −2.0 %). 2k fresh-TLS and 200 × 1 MB concurrent: within noise.

Retry path (install/S3 re-schedule the same AsyncHTTP): RequestCell::arm() runs on every schedule() — first time it captures the owner's config (flags, redirect budget, hostname, if_modified_since, ownership of http_proxy/unix_socket_path), every time it resets the client to that config, rebuilds headers/url/method from the AsyncHTTP, and installs the result callback afresh. Per-attempt resources are still released on the HTTP thread at completion. Intended delta: a retry starts from the owner's forced_protocol/disable_keepalive/redirect budget rather than the previous attempt's learned values. test/cli/install/bun-install-retry.test.ts 10/10.

Testing

Debug+ASAN, --timeout 60000: 84 files across test/js/web/fetch/* (incl. http2/http3/proxy/redirect/keepalive/leak/unix/upgrade), test/js/bun/http/{proxy*,fetch*,tls-keepalive}, test/js/bun/s3/*, test/js/node/http/*, 445 test/js/node/test/parallel/test-http*/https-* — green except known sandbox reds (fetch.test.ts root-permission ×4 + 150 ms AbortSignal.timeout; fetch-tcp-stress/fetch-leak › URLSearchParams debug timeouts that pass on release; test-http-client-req-error-dont-double-fire.js identical under system bun). bun-install-registry.test.ts 245/246. Hand-driven: 1000 keep-alive fetches on one socket; redirect ×20; gzip/br/zstd/deflate; chunked streaming upload; 500× abort mid-body + GC; self-signed reject / rejectUnauthorized:false / CA / checkServerIdentity; NODE_TLS_REJECT_UNAUTHORIZED=0; HTTP + CONNECT proxy with tunnel reuse; server close mid-response/mid-chunk; Worker terminated mid-download ×5 — exit 0, no ASAN output. clippy clean; rust-check-all windows-msvc + apple-darwin pass.