skillguard/.agents/memory/ndjson-streaming-express-replit.md

name	description
NDJSON streaming under Express behind the Replit proxy	Pitfalls for live/streaming HTTP responses (NDJSON/SSE) from an Express API on Replit — client-disconnect detection, persistence, and proxy buffering

Streaming responses (NDJSON) from Express on Replit

For a long-lived POST that streams progress (e.g. application/x-ndjson with res.flushHeaders() then line-delimited JSON writes):

Detect a real client abort with res.on("close") + res.writableFinished — never req.on("close"). req.on("close") fires as soon as the POST request body has been fully consumed, which is normal and happens immediately. Gating writes on that wrongly suppresses the entire stream. Use the response close event, and treat it as an abort only when !res.writableFinished.

Persist the result regardless of disconnect. Run the analysis and the DB write inside the handler's try unconditionally; only guard the res.write() calls on "connection still open". This way a client that disconnects mid-stream still gets its result persisted (and the client must NOT re-submit on a mid-stream error, or it creates a duplicate row — see below).

The Replit path-proxy does NOT buffer the stream. Incremental flushes arrive line by line through $REPLIT_DEV_DOMAIN, same as localhost. No special proxy header or chunk-padding needed.

Client fetch must be root-relative (/api/...), not import.meta.env.BASE_URL. That matches the generated API client and is what routes correctly through the proxy.

Fallback gating to avoid duplicate persists: a client that streams and also falls back to a non-streaming POST on error must only fall back when the server never processed the request (fetch rejected, or !res.ok with a 5xx). Once the response body has started streaming, any read error means the server is already persisting — falling back then duplicates the record. Distinguish 4xx (show message, no retry) from transport failures.