OpenAI's Responses API now supports WebSockets and connection-scoped caching, measurably reducing overhead in multi-turn agentic workflows like Codex.
OpenAI published a technical deep dive into how the Codex agent loop was optimized using WebSockets in the Responses API. The key changes include persistent connections (eliminating repeated HTTP handshake overhead) and connection-scoped caching that avoids redundant context re-transmission between agent turns. The result is reduced API latency and lower overhead for agentic loops that make many sequential calls. This is available in the Responses API today.
WebSocket support in the Responses API means your agent loop no longer pays TCP + TLS handshake costs on every turn. Connection-scoped caching means shared context (system prompts, tool definitions) isn't re-sent on each request. For high-frequency agentic workflows — code agents, orchestration loops, multi-step reasoning — this is a direct latency and cost reduction with no model change required.
Swap your Responses API calls from HTTP polling to WebSocket connections in your existing agent loop this week, benchmark the p50/p95 latency on a 10-turn sequence, and measure the delta before updating your architecture docs.
Suggested exercise · not independently tested. Access requirements and costs may vary.
Install the latest OpenAI SDK: pip install --upgrade openai
Paste this snippet to open a WebSocket session: from openai import OpenAI; client = OpenAI(); stream = client.responses.create(model='gpt-4o', input=[{'role':'user','content':'Write a Python function to sort a list'}], stream=True); [print(event) for event in stream]
Compare wall-clock time of 5 sequential turns over WebSocket vs your current HTTP implementation using time.perf_counter()
Tags