Hheadroom/PROXY LAB
Back to coding agent demo ↗Connecting…
CONTEXT ENGINEERING / QUICK TEST

Quick compression test

Compare a fixed prompt and context with and without compression. No autonomous agent or tool execution.

COMPRESSION AT THE PROXY LAYERNo client changes
⌘ClientOpenAI request
→
⇄LiteLLMProxy + policy
→
≋HeadroomThreshold-based
→
✧ModelSame provider

Comparison READY

Run a scenario to see measured results.

Input token reduction—%

Before → after compression

Estimated input saving—

At your configured token rate

End-to-end latency Δ—

Compressed minus baseline

Facts retained in answer—

Scenario-specific exact checks

Context footprint

ESTIMATED INPUT TOKENS
Original
—
Headroom
—
Instructions stay intact. Large tool payloads are eligible.

Original

BASELINE
The uncompressed model response will appear here.

With Headroom

POLICY ON
The model response with proxy-side compression will appear here.
ⓘ

Token counts use a consistent local estimator. Provider usage is shown after each run. Exact fact checks help compare these scenarios; they do not prove general response quality.

REQUEST INSPECTOR

The requests before provider adaptation

Captured inside LiteLLM’s pre-call hook. Transport headers and provider-specific formatting are not included.

Original request

Headroom request


A small hook. A visible difference.
The dashboard sends two standard chat completion requests through the real LiteLLM Proxy. A custom async_pre_call_hook evaluates the compression policy before the upstream call.
  1. Count: estimate tokens in messages and tool definitions.
  2. Decide: skip requests below the configurable threshold.
  3. Compress: Headroom processes eligible tool content locally, without an extra LLM call.
  4. Protect: preserve system and user instructions, assistant messages, and tool-call IDs. Keep the original if compression fails or increases tokens.
  5. Compare: show both responses, token estimates, actual provider usage, timing, and exact fact checks.
Follow-ups keep separate, uncompressed client histories for the two branches. Add more context to simulate additional tool or file reads.
Demo scope: structured tool context, no database, one backend container. ML text compression and automatic context retrieval are not enabled. Compressed content may lose information; inspect the answers and test follow-up questions.
Cloudflare Static Assets → Worker → Container → LiteLLM + Headroom → upstream provider