Query-aware context compression for better snippets

Perplexity describes a query-aware extractive context-compression system designed to remove irrelevant material from retrieved search results before they reach answer models, improving accuracy, latency, and cost while preserving source wording for reliable citations. Rather than generating summaries, the system selects the most useful spans from each query-document pair, using a bidirectional pplx-diffusion encoder and a lightweight compression head trained on 750,000 pairs labeled through a two-stage LLM judging pipeline. A distilled 17-layer production model matched the quality of a 28-layer teacher while reducing inference latency by 35–40% and GPU compute by 40–45%, with p99 serving latency below 20 milliseconds. Tests on BrowseComp and SimpleQA found that compression reduced token usage by 10–70% while improving accuracy, and an analysis of 1,000 queries showed a 63% increase in vital content alongside declines in irrelevant navigation, metadata, advertising, and duplicate text. The approach gives systems and API users a way to adjust compression according to desired tradeoffs among evidence recall, speed, and cost.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.