UltraBERT Fits 32K Tokens in One Vector. RIP Chunking?

UltraBERT-Embed-150M, a 151-million-parameter embedding model supporting inputs up to 32,000 tokens, enables entire long documents to be represented by a single 768-dimensional vector, prompting tests of whether it can replace conventional text chunking. Evaluations in Qdrant across LongEmbed tasks and the 200,000-article MLDR benchmark found that whole-document embeddings excelled when queries described a document’s overall theme, such as TV episode summaries, and performed nearly perfectly on passkey retrieval, but 512-token chunks generally retrieved localized facts and passages more effectively. Combining whole-document and chunk vectors produced limited, inconsistent gains, suggesting that hybrid retrieval requires further tuning. Although chunking greatly increases vector counts, 1-bit TurboQuant compressed MLDR chunk embeddings by roughly 30 times to about 140 MB while preserving 95% of retrieval quality without rescoring and 98% with rescoring, albeit with additional latency. The findings conclude that chunking remains the stronger default for detail-oriented search, while UltraBERT’s long context window offers flexibility for multi-scale retrieval approaches, such as identifying relevant documents first and then searching their chunks.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.