April 2023 Summaries

A project involved scraping over 100,000 pages of IRS manuals, primarily in PDF format, from the IRS website and using the Unstructured API to preprocess these documents into structured JSON data. This preprocessing allows the data to be organized in a way that benefits large language models (LLMs), facilitating experiments with various downstream libraries for different applications. The team used tools such as Pinecone for data storage, OpenAI for embeddings, and LangChain as a programming framework, demonstrating flexibility in choosing alternatives like Hugging Face or Llama Index. Once the data is structured and stored in a vector database, it can be queried to answer questions about IRS policies and procedures, enabling enterprises to leverage their internal data effectively with LLMs. The project emphasizes the growing capabilities of natural language processing and data connectivity offered by Unstructured, encouraging users to engage with the data through a hosted instance or by running a command-line interface application themselves.

In the field of document understanding, employing larger images necessitates algorithmic improvements to maintain efficiency while processing them through vision transformers (ViTs), which inherently have a quadratic cost related to input length. Standard optimization techniques such as quantization and pruning can double processing speed, and adaptations like EfficientFormer combine CNNs with transformers to increase speed, albeit with reduced performance compared to more resource-intensive networks. Techniques like Swin ViT improve efficiency by splitting images into smaller patches, and sparse attention approaches reduce computational complexity significantly, though their practical effectiveness remains uncertain. Other innovations, such as Performer, achieve linear performance by rearranging matrices used in attention calculations, but still lag behind more advanced models like Swin ViT. Knowledge distillation from complex networks to more efficient ones could bridge performance gaps, and simpler models may identify main text areas for decoder transformers. The Unstructured team is actively exploring these methods to create faster document preprocessing applications using ViTs, encouraging followers to engage with their ongoing research efforts on platforms like LinkedIn, Huggingface, and GitHub.

In the context of document understanding, the challenge lies in processing larger images without losing information, and while there are existing hardware-dependent optimizations, Unstructured focuses on algorithmic improvements for vision transformers (ViTs). ViTs, which split images into patches before feeding them to a transformer, face a quadratic cost issue related to input length. Existing solutions to enhance ViT processing speed include quantization, pruning, and adapting transformers for environments with limited computing capabilities, such as mobile networks exemplified by EfficientFormer. Techniques like sparse attention matrices and matrix decomposition offer potential reductions in computational complexity. Despite some methods not yet being fully evaluated for document understanding, combining vision transformer approaches with recent advancements in attention calculation cost reduction may yield benefits. The Unstructured team is exploring various strategies, including knowledge distillation and using simpler models to pinpoint main text areas, to develop efficient vision transformers for real-world document preprocessing.