Company
Date Published
Author
Ben Esh
Word count
585
Language
English
Hacker News points
None

Summary

Pinecone has introduced a new capability for bulk importing data from object storage to its serverless infrastructure, which enhances efficiency for developers working on AI applications by simplifying large-scale data ingestion. This feature allows for up to 6x cost savings compared to traditional upsert methods, as well as the ability to handle billions of records with reduced overhead. Import operations are asynchronous, eliminating the need for constant performance monitoring, and are limited to 100 million records at a time during the public preview phase. Data is securely accessed from object storage, providing users with control over data access and security. The service currently supports Amazon S3, with plans to extend compatibility to Google Cloud Storage and Azure Blob Storage. The bulk import is available for Standard and Enterprise users at a flat rate, and involves setting up storage integrations via the Pinecone console.