Fast and Easy Vector Ingestion from Python DataFrames
Blog post from SingleStore
To ingest embeddings from Pandas DataFrames into SingleStoreDB using Python, you can convert the array containing the vector embedding into a packed byte string using numpy. This approach offers speed and simplicity when loading your embeddings as vectors into SingleStoreDB. To get started, create a connection to SingleStoreDB or use a SingleStore notebook, select the database you want to access, install necessary libraries, and get a sample dataset from the 2022 Beijing Winter Olympics that includes text chunks and their embeddings. Then, create a simple table using SQL and load your DataFrame into the SQL table using the `to_sql` method. Note that this approach assumes the data is in a form of array; if not, additional conversion code may be required. The pandas documentation provides more information about options available through the `to_sql` method.