Chaining Activities — from text to vectors

Houman Kargaran describes a PII-compliant RAG pipeline orchestrated with Temporal, extending an earlier ingestion stage in which complaint redaction, classification, and storage remain within a single Activity and only a non-sensitive file ID proceeds through the workflow. The ComplaintWorkflow sequences separate, retryable Activities to generate on-premises embeddings with the all-MiniLM-L6-v2 model, cache vectors in Redis, upsert them into Pinecone, mark successfully stored PostgreSQL records as embedded, perform a lightweight vector-query smoke test, and delete the temporary cache. Redis functions as external payload storage, similar to the Claim Check pattern, so Temporal passes only small references rather than vectors or PII. Provider interfaces allow embedding and storage implementations to be replaced independently, although changing models may require index changes and re-embedding. Reliability is supported by stable case IDs, duplicate-safe PostgreSQL inserts, Pinecone upserts, and an embedded flag that enables reconciliation and selective reruns, while Temporal provides durable orchestration and retry behavior. Although vectors originate only from redacted text, the approach treats embeddings as sensitive application data and removes cached copies after the workflow completes.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.