Moving away from Segment to a cost-effective do-it-yourself event streaming pipeline with Cloud Pub/Sub and dlt.

The proposed solution to replace Segment involves using dlt with Cloud Pub/Sub to create a simple, scalable event streaming pipeline. The pipeline's overall architecture is as follows: events are published to a Pub/Sub topic, where they are aggregated and then forwarded to Cloud Storage. A Cloud Function reads the data from storage and uses dlt to ingest it into BigQuery. The solution aims to provide a cost-effective alternative to Segment, with estimated costs ranging from $37 for 1 million ingested events, compared to $1000 for Segment's equivalent service. The proposed pipeline can offer up to 18x in savings in practice, making it an attractive option for organizations looking to reduce their event streaming costs.