Modeling the City of the Future with Kafka and Spark

Spark Summit showcased MemCity, a simulation measuring energy consumption across 1.4 million households in a futuristic city, using real-time data processing and analytics. MemCity tracks various devices in homes, analyzing data from energy devices measured by the minute, providing insights into energy usage trends and solutions for smarter energy systems. It is built with Apache Kafka, Apache Spark, and SingleStore, solving the problem of ingesting, processing, and serving real-time data across an organization. The simulation runs on four Amazon EC2 nodes, processes 186.67k transactions per second, and enables simultaneous real-time SQL queries. MemCity highlights the importance of smart cities and gathering meaningful insights from sensors to inform urban planning and energy solutions, making long-term planning viable with low-cost hardware.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.