January 2018 Summaries

PubNub has announced its compliance with the European General Data Protection Regulation (GDPR), which requires businesses that handle personal data of European Union (EU) residents to adhere to stringent data protection standards, effective May 25, 2018. This regulation applies to all companies, regardless of their physical presence in Europe, if they engage with EU residents' data, classifying them as either "controllers" or "processors" based on their role in handling data. PubNub has implemented measures such as appointing a Data Protection Officer and incorporating GDPR principles into product development, alongside processes for data protection, deletion, retention, and pseudonymization. The company also suggests additional steps for compliance, like creating a Data Map and enabling users to withdraw consent for data processing, and emphasizes that further regulations may be introduced.

PubNub has enhanced its support for MQTT devices by integrating its Data Stream Network (DSN) features, allowing IoT developers to utilize capabilities such as Functions, Storage & Playback, and Mobile Push without additional coding. This integration enables real-time message filtering, routing, and enhancement, as well as seamless cloud integration with providers like AWS, Azure, and GCP. PubNub's platform supports scalability, covering 9% of the world's Internet-connected devices, and offers pre-built integrations for IoT APIs, facilitating anomaly detection in various use cases, such as industrial sensors and home alarms. The MQTT protocol, originally developed by IBM, is known for its low power consumption, making it ideal for IoT applications, and the collaboration with PubNub allows for easy and rapid setup, as demonstrated by Soracom's integration within two days.

In a globally distributed system, the challenge of achieving real-time data consistency is addressed by implementing a two-tier caching strategy that enhances the speed of data acquisition beyond the capabilities of CassandraDB's eventual consistency model. The system employs a storage cache to minimize read times and a volatile cache to handle unverified user actions, allowing data updates to occur at a rate of 1/10 of a second, similar to the Pub/Sub service. This new architecture, featuring a list of channels with timestamps and statuses, ensures faster response times while maintaining the database as the ultimate source of truth. However, the volatility and potential for discrepancies in the cache, due to factors like clock skew and simultaneous updates in different regions, require careful management through features like a configurable age-out time and a feature flag mechanism to mitigate risk and reduce memory overhead.