October 2019 Summaries

CARTO, a company committed to Open Source Open Data and Open Science, has joined the Urban Computing Foundation (UCF) Technical Advisory Committee. The economic growth of cities is causing an increase in urban challenges such as environmental degradation, traffic congestion, and social inequality. Spatial analytics is crucial for understanding and addressing these issues. The UCF aims to accelerate open-source community development and enable a fully interconnected system where data can flow seamlessly from sensors to simulation models. CARTO's Founder and Chief Strategy Officer, Javier de la Torre, will contribute expertise in three main areas: Open Source software components, connecting the community, and promoting next-generation Spatial Data Infrastructure Architecture, Interoperability, and standards.

BigQuery is now supporting next generation spatial data infrastructure stack with new spatial functions, including Uber H3-js, which are available as an open source project and can be used directly on queries without any installation required. The functions allow users to perform various spatial operations such as joins, buffers, and more, making it easier to work with geospatial data in BigQuery. The functions are deployed and ready to use, and can be visualized using tools like BigQuery Geo Viz QGIS or CARTOframes library. While there is no need to worry about information security since the functions are stored in a separate project, users can propose new functions or libraries for porting by sending a pull request on the project.

PostGIS 3.0.0 has been released, marking a significant milestone for CARTO and its clients, as they have been benefiting from this version's features for some time through their custom implementation. The release includes numerous enhancements, bug fixes, and new features, such as improved speed and parallelization of functions, which significantly impact performance, particularly in rendering raster images and preprocessing geometries. Additionally, GEOS 3.8 has introduced fewer topological errors and self-intersecting issues, while the continuous integration cycle has been strengthened with new bots and tasks to better test against PostGIS dependencies across multiple architectures. This release represents a great step forward for users and projects relying on PostGIS, and CARTO is excited to join efforts with the Open Source community to continue improving this powerful tool.

CARTO's Data Observatory 2.0, now powered by Google BigQuery, is a new platform for discovering and consuming location data for spatial analysis. The platform provides a scalable and robust data warehouse with strong geospatial support, utilizing Google Cloud and BigQuery capabilities. It hosts all data within the DO in BigQuery, using a smart metadata system to register thousands of datasets, which are spatially indexed and fully cataloged. The platform relies on cloud-based components to populate, process, and create different data products. CARTO and BigQuery's collaboration is seen as a game-changer for spatial data infrastructure, offering benefits such as separation of computation from storage, scalability, multitenancy, extensibility, and geo support. Public data will be made available through the platform, including US Census Bureau American Community Survey and environmental datasets. The partnership aims to make spatial data more accessible to Data Scientists and geospatial experts, while leveraging tools like CARTO and BigQuery to create a fully invested GIS data access analysis and visualization solution.

Spatial Data Simplified: Introducing Data Observatory 2.0` introduces the concept of spatial data analysis, which relies heavily on access to relevant data. However, gathering and cleaning data is often time-consuming, taking up to 80% of a Data Scientist's total analysis time. This is particularly challenging for Spatial Data Science, where data sources are highly varied. CARTO aims to improve efficiency by releasing Data Observatory 2.0, a private beta version that simplifies access to public data and commercial datasets, enables easier data joins through a common geographic base, and streamlines Data Science workflows. The new platform addresses three key issues: discovery, licensing, and ETL/joining, reducing the time spent on these tasks from months to weeks. By reducing the cost of accessing and using data, Data Observatory 2.0 aims to increase the use of data, ultimately leading to a continuous virtuous cycle of reduced data costs and increased usage. The platform is designed to support Data Scientists in Python Notebooks, providing an integrated experience for managing data discovery, purchase, and access without leaving their analysis environment.

CARTO and SafeGraph have teamed up to provide behavior pattern insights, which can help cities and governments optimize infrastructure, companies engage with customers more effectively, and investors make informed decisions. The partnership combines premium datasets from SafeGraph with CARTO's Location Data Stream Catalogue, enabling users to analyze anonymized population movement patterns in aggregate. By exploring different dimensions of the data, such as time, space, and brand relationships, users can gain insights into visit patterns, dwell times, and demographic segments, ultimately informing geomarketing decisions and uncovering growth opportunities. The partnership offers a wealth of data streams, including origin-destination matrices, census block group analysis, and cross-brand affinity exploration, allowing businesses to create a deeper understanding of their customers and optimize their strategies accordingly.