Upload Anything: how we revolutionized data upload

Felt's development of the "Upload Anything" feature aims to revolutionize map-making by allowing users to easily drag and drop geospatial datasets into their browser for quick visualization. Initially, the platform used a combination of technologies such as FastAPI, Celery, and Python tools for processing datasets, but faced challenges with speed, concurrency, and limited data format support. To overcome these, Felt transitioned to a full AWS architecture, employing AWS Lambda, Step Functions, and S3, which significantly enhanced their pipeline's efficiency and flexibility. This new setup supports a wide range of geospatial data, facilitates easy internal development and experimentation, and ensures fast production shipping. Additionally, by adopting the pmtiles format, Felt improved global performance by serving vector tiles through a CDN, providing a seamless user experience without disrupting internal workflows. This comprehensive approach not only enhances the map-making process but also allows for the handling of large datasets, ultimately enabling fast and frequent feature development and deployment.