Building an Open-Source Project Monitoring Tool with Firecrawl and Streamlit
Blog post from Firecrawl
OS-watch is a Python-based application designed to keep users updated on trending open-source projects by scraping GitHub's trending repositories and sending notifications via Slack. The application allows users to customize their searches by keywords, programming language, and time period, and it automates this process using a scheduler to run checks at set intervals. OS-watch employs Firecrawl for web scraping and Streamlit for the user interface, facilitating interaction through a web browser. Notifications are delivered to Slack channels using webhooks, formatted to provide detailed repository information. The system's modular design, using Python modules like config.py and scraper.py, ensures maintainability and ease of updates, while state management and error handling contribute to its robustness. The application exemplifies modern Python development, combining web scraping, user interface design, and automation to monitor open-source trends effectively.