Home / Companies / Firecrawl / Blog / September 2025

September 2025 Summaries

3 posts from Firecrawl

Filter
Month: Year:
Post Summaries Back to Blog
AI is transitioning from simple prompt-based interactions to agentic systems, which are autonomous software capable of understanding their environment, reasoning, utilizing tools, and working until specific goals are achieved. Unlike traditional static bots, AI agents can make independent decisions, solve problems flexibly, and collaborate across various platforms, making them valuable in fields like finance and operations for task automation and process acceleration. Aimed at different skill levels, this article provides 11 hands-on projects to teach the creation of AI agents, ranging from no-/low-code projects for beginners to advanced projects using agentic frameworks. These projects guide users through building autonomous systems that can perform tasks such as data analysis, travel planning, and website monitoring, while imparting essential skills like system design, technical foundations, and practical judgment. The projects emphasize the importance of starting small and iterating, using prebuilt templates and SDKs, and understanding the fundamentals of AI agents to successfully implement them in production environments. Through hands-on experience, users learn to build scalable, reliable AI systems, bridging the gap between theoretical understanding and practical application.
Sep 18, 2025 3,321 words in the original blog post.
OpenAI's GPT-OSS 20B, a refined open-source AI model, is particularly adept at instruction-following and can be efficiently fine-tuned for specialized tasks using LoRA (Low-Rank Adaptation), which updates a minimal portion of the model's parameters to enable fine-tuning on consumer-grade hardware. This tutorial provides a step-by-step guide to fine-tuning GPT-OSS 20B into a specialized dermatology assistant using a custom Firecrawl Dermatology Q&A dataset. The process involves setting up a suitable environment, loading the model and tokenizer, and applying the OpenAI Harmony prompt format to ensure structured, role-based training. Fine-tuning with LoRA adapters updates only a small percentage of parameters, making the process resource-efficient, and results in a model that generates concise and clinically relevant responses. The tutorial also highlights the importance of prompt engineering for achieving precise and domain-specific answers during inference, demonstrating how modern web scraping APIs can facilitate the creation of high-quality datasets and accelerate AI development.
Sep 05, 2025 4,528 words in the original blog post.
Website mapping is crucial in modern web development due to the complexity of sites with dynamic content and single-page applications, impacting SEO, maintenance, and user experience. Firecrawl's /map endpoint offers an automated solution for website mapping and URL discovery, addressing challenges such as maintaining site structure, identifying broken links, and ensuring proper indexing by search engines. Sitemaps, either visual or XML, are essential tools; visual sitemaps help in understanding site hierarchy and planning, while XML sitemaps aid search engines in indexing site content efficiently. The /map endpoint is particularly effective for rapid URL discovery and sitemap generation, providing flexibility with parameters like search, sitemap, and include_subdomains to tailor mapping needs. It is faster than traditional crawling methods and is ideal for creating XML sitemaps and interactive visual sitemaps. The endpoint’s speed, adaptability, and practical applications make it a valuable tool for web developers aiming to optimize website structure, SEO, and content strategy.
Sep 03, 2025 3,341 words in the original blog post.