January 2025 Summaries

Semgrep Assistant has introduced two new features aimed at enhancing its semi-autonomous capabilities to significantly reduce triage workloads for developers and security engineers, with a reported immediate workload reduction of 20% and up to 40% after a week of use. The first feature, Noise Filtering, employs Semgrep's deterministic SAST engine and LLMs to identify and filter false positives, maintaining a high accuracy rate that aligns with user and security researcher assessments, thus addressing the persistent issue of signal-to-noise ratio in static analysis tools. The second feature, Autotriage Memories, allows the Assistant to learn and remember organization-specific security contexts without requiring custom rule creation, further streamlining the triage process and improving efficiency, as exemplified by a Fortune 500 company experiencing a 2.8-fold improvement. These advancements are built on principles of transparency and user control, ensuring that all contextual data and findings are auditable and reversible, thereby building trust in the tool's AI-powered functionalities. Additionally, a new Pre-production tab and improved secrets detection through a combination of static analysis and AI context analysis further enhance the platform's capability, enabling it to provide accurate, low-noise results and alleviating the burden on security teams to focus on more critical security challenges.

Semgrep Assistant, by 2025, achieves a 96% alignment with security researchers in identifying true positive security findings, making it a reliable tool for filtering non-exploitable issues and alerting developers only when real problems exist. This high accuracy was reached through continuous benchmarking and refinement of the Assistant's decision-making processes using large language models (LLMs), which excel at reading and interpreting code. Despite being conservative in identifying false positives, leading to a 41% agreement rate in this area, the Assistant rarely overlooks true positives, ensuring critical issues are not ignored. The system effectively combines AI triage with human review, increasing developer efficiency by focusing attention on actionable findings and reducing noise, ultimately enhancing the overall security posture by encouraging developers to address more issues promptly. This AI-driven approach offers a significant advantage, especially for AppSec teams dealing with extensive backlogs, as it saves time and resources by automating parts of the triage process, though it does not replace the need for human expertise entirely.

In 2023, a significant shift in the belief system surrounding traditional static application security testing (SAST) tools occurred, with the integration of large language models (LLMs) anticipated to revolutionize the landscape, rendering many existing tools obsolete, including Semgrep. Initially skeptical about LLMs extending beyond remediation guidance, the realization in 2024 was that LLMs could enhance the findings engine itself, improving the signal-to-noise ratio (SNR), crucial for reducing false positives that lead to engineers disregarding alerts. Semgrep's open-source nature allows leveraging LLMs for writing rules, but domain-specific knowledge remains necessary. A major breakthrough was the introduction of the "Memories" feature, which applies user triage data to refine rules and findings, enabling customization without requiring YAML expertise. The private beta demonstrated the potential of this approach, with users able to significantly reduce their backlog of security findings by encoding organization-specific contexts into the deterministic SAST engine. This development suggests a future where security practitioners can rely on tools that autonomously understand security nuances, reducing the need for "security tool experts" and allowing teams to focus more on strategic tasks.

Managing security policies at scale is often a complicated and error-prone task, especially when using static application security testing (SAST). The Semgrep Policy Management API offers a solution to streamline and automate policy management workflows, making them more efficient, reliable, and scalable. This API allows users to programmatically add, update, or disable rules across multiple policies and integrate these processes into existing CI/CD pipelines, enhancing consistency and reducing manual errors. The API's key features include endpoints that provide detailed visibility into policy structures, control over policy rules, and the ability to apply rules in various modes to suit specific security needs. Designed with scalability and flexibility in mind, the API ensures that security policies can evolve alongside codebases, supporting shift-left security practices and enabling teams to address vulnerabilities earlier in the development cycle. By emphasizing automation and integration, the Semgrep Policy Management API empowers application security teams to focus on delivering secure, high-quality software with reduced friction and improved accuracy.