January 2023 Summaries

Socket has been leveraging GPT as an internal tool to aid in the triage of the npm package firehose, particularly after the introduction of ChatGPT, which has heightened enthusiasm for large language models. While these models offer significant strengths, such as providing quick summaries to prioritize auditing, they also have limitations, including susceptibility to false positives and negatives and constraints on analyzing large files due to token limits. Socket addresses these challenges by segmenting code for analysis and adjusting AI parameters like temperature to maintain a balance between accuracy and paranoia. Although GPT is not poised to replace human roles entirely due to its vulnerability to evasion tactics like misleading comments, it serves as a valuable tool for summarization and triaging, even as humans must still verify the results. The use of GPT in this context reflects a broader trend of integrating AI into existing workflows while acknowledging its current limitations and the ongoing need for human oversight.

In an investigation by Socket Security, the npm package "Binky" raised eyebrows when it was discovered to have an anomalous 11,460 versions published in just four months, suggesting a daily update rate that would take 30 years to match manually. Despite its impressive version count, Binky is a minimal package with only 31 lines of code, primarily referencing another package, "random-seed," which itself is unremarkable. The mystery behind Binky's prolific updates was traced to a PowerShell script in the package's files, which continuously looped to publish new versions by incrementing a counter in the package.json file. This script managed to bypass npm's rate limits to achieve its extraordinary publication frequency, though it has since been halted. The situation highlights the whimsical and curious behaviors some developers exhibit within the npm ecosystem.

In January 2023, Socket for GitHub announced a new permission requirement for its users, specifically targeting GitHub organization admins. The app now requires the "Members" permission to access organization member lists, facilitating accurate representation in its dashboard and streamlining member onboarding. Concurrently, Socket has removed the "Single file" permission, which is no longer necessary, while maintaining its policy of only reading package manifest files without transmitting source code to its servers. Organization admins will receive an email regarding this update and are encouraged to review and accept the new permissions. For any queries, users can contact Socket support for assistance.