August 2014 Summaries

Gianluca Borello explores the concept of hiding processes in Linux from monitoring tools like `ps` and `top`. By leveraging the dynamic linker's preloading feature, Borello demonstrates how to create a custom shared library that overrides the `readdir()` function in libc, effectively concealing a malicious script from standard process monitoring. The post discusses various methods to achieve process hiding, including modifying binaries and kernel system calls, but settles on the preloading technique for its simplicity and effectiveness. Despite successfully hiding the process from traditional tools, Borello shows that Sysdig, a system call-based monitoring tool, remains capable of detecting the hidden process due to its ability to inspect system calls directly. The experiment highlights the versatility of Linux's /proc file system and the challenges of completely concealing system activity, emphasizing the importance of monitoring both through system calls and file system data.

The blog post by Loris Degioanni introduces the use of sysdig, a powerful tool that enhances the capabilities of basic command-line utilities like lsof, ps, and netstat, by incorporating features such as Wireshark-like filtering and the ability to analyze data from any point in time. Sysdig differs from traditional tools by maintaining a constantly updated table of system calls, enabling users to troubleshoot issues with precision and minimal overhead. It allows for detailed queries and time navigation, offering insights into system states during specific events, such as process exits or errors, which traditional tools cannot achieve. The post also highlights practical use cases and the advantages of sysdig's low overhead and flexibility through its chisels and Lua API, making it an invaluable tool for system monitoring and troubleshooting.

Sysdig 0.1.88 introduces new features, including the ability to access the process table through the Lua interface, enhancing its capabilities for users. This update includes three new chisels—lsof, ps, and netstat—designed to leverage this functionality, with a forthcoming blog post promising more detailed insights. Additionally, the release notes highlight Falco Feeds, which offer regularly updated, expert-written security rules to help open source-focused companies stay ahead of new threats. Community support is accessible via the sysdig mailing list, and users can report bugs and issues through GitHub, while installation and update instructions, as well as the source code, are readily available for users.

Sysdig, an advanced log analysis tool, enhances log collection and inspection by integrating with system processes to provide comprehensive insights into system activity. It offers features like syslog and application log monitoring through chisels such as spy_syslog and spy_logs, allowing users to tail logs, correlate them with system events, and troubleshoot issues without extensive configuration. A practical example demonstrates its utility in diagnosing a "database error" on a web server by capturing a system snapshot and isolating the activity leading to the error, revealing incorrect MySQL credentials as the root cause. Sysdig's capabilities highlight the potential to unlock valuable information from system logs, promoting efficient problem-solving and system monitoring, with future enhancements anticipated to further expand its functionality.

Sysdig 0.1.87 introduces enhanced functionalities for log management, featuring updated tools known as chisels that improve the way users interact with log data. Key updates include the introduction of the "spy_logs" chisel, which intercepts and formats log file writes, and the updated "spy_syslog" chisel, which logs syslog messages. Additionally, the release highlights Falco Feeds, a service providing continuously updated, expert-written security rules to help open-source companies stay ahead of emerging threats. The blog hints at a forthcoming detailed post to further explore these new capabilities, while community support is available through the Sysdig mailing list and GitHub for addressing bugs and issues.