February 2017 Summaries

Rescale's ScaleX Enterprise offers a comprehensive solution for IT and engineering management teams to monitor and control their high-performance computing (HPC) cloud infrastructure, addressing concerns about budget management and resource oversight. The platform provides a customizable IT dashboard with administrative capabilities, allowing users to set permissions, manage budgets, provision hardware and software, and track usage across global and regional levels. Detailed usage reports and budget controls are available to ensure spending remains within constraints, with the ability to set budgets for companies, groups, users, and projects. In cases where resource usage exceeds budget limits, the system places jobs in a queue or terminates active jobs until funds are replenished, ensuring financial discipline and efficient resource allocation.

The text outlines techniques for testing asynchronous code, particularly focusing on scenarios involving promise resolution and side effects within JavaScript frameworks. It presents a class, PromisesHaveFinishedIndicator, which sets a property to true once all promises have resolved, and explores how to test this behavior using the Jest framework. The challenge of asynchronous callbacks, which run after test completion due to JavaScript's event loop and microtask queue handling, is addressed by using setTimeout to ensure promise callbacks execute before the test concludes. The article further refines this testing approach by introducing async/await syntax and discusses the potential use of setImmediate for environments such as Node.js, providing insights into how understanding task scheduling and queues is crucial for reasoning about asynchronous code.

Rescale has introduced a new In-Browser SSH feature on its platform, allowing users to connect to running Linux compute clusters via SSH directly from the browser. This feature is accessible even without pre-configured SSH public keys, although setting them up is recommended for greater flexibility. Users can launch SSH sessions either within the platform or in a separate browser window, with ANSI color support and connection links available for each node in multi-node clusters. However, SSH tabs will only remain active while in the Job Status view, and switching views will close the sessions unless launched in a new browser window or tab. The SSH terminal operates in text mode using an HTML canvas, requiring an intermediate clipboard for copy-pasting, and does not support X11 forwarding. The feature is exclusive to job owners, as shared access does not extend SSH privileges to recipients.

Navigating the challenges of developing a new feature at Rescale, an eager developer delves into the intricacies of implementing persistent clusters, which allow users to run multiple jobs throughout the day on a single cluster. This feature required new API endpoints via Python Django and UI changes with Angular, particularly focusing on reusing the "clusterList Directive" for rendering and managing clusters. The developer faced a critical design decision between using a publish-subscribe (pub-sub) architecture and dependency injection to handle communication between components. While pub-sub offered simplicity and loose coupling, it posed issues with potential interference and implicit dependencies, which could complicate maintenance as the application scaled. In contrast, dependency injection provided explicit dependencies, enhancing reusability and maintainability by clearly delineating component interactions. Through this experience, the developer learned the importance of thoughtfully considering design choices with an emphasis on scalability and clarity.

Advancements in technology over the past decade have drastically reduced the cost of sequencing the human genome, enabling researchers to conduct large-scale population studies that generate vast amounts of data. These studies require substantial computing resources, often straining on-premise systems due to storage expansion logistics, command line interface challenges, and compute resource limitations. Graduate students, often inexperienced with command line interfaces, face a steep learning curve, while IT teams must manage and optimize the high workload of an on-premise HPC system, including archiving and disaster recovery management. Cloud services like Rescale offer a solution by providing scalable, flexible, and instantly available compute and storage resources, allowing researchers to efficiently manage data and meet deadlines. By migrating data to cloud platforms, researchers can enhance the speed and scale of their analyses, leading to quicker discoveries that contribute to advancements in personalized medicine and potentially improve individual health outcomes.