September 2014 Summaries

In the realm of high-performance computing (HPC) in the cloud, efficient data management is crucial for engineers and scientists dealing with large data volumes. The emphasis lies on minimizing the time between data generation and its practical usability, rather than simply downloading vast datasets. Rescale provides solutions such as command-line tools and post-processing options to streamline this process. For instance, engineers using LS-DYNA can reduce data size significantly by using specific commands to compile necessary information into smaller files, while Converge CFD users can utilize SSH credentials to access and convert data on clusters. Additionally, Rescale offers a Java-based utility for automatic data downloading upon job completion. The company is also developing a remote desktop solution for improved GUI-based post-processing, aligning with its goal to integrate cloud capabilities seamlessly into existing engineering workflows.

Rescale addressed a common user interface challenge of selecting a large set of files for group actions by developing an innovative inclusion/exclusion rule system, as traditional methods like checkboxes for each item proved inefficient and could crash browsers when handling extensive data sets. Inspired by Gmail's approach to managing large email selections, Rescale implemented a solution where users initially see only a limited number of files. Users can select files, which are added to an inclusion set, and utilize a "Select all N files" button, which clears the inclusion set and sets a flag for selecting all files, allowing deselected files to be added to an exclusion set. This method enables users to perform group actions efficiently, such as deleting or downloading files, with the flexibility of excluding a few, thus improving the user experience for managing large file outputs from simulations on Rescale's platform.

Stephen Jones, a lead software engineer at SpaceX and former CUDA architect at NVIDIA, shares insights on the evolution of computing technology, particularly focusing on parallel computing and high-performance computing (HPC). He notes how computing has shifted towards parallelism due to the stagnation in individual processor speed, emphasizing the role of automated tools over human programming in managing this complexity. Jones highlights the game-changing impact of cloud technology, like Amazon's AWS, in democratizing access to HPC, enabling startups to engage in large-scale computing tasks previously reserved for larger entities. He discusses CUDA's niche success in pushing programming boundaries due to its close integration with NVIDIA's hardware, contrasting it with the broader but less specialized programming languages. Jones advocates for high-level programming languages like Python for their maintainability and broad talent pool, while acknowledging the ongoing relevance of languages like Haskell for their natural parallelism. He predicts a decline in FORTRAN usage due to its lack of intrinsic parallelism, foreseeing a future where software increasingly writes software. Jones also stresses the importance of bridging the gap between software engineering and traditional engineering fields, advocating for better education in computer programming for engineers to enhance innovation and productivity.