RMIT University & The Data Foundry – Amazon Web Services (AWS)

Executive Summary

The Data Foundry, an Australian AWS Partner, implemented an AWS-based HPC platform to help RMIT University researchers visualize molecular structure 100 times faster, simulate photonic chips 10 times faster, reduce wait times, and gain better visibility into costs. The Data Foundry used Service Workbench on AWS to create the RMIT AWS Cloud Supercomputing Hub.

Overcoming the Limitations of an On-Premises HPC Environment

The Royal Melbourne Institute of Technology (RMIT University), founded in Melbourne, Australia, in 1887, is a leading public research university with 97,000 students. Named one of the world’s top 250 universities, RMIT focuses on art and design, architecture, education, engineering, technology, business, and communications.

For years, RMIT researchers relied on a distributed high performance computing (HPC) environment, which could not scale sufficiently to support increasingly complex research by both researchers and students in areas such as photonics, battery technologies, and geospatial science. “Many of our researchers faced compute, storage, and network constraints that impacted their research,” says Dr. Robert Shen, director of RMIT AWS Cloud Supercomputing. “Some researchers couldn’t analyze multidimensional datasets or run large computationally intensive data modeling, and a few struggled to even run simulations using small datasets. We needed more scalability and permanent data storage options for researchers.”

RMIT also wanted to provide self-service HPC access to researchers, so they wouldn’t have to rely on external HPC facilities such as the Australia’s National Computational Infrastructure (NCI), which allocated public research resources on a quarterly basis. “NCI is very competitive, and not all researchers can get resources,” Shen says. “Also, even if you do get resources, you have to submit your job in a queue.”

RMIT sought to move to a cloud-based HPC environment to overcome its challenges. “We knew the cloud would provide scalability and on-demand access,” says Shen.

RMIT researchers using the RACE platform on AWS are able to test ideas and solutions up to 100 times faster compared to our distributed HPC environment.”

Dr. Robert Shen
Director, AWS Cloud Supercomputing Hub, RMIT University

Building Australia’s First Dedicated Cloud Supercomputing Facility

Because RMIT wanted to offer the first dedicated cloud supercomputing facility at an Australian university, it needed to get its new HPC platform up and running as quickly as possible. “We understood that we had a limited capacity internally to build our own environment quickly,” says Nick Balkin, technology program manager at RMIT. For this reason, RMIT engaged with Amazon Web Services (AWS), which introduced RMIT to AWS Partner The Data Foundry, an Australian technology and services solution provider.

The Data Foundry team worked directly with RMIT researchers to understand their needs around processing power, speed, and data storage. The Data Foundry then implemented Service Workbench on AWS, a solution that offers prebuilt AWS environments with scalable governance and security. With these capabilities, RMIT researchers have self-serve access to AWS resources through a web-based catalog of preconfigured environments. Through the Service Workbench study portal, researchers can upload their study data or software directly into Amazon Simple Storage Service (Amazon S3) for storage.

RMIT and The Data Foundry used Service Workbench on AWS to create the RMIT AWS Cloud Supercomputing Hub (RACE), which can scale from 10 Gbps to 400 Gbps, enabling significantly faster data upload times. AARNet provisioned connectivity to AWS from the RACE facility in Melbourne using AWS Direct Connect services. The Data Foundry helped RMIT put the Service Workbench solution into place in less than two months, working closely with the initial researchers to ensure they had all the required software and configurations to continue their research using RACE. RMIT became the first Australian university to go live with a dedicated cloud supercomputing facility. “We wanted to go live quickly because we knew we had an opportunity to build something here that was somewhat groundbreaking in the sector,” says Balkin. More than 150 RMIT researchers use RACE, which opened in July 2022, to drive advances in research.

Accelerating Innovative Scientific Research

Relying on the AWS-based RACE platform, RMIT has the scalability and performance to drive faster research outcomes. Researchers can now access greater computing power on demand to address complex challenges in areas such as battery technologies, photonics, and geospatial science. “RMIT researchers using the RACE platform on AWS are able to test ideas and solutions up to 100 times faster compared to our distributed HPC environment,” Shen says. One researcher, Professor Michelle Spencer, is using RACE to analyze data and communicate a new, faster way to screen hundreds of potential molecules that could make electrolytes for lithium-metal batteries. “Professor Spencer can visualize molecular structure 100 times faster than with the on-premises environment, which means she can more quickly analyze how molecules impact each other,” says Shen.

Associate Professor Thach Nguyen at RMIT’s Integrated Photonics and Applications Center is simulating photonic chips 10 times faster than before by using RACE. The tiny chips can plug into optical fiber networks to make the internet faster or plug into medical diagnostic tools to quickly analyze how cancer cells spread.

RMIT Professor Matt Duckham is using RACE to design new ways to automatically pinpoint a person’s exact location using only a verbal description of the features around them. With RACE, Duckham’s team can now process massive information streams including drone imagery, satellite data, and data from sensor networks.

Reducing Wait Times and Improving Visibility into Costs

RMIT researchers no longer need to wait in queues to access HPC resources. Instead of waiting up to 100 hours, researchers only spend a few hours provisioning compute and storage and setting up research parameters. “Rather than waiting and getting approval, our researchers can do their work in a few hours because they no longer need to wait on resources from NCI,” says Shen.

In addition, the RACE portal gives researchers visibility and control over cloud spend. “Our researchers can see their exact cloud resource usage by logging in to the portal,” says Shen. “As a result, they get more accurate cost estimates in a browsable service catalog, which makes it easier to estimate costs and meet budgets.”

RMIT is now considering the deployment of additional AWS services such as AWS ParallelCluster, which will help researchers access more distributed computing. “The partnership with AWS and the RACE team has been great, and we have enjoyed being the technical enablers of this platform,” says Joel Macey, lead data engineer at The Data Foundry. “We look forward to continue working closely with RMIT to expand the university’s research capabilities on AWS.”