Compute Canada demonstrates one of the fastest, long-distance, academic big data transfers in Canada
Salt Lake City, November 17, 2016 – Several leading Canadian and international organizations have successfully tested a next-generation, ultra-fast data transfer at SC16, the world’s largest supercomputing conference.
Led by Compute Canada, this demonstration is the one of the fastest, long-distance, academic data transfers known in Canada. Data-intensive research in sectors like genomics and personalized medicine, environmental science and advanced manufacturing require stable, dedicated, high bandwidth connections for data exchange, analysis and visualization. This demonstration was a direct end-to-end data transfer over Canada’s National Research and Education Network.
“Supercomputing is innovation’s infrastructure. In the digital age, access to tools that can manage massive data sets is critical for our research and innovation community and their industrial and international partners. We need to continue to work with our partners to remain at the forefront of this technology to remain competitive internationally,” says Mark Dietrich, President and Chief Executive Officer, Compute Canada.
For this demonstration, scientific datasets were transported over a dedicated 100Gbps network from Toronto and Montréal to the StarLight International/National Communications Exchange Facility in Chicago and then to the final destination at the SC16 venue in Salt Lake City, Utah.
“With the explosion of data in genomics, rapid data transfer capabilities are absolutely essential. For example, we have a project with Japanese collaborators who are sending us 200TB of whole-genome data. For such projects, the data transfer rate has become one of the main limiting factors to this collaboration,” says Dr. Guillaume Bourque, Associate Professor in the Department of Human Genetics at McGill University and the Director of the Canadian Centre for Computational Genomics.
The collaboration included Calcul Québec, CANARIE, Ciena®, Compute Canada, Compute Ontario, ETS, Globus, GTAnet, Juniper Networks, McGill University, Mellanox Technologies, Nokia, Northwestern University, Ontario Institute for Cancer Research, Open Commons Consortium, ORION, RISQ, SAK Data, SmartOptics, Seagate Technology, StarLight International/National Communications Exchange Facility, Université Laval, and the University of Toronto.
“The possibility to transfer data at great speed and reliability opens up a whole new perspective for scientists in general and for us specifically in Metagenomic sciences that generally deal with large databases. Scientists need to be able to easily transfer and process this large-scale information in order to keep Canada’s competitive edge and democratize access to large datasets. As an example of large datasets, metagenomic sequencing provides insight into the biology of microbial communities that can be applied to understand human diseases or environmental issues. Datasets relating to these projects generate hundreds of terabytes of data – essentially the equivalent of 10 million movie downloads. This type of research provides new abilities to seek better insights into our biological environment and how we interact with it,” says Dr. Jacques Corbeil, Canada Research Chair in Medical Genomics Tier 1, Professor of Medicine,Department of Molecular Medicine,Big Data Research Centre, Laval University.