Research Portal2019 Resource Allocations Competition Results

French

List of Resource Allocation Competitions 2019 Awards

Summary
Computational Resources
– CPU Allocations
GPU Allocations
– Cloud Allocations
Persistent Disk Storage Allocations
Review Process
Scaling for Compute Requests
Monetary Value of the 2019 Allocations

Summary

Canada’s national advanced research computing (ARC) platform is delivered through the Compute Canada Federation (CCF), which is a partnership of Compute Canada, regional organizations (WestGrid, Compute Ontario, Calcul Québec and ACENET) and institutions across Canada. Providing researchers with access to the infrastructure and expertise they need to accomplish globally competitive, data-driven, transformative research, it serves the needs of nearly 14,000 researchers, including over 4,100 faculty based at Canadian institutions as of January 1, 2019.

Recent investments have enabled a renewal of Canada’s national ARC platform — the incorporation of the new systems, Cedar (Simon Fraser University), Graham (Waterloo University), Niagara (University of Toronto), and Béluga (Calcul Québec) yielded over 90PB of new online storage capacity and approximately 170,300 core years.

However, the dual challenge of the retirement of legacy systems and ongoing growth in researcher demand for resources means that demand continues to outstrip supply. The 2019 RAC competition received the highest number of applications in its history with 507 projects applying for an allocation — 8.1% more applications than 2018. Due to the challenges discussed above, unfortunately, this year’s RAC was only able to award 40% of the total compute requested, 75% of the total storage requested, and 20% of the total GPUs requested. It is important to note that virtual CPUs and cloud allocations are included in the total compute allocation and that this year’s RAC was able to allocate 95% of the total virtual CPUs requested.

While 80% of the resources available through the CCF are allocated through the Resource Allocation Competition (RAC), the CCF reserves 20% for researchers to use through the Rapid Access Service (RAS) and all users have access to these modest quantities of compute, storage and cloud resources as soon as they have a Compute Canada account.

If you have questions about the terminology used in this page, please consult the Technical Glossary.

Table 1: Applications submitted to the Resource Allocation Competitions

Year Total Year-on-Year Increase
2019 507 8.1%
2018 469 14.7%
2017 409 11.7%
2016 366 4.6%
2015 350 20.3%
2014 291 37.9%
2013 211 32.7%
2012 159 17.8%

Computational Resources

CPU Allocations

Based on available computing resources, 40% of CPU (core year) requests were met by RAC 2019, nearly 15% less than in 2018. New systems (Cedar, Graham, Niagara and Béluga), which are faster and have more memory than older systems, provided nearly 85% of the available capacity or approximately 170,300 cores. This resulted in a modest increase in available cores.

The new systems were allocated at close to 80% capacity, leaving approximately 20% capacity for new users and smaller development projects to access resources via RAS.

Table 2: 2019 Compute Allocations per System

CPU Resource Supply: Allocatable Core Years (100% capacity) Need: Total Core Years Requested Provided: Total Core Years Allocated % of CPU Capacity Allocated
Béluga 28,000 47,283 19,656 70%
Cedar 53,888 107,533 44,239 82%
Graham 28,448 65,743 23,166 81%
MP2 30,984 41,034 13,815 45%
Niagara 60,000 128,759 56,385 94%
Total 201,320 390,352 157,262 78%

 As of February 18, 2019

Table 3: Historical Compute Ask vs. Allocation

Year Supply: Allocatable CPU Core Years  Need: Total Core Years Requested Provided: Total Core Years Allocated Shortfall Capacity Core Years % of the Demand Awarded
2019 201,320 390,352 157,262 233,089 40.3%
2018 211,020 287,957 158,632 129,325 55.1%
2017 182,760 255,63 148,10 107,538 57.9%
2016 155,952 237,862 128,463 109,399 54.0%
2015 161,888 191,690 123,699 67,991 64.5%
2014 190,466 172,989 133,508 39,481 77.2%
2013 187,227 142,106 126,677 15, 429 89.1%
2012 189, 024 103, 845 87, 312 16, 533 84.1%

 

GPU Allocations

The allocation of GPU resources was more competitive this year than it was for CPUs. As Table 4 shows, requests for GPUs have increased 5 fold since 2016. In 2019, 688 new GPU devices became available as part of the Béluga cluster. However, due to increased demand for GPU resources, the allocation success rate in 2019 was 20.3%, slightly lower than the 20.5% in 2018.

Table 4: Historical GPU demand vs. supply (GPU years)

Year Supply: Allocatable GPUs  

Need: GPUS Requested
Provided: Total GPUs Allocated Shortfall Capacity GPUs % of the Need Awarded
2019 1,664 6,555 1,331 5,224 20.3%
2018 976 4,092 840 3,252 20.5%
2017 1,420 2,790 1,047 1,743 37.5%
2016 373 1,357 269 1,088 19.8%
2015 482 608 300 308 49.3%
2014 n/a 420 308 112 73.3%
2013 n/a 390 259 131 66.4%
2012 n/a 10 10 0 100%

 

Cloud Allocations

The Arbutus cluster at the University of Victoria has 24,960 allocatable virtual CPUs. These are available via RAC and RAS and are also utilized for internal Compute Canada services such as software development and hosting. A further 36 legacy nodes remain in service as part of Cloud East at l’Université de Sherbrooke, and relatively small cloud offerings are implemented alongside Cedar and Graham. RAC 2019 received a 56% increase in requests for virtual CPUs. Between Arbutus and the additional nodes at Cloud East (UdeS), Cedar (SFU) and Graham (Waterloo), this year’s RAC was able to allocate 95% of the total virtual CPUs requested. In total, cloud storage was allocated at 74% of its capacity for 2019.

Persistent Disk Storage Allocations

Storage integrated with Cedar, Graham, Arbutus, Niagara and Béluga yielded a total of approximately 100PB of online capacity available for 2019. This meant that, across all types of storage, the CCF was able to allocate a total of 86% of its available storage capacity.

Table 5: 2019 Storage Need vs. Supply by Storage Type (TB)

Storage Type Supply Need: Storage Requested TB Provided:  Storage Allocated % of the Demand Awarded
Project 54,600 44,114 33,673 76%
dCache 9,060 9,459 9,376 99%
Cloud 5,184 4,802 3,850 80%
Nearline  32,500 30,687 29,400 96%
Total 101,344 89,063 76,299 86%

Review Process

The majority of RAC applicants request resources to support research programs and highly qualified personnel (HQP) that are already funded through other Tri-Council and peer-reviewed sources.

Submissions are evaluated for the merit and feasibility of the application. For the 2019 competitions, there were 507 submissions to RAC — all of them received a technical evaluation and 239 of them, which were new applications, were peer-reviewed.

Please find here the list of 2019 RAC Expert Review Committees.

Technical Review Technical Staff
  • Technical adjustments are made to ensure requests are compliant with policy and aligned with the technical capabilities of available resources.
Expert Review Disciplinary peer review panel evaluates each proposal
  • Each proposal receives multiple independent reviews;
  • Expert Review Committees meet to discuss the applications;
  • The peer review panel may or may not recommend specific cuts for an application;
  • The peer review panel gives an overall score.

Scaling for Compute Requests

As described above, there were insufficient ARC resources to fully meet the allocations requested through RAC 2019.

As a result, a scaling function was applied to the RAC 2019 to provide a means by which decisions on RAC allocations, in a context of insufficient capacity, could be made. This function, which is endorsed by the RAC Chairs Committee, was set so that only applications with an overall score of 2.3 or higher (out of 5) received an allocation. Applicants who did not receive a compute allocation can still make opportunistic use of system resources via the Rapid Access Service.

Until this year, the scaling function took into account only the overall score. However, researchers raised concerns that this did not address differences in the size of requests. In response to this feedback, this year, the CCF implemented a more complex function that takes into account two variables: the overall score and the size of the request. For example, applications with large requests and low scores received larger cuts, while applications with low scores but smaller requests received smaller cuts.

If you would like details of the mathematical implementation, please contact rac@computecanada.ca.

Monetary Value of the 2019 Allocations

These values represent an average across the national ARC platform’s facilities and include total capital and operational costs incurred to deliver the resources and associated services. These are not commercial or market values. For the 2019 competition, the value of the resources allocated was calculated on a per-year basis using the following rates.

Table 6: Historical Financial Value of RAC Awards

 

Financial Value of Award 2019 2018 2017 2016 2015
1 core year $121.34 $156.78 $188.84 $279.00 $275.00
1 GPU year $2,435.89 $2,960.77 $566.52 $1,100.00 $1,100.00
1 TB of project storage / year $54.96 $36.48 $128.00 $173.00 $190.00
1 TB of nearline / year $25.66 NA NA NA NA
1 VCPU year $80.93 $91.05 $40.50 NA NA
1 TB of cloud storage (Ceph) / year $117.70 $236.81 $178.50 NA NA

Costs for CPUs also reflect the inclusion of 30,984 legacy CPU cores valued at $279/year each. Capital costs are not included for legacy cores, only operational costs. The overall calculation methodology was improved from 2018 to be more accurate.

The valuation of each of these resources decreases each year as older, more expensive, resources are retired and replaced with newer, more cost-effective, resources.

Top