Research PortalResource Allocation Competitions 2017 – Frequently Asked Questions
Compute Canada’s resource competitions have seen significant growth in demand since the first call for applications in 2011. For more information on Compute Canada’s Resource Allocation Competitions (RAC) process, please refer to these Frequently Asked Questions or send questions to firstname.lastname@example.org.
Table of Contents:
- Resource Allocation Competitions (RAC) FAQs
- Resources for Research Groups (RRG) FAQs
- Research Platforms and Portals (RPP) FAQs
RESOURCE ALLOCATION COMPETITION (RAC) FAQs
The advanced research computing (ARC) needs of the Canadian research community continue to grow as the next generation of scientific instruments is deployed, as ARC becomes relevant to answering key questions in an ever broader list of disciplines, as new datasets are gathered and mined in innovative ways, and as technological advances allow researchers to construct ever more precise models of the world around us. The current CC infrastructure must keep pace with the needs of Canadian researchers. Demand for resources has grown each year and our systems are currently running at maximum capacity. To learn more about Compute Canada’s technical refresh planning click here.
Any individual who is eligible to apply to national granting councils for funding is eligible to apply for an allocation. Typically, this means that regular faculty are eligible to apply while postdoctoral fellows and graduate students are not. Individuals who have adjunct status may apply for an allocation if the project for which the allocation is requested is eligible for funding from CFI or the granting councils. A lead PI cannot submit more than one individual application, but can be a participant in other submissions. A lead PI may simultaneously hold an individual RAC allocation and an RPP allocation.
Compute Canada’s annual Resource Allocation Competitions (RAC) are peer-reviewed competitions that provide priority computing time and storage space on Compute Canada systems. These competitions are open to projects from all disciplines, from humanities to subatomic physics. The allocation requests are peer-reviewed and awarded based on scientific merit and the quality of the research applicant.
No, Compute Canada is not a funding agency. Through our Resource Allocation Competitions, we offer computing time and storage space in our systems.
- Have moderate-sized allocations.
- Request the same amount or less of the resources allocated previously. Increase requests will not be considered.
- Received better than average science score from the RAC.
- Have used their allocation consistently over the previous year.
Applicants who have received resource allocations via Fast Track for two consecutive years MUST apply for resources by submitting a full application to the RAC or RPP competitions.
Compute Canada provides a broad spectrum of advanced computing services in support of research, including technical support, training and application. In this call for proposals, you will be able to request access to:
- Capability clusters
- Capacity clusters
- Large shared memory systems
- General-purpose computing on graphics processing units (GPGPU)
- Local and non-local storage
For more information on the resources and services Compute Canada provides, visit the National Services page on our website.
The Compute Canada team of experts is available to assist researchers in making use of their allocations. This may include installing and configuring software packages, assistance in code optimization, and detailed advice on how to solve technical issues. While the CC team member may end up writing some code for your group, your allocation does not entitle you to deep “embedded” programming support. As a rule of thumb, our staff will help solve problems requiring a few days of their effort, not a few months of their effort.
Yes. It is strongly recommended that all applicants consult with Compute Canada technical staff responsible for the systems they intend to use. This ensures the technical aspects of the proposals match well with the requested systems, to the PI’s benefit. PIs who have not previously used Compute Canada systems MUST discuss their proposal with Compute Canada technical staff before submission.
All consultations with Compute Canada staff should take place in advance of completing your proposal, to allow time for discussion or revisions to the proposal if needed. If you do not know the appropriate technical staff to contact, please email email@example.com.
Yes, you may submit or participate in proposals to both competitions; however, it is not recommended to submit the same project to both competitions. Each competition has specific eligibility criteria, which are outlined in the Competition Guides. Prior to submitting a proposal, you should review each competition’s criteria to ensure your project aligns with the focus. If after reviewing the competition guides you still have questions or concerns, please contact us at firstname.lastname@example.org.
The CCV is now required for all Compute Canada account renewals and Resource Allocation Competitions. Integrating CCV into the Compute Canada DataBase (CCDB) makes it easier for applicants to upload their CVs and standardize their format. It also helps the expert review committees evaluate the quality of each project’s research team.
Yes. A comprehensive list of terms and acronyms used in the competition and application process is available here. Please consult this Glossary when preparing your application.
Compute Canada’s Resource Allocation Competitions only grant access to Compute Canada infrastructure. If you have questions about whether the infrastructure you are using is part of Compute Canada’s national platform, please contact us at email@example.com.
No. You can request access to a service and we will find the best location in our network to host that service. If you did happen to have an existing collaboration, you could mention it and we can pick the most convenient option, but it is not a requirement.
No. The published deadline for submitting final applications to the RRG and RPP competitions is a final deadline. There will be no extensions and late submissions will not be accepted. Any concerns with this policy may be directed to firstname.lastname@example.org.
All applicants will be notified of their competition results in March. You will be contacted by email with any additional information you may need or action items required of you to finalize your allocation. Once all recipients have been privately notified, Compute Canada will publish a list of recipients, project allocation amounts, and allocation values to its public website.
It is possible to assign a monetary value to each allocation. This is useful for researchers to assign an in-kind value of the contribution of Compute Canada to their research program (please note: this in-kind valuation cannot be used as match for a CFI award). These values represent an average across all Compute Canada facilities and include the total capital and operational costs incurred by Compute Canada to deliver the resources and associated services. These are not commercial or market values.
For the 2016 competition, the value of the resources allocated was calculated using the following:
The monetary values for the 2017 resource allocations will be made available when the 2017 competition results are announced in March.
All successful allocations are activated in April. The exact starting date will be confirmed in your notification letter. RRG allocations that were awarded in the 2016 RAC Competition will be extended until March 2017.
It means that you will be given priority on a shared Compute Canada system. If you make use of that priority by submitting a steady load of computational “jobs” throughout the year, you should get at least the level of core-years you have been allocated. This is something known as a “fair-share” system, not a “dedicated use” system. If your workload is not steady, but will rather come as a burst during a certain time of year, you are asked to note this in your application and we will attempt to accommodate your workflow.
Each year, our infrastructure is heavily oversubscribed. The 2016 RAC process was especially competitive due to an increase in the quality and size of requests by the applications received from across the country. In order to support as many high-quality proposals as possible, final allocations were decided based on the caliber of the project’s science, the merits of the project team, and what impacts a 25-50% cut in resources would have on the project. Many projects, across all disciplines, received final allocations less than their original request. The 2017 RAC process is expected to be very competitive as well, and so the final allocation amounts for 2017 competition may be subject to the same decisions around final allocation amounts.
Compute Canada users will be able to request additional resources without needing to submit a full RAC application starting in April 2017. Any Compute Canada user can access modest quantities of compute, storage and cloud resources through Compute Canada’s Rapid Access Service. Unlike the RAC, the Rapid Access Service is not a guaranteed allocation of certain computational resources. It is a shared pool of unallocated resources.
For compute allocations, generally speaking, Compute Canada allocates its batch processing priority based on a fair-share algorithm. Each user is allocated a share of the total system resources, which effectively translates into priority access to the system. If you have used a large fraction of the system recently (ie. larger than your fair-share), your priority drops. However, the scheduling system has a limited time window over which it calculates priority.
After some time (e.g., weeks) of reduced usage, it gradually “forgets” that you overused in the past. This is designed to ensure full system usage and not to penalize users who take advantage of idle compute resources. A consequence is that your total allocation is not a limit on how many compute resources you can consume. Rather, your total allocation represents what you should be able to get over the course of the year if you submit a constant workload to the system and it is fully busy. In other words, once your “total allocation” is used, just keep working.
Each annual Resource Allocation Competition (RAC) is handled as a separate process. The current projects have an advantage in that they have demonstrated usage and success stories, but not an advantage by policy.
Compute Canada’s annual Resource Allocation Competitions announce their formal Calls for Submissions in September and October each year with final submissions due in November. Compute Canada will only consider requests outside of this time frame under very special circumstances. Two such examples include:
- a sudden chance for a major breakthrough
- a new faculty member who just moved to Canada
In these special situations a researcher can submit an “Out-of-Round” application at any time of year. Compute Canada staff will review the project’s requirements and if the researcher qualifies, he/she will be granted access to complete an Out-of-Round application through a link administered through the Compute Canada DataBase (CCDB).
Out-of-Round requests are reviewed based on the quality of the science outlined in the proposal, the qualifications of the principal investigators, and the technical justification for the requested resources. Following submission, the resource requirements and justification documents will undergo a review process and the user will be notified of next steps. For more information, or to request an Out-of-Round Application, please contact email@example.com.
Yes. Any individual who is eligible to apply to national granting councils for funding is eligible to apply for an allocation, irrespective of the size of the research team or the funding received.
The Expert Review Committees are created annually to evaluate the applications for the Resources for Research Groups (RRG) and Research Platforms and Portals (RPP) competitions. In 2016, there were eight Expert Review Committees:
- Astronomy and Subatomic Physics
- Bioinformatics, Neuroscience and Medical Imaging
- Chemistry, Biochemistry and Biophysics
- Environmental and Earth Sciences
- Engineering, Mathematical and Computer Sciences
- Humanities and Social Sciences
- Nano, Materials and Condensed Matter
- Research Platforms and Portals
The members of the 2017 National Resource Allocation Competitions Committee (NRAC) are listed here.
Yes, applicants may choose the scientific committee they would like to review their proposals; however, Compute Canada reserves the right to assign the proposal to another committee if necessary.
Led by Compute Canada’s Chief Science Officer, the Resource Allocation Competition (RAC) Administrative Committee decides the competition policies, oversees the entire competition process and provides technical and administrative support to the Expert Review Committees. While the expert review committees determine the merit of the proposal and the appropriate resource allocation, the RAC Administrative Committee is responsible for making those allocations “fit” on existing Compute Canada resources. This may involve applying a scaling to all allocations, but that scaling will be based on the science score from the expert committee, not an evaluation by the RAC Administrative Committee. This RAC Administrative Committee is composed of individuals who are knowledgeable of Compute Canada resources and are significantly experienced in the RAC process. It is comprised of technical and administrative leaders representing each region (WestGrid, Compute Ontario, Calcul Québec, and ACENET).
The growing demand for Compute Canada resources is reflective of global trends of increased data-intensive research. Within Canada’s research community, we are seeing computational tools and services increasingly being used to identify patterns, undertake modelling and simulations, and support the collection and distribution of massive volumes of data. Table 1 below shows the increase in size of compute and storage resource requests, as well as the average amount awarded, since 2011.
|Year||Cluster size||Total CY Requested||Total CY allocated||Shortfall capacity CY||Shortfall %||Average awarded|
Compute Canada is leading one of the biggest advanced research computing renewals in Canada’s history. Over the course of 2015-2018, Compute Canada will use funding from the Canada Foundation for Innovation (CFI) to replace ageing systems across the country with four new systems that will consolidate resources and centralize services. The institutions nominated to host these new systems are:
- University of Toronto – Large Parallel (LP) Site
- University of Victoria – General Purpose (GP1) Site
- Simon Fraser University – General Purpose (GP2) Site
- University of Waterloo – General Purpose (GP3) Site
Please refer to Compute Canada’s Technology Briefing for more information on this renewal process.
Compute Canada’s capital and operations priorities are driven by the needs of the Canadian research community. Alongside Compute Canada’s renewal and consolidation planning, its Sustainable Planning for Advanced Research Computing Phase 2 (SPARC2) initiative is a consultation process to gather feedback from users and researchers on the types of investment required to enable Canada’s excellence and leadership in science and innovation.
Through a series of in-person and online feedback sessions, an online survey, and a call for White Papers, Compute Canada is gathering feedback from researchers on community needs from 2017-2022. This will inform capital planning (technology choices), service offerings and the setting of operational priorities.
Collectively, we all have a role to ensure advanced research computing resources keep pace with the needs of Canadian researchers and their international and industrial partners. You can help by:
- Letting decision makers in your institution know that advanced research computing is essential to your research.
- Ensuring your President and VPR are aware of how digitally intensive research is transformative and that it requires sustainable predictable funding models.
- Mentioning the resources you use in your publications and presentations.
- Writing your local MP and share news of your research and how you use advanced research computing.
- Working with your communications department on success stories highlighting the advances in digitally intensive research.
- Participating in Compute Canada’s Sustainable Planning for Advanced Research Computing (SPARC) consultations.
RESOURCES FOR RESEARCH GROUPS (RRG) COMPETITION FAQs
Prior to 2016, this competition was referred to as the Resource Allocation Competition (RAC). The name has been changed to more clearly distinguish it from the overall Research Allocation Competition process and the Research Platforms and Portals (RPP) Competition.
The Resources for Research Groups (RRG) Competition was created to enable individual faculty members, and their sponsored research groups, to access compute and storage resources beyond what can be obtained via the Rapid Access Service. Allocations are awarded based on scientific merit, quality of the research team, and development of highly-qualified personnel (HQP). This competition is peer-reviewed and the allocations are available for one (1) year starting in early April.
The RRG Competition is open to projects from all disciplines. Depending on your request, your RRG application will fall into one of two streams:
- Compute between 50-1999 Core Years
- Storage between 10-999 TBs
- GPUs between 10-199 GPU years
- Cloud Compute between 80-499 VCPUs
- Persistent Cloud between 10-99 VCPUs
- exceeds at least one of the resource limits for Regular as defined above
The purpose of streaming is to make access to resources easier for Compute Canada users, as well as to reduce the documentation required for allocations below a certain amount and the complexity of the corresponding evaluation process for the Regular Stream relative to the Large Stream.
Differences in the evaluation process are described in the table below:
If you require resources which are less than the lower bound on the RRG Competition’s Regular Stream, you do not need to submit an RRG application. Instead, you should make use of Compute Canada’s Rapid Access Service, which allows any Compute Canada user to access modest quantities of compute, storage and cloud resources after they register for an account. Many research groups can meet their needs through RAS only.
No. While Principal investigators (PIs) can participate as collaborators on other projects submitting RRG proposals, they can only be the lead applicant for one RRG application.
Yes. Allocations in the Resources for Research Groups (RRG) Competition are granted on a one-year basis, so applicants must re-apply each year for a new allocation.
No. A Notice of Intent (NOI) is NOT required for this competition. This is different from the RPP competition, which does have a NOI stage (please refer to the RPP Competition details here). To apply for the RRG, you simply need to complete an application form in the Compute Canada DataBase (CCDB). The RRG Competition announces its formal Call for Submissions in October with final submissions due in November.
No. Due to the imbalance of demand versus available resources, the RRG has increasingly become a more competitive process. In 2016, 209 applications were submitted to the former RAC (now called RRG) and 187 allocations were awarded. The table below shows the comparison of applications submitted versus allocations granted.
|RAC 2016||Applications Submitted||Allocations Granted|
Using data from the 2016 competition, RRG applications on average received 57% of their requested compute amount and 60% of their requested storage.
All proposals are reviewed based on their scientific and technical merits. In the scientific review, the two areas of focus are: Quality of the Science (60%) and Quality of the Team (40%).
The Resources for Research Groups (RRG) Competition is getting more and more competitive each year given Compute Canada’s fixed resources. New infrastructure and resources will become available over the next few years through Compute Canada’s platform renewal. Over the course of this migration to the new national systems, Compute Canada will continue to allocate what it can within its fixed resource base.
RESEARCH PLATFORMS AND PORTALS (RPP) COMPETITION FAQs
The Research Platforms and Portals (RPP) Competition was created in 2014 to enable communities of researchers to develop and benefit from projects that improve access to shared datasets, enhance existing online research tools / facilities, or advance national or international research collaborations.
The RPP competition is targeted specifically at applications that create new or support existing research platforms or portals. Allocations in the RPP competition may be awarded over multiple years (maximum of three years). Multi-year allocations are subject to an annual review prior to the start of subsequent years and to the availability of resources. This is different from the Resources for Research Group (RRG) Competition, which is targeted at individuals and small groups requesting access to Compute Canada resources and which grants allocations on a one-year basis.
Groups are encouraged to use the RPP competition if their application falls within any of the following categories:
- Resources requested on behalf of a large community of users that will be reallocated to individuals and small groups following the award.
- Applications that provide a public platform that will make use of Compute Canada computing or storage.
- Groups engaging in international agreements to provide multi-year computing or storage solutions based in Canada.
- Groups that are providing shared data sets accessible using a third party (non-Compute Canada) interface.
If your project does not fall into any of the categories above, you are encouraged to access Compute Canada resources through other means:
- All Compute Canada users have access to modest quantities of resources without scientific review through a Rapid Access Service . (Note: This replaces what was formerly known as a default allocation.)
- If you need more resources than can be obtained through Rapid Access but do not meet the RPP criteria above, please apply to the Resources for Research Groups (RRG) Competition.
If you have any questions about whether your resource needs fit within the RPP Competition, please contact us at firstname.lastname@example.org to discuss.
The RPP competition has two application streams:
- Service Portal:
A research web portal providing datasets or tools to a broad research community. Portals generally do not require large computing or storage resources, but may require support effort by the Compute Canada technical team. Groups applying for a service portal often use the Compute Canada Cloud, generally require a static IP address, and may (or may not) have more stringent up-time requirements than most research projects. This option is shown as “Portal” in the online NOI form.
- Compute and Storage Platform:
Platforms usually provide significant compute and storage resources to a community of users. This may include a user-friendly front-end that submits traditional batch jobs on the back-end. Please note: Any proposal requiring at least 50 core years (or 50 VCPUs in the cloud) OR more than 50 TB of storage is considered to be a Compute and Storage Platform. This option is shown as “Platform” in the online form.
Generally speaking, Platforms consume much larger physical resources than Service Portals and as such, they will be subjected to more rigorous review. To learn more about the types of projects that apply to the RPP Competition, visit the RPP Case Studies page on the Compute Canada website.
Yes. Each RPP represents a single platform or a single portal. Researchers are free to submit and participate on more than one application. If the platforms or portals are related to each other, you are also free to combine them in a single application.
Yes. The Notice of Intent (NOI) is a non-competitive stage of the RPP Competition but is still required to verify that your proposal aligns with the competition criteria and to enable Compute Canada to anticipate and prepare for expected full applications.
The NOI should present an overview of the platform / portal being created or supported, the project’s scope and scale, and the rationale for applying to the RPP Competition. More comprehensive instructions of what to include and how to complete the NOI are included in the RPP Competition Guide.
No. RPP projects that are awarded multi-year allocations are required to submit an annual progress report (usually in the fall), however they do not need to submit a new application each year to renew. This annual progress report must be uploaded directly on CCDB.
Yes. Only those projects that requested and were awarded resources for more 2 or 3 years will be exempted from re-applying. Please consult the RPP Competition Guide for further details.
Generally speaking, Compute and Storage Platforms consume much larger physical resources than Service Portals. As such, they will be subjected to more rigorous review.
|Compute and Storage Platform||
You have the option of asking for a three-year allocation and then adjusting your ask after the first year. However, that model works well if the scope of the project is very similar and you are tweaking. If you do not need any computational cores in year one but suddenly request to running 128-core jobs in the middle of your grant, that would be a separate proposal. It is common for projects to ask for a single-year award and then reassess.