Comparison of Commercial Cloud and Compute Canada’s National Advanced Research Computing Systems
Compute Canada has performed a detailed analysis of two recent large compute cluster purchases along with storage purchases, totalling over $15M (Cdn) in capital expenditures.
Comparison pricing was used from commercial offerings of a very large commercial cloud provider. Configurations were selected which most closely match CPUs, storage, and other characteristics of Compute Canada’s new systems. Applicable volume discounts were applied, and US funds were converted to Canadian Funds.
The key result was that the total costs, over five years, of the commercial provider were significantly higher than the total cost of ownership of newly-purchased systems over the same five years. In addition, the commercial cloud provider could not provide the range of services required to serve a large portion of the Canadian research community.
Cloud-based costs ranged from 4x to 10x more than the cost of owning and operating our own clusters. Some components were dramatically more expensive, notably persistent storage which was 40x the cost of Compute Canada’s storage.
Furthermore, the cloud provider did not offer all of the key characteristics of two of Compute Canada’s new national sites namely ARBUTUS at the University of Victoria and CEDAR at Simon Fraser University. Compared to ARBUTUS, the cloud provider did not offer adequate local/dedicated storage. Researchers with needs for very high performance on-node storage would be dramatically underserved by the commercial provider. These include research fields such as physics, environmental science and other fields
Compared to CEDAR, the cloud provider did not offer tightly coupled sets of nodes, with a low-latency high-performance interconnect. This would create great challenges for much of the CEDAR workload, which is dominated by parallel multi-node computation. This type of configuration is standard in data intensive research and is the basis for a large majority of advanced research computing in areas of importance such as environmental science, complex system modelling, physics etc.
The overall cost-comparison outcome may be summarized as follows:
- For ARBUTUS, outsourced direct (capital) costs of 5 years of the closest comparable configuration was approximately 7.4x the cost of the GP1 purchase. However, the outsourced solution would not meet the requirements of the ARBUTUS workload, due to the lack of local node storage.
- For ARBUTUS, when 5 years of operational costs are added, the 7.4x decreases to approximately a 4x cost increase, to outsource versus purchase.
- For CEDAR, outsourced direct (capital) costs of 5 years of the closest comparable configuration was approximately 15.4x the cost of the CEDAR purchase, reflecting 6.6x for compute, and 40x for storage. However, the outsourced solution would not meet the requirements of the CEDAR workload, due to the lack of local node storage (for the OpenStack partition) and absence of a high performance low latency interconnect.
- For CEDAR, when 5 years of operational costs are added, the 15.4x decreases to approximately a 10.9x cost increase, to outsource versus purchase.