HPC Market Update, HPC Trends In the Oil/Gas Sector and IDC's Top 10 Predictions for 2014 Earl Joseph, HPC Program Vice President ejoseph@idc.com
IDC Has >1,000 Analysts In 52 Countries 2
IDC s HPC Team Earl Joseph IDC HPC research studies, HPC User Forum and strategic consulting Steve Conway Strategic consulting, HPC User Forum, market trends, Big Data Chirag DeKate HPC QView, new technology trends, Big Data, innovation awards program Lloyd Cohen HPC data analysis and workstations Mike Thorp Government account support and special projects Kurt Gantrish Government account support and special projects Gabrielle Cattaneo IDC EMEA Charlie Hayes Government HPC issues, DOE and special studies Mary Rolph HPC User Forum conference planning and logistics 3
About IDC: IDC HPC Activities Track all HPC servers sold each quarter Conduct 4 HPC User Forum meetings each year Publish 65 plus research reports each year Visit all major supercomputer sites & write reports Assist in collaborations between buyers/users and vendors Assist governments in HPC plans, strategies and direction Assist buyers/users in planning and procurements Maintain 5 year forecasts in many areas/topics Conduct special research studies 4
Key Trends in HPC The global economy in HPC is growing well, but will likely see a decline in 2013 HPC servers grew 7.7% in 2012 to reach $11.1B Supercomputers jumped 29.6% year/year in 2012 The HPC ecosystem (servers, storage, etc.) is expected to exceed $20B in 2013 The lower half of the market regained momentum in 2013 Major challenges remain for datacenters Power, cooling, real estate, system management Software hurdles: costs; programming for scale and heterogeneity Finding the right compute mix as coprocessors, accelerators grow Storage and data management are more crucial High performance data analysis (HPC Big Data) is growing fast 5
IDC HPC Competitive Segments (in 2012) HPC Servers $11B Supercomputers (Over $500K) $5.6B Workgroup (under $100K) $1.2B Divisional ($250K - $500K) $1.2B Departmental ($250K - $100K) $3.0B 6
HPC WW Market Trends ($K): By Competitive Segments 2010 2011 2012 1H2013 2012/ 2011 Supercomputers 3,475,577 4,370,194 5,654,960 1,725,756 29.4% Divisional 1,268,735 1,236,684 1,216,187 702,067-1.7% Departmental 3,342,747 3,467,271 2,979,230 1,853,790-14.1% Workgroup 1,411,264 1,225,910 1,247,366 809,349 1.8% Total 9,498,323 10,300,058 11,097,743 5,090,962 7.7% 7
HPC WW Market Trends: By System Units Sold 2010 2011 2012 1H2013 2012/ 2011 Supercomputers 2,560 2,908 2,400 748-17.5% Divisional 3,914 3,724 3,663 2,189-1.6% Departmental 20,382 20,625 16,981 11,048-17.7% Workgroup 92,988 84,294 81,104 50,963-3.8% Total 119,844 111,551 104,148 64,947-6.6% 8
HPC WW Market Trends ($K): By OEM 2011 2012 1H2013 IBM 3,362,098 3,551,723 1,563,771 HP 3,307,427 3,419,554 1,573,560 Dell 1,493,289 1,493,172 737,733 Cray 155,620 353,800 122,220 SGI 225,741 274,693 216,743 Fujitsu 120,351 686,657 65,139 NEC 84,141 64,112 37,106 Appro 135,360 111,648 - Dawning 102,923 115,359 74,906 Bull 327,536 60,494 40,049 Other 847,140 966,531 659,736 Total 10,300,058 11,097,743 5,090,962 9
HPC Forecasts: By Competitive Segment 18000 16000 14000 12000 Supercomputer Divisional Departmental Workgroup 10000 8000 6000 4000 2000 0 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 10
Buyer Insights: High Performance Data Analysis (HPDA) Is Transforming HPC Architectures Will Need to Shift Away from Extreme Compute-Centrism Static searches will give way to dynamic pattern discovery 67% of surveyed HPC sites are running big data workloads HPDA includes data-intensive simulation & advanced analytics Most HPDA work will happen on clusters, but not everything Data movement is a big challenge Storage vendors will benefit 11
The HPC Market Beyond The Servers: The Broader HPC Market Worldwide HPC Compute, Storage, Middleware, Application and Service Revenues ($M) 2011 2012 2017 CAGR (12-17) Server 10,300 11,098 15,441 6.8% Storage 3,664 4,059 6,008 8.2% Middleware 1,147 1,254 1,568 4.6% Applications 3,370 3,621 4,837 6.0% Service 1,801 1,877 2,368 4.8% Total 20,282 21,909 30,223 6.6% 12
HPC Trends In The Oil/Gas Sector 13
Worldwide (WW) HPC Server REVENUE Growth By Sector 14
Worldwide (WW) HPC Server Growth By Sector -- PROCESSORS 15
Worldwide (WW) HPC Server Growth By Sector BY NODES 16
The Current HPC Situation: Use Of GPUs/Co-processors Today What percentage of the applications on your HPC systems exploit accelerators or co-processors today? 17
The Current HPC Situation: Budgets in the Oil/Gas Sector 18
The Current HPC Situation: What Is Wanted In Partners What are the key technical or business-partner-related attributes that would cause you to rank one vendor's product over another: 19
The Current HPC Situation: Barriers To Expanding The Use Of HPC What do you see as the barriers to expanding your use of HPC/ technical computing?: 20
IDC s Top 10 HPC Predictions for 2014 21
IDC Top 10 HPC Predictions for 2014 1. HPC Server Market Growth Will Continue in 2014, after a decline in 2013 2. The Global Exascale Race Will Pass the 100PF Milestone 3. High Performance Data Analysis Will Enlarge Its Footprint in HPC 4. ROI Arguments Will Become Increasingly Important for Funding Systems 5. Industrial Partnerships Will Proliferate, with Mixed Success 6. x86 Base Processor Dominance Will Grow and Competition Will Heat Up 7. Storage and Interconnects Will Benefit as HPC Architectures Gradually Course-Correct from Today s Extreme Compute Centrism 8. More Attention Will Be Paid to the Software Stack 9. Cloud Computing Will Experience Steady Growth 10.HPC Will Be Used More for Managing IT Mega-Infrastructures 22
1. HPC Server Market Growth Will Continue in 2014, after a Decline in 2013 2010-12: 3 successive years of record revenue growth 2013: A $1 billion dip from exceptional 2012, but the lower half came back strong We forecast that all HPC competitive segments will grow in 2014 23
2. The Global Exascale Race Will Pass the 100PF Milestone China, the U.S., Europe (PRACE) and Japan will likely deploy 100PF systems in 2H 2014 to 2015 Watch for the roles played by indigenous Chinese, Japanese processors Peak ES systems will start arriving ~2020 Power efficient (20 to 30MW), early ES systems will wait till 2022-24 The ES race will be as much a funding competition as a technology competition 24
3. High Performance Data Analysis Will Enlarge Its Footprint HPDA = Big Data Using HPC Data-intensive modeling/simulation + newer analytics methods Growing in established HPC domains + new wave of commercial firms 67% of HPC sites are using HPDA today Data analysis uses 30% of the HPC compute cycles on average 2016 forecast: HPDA servers: $1.2B HPDA storage: $800M 25
High Performance Data Analysis Will Enlarge Its Footprint 26
There Are New Technologies That Will Likely Cause A Mass Explosion In Data Requiring HPDA Solutions 27
4. ROI Arguments Will Become Increasingly Important for Funding Big Systems The former Cold War arms race is becoming an economic race HPC is a proven accelerator of economic competitiveness High-end supercomputers now cost $200-500 million ROI can be a scientific advance or corporate profit, revenues, new jobs or retaining jobs More large HPC centers have industry outreach programs IDC s HPC ROI pilot study for DOE quantified 208 examples 28
5. Industrial Partnerships Will Proliferate, with Mixed Success Many national labs/centers added industrial outreach programs in recent years Partnerships typically have an ROI component (technology transfer, economic development) Some labs/centers have had shining successes (e.g., INCITE, SciDAC), while others are at the start of the learning curve and struggling It is important to share what has and hasn t worked The HPC User Forum is one platform for sharing 29
6. x86 Base Processor Dominance Will Grow and Competition Will Heat Up Base Processors x86-based systems already capture about 80% of all HPC server revenue The acquisition of IBM s x86 server business should enable Lenovo to further advance its x86 position To grow share, other base processors (e.g., Power, ARM) will need to step up innovation and provide clearly differentiated value Coprocessors (2013 MCS) Sites using coprocessors/ accelerators jumped from 28% in 2011 to 77% in 2013 Nvidia leads the pack today Future purchase intent is strong for both Nvidia and Phi -- FPGAs are a distant third Most are still experimental Growth barriers remain in programming difficulty and lack of strong software ecosystems 30
7. Storage and Interconnects Will Benefits As Architectures Course-Correct from Today s Extreme Compute Centrism Storage The fastest-growing HPC market segment $4.1B in 2012, $6.0B in 2017 (8.2% CAGR) $6B = size of HPC server market in 2000 HPC storage revenue will grow to record levels The HPC storage market remains fragmented The big players are turning their attention to this market HPDA will boost storage budgets Interconnects The HPC interconnect market is in transition Data movement/ management is a major paint point Multi-year shift away from today s extreme computecentrism The big players are turning their attention to this market Established players are advancing hard Much anticipation about Intel s fabric plans 31
8. More Attention Will Be Paid to the Software Stack Growing needs can no longer be postponed Robustness/resiliency in mega-compute and storage systems where there may always be some components in failure mode Autonomic and machine learning functions to relieve programmers/users New collaboration modes and environments (teams working across distances, cloud computing) Rebalancing the stack as architectures shift from extreme compute centrism Vendors are already putting more focus on the stack IDC forecast: HPC systems software will grow to $1.5B in 2017 32
9. Public Cloud Computing Will Experience Steady Growth Sites exploiting cloud computing to address parts of their HPC workloads rose from 13.8% in 2011 to 23.5% in 2013 Public and private clouds were about equally represented Today s public clouds are still best suited for EP workloads More private- and public-sector organizations are using public clouds for drug candidate screening, other EP jobs Main cloud use scenarios: surge workloads, R&D projects, SMBs without HPC data centers Public cloud use will accelerate as clouds overcome barriers: Data security, data transfer times, non-ep performance 33
10. HPC Will Be Used More for Managing Mega-IT Infrastructures For managing large and diverse mega-it environments Dealing highly mixed systems (hardware, software, different user access devices, etc.) Mega-IT centers linked between major geographies For security and RAS Dealing with constantly failing components Monitoring the system complex for intrusion vs. failures For example: PayPal using HPC to mange their IT infrastructure Google hiring HPC experts to design their next-generation architectures 34
HPC Will Be Used More for Managing Mega-IT Infrastructures 35
In Summary 36
Conclusions HPC is still expect to be a strong growth market Growing recognition of HPC s strategic value is helping to drive high-end sales Low-end buyers are back into a growth mode HPC vendor market share positions will likely shifted greatly in 2014 and 2015 Recognition of HPC s strategic/economic value will drive the exascale race, with 100PF systems in 2H 2014/2015 20/30MW exascale systems will wait till 2022-2024 The formative HPDA market will expand opportunities for vendors 37
Essential Guidance Major Customer Pain Points = Opportunity Areas For Vendors New challenges affecting HPC data centers -- the increase in CPUs, heterogeneous processing, server units and data is creating significant IT challenges in: Managing HPC system complexity Balancing user needs & peak/linpack performance expectations Managing data volumes and types Specifying and managing storage Planning for power/cooling and space Application scaling and hardware utilization Making optimal use of new processor and system designs Finding the appropriate role for private/public cloud computing 38
In Conclusion: Why HPC Is Projected To Keep Growing 1. The low half of the market is finally back to a recovery mode 2. It has become a competitive weapon 3. Governments view HPC leadership as critical For national pride, but more importantly for economic prosperity It use to be 1 large supercomputer now its multiple ones 4. There are very critical HPC issues that need to be solved Global warming, alternative energy, safe NE, financial disaster modeling, healthcare, homeland security, And 3D movies and large scale games are fun 5. The combination of big data and HPC is creating many new opportunities 6. At the same time, live science and live engineering costs have escalated And time-to-solution is months faster with simulations 39
Questions? Please email: hpc@idc.com Or check out: www.hpcuserforum.com www.hpcuserforum.com/roi www.idc.com 40