Comparison of interconnection networks

Size: px
Start display at page:

Download "Comparison of interconnection networks"

Transcription

1 Department of Information Technology CT30A7001 Concurrent and parallel computing Document for seminar presentation Comparison of interconnection networks Otso Lonka Student ID: Andrey Naralchuk Student ID: B Lappeenranta,

2 TABLE OF CONTENTS 1. Introduction Background knowledge Goals and limitations The structure of the work Interconnect network architectures Main concept Different interconnection networks in general Comparison of interconnection networks In the scientific publications before In the scientific publications after 1994 till nowadays TOP500.org statistics Summary References

3 1. Introduction 1.1. Background knowledge This document is a seminar presentation for a course Concurrent and Parallel Computing. The topic of the presentation is Comparison of interconnection networks. Our task is to compare interconnection network solutions. The comparison is done based on scientific publications that were available Goals and limitations The key question in this work is to gain information of interconnection networks and report the results of comparison. We will overview interconnect networks and clarify which features of interconnection network are needed for supercomputing. We will also introduce some interconnection networks as an example. There are many interconnection networks available and this paper will not consider all of them The structure of the work The first part of the document considers of architectures of interconnection networks at general level. The second chapter consists of the comparison of interconnection networks based on scientific articles and it will also explain why it is difficult to compare different interconnection networks. The fourth chapter will introduce statistics and charts from top500.org and explain them. At the last chapter, we will summarize what we did and what we discovered of interconnection networks. 3

4 2. Interconnect network architectures 2.1. Main concept Parallel computer consists of processing units, memory banks and interconnection network. Processing units are responsible of data processing and interconnection network is responsible of data transfer between processing units and memory banks. [1] Interconnection network provides connections between processing nodes in a parallel processing system. Interconnection networks can be classified as static or dynamic Picture 1. A static network is a point-to-point network between processing nodes. A processing node is connected to another processing node without use of any external switching elements. Static interconnection network can be also referred as direct interconnection network. Dynamic interconnection networks are built by using switches and cables between processing elements. Network switches and connections form an interconnection network and the processing units are separate from the network. Dynamic networks are called as indirect networks. [1] Dynamic interconnection networks have scalability features in general. Picture 1. Static and Indirect networks [1]. The links of an interconnection network can be either based on conducting material or fiber. [1] For the links built of conducting material, there is always some amount of signal loss and noise. A capacitance between a signal path and a ground limits the frequency range which can be used for data transfer. The cable works itself as a low pass filter, which filters high frequencies. The longer the cable is, the more capacitance it contains. [1] The higher frequencies are needed to achieve high data transfer rate. Thus 4

5 the length of the link limits the capacity of data transfer when using links based on conducting material. For fiber cables there is no such a limitation as the signal is transferred in a non-electrical form. By using fiber cables, the lengths of the links are not so critical and this is probably one of the reasons, why clustering is so popular in modern supercomputers instead of using PCBs Different interconnection networks in general Multistage crossbar is an interconnection network in which all input ports are directly connected to all output ports without interference from messages from other ports. In a one-stage crossbar this has the effect that for instance all memory modules in a computer system are directly coupled to all CPUs. This is often the case in multi-cpu vector systems. In multistage crossbar networks the output ports of one crossbar module are coupled with the input ports of other crossbar modules. In this way one is able to build networks that grow with logarithmic complexity, thus reducing the cost of a large network [2]. A hypercube is the family of direct or indirect binary n-cubes is a special case of the family of k-ary n- cubes with n dimensions and k nodes in each dimension. n-cube is used to denote the binary n-cube. n The n-dimensional hypercube is composed of 2 nodes and has n edges per node. If unique n-bit binary addresses are assigned to the nodes of the hypercube, then an edge connects two nodes if and only if their binary addresses differ in a single bit [3]. A mesh is the family of interconnection networks that are a special case of k-ary n-cube networks in which the number of dimensions, n, is two. Until recently Myrinet was the market leader in fast cluster networks and is still one of the largest. The first Myrinet implementation as an alternative for Ethernet to connect the nodes in a cluster. Myrinet uses cut-through routing for an efficient utilization of the network. Also RDMA is used to write to/read from the remote memory of other host adaptor cards, called Lanai cards. These cards interface with the PCI-X bus of the host they are attached to. The latest Myrinet implementation only uses fibers as signal carriers. Since the start of 2006 Myricom provides a multi-protocol switch (and adapters): The Myri-10G. Apart from Myricom's own MX protocol it also supports 10 Gigabit Ethernet which makes it easy to connect to external nodes/clusters. The specifications as given by Myricom are quite good: 1.2 GB/s for the uni-directional theoretical bandwidth for both its MX protocol and about the same for the MX 5

6 emulation of TCP/IP on Gigabit Ethernet. According to Myricom there is no difference in bandwidth between MX and MPI and also the latencies [4]. Infiniband has become rapidly a widely accepted medium for internode networks. The specification was finished in June Infiniband is employed to connect various system components within a system. Via Host Channel Adapters (HCAs) the Infiniband fabric can be used for interprocessor networks, attaching I/O subsystems, or to multi-protocol switches like Gbit Ethernet switches. Because of this versatility, the market is not limited just to the interprocessor network segment and so Infiniband is expected to become relatively inexpensive because a higher volume of sellings can be realised. The characteristics of Infiniband are rather nice. Conceptually, Infiniband knows of two types of connectors to the system components, the Host Channel Adapters (HCAs), already mentioned, and Target Channel Adapters (TCAs). Infiniband defines a basic link speed of 2.5 Gb/s (312.5 MB/s) but also a 4 and 12 speed of 1.25 GB/s and 3.75 GB/s, respectively. Also HCAs and TCAs can have multiple ports that are independent and allow for higher reliability and speed [5]. QsNet(Quadrics) is the main product of Quadrics, a company that initially started in the 1990s. The network has effectively two parts: the ELAN interface cards, comparable to Infiniband Host Bus Adaptors or Myrinet's Lanai interface cards, and the Elite switch, comparable to an Infiniband switch/router or a Myrinet switch. The topology that is used is a (quaternary) fat tree. The ELAN card interfaces with the PCI-X port of a host computer. The second generation is QsNet II. The structure, protocols, etc., are very similar to those of the former QsNet but much faster: where in a speed of 1.3 GB/s. In this case the PCI-X bus speed is a limiting factor: it allows for somewhat more than about 900 MB/s. However, with the advent of PCI Express this limitation may be over. Also the latency for short messages has improved from 5 µs. Since early 2006 Quadrics also offers 10 Gbit Ethernet cards and switches under the name QSTenG. These products capitalise on the experience obtained in the development of the two generations of QsNet but they are not meant for inter-processor communication [6]. SCI, standing for Scalable Coherent Interface. SCI became by October It was born as a reaction to the limitations that were encountered in bus-based multiprocessor systems. In clusters SCI is always used as just an internode network. Because of the ring structure of SCI this 6

7 means that the network is arranged as a 1-D, 2-D, or 3-D torus. One of the SCI vendors provides software to reconfigure the torus in such a way that the minimal number of nodes becomes unavailable. Furthermore, it is not possible to add or remove an arbitrary number of nodes in the cluster because of the torus topology. Bandwidths are reported for the SCI-based clusters: up to about 320 MB/s for a Ping-Pong experiment over MPI with very low latencies of 1 2 µs for small messages [7]. 3. Comparison of interconnection networks 3.1. In the scientific publications before 1994 I have found some scientific article about problems to compare interconnection networks. One of the problems is which metrics of the network should be determined. A sampling of some of the metrics found in the literature include: message delay characteristics, permuting ability, partitionability, fault tolerance, bisection width, graph theoretic measures, throughput, complexity of control, and costeffectiveness. In general, it is difficult to select the important metrics, because they may vary with the intended application and operating assumptions. Basing design decisions on different sets of metrics can result in different design choices [12]. Another difficulty in comparing two different network designs is determining if they are of equal cost, so that the comparison will be fair. The cost should be defined in terms of hardware required, rather than actual dollar cost [12]. In any way let s to define what the tendency is to choice of type s interconnection network. One of the important characteristics of a supercomputer is a supercomputer cost. The customers look for a supercomputer with high-performance and low cost of it. And a cost of interconnection networks plays not the last role in the full cost of supercomputer. How are characteristics of supercomputers in part of interconnections network been described in the publications? One of the publications was about comparison bus and crossbar [11]. Authors said that traditionally, the choice for interconnection architecture has been a bus. Buses are relatively simple and the hardware cost is small. The drawback of a single bus is that only one agent can transmit at a time. One alternative 7

8 to bus-based architectures is a switched network that is described into a circuit-switched crossbar. Crossbar is a low latency, non-blocking interconnection network for large data transfers. However, the hardware cost of a crossbar is relatively large. One of the author opinions is to use hierarchical structures where interconnection segments are connected with bridges (bus) or hierarchical switches (crossbar) that was implemented in big supercomputer. Crossbar is relatively complex system architecture. However, if the data transmissions are long and if they can be done in parallel, the hardware complexity can be justified [11]. The publication about analysis of the network characteristics that was presented by Anant Agarwal [13] is based on model. The model includes the effects of packet size, communication locality and other characteristics. Network analysis was made under various constraints. After comparison author found out that a message length plays an important role in the tradeoff between low and high-dimensional networks. Longer messages (such as those in message passing machines) make the relative effect of network distance from source to destination less important, while the lower expected message lengths in shared memory machines increase the relative influence of network distance, and tend to favor networks with more dimensions. Communication locality depends on several factors including the architecture of the multi processor, the compiler and runtime systems, and the characteristics of parallel applications. If the communication locality of parallel applications can be enhanced through better algorithms and systems architectures, parallel machines with significantly higher performance can be built without incurring the high cost of expensive networks [13]. One of the interesting publications is the performance analysis of the supercomputers with different interconnection topologies [14]. It was actually in days when the article was written. Author measured and grid topologies on the SUPRENUM supercomputer. Those measurements utilized the original 4- ring interconnection topology. Author presents the measured result in the Table 1. The table indicates the number of processors P, their arrangement as a logical Px Py rectangular processor array, the computational domain size Mx My, the resulting computational efficiency (comparison to P times the single node performance) and the Mflops generated. 8

9 Each node contains a grid and the processors are arranged in a line parallel to the Y direction. The peak performance is seen to be 1272 Mflops. Table 1. Processor grid aligned with Y axis (double matrix) [14] He compares the effect of varying the shape of the processor grid for 256 node computations Table 2. The almost square grid on a processor array is seen to deliver 1042 Mflops. Table node performance as function of processor grid (full connect) [14] I can make conclusion of this research that a grid topology has different efficiency and performance with different characteristics of the grid. Also I meet articles about possibility to achieve high performance in a supercomputer. One of the articles was presented by Sotirios G. Ziavras [15]. He said that the direct binary hypercube has been one of the most popular interconnection networks because it provides small diameter, and is so robust that it can simulate very efficiently a wide variety of other frequently used interconnection networks. Newetheless, a major drawback of the hypercube is the dramatic increase in the total number of communication channels with any increase in the total number of processors in the system. This drawback has significant impact on the very large scale integration (VLSI) complexity of hypercube systems. For this reason, existing commercial hypercube systems have either a large number 9

10 of primitive processors (for example, the Connection Machine system CM-2 has 65,536 simple processors) or a relatively small number of powerful processors (for example, the ncube with 1,024 processors). These systems, however, cannot achieve 1 Teraflop performance; the development of technologies for the construction of massively parallel systems with thousands of processors that could achieve 1 Teraflop peak performance. This performance goal could be achieved in this decade only with systems containing thousands of powerful processors [15]. In those days several hypercube variations have been proposed in the literature. Several of them were developed to improve the topological properties of the hypercube. It is well-known that the hypercube does have very good topological properties, therefore, the viability of the latter variations is questionable; in fact, they often increase its already high VLSI complexity! Other variations employ hypercube building blocks that contain a special communication processor. Building blocks are connected together through their communication processors. Not only are such structures irregular but for large building blocks the employment of the communication processors results in substantial bottlenecks. In contrast, author has introduced the family of reduced hypercube (RH) interconnection networks that have smaller VLSI complexity than hypercubes with the same number of processors. RHs are produced from regular hypercubes by a uniform reduction in the number of channels for each node [15]. Extensive comparison of RH interconnection networks with conventional hypercubes of the same size has proven that they achieve comparable performance. It has been also shown that any RH can simulate simultaneously in an optimal manner several popular cube-connected cycles interconnection networks. Additionally, techniques have been developed for the efficient embedding of linear arrays, multidimensional meshes, and binary trees into RHs. Finally, generalized reduced hypercube interconnection networks have been introduced for even more versatility. I have described opinion about most popular interconnection network till 1994 year. I will present comparisons from 1994 till nowadays in the next subchapter In the scientific publications after 1994 till nowadays The article [16] compares and evaluates the multicast performance of two System-Area Networks (SANs), Dolphin s Scalable Coherent Interface (SCI) and Myricom s Myrinet. Both networks deliver 10

11 low latency and high bandwidth to applications, but do not support multicast in hardware. Authors compared SCI and Myrinet in terms of their user level performance using various software-based multicast algorithms under various networking and multicasting scenarios. The recent Dolphin SCI networks are capable of achieving low latencies (smaller than 2µs) and high throughputs (5.3 Gbps peak link throughput) over point-to-point links with cut-through switching. SCI architecture is presented on the Picture 2. Using the unidirectional ringlets as a basic block, it is possible to obtain a large variety of topologies, such as counter-rotating rings and unidirectional and bidirectional torus [16]. Picture 2. Architectural block diagram of SCI NIC (2D) [16]. Myrinet provides low latencies (as low as about 6µs) and high data rates (up to 2.0 Gbps). Picture 3 shows the architectural block diagram of a PCI-based Myrinet NIC. 11

12 Picture 3. Architectural block diagram of Myrinet NIC [16]. For the Dolphin SCI experiments, Dolphin PCI-64/66/D330 SCI NICs with 5.3 Gb/s link speed are used along with Scali s SSP (Scali Software Platform) communication software. The nodes are interconnected to form a 4x4 unidirectional 2D torus using 2m cables. The Myrinet experiments are performed using M2L-PCI64A-2 Myrinet NICs with a 1.28 Gb/s link speed along with Myricom s GM communication software. The nodes are interconnected to form a 16-node star network using 16 3m Myrinet LAN cables and a 16-port Myrinet switch for the Myrinet experiments [16]. The result is shown on the Picture 4. 12

13 Picture 4. Large-message multicast completion latency. The conclusion of the article is that the analysis of most widely deployed interconnects for cluster computing takes different results. For example the SCI separate addressing algorithm is found to be the best choice for small-message. For large messages, Dolphin s SCI has a clear advantage compared to Myricom s Myrinet (i.e. 5.3 Gb/s of SCI vs Gb/s of Myrinet used in this experiment). Although the newest Myrinet hardware features higher data rates (for example 2.0 Gb/s) than tested, these rates are still significantly lower than SCI. The article [17] also compared SCI and Myrinet. In their study, they concluded that, in terms of their performance and robustness analyses, Myrinet is a better choice compared to SCI. One of the articles was about bandwidth and latency analysis on a Cray XT4 system [18]. It is registered with communication bandwidth point-to-point 7.6 GB/s and bisectional/cabinet 667 GB/s into top500 list. The Cray XT4 interconnection network is arranged in a 3-D mesh topology (some systems employ wraparound links, producing a torus in some or all dimensions). Each node in the mesh is nominally connected to its six nearest neighbors by point-to-point communication links. Normally, these links are configured at boot time by the Cray route manager to operate at full speed. [18]. 13

14 Picture 5. MPI ping-pong bandwidth and latency for the three supported link bandwidth modes [18]. Three degraded modes are supported: 1/4 bandwidth, 1/2 bandwidth, and 3/4 bandwidth. Picture 5 shows the resulting MPI uni-directional bandwidth and latency for the degraded modes, as well as full bandwidth. The 3/4 and full link bandwidth results are limited by the compute node s injection bandwidth which is lower than the network link bandwidth. Based on the 1/4 bandwidth results, the effective link bandwidth is approximately 3092 MB/s in each direction. Small-message MPI latency is virtually unchanged for the degraded modes, indicating that link bandwidth can be controlled independently of latency [18]. Picture 6 shows MPI ping-pong bandwidth and latency for the four tested HyperTransport link configurations: 200-MHz 8-bit (400 MB/s per direction), 200-MHz 16-bit (800 MB/s), 400-MHz 16-bit (1600 MB/s), and 800-MHz 16-bit (3200 MB/s) Picture 6. MPI ping-pong bandwidth and latency for the four link configurations [18]. On the author s opinion there were there several network interconnects in 2003 year that provide low latency (less than 10µs) and high bandwidth (multiple Gbps). Two of the leading products are Myrinet 14

15 and Quadrics. In the high performance computing area, MPI has been the de facto standard for writing parallel applications. To achieve optimal performance in a cluster, it is very important to implement MPI efficiently on top of the cluster interconnect. Myrinet and Quadrics were designed for high performance computing environments. As a result, their hardware and software are specially optimized to achieve better MPI performance [19]. More recently, InfiniBand has entered the high performance computing market. Unlike Myrinet and Quadrics, InfiniBand was initially proposed as a generic interconnect for inter-process communication and I/O. However, its rich feature set, high performance and scalability make it also attractive as a communication layer for high performance computing [19]. Authors present a comprehensive performance comparison of MPI implementations over InfiniBand, Myrinet and Quadrics. InfiniBand platform consisted of InfiniHost HCAs and an InfiniScale switch from Mellanox. InfiniScale is a full wire-speed switch with eight 10 Gbps ports. The InfiniHost MT23108 HCA connects to the host through PCIX bus. It allows for a bandwidth of up to 10 Gbps over its ports. Myrinet network consists of Myrinet M3F-PCIXD-2 network interface cards connected by a Myrinet switch. The link bandwidth of the Myrinet network is 2Gbps in each direction. The Myrinet-2000 switch is an 8-port crossbar switch. The network interface card has a 133MHz/64bit PCI-X interface. It has a programmable LANai-XP processor running at 225 MHz with 2MB onboard SRAM. The LANai processor on the NIC can access host memory via the PCI-X bus through the DMA controller. Quadrics network consists of Elan3 QM-400 network interface cards and an Elite 16 switch. The Quadrics network has a transmission bandwidth of 400MB/s in each link direction. Picture 7 shows the MPI-level latency results. The test is conducted in a ping-pong fashion and the latency is derived from round-trip time. For small messages, Quadrics shows excellent latencies, which are around 4.6µs. The smallest latencies for InfiniBand and Myrinet are 6.8µs and 6.7µs, respectively. For large messages, InfiniBand has a clear advantage because of its higher bandwidth [19]. 15

16 Picture 7. MPI Latency across Three interconnects [19]. Picture 8 shows the bi-directional bandwidth results. We can see that InfiniBand bandwidth increases from 841MB/s uni-directional to 900MB/s. Then it is limited by the bandwidth of the PCI-X bus. Quadrics bandwidth improves from 308MB/s to 375MB/s. Myrinet shows even more improvement. Its peak bandwidth increases from 235MB/s to 473MB/s. However, Myrinet bandwidth drops to less than 340MB/s when the message size is larger than 256KB. Picture 8. MPI BI-Directional bandwidth [19]. Conclusion shows that although InfiniBand is able to deliver very good performance at the MPI level. Results on the 8 node OSU cluster and 16 node Topspin cluster also show that InfiniBand has very 16

17 good scalability. InfiniBand can still outperform other interconnects if the application is bandwidthbound. The article that was presented by DELL [20]. The lab set up a test environment that comprised a cluster of 32 identically configured Dell PowerEdge 1750 servers inter connected with Gigabit Ethernet and InfiniBand HCAs and switches. Each PowerEdge 1750 server had two Intel Xeon processors running at 3.06 GHz with 512 KB level 2 (L2) cache, 1 MB level 3 (L3) cache, and 4 GB of DDR RAM operating on a 533 MHz front-side bus (FSB). The chip set of the PowerEdge 1750 was the ServerWorks GC-LE, which accommodates up to four registered DDR PC2100 (266 MHz) DIMMs with two-way interleaved memory architecture. Each PowerEdge 1750 server was also equipped with two integrated Broadcom Gigabit2 Ethernet adapters, dual 64-bit 133 MHz PCI-X slots for the interconnects, and the appropriate NICs. The 32 PowerEdge 1750 servers functioned as the compute nodes of the cluster. An Extreme Networks Summit t switch with 48 non-blocking Gigabit Ethernet ports was used for Gigabit Ethernet tests. The InfiniBand components consisted of two Topspin 120 InfiniBand switches and Topspin InfiniBand 10 Gbps PCI-X HCAs. Each of the Topspin 120 switches was connected to 16 nodes and to each other with eight interswitch links between the two switches. Picture 9. Latency comparison between Gigabit Ethernet and InfiniBand [20]. 17

18 Picture 10. Bandwidth comparison between Gigabit Ethernet and InfiniBand [20]. Picture 9 shows that the latency for the InfiniBand interconnect is fairly low compared to that of the Gigabit Ethernet interconnect in this study. For a small message size, InfiniBand has a latency of approximately 7 microseconds (μs) as compared to Gigabit Ethernet, which shows a latency of approximately 65 μs for the same message size. Picture 10 shows the bandwidth obtained in this study. Gigabit Ethernet peaks out at roughly 85 MB/sec to 90 MB/sec as compared to InfiniBand, which shows a peak of approximately 700 MB/sec for large message sizes. The results in this study were obtained using PCI-X interfaces for both InfiniBand and Gigabit Ethernet on separate PCI-X buses. Another approach based on price of the supercomputer to achieve exact level of performance. The result after testing is shown on the Table 3. Table 3. Comparison between Gigabit Ethernet and InfiniBand Clusters from Nov2003 Top500 list of Computers [21]. In this case, the price/performance equation is simple since InfiniBand switch ports, at less than $300 per port, are far more cost effective than buying servers costing several thousand dollars or more. 18

19 Table 4. Comparison of InfiniBand and GE / 10GE Switches Table 4 shows that Gigabit Ethernet and 10 Gigabit Ethernet are more expensive interconnection networks than Infiniband. At the same time, Infiniband provides more performance. 4. TOP500.org statistics Top500.org has listed the most used interconnection families in 500 most powerful supercomputers in the world. Table 1 describes interconnect family, count, percentual share, sum of maximum ranges, sum of peak ranges and total amount processors which are connected to a interconnect network family. Picture 11 and 12 present graphically percentual share of different interconnection networks listed in top500.org. Picture 11. Interconnect share for 11/2008 part 1. 19

20 Picture 12. Interconnect share for 11/2008 part 2. The table reveals that Gigabit Ethernet and Infiniband are clearly the most popular interconnection networks in current supercomputers. Over 50% of them are using Gigabit Ethernet and Infiniband s share is almost 30%. Table 3 shows a total performance for supercomputers based on used interconnection family. [9] For an interconnection network comparison purposes, we will calculate interconnects average performance in a system and interconnect network s average performance per processor. Table 5. Average performance for supercomputers based on used interconnection family. Interconnect family Rmax Sum/Count (GF/system) Rpeak Sum/Count (GF/system) Rmax Sum/Processor Sum (GF/processor) Rpeak Sum/Processor Sum (GF/processor) Average amount of processors in a system Myrinet Quadrics Gigabit Ethernet Infiniband Crossbar Mixed NUMALink SP Switch

21 Proprietary Cray Interconnect According to Table 5, it seems when Infiniband interconnection network have been used, have the best average performance relative to number of processors been achieved. Average maximum performance per system states that the computers that use Infiniband, are also quite powerful. Infiniband is used in 142 supercomputers and there are also smaller clusters included. Supercomputers that use Gigabit Ethernet are less powerful at average but on the other hand, Gigabit Ethernet is currently the most used interconnection network and probably quite popular solution for smaller clusters as well. Proprietary interconnection networks are used in 42 supercomputers. They seem to be the most powerful at average, but it is easy to understand that when some type of special solution is applied, the computer is supposed to be expensive and powerful. Table 4. Approximate bandwidths and latencies of interconnection networks [10]. Top500.org has listed approximate bandwidths and latencies of interconnection networks to Table 4. Gigabit Ethernet is available with maximum theoretical bandwidth of 125 MB per second. Gigabit Ethernet in-switch latencies are approximately µs [10]. This could be suitable configuration for some applications that are not latency-bound at all [10]. Otherwise network choice should be somewhat different. When using high bandwidth interconnection networks, should a processing computer be able to deliver 21

22 data as fast as the network. Available bandwidths of PCI lane are MB/s. PCI-X supports bandwidth of 1 GB/s and PCI-X v2 supports also bandwidths of 2 and 4 GB/s. The recent PCI-Express has multiple data transfer lanes which bandwidths are 200MB/s. PCI-Express supports x2, x4, x8, x16 or x32 data lanes [10]. 5. Summary As a conclusion I can mark that there is two most used interconnection topologies: Infiniband and Gigabit Ethernet in the supercomputer implementations. We can make this conclusion also based on scientific articles. Gigabit Ethernet takes first place in utilization by virtue of the fact that it is cheaper and simply to implement. 22

23 References [1] Grama Ananth, Gupta Anshul, Karypis George, Kumar Vipin: Introduction to Parallel Computing, Second Edition, Addison-Wesley, 2003, ISBN [2] actually 20/11/2008 [3] Sotitios G. Ziavras, RH: A Versatile Family of Reduced Hypercube Interconnection Networks, [4] actually 20/11/2008 [5] actually 20/11/2008 [6] actually 20/11/2008 [7] actually 20/11/2008 [9] Interconnect Family share for 11/2008, actually 21/11/2008 [10] Overview of Recent Supercomputers, actually 18/11/2008 [11] V. Lahtinen, E. Salminen, K. Kuusilinna, T. Hamalainen, Comparison of synthesized bus and crossbar interconnection architectures., Tampere, Finland. [12] Howard Jay Siegel, Panel 2:Is It Possible to Fairly Compare Interconnection Networks?, USA. [13] Anant Agarwal, Limits on Interconnection Network Performance., USA. [14] Oliver A. McBryan. Perfomance of the SUPRENUM-1 supercomputer with the full bus interconnection network., USA. [15] Sotirios G. Ziavras, Reduces hypercube interconnection networks for massively parallel computers. [16] Sarp Oral and Alan D. George, A User-level Multicast Performance Comparison of Scalable Coherent Interface and Myrinet Interconnects. USA. [17] M. Fischer, U. Brüning, J. Kluge, L. Rzymianowicz, P. Schulz, and M. Waack, ATOLL a new 23

24 switched, highspeed Interconnect in Comparison to Myrinet and SCI,, Mexico, [18] Kevin T. Pedretti, Courtenay Vaughan, K. Scott Hemmert, and Brian Barrett, Application Sensitivity to Link and Injection Bandwidth on a Cray XT4 System, Finland, [19] Jiuxing Liu, Balasubramanian, Chandrasekaran Jiesheng, Wu Weihang Jiang, Performance Comparison of MPI Implementations over InfiniBand, Myrinet and Quadrics, [20] Dell Power Solutions, Exploring InfiniBand as an HPC Cluster Interconnect, [21] Mellanox Technologies Inc., InfiniBand Clustering Delivering Better Price/Performance than Ethernet, USA 24

Interconnection Networks Programmierung Paralleler und Verteilter Systeme (PPV)

Interconnection Networks Programmierung Paralleler und Verteilter Systeme (PPV) Interconnection Networks Programmierung Paralleler und Verteilter Systeme (PPV) Sommer 2015 Frank Feinbube, M.Sc., Felix Eberhardt, M.Sc., Prof. Dr. Andreas Polze Interconnection Networks 2 SIMD systems

More information

System Interconnect Architectures. Goals and Analysis. Network Properties and Routing. Terminology - 2. Terminology - 1

System Interconnect Architectures. Goals and Analysis. Network Properties and Routing. Terminology - 2. Terminology - 1 System Interconnect Architectures CSCI 8150 Advanced Computer Architecture Hwang, Chapter 2 Program and Network Properties 2.4 System Interconnect Architectures Direct networks for static connections Indirect

More information

Performance Evaluation of InfiniBand with PCI Express

Performance Evaluation of InfiniBand with PCI Express Performance Evaluation of InfiniBand with PCI Express Jiuxing Liu Server Technology Group IBM T. J. Watson Research Center Yorktown Heights, NY 1598 jl@us.ibm.com Amith Mamidala, Abhinav Vishnu, and Dhabaleswar

More information

Lecture 2 Parallel Programming Platforms

Lecture 2 Parallel Programming Platforms Lecture 2 Parallel Programming Platforms Flynn s Taxonomy In 1966, Michael Flynn classified systems according to numbers of instruction streams and the number of data stream. Data stream Single Multiple

More information

Interconnection Networks. Interconnection Networks. Interconnection networks are used everywhere!

Interconnection Networks. Interconnection Networks. Interconnection networks are used everywhere! Interconnection Networks Interconnection Networks Interconnection networks are used everywhere! Supercomputers connecting the processors Routers connecting the ports can consider a router as a parallel

More information

Sockets vs. RDMA Interface over 10-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck

Sockets vs. RDMA Interface over 10-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck Sockets vs. RDMA Interface over 1-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck Pavan Balaji Hemal V. Shah D. K. Panda Network Based Computing Lab Computer Science and Engineering

More information

Interconnection Network

Interconnection Network Interconnection Network Recap: Generic Parallel Architecture A generic modern multiprocessor Network Mem Communication assist (CA) $ P Node: processor(s), memory system, plus communication assist Network

More information

Scalability and Classifications

Scalability and Classifications Scalability and Classifications 1 Types of Parallel Computers MIMD and SIMD classifications shared and distributed memory multicomputers distributed shared memory computers 2 Network Topologies static

More information

Components: Interconnect Page 1 of 18

Components: Interconnect Page 1 of 18 Components: Interconnect Page 1 of 18 PE to PE interconnect: The most expensive supercomputer component Possible implementations: FULL INTERCONNECTION: The ideal Usually not attainable Each PE has a direct

More information

White Paper Solarflare High-Performance Computing (HPC) Applications

White Paper Solarflare High-Performance Computing (HPC) Applications Solarflare High-Performance Computing (HPC) Applications 10G Ethernet: Now Ready for Low-Latency HPC Applications Solarflare extends the benefits of its low-latency, high-bandwidth 10GbE server adapters

More information

Using PCI Express Technology in High-Performance Computing Clusters

Using PCI Express Technology in High-Performance Computing Clusters Using Technology in High-Performance Computing Clusters Peripheral Component Interconnect (PCI) Express is a scalable, standards-based, high-bandwidth I/O interconnect technology. Dell HPC clusters use

More information

Current Trend of Supercomputer Architecture

Current Trend of Supercomputer Architecture Current Trend of Supercomputer Architecture Haibei Zhang Department of Computer Science and Engineering haibei.zhang@huskymail.uconn.edu Abstract As computer technology evolves at an amazingly fast pace,

More information

PCI Technology Overview

PCI Technology Overview PCI Technology Overview February 2003 February 2003 Page 1 Agenda History and Industry Involvement Technology Information Conventional PCI PCI-X 1.0 2.0 PCI Express Other Digi Products in PCI/PCI-X environments

More information

COMP 422, Lecture 3: Physical Organization & Communication Costs in Parallel Machines (Sections 2.4 & 2.5 of textbook)

COMP 422, Lecture 3: Physical Organization & Communication Costs in Parallel Machines (Sections 2.4 & 2.5 of textbook) COMP 422, Lecture 3: Physical Organization & Communication Costs in Parallel Machines (Sections 2.4 & 2.5 of textbook) Vivek Sarkar Department of Computer Science Rice University vsarkar@rice.edu COMP

More information

Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand

Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand P. Balaji, K. Vaidyanathan, S. Narravula, K. Savitha, H. W. Jin D. K. Panda Network Based

More information

Cray Gemini Interconnect. Technical University of Munich Parallel Programming Class of SS14 Denys Sobchyshak

Cray Gemini Interconnect. Technical University of Munich Parallel Programming Class of SS14 Denys Sobchyshak Cray Gemini Interconnect Technical University of Munich Parallel Programming Class of SS14 Denys Sobchyshak Outline 1. Introduction 2. Overview 3. Architecture 4. Gemini Blocks 5. FMA & BTA 6. Fault tolerance

More information

Boosting Data Transfer with TCP Offload Engine Technology

Boosting Data Transfer with TCP Offload Engine Technology Boosting Data Transfer with TCP Offload Engine Technology on Ninth-Generation Dell PowerEdge Servers TCP/IP Offload Engine () technology makes its debut in the ninth generation of Dell PowerEdge servers,

More information

Micro-Benchmark Level Performance Comparison of High-Speed Cluster Interconnects

Micro-Benchmark Level Performance Comparison of High-Speed Cluster Interconnects Micro-Benchmark Level Performance Comparison of High-Speed Cluster Interconnects Jiuxing Liu Balasubramanian Chandrasekaran Weikuan Yu Jiesheng Wu Darius Buntinas Sushmitha Kini Peter Wyckoff Ý Dhabaleswar

More information

Interconnection Network Design

Interconnection Network Design Interconnection Network Design Vida Vukašinović 1 Introduction Parallel computer networks are interesting topic, but they are also difficult to understand in an overall sense. The topological structure

More information

Lecture 18: Interconnection Networks. CMU 15-418: Parallel Computer Architecture and Programming (Spring 2012)

Lecture 18: Interconnection Networks. CMU 15-418: Parallel Computer Architecture and Programming (Spring 2012) Lecture 18: Interconnection Networks CMU 15-418: Parallel Computer Architecture and Programming (Spring 2012) Announcements Project deadlines: - Mon, April 2: project proposal: 1-2 page writeup - Fri,

More information

Maximizing Server Storage Performance with PCI Express and Serial Attached SCSI. Article for InfoStor November 2003 Paul Griffith Adaptec, Inc.

Maximizing Server Storage Performance with PCI Express and Serial Attached SCSI. Article for InfoStor November 2003 Paul Griffith Adaptec, Inc. Filename: SAS - PCI Express Bandwidth - Infostor v5.doc Maximizing Server Storage Performance with PCI Express and Serial Attached SCSI Article for InfoStor November 2003 Paul Griffith Adaptec, Inc. Server

More information

Topological Properties

Topological Properties Advanced Computer Architecture Topological Properties Routing Distance: Number of links on route Node degree: Number of channels per node Network diameter: Longest minimum routing distance between any

More information

Why the Network Matters

Why the Network Matters Week 2, Lecture 2 Copyright 2009 by W. Feng. Based on material from Matthew Sottile. So Far Overview of Multicore Systems Why Memory Matters Memory Architectures Emerging Chip Multiprocessors (CMP) Increasing

More information

Parallel Programming

Parallel Programming Parallel Programming Parallel Architectures Diego Fabregat-Traver and Prof. Paolo Bientinesi HPAC, RWTH Aachen fabregat@aices.rwth-aachen.de WS15/16 Parallel Architectures Acknowledgements Prof. Felix

More information

Scaling 10Gb/s Clustering at Wire-Speed

Scaling 10Gb/s Clustering at Wire-Speed Scaling 10Gb/s Clustering at Wire-Speed InfiniBand offers cost-effective wire-speed scaling with deterministic performance Mellanox Technologies Inc. 2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400

More information

PCI Express* Ethernet Networking

PCI Express* Ethernet Networking White Paper Intel PRO Network Adapters Network Performance Network Connectivity Express* Ethernet Networking Express*, a new third-generation input/output (I/O) standard, allows enhanced Ethernet network

More information

Lecture 23: Interconnection Networks. Topics: communication latency, centralized and decentralized switches (Appendix E)

Lecture 23: Interconnection Networks. Topics: communication latency, centralized and decentralized switches (Appendix E) Lecture 23: Interconnection Networks Topics: communication latency, centralized and decentralized switches (Appendix E) 1 Topologies Internet topologies are not very regular they grew incrementally Supercomputers

More information

Introduction to Parallel Computing. George Karypis Parallel Programming Platforms

Introduction to Parallel Computing. George Karypis Parallel Programming Platforms Introduction to Parallel Computing George Karypis Parallel Programming Platforms Elements of a Parallel Computer Hardware Multiple Processors Multiple Memories Interconnection Network System Software Parallel

More information

Performance Characteristics of a Cost-Effective Medium-Sized Beowulf Cluster Supercomputer

Performance Characteristics of a Cost-Effective Medium-Sized Beowulf Cluster Supercomputer Res. Lett. Inf. Math. Sci., 2003, Vol.5, pp 1-10 Available online at http://iims.massey.ac.nz/research/letters/ 1 Performance Characteristics of a Cost-Effective Medium-Sized Beowulf Cluster Supercomputer

More information

I/O Virtualization Using Mellanox InfiniBand And Channel I/O Virtualization (CIOV) Technology

I/O Virtualization Using Mellanox InfiniBand And Channel I/O Virtualization (CIOV) Technology I/O Virtualization Using Mellanox InfiniBand And Channel I/O Virtualization (CIOV) Technology Reduce I/O cost and power by 40 50% Reduce I/O real estate needs in blade servers through consolidation Maintain

More information

Local-Area Network -LAN

Local-Area Network -LAN Computer Networks A group of two or more computer systems linked together. There are many [types] of computer networks: Peer To Peer (workgroups) The computers are connected by a network, however, there

More information

InfiniBand Clustering

InfiniBand Clustering White Paper InfiniBand Clustering Delivering Better Price/Performance than Ethernet 1.0 Introduction High performance computing clusters typically utilize Clos networks, more commonly known as Fat Tree

More information

Solving I/O Bottlenecks to Enable Superior Cloud Efficiency

Solving I/O Bottlenecks to Enable Superior Cloud Efficiency WHITE PAPER Solving I/O Bottlenecks to Enable Superior Cloud Efficiency Overview...1 Mellanox I/O Virtualization Features and Benefits...2 Summary...6 Overview We already have 8 or even 16 cores on one

More information

Communicating with devices

Communicating with devices Introduction to I/O Where does the data for our CPU and memory come from or go to? Computers communicate with the outside world via I/O devices. Input devices supply computers with data to operate on.

More information

Chapter 2. Multiprocessors Interconnection Networks

Chapter 2. Multiprocessors Interconnection Networks Chapter 2 Multiprocessors Interconnection Networks 2.1 Taxonomy Interconnection Network Static Dynamic 1-D 2-D HC Bus-based Switch-based Single Multiple SS MS Crossbar 2.2 Bus-Based Dynamic Single Bus

More information

Interconnection Networks

Interconnection Networks CMPT765/408 08-1 Interconnection Networks Qianping Gu 1 Interconnection Networks The note is mainly based on Chapters 1, 2, and 4 of Interconnection Networks, An Engineering Approach by J. Duato, S. Yalamanchili,

More information

HP reference configuration for entry-level SAS Grid Manager solutions

HP reference configuration for entry-level SAS Grid Manager solutions HP reference configuration for entry-level SAS Grid Manager solutions Up to 864 simultaneous SAS jobs and more than 3 GB/s I/O throughput Technical white paper Table of contents Executive summary... 2

More information

Cluster Grid Interconects. Tony Kay Chief Architect Enterprise Grid and Networking

Cluster Grid Interconects. Tony Kay Chief Architect Enterprise Grid and Networking Cluster Grid Interconects Tony Kay Chief Architect Enterprise Grid and Networking Agenda Cluster Grid Interconnects The Upstart - Infiniband The Empire Strikes Back - Myricom Return of the King 10G Gigabit

More information

Fibre Channel over Ethernet in the Data Center: An Introduction

Fibre Channel over Ethernet in the Data Center: An Introduction Fibre Channel over Ethernet in the Data Center: An Introduction Introduction Fibre Channel over Ethernet (FCoE) is a newly proposed standard that is being developed by INCITS T11. The FCoE protocol specification

More information

Lustre Networking BY PETER J. BRAAM

Lustre Networking BY PETER J. BRAAM Lustre Networking BY PETER J. BRAAM A WHITE PAPER FROM CLUSTER FILE SYSTEMS, INC. APRIL 2007 Audience Architects of HPC clusters Abstract This paper provides architects of HPC clusters with information

More information

How PCI Express Works (by Tracy V. Wilson)

How PCI Express Works (by Tracy V. Wilson) 1 How PCI Express Works (by Tracy V. Wilson) http://computer.howstuffworks.com/pci-express.htm Peripheral Component Interconnect (PCI) slots are such an integral part of a computer's architecture that

More information

ECLIPSE Performance Benchmarks and Profiling. January 2009

ECLIPSE Performance Benchmarks and Profiling. January 2009 ECLIPSE Performance Benchmarks and Profiling January 2009 Note The following research was performed under the HPC Advisory Council activities AMD, Dell, Mellanox, Schlumberger HPC Advisory Council Cluster

More information

LS DYNA Performance Benchmarks and Profiling. January 2009

LS DYNA Performance Benchmarks and Profiling. January 2009 LS DYNA Performance Benchmarks and Profiling January 2009 Note The following research was performed under the HPC Advisory Council activities AMD, Dell, Mellanox HPC Advisory Council Cluster Center The

More information

Introduction to PCI Express Positioning Information

Introduction to PCI Express Positioning Information Introduction to PCI Express Positioning Information Main PCI Express is the latest development in PCI to support adapters and devices. The technology is aimed at multiple market segments, meaning that

More information

Cluster Implementation and Management; Scheduling

Cluster Implementation and Management; Scheduling Cluster Implementation and Management; Scheduling CPS343 Parallel and High Performance Computing Spring 2013 CPS343 (Parallel and HPC) Cluster Implementation and Management; Scheduling Spring 2013 1 /

More information

VMWARE WHITE PAPER 1

VMWARE WHITE PAPER 1 1 VMWARE WHITE PAPER Introduction This paper outlines the considerations that affect network throughput. The paper examines the applications deployed on top of a virtual infrastructure and discusses the

More information

Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks

Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks WHITE PAPER July 2014 Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks Contents Executive Summary...2 Background...3 InfiniteGraph...3 High Performance

More information

Hyper Node Torus: A New Interconnection Network for High Speed Packet Processors

Hyper Node Torus: A New Interconnection Network for High Speed Packet Processors 2011 International Symposium on Computer Networks and Distributed Systems (CNDS), February 23-24, 2011 Hyper Node Torus: A New Interconnection Network for High Speed Packet Processors Atefeh Khosravi,

More information

Microsoft Exchange Server 2003 Deployment Considerations

Microsoft Exchange Server 2003 Deployment Considerations Microsoft Exchange Server 3 Deployment Considerations for Small and Medium Businesses A Dell PowerEdge server can provide an effective platform for Microsoft Exchange Server 3. A team of Dell engineers

More information

Optimizing Infrastructure Support For Storage Area Networks

Optimizing Infrastructure Support For Storage Area Networks Optimizing Infrastructure Support For Storage Area Networks December 2008 Optimizing infrastructure support for Storage Area Networks Mission critical IT systems increasingly rely on the ability to handle

More information

Using High Availability Technologies Lesson 12

Using High Availability Technologies Lesson 12 Using High Availability Technologies Lesson 12 Skills Matrix Technology Skill Objective Domain Objective # Using Virtualization Configure Windows Server Hyper-V and virtual machines 1.3 What Is High Availability?

More information

Building Clusters for Gromacs and other HPC applications

Building Clusters for Gromacs and other HPC applications Building Clusters for Gromacs and other HPC applications Erik Lindahl lindahl@cbr.su.se CBR Outline: Clusters Clusters vs. small networks of machines Why do YOU need a cluster? Computer hardware Network

More information

Introduction to Infiniband. Hussein N. Harake, Performance U! Winter School

Introduction to Infiniband. Hussein N. Harake, Performance U! Winter School Introduction to Infiniband Hussein N. Harake, Performance U! Winter School Agenda Definition of Infiniband Features Hardware Facts Layers OFED Stack OpenSM Tools and Utilities Topologies Infiniband Roadmap

More information

A Flexible Cluster Infrastructure for Systems Research and Software Development

A Flexible Cluster Infrastructure for Systems Research and Software Development Award Number: CNS-551555 Title: CRI: Acquisition of an InfiniBand Cluster with SMP Nodes Institution: Florida State University PIs: Xin Yuan, Robert van Engelen, Kartik Gopalan A Flexible Cluster Infrastructure

More information

Design and Implementation of an On-Chip timing based Permutation Network for Multiprocessor system on Chip

Design and Implementation of an On-Chip timing based Permutation Network for Multiprocessor system on Chip Design and Implementation of an On-Chip timing based Permutation Network for Multiprocessor system on Chip Ms Lavanya Thunuguntla 1, Saritha Sapa 2 1 Associate Professor, Department of ECE, HITAM, Telangana

More information

Low Latency 10 GbE Switching for Data Center, Cluster and Storage Interconnect

Low Latency 10 GbE Switching for Data Center, Cluster and Storage Interconnect White PAPER Low Latency 10 GbE Switching for Data Center, Cluster and Storage Interconnect Introduction: High Performance Data Centers As the data center continues to evolve to meet rapidly escalating

More information

SAN Conceptual and Design Basics

SAN Conceptual and Design Basics TECHNICAL NOTE VMware Infrastructure 3 SAN Conceptual and Design Basics VMware ESX Server can be used in conjunction with a SAN (storage area network), a specialized high speed network that connects computer

More information

Network Design. Yiannos Mylonas

Network Design. Yiannos Mylonas Network Design Yiannos Mylonas Physical Topologies There are two parts to the topology definition: the physical topology, which is the actual layout of the wire (media), and the logical topology, which

More information

Cluster Scalability of ANSYS FLUENT 12 for a Large Aerodynamics Case on the Darwin Supercomputer

Cluster Scalability of ANSYS FLUENT 12 for a Large Aerodynamics Case on the Darwin Supercomputer Cluster Scalability of ANSYS FLUENT 12 for a Large Aerodynamics Case on the Darwin Supercomputer Stan Posey, MSc and Bill Loewe, PhD Panasas Inc., Fremont, CA, USA Paul Calleja, PhD University of Cambridge,

More information

Achieving Nanosecond Latency Between Applications with IPC Shared Memory Messaging

Achieving Nanosecond Latency Between Applications with IPC Shared Memory Messaging Achieving Nanosecond Latency Between Applications with IPC Shared Memory Messaging In some markets and scenarios where competitive advantage is all about speed, speed is measured in micro- and even nano-seconds.

More information

Architectural Level Power Consumption of Network on Chip. Presenter: YUAN Zheng

Architectural Level Power Consumption of Network on Chip. Presenter: YUAN Zheng Architectural Level Power Consumption of Network Presenter: YUAN Zheng Why Architectural Low Power Design? High-speed and large volume communication among different parts on a chip Problem: Power consumption

More information

All-Flash Arrays Weren t Built for Dynamic Environments. Here s Why... This whitepaper is based on content originally posted at www.frankdenneman.

All-Flash Arrays Weren t Built for Dynamic Environments. Here s Why... This whitepaper is based on content originally posted at www.frankdenneman. WHITE PAPER All-Flash Arrays Weren t Built for Dynamic Environments. Here s Why... This whitepaper is based on content originally posted at www.frankdenneman.nl 1 Monolithic shared storage architectures

More information

QUADRICS IN LINUX CLUSTERS

QUADRICS IN LINUX CLUSTERS QUADRICS IN LINUX CLUSTERS John Taylor Motivation QLC 21/11/00 Quadrics Cluster Products Performance Case Studies Development Activities Super-Cluster Performance Landscape CPLANT ~600 GF? 128 64 32 16

More information

EDUCATION. PCI Express, InfiniBand and Storage Ron Emerick, Sun Microsystems Paul Millard, Xyratex Corporation

EDUCATION. PCI Express, InfiniBand and Storage Ron Emerick, Sun Microsystems Paul Millard, Xyratex Corporation PCI Express, InfiniBand and Storage Ron Emerick, Sun Microsystems Paul Millard, Xyratex Corporation SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies

More information

D1.2 Network Load Balancing

D1.2 Network Load Balancing D1. Network Load Balancing Ronald van der Pol, Freek Dijkstra, Igor Idziejczak, and Mark Meijerink SARA Computing and Networking Services, Science Park 11, 9 XG Amsterdam, The Netherlands June ronald.vanderpol@sara.nl,freek.dijkstra@sara.nl,

More information

3G Converged-NICs A Platform for Server I/O to Converged Networks

3G Converged-NICs A Platform for Server I/O to Converged Networks White Paper 3G Converged-NICs A Platform for Server I/O to Converged Networks This document helps those responsible for connecting servers to networks achieve network convergence by providing an overview

More information

Understanding PCI Bus, PCI-Express and In finiband Architecture

Understanding PCI Bus, PCI-Express and In finiband Architecture White Paper Understanding PCI Bus, PCI-Express and In finiband Architecture 1.0 Overview There is some confusion in the market place concerning the replacement of the PCI Bus (Peripheral Components Interface)

More information

PCI Express and Storage. Ron Emerick, Sun Microsystems

PCI Express and Storage. Ron Emerick, Sun Microsystems Ron Emerick, Sun Microsystems SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature

More information

Can High-Performance Interconnects Benefit Memcached and Hadoop?

Can High-Performance Interconnects Benefit Memcached and Hadoop? Can High-Performance Interconnects Benefit Memcached and Hadoop? D. K. Panda and Sayantan Sur Network-Based Computing Laboratory Department of Computer Science and Engineering The Ohio State University,

More information

Principles and characteristics of distributed systems and environments

Principles and characteristics of distributed systems and environments Principles and characteristics of distributed systems and environments Definition of a distributed system Distributed system is a collection of independent computers that appears to its users as a single

More information

Private cloud computing advances

Private cloud computing advances Building robust private cloud services infrastructures By Brian Gautreau and Gong Wang Private clouds optimize utilization and management of IT resources to heighten availability. Microsoft Private Cloud

More information

PCI Express Overview. And, by the way, they need to do it in less time.

PCI Express Overview. And, by the way, they need to do it in less time. PCI Express Overview Introduction This paper is intended to introduce design engineers, system architects and business managers to the PCI Express protocol and how this interconnect technology fits into

More information

Computer Systems Structure Input/Output

Computer Systems Structure Input/Output Computer Systems Structure Input/Output Peripherals Computer Central Processing Unit Main Memory Computer Systems Interconnection Communication lines Input Output Ward 1 Ward 2 Examples of I/O Devices

More information

enabling Ultra-High Bandwidth Scalable SSDs with HLnand

enabling Ultra-High Bandwidth Scalable SSDs with HLnand www.hlnand.com enabling Ultra-High Bandwidth Scalable SSDs with HLnand May 2013 2 Enabling Ultra-High Bandwidth Scalable SSDs with HLNAND INTRODUCTION Solid State Drives (SSDs) are available in a wide

More information

Building High-Performance iscsi SAN Configurations. An Alacritech and McDATA Technical Note

Building High-Performance iscsi SAN Configurations. An Alacritech and McDATA Technical Note Building High-Performance iscsi SAN Configurations An Alacritech and McDATA Technical Note Building High-Performance iscsi SAN Configurations An Alacritech and McDATA Technical Note Internet SCSI (iscsi)

More information

Storage at a Distance; Using RoCE as a WAN Transport

Storage at a Distance; Using RoCE as a WAN Transport Storage at a Distance; Using RoCE as a WAN Transport Paul Grun Chief Scientist, System Fabric Works, Inc. (503) 620-8757 pgrun@systemfabricworks.com Why Storage at a Distance the Storage Cloud Following

More information

Achieving a High-Performance Virtual Network Infrastructure with PLUMgrid IO Visor & Mellanox ConnectX -3 Pro

Achieving a High-Performance Virtual Network Infrastructure with PLUMgrid IO Visor & Mellanox ConnectX -3 Pro Achieving a High-Performance Virtual Network Infrastructure with PLUMgrid IO Visor & Mellanox ConnectX -3 Pro Whitepaper What s wrong with today s clouds? Compute and storage virtualization has enabled

More information

InfiniBand -- Industry Standard Data Center Fabric is Ready for Prime Time

InfiniBand -- Industry Standard Data Center Fabric is Ready for Prime Time White Paper InfiniBand -- Industry Standard Data Center Fabric is Ready for Prime Time December 2005 Server and storage clusters benefit today from industry-standard InfiniBand s price, performance, stability,

More information

The Advantages of Multi-Port Network Adapters in an SWsoft Virtual Environment

The Advantages of Multi-Port Network Adapters in an SWsoft Virtual Environment The Advantages of Multi-Port Network Adapters in an SWsoft Virtual Environment Introduction... 2 Virtualization addresses key challenges facing IT today... 2 Introducing Virtuozzo... 2 A virtualized environment

More information

PCI Express Impact on Storage Architectures and Future Data Centers. Ron Emerick, Oracle Corporation

PCI Express Impact on Storage Architectures and Future Data Centers. Ron Emerick, Oracle Corporation PCI Express Impact on Storage Architectures and Future Data Centers Ron Emerick, Oracle Corporation SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies

More information

A SIMULATOR FOR LOAD BALANCING ANALYSIS IN DISTRIBUTED SYSTEMS

A SIMULATOR FOR LOAD BALANCING ANALYSIS IN DISTRIBUTED SYSTEMS Mihai Horia Zaharia, Florin Leon, Dan Galea (3) A Simulator for Load Balancing Analysis in Distributed Systems in A. Valachi, D. Galea, A. M. Florea, M. Craus (eds.) - Tehnologii informationale, Editura

More information

Unified Fabric: Cisco's Innovation for Data Center Networks

Unified Fabric: Cisco's Innovation for Data Center Networks . White Paper Unified Fabric: Cisco's Innovation for Data Center Networks What You Will Learn Unified Fabric supports new concepts such as IEEE Data Center Bridging enhancements that improve the robustness

More information

Oracle Database Scalability in VMware ESX VMware ESX 3.5

Oracle Database Scalability in VMware ESX VMware ESX 3.5 Performance Study Oracle Database Scalability in VMware ESX VMware ESX 3.5 Database applications running on individual physical servers represent a large consolidation opportunity. However enterprises

More information

Chapter 4 Multi-Stage Interconnection Networks The general concept of the multi-stage interconnection network, together with its routing properties, have been used in the preceding chapter to describe

More information

Interconnect Efficiency of Tyan PSC T-630 with Microsoft Compute Cluster Server 2003

Interconnect Efficiency of Tyan PSC T-630 with Microsoft Compute Cluster Server 2003 Interconnect Efficiency of Tyan PSC T-630 with Microsoft Compute Cluster Server 2003 Josef Pelikán Charles University in Prague, KSVI Department, Josef.Pelikan@mff.cuni.cz Abstract 1 Interconnect quality

More information

Intelligent Content Delivery Network (CDN) The New Generation of High-Quality Network

Intelligent Content Delivery Network (CDN) The New Generation of High-Quality Network White paper Intelligent Content Delivery Network (CDN) The New Generation of High-Quality Network July 2001 Executive Summary Rich media content like audio and video streaming over the Internet is becoming

More information

Cloud-Based Apps Drive the Need for Frequency-Flexible Clock Generators in Converged Data Center Networks

Cloud-Based Apps Drive the Need for Frequency-Flexible Clock Generators in Converged Data Center Networks Cloud-Based Apps Drive the Need for Frequency-Flexible Generators in Converged Data Center Networks Introduction By Phil Callahan, Senior Marketing Manager, Timing Products, Silicon Labs Skyrocketing network

More information

Binary search tree with SIMD bandwidth optimization using SSE

Binary search tree with SIMD bandwidth optimization using SSE Binary search tree with SIMD bandwidth optimization using SSE Bowen Zhang, Xinwei Li 1.ABSTRACT In-memory tree structured index search is a fundamental database operation. Modern processors provide tremendous

More information

Windows 8 SMB 2.2 File Sharing Performance

Windows 8 SMB 2.2 File Sharing Performance Windows 8 SMB 2.2 File Sharing Performance Abstract This paper provides a preliminary analysis of the performance capabilities of the Server Message Block (SMB) 2.2 file sharing protocol with 10 gigabit

More information

Where IT perceptions are reality. Test Report. OCe14000 Performance. Featuring Emulex OCe14102 Network Adapters Emulex XE100 Offload Engine

Where IT perceptions are reality. Test Report. OCe14000 Performance. Featuring Emulex OCe14102 Network Adapters Emulex XE100 Offload Engine Where IT perceptions are reality Test Report OCe14000 Performance Featuring Emulex OCe14102 Network Adapters Emulex XE100 Offload Engine Document # TEST2014001 v9, October 2014 Copyright 2014 IT Brand

More information

Cluster Computing at HRI

Cluster Computing at HRI Cluster Computing at HRI J.S.Bagla Harish-Chandra Research Institute, Chhatnag Road, Jhunsi, Allahabad 211019. E-mail: jasjeet@mri.ernet.in 1 Introduction and some local history High performance computing

More information

Computer Organization & Architecture Lecture #19

Computer Organization & Architecture Lecture #19 Computer Organization & Architecture Lecture #19 Input/Output The computer system s I/O architecture is its interface to the outside world. This architecture is designed to provide a systematic means of

More information

Dell High-Performance Computing Clusters and Reservoir Simulation Research at UT Austin. http://www.dell.com/clustering

Dell High-Performance Computing Clusters and Reservoir Simulation Research at UT Austin. http://www.dell.com/clustering Dell High-Performance Computing Clusters and Reservoir Simulation Research at UT Austin Reza Rooholamini, Ph.D. Director Enterprise Solutions Dell Computer Corp. Reza_Rooholamini@dell.com http://www.dell.com/clustering

More information

Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building Blocks. An Oracle White Paper April 2003

Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building Blocks. An Oracle White Paper April 2003 Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building Blocks An Oracle White Paper April 2003 Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building

More information

Large Scale Clustering with Voltaire InfiniBand HyperScale Technology

Large Scale Clustering with Voltaire InfiniBand HyperScale Technology Large Scale Clustering with Voltaire InfiniBand HyperScale Technology Scalable Interconnect Topology Tradeoffs Since its inception, InfiniBand has been optimized for constructing clusters with very large

More information

The Bus (PCI and PCI-Express)

The Bus (PCI and PCI-Express) 4 Jan, 2008 The Bus (PCI and PCI-Express) The CPU, memory, disks, and all the other devices in a computer have to be able to communicate and exchange data. The technology that connects them is called the

More information

PCIe Over Cable Provides Greater Performance for Less Cost for High Performance Computing (HPC) Clusters. from One Stop Systems (OSS)

PCIe Over Cable Provides Greater Performance for Less Cost for High Performance Computing (HPC) Clusters. from One Stop Systems (OSS) PCIe Over Cable Provides Greater Performance for Less Cost for High Performance Computing (HPC) Clusters from One Stop Systems (OSS) PCIe Over Cable PCIe provides greater performance 8 7 6 5 GBytes/s 4

More information

- An Essential Building Block for Stable and Reliable Compute Clusters

- An Essential Building Block for Stable and Reliable Compute Clusters Ferdinand Geier ParTec Cluster Competence Center GmbH, V. 1.4, March 2005 Cluster Middleware - An Essential Building Block for Stable and Reliable Compute Clusters Contents: Compute Clusters a Real Alternative

More information

Parallel Programming Survey

Parallel Programming Survey Christian Terboven 02.09.2014 / Aachen, Germany Stand: 26.08.2014 Version 2.3 IT Center der RWTH Aachen University Agenda Overview: Processor Microarchitecture Shared-Memory

More information

The proliferation of the raw processing

The proliferation of the raw processing TECHNOLOGY CONNECTED Advances with System Area Network Speeds Data Transfer between Servers with A new network switch technology is targeted to answer the phenomenal demands on intercommunication transfer

More information