SGI 10-Gigabit Ethernet Network Adapter Manuel d'utilisateur Page 142
- Page / 150
- Table des matières
- MARQUE LIVRES
Noté. / 5. Basé sur avis des utilisateurs
Hadoop: Java Communication Benchmark
• Sockets level ping-pong bandwidth test
• Java performance depends on usage of NIO (allocateDirect)
• C and Java versions of the benchmark have similar performance
• HDFS does not use direct allocated blocks or NIO on DataNode
S. Sur, H. Wang, J. Huang, X. Ouyang and D. K. Panda
“Can High-Performance
Interconnects Benefit Hadoop Distributed File System?”
, MASVDC „10 in conjunction with
MICRO 2010, Atlanta, GA.
0
200
400
600
800
1000
1200
1400
Bandwidth (MB/s)
Message Size (Bytes)
Bandwidth with C version
0
200
400
600
800
1000
1200
1400
1
4
16
64
256
1K
4K
16K
64K
256K
1M
4M
Bandwidth (MB/s)
Message Size (Bytes)
Bandwidth Java Version
1GE
IPoIB
SDP
10GE-TOE
SDP-NIO
CCGrid '11
142
- A Tutorial at CCGrid ’11 1
- Computing Systems 3
- Cluster Computing Environment 4
- (http://www.top500.org) 5
- Grid Computing Environment 6
- Compute cluster 8
- Cloud Computing Environments 9
- Hadoop Architecture 10
- Memcached Architecture 11
- • Hardware components 13
- • Software components 13
- • Ex: TCP/IP, UDP/IP 14
- – E.g., PCI, PCI-X 15
- – One bit per wire 15
- – Not scalable: 15
- Myrinet (1993 -) 1 Gbit/sec 16
- IB Trade Association 19
- • Network speed bottlenecks 21
- • I/O interface bottlenecks 21
- (not shown) 24
- • Fast Messages (FM) 27
- • Myricom GM 27
- IB Hardware Acceleration 28
- • Interrupt Coalescing 29
- • Jumbo Frames 29
- • Hardware Checksum Engines 29
- TOE and iWARP Accelerators 30
- 2006 (v3.0), 2008 (v3.1) 34
- PCI-Express (PCIe) 34
- 2003 (Gen1), 2007 (Gen2) 34
- IB, HSE and their Convergence 36
- Traditional Ethernet 37
- IB Overview 38
- Components: Channel Adapters 39
- • Switches: intra-subnet 40
- – Not directly addressable 41
- IB Communication Model 43
- Queue Pair Model 44
- Memory Registration 45
- Memory Protection 46
- (Send/Receive Model) 47
- InfiniBand Device 48
- Hardware ACK 48
- Hardware Protocol Offload 51
- • Buffering and Flow Control 52
- • Switching and Multicast 52
- Buffering and Flow Control 53
- Virtual Lanes 54
- Service Levels and QoS 55
- Traffic Segregation Benefits 56
- Identifiers) 57
- Switch Complex 58
- – 3D Torus (Sandia Red Sky) 59
- More on Multipathing 60
- IB Multicast Example 61
- IB Transport Services 63
- Reliability 64
- Transport Layer Capabilities 65
- Data Segmentation 66
- Transaction Ordering 67
- Message-level Flow-Control 68
- Concepts in IB Management 71
- Subnet Manager 72
- HSE Overview 74
- Differences 75
- – Multi Stream Semantics 76
- Offloaded TCP 83
- Myrinet Express (MX) 85
- Datagram Bypass Layer (DBL) 86
- • Solarflare approach: 87
- Ethernet 90
- IB Network 90
- IB Verbs 90
- Application 90
- Hardware 90
- IB Transport 91
- IB iWARP/HSE RoE RoCE 92
- IB Hardware Products 94
- Tyan Thunder S2935 Board 95
- IB Hardware Products (contd.) 96
- – Nortel Networks 97
- • Support for VPI and RoCE 98
- – www.openfabrics.org 99
- – OFED 1.6 is underway 99
- Verbs Interface 100
- (libibverbs) 100
- • Within the hardware: 101
- OpenFabrics Software Stack 102
- InfiniBand in the Top500 103
- Cray Interconnect 104
- NUMAlink 104
- SP Switch 104
- CCGrid '11 106
- • Other small clusters 107
- • Integrated Systems 107
- Other HSE Installations 108
- Presentation Overview 109
- 1/10 GigE 110
- InfiniBand 110
- Case Studies 111
- Latency (us) 112
- Message Size (bytes) 112
- Bandwidth (MBps) 113
- MVAPICH/MVAPICH2 Software 115
- One-way Latency: MPI over IB 116
- Bandwidth: MPI over IB 117
- Bandwidth: MPI over iWARP 119
- Convergent Technologies: 121
- Traditional Model 123
- Possible SDP Model 123
- InfiniBand CA 123
- SDP vs. IPoIB (IB QDR) 124
- IB on the WAN 126
- Features 127
- • Supports SONET OC-192 or 128
- “appear” local 128
- Sunnyvale 129
- 700 miles 130
- 3300 miles 4300 miles 130
- Cluster B 131
- Communication Options in Grid 132
- Interface 133
- Modern WAN 133
- Performance of Memory Based 134
- Data Transfer 134
- ADTS-64MB 135
- IPoIB-64MB 135
- Application Level Performance 136
- Memcached Design Using Verbs 139
- Memcached Get Latency 140
- Memcached Get TPS 141
- Bandwidth with C version 142
- Hadoop Sort Benchmark 145
- Concluding Remarks 147
- Funding Acknowledgments 148
- Personnel Acknowledgments 149
- Web Pointers 150
Produits connexes et manuels pour Mise en réseau SGI 10-Gigabit Ethernet Network Adapter
(34 pages)© 2020, manymanuals.fr. Tous droits réservés | 0.152 s |
Manymanuals.com
Manymanuals.de
Manymanuals.fr
Manymanuals.it
Manymanuals.pl
Manymanuals.cz
Manymanuals.es
Manymanuals-pt.com
Commentaires sur ces manuels