How to build a Beowulf Cluster Applied to Computational Electromagnetic

Transcription

1 How to build a Beowulf Cluster Applied to Computational Electromagnetic Carlos Henrique da Silva Santos Leonardo André Ambrosio Hugo Enrique Hernández Figueroa Department of Microwaves and Optics (DMO) School of Electrical and Computer Engineering (FEEC) State University of Campinas (UNICAMP)

2 Summary Motivation; Objectives; 1. Preview: Operation Systems and Network concepts; 2. Installing Linux Fedora distribution; 3. Configuring a computer cluster; 4. HPC cluster applications in electromagnetism; Conclusions; Main references; How to Build a Cluster Carlos Henrique 2

3 Motivations Different concepts over High Performance Computing (HPC) applied to Computational Electromagnetism; Low computational costs solutions for HPC; New computational technologies and resources to build a HPC cluster; Some large computational electromagnetic application. How to Build a Cluster Carlos Henrique 3

4 Objectives Presenting some computational concepts: Operating system layers Network structures; Linux distributions and services (NFS and SSH); Sophisticated and easy ways to build a computer cluster, focusing on data security; How to configure services and connections. How to Build a Cluster Carlos Henrique 4

5 1 st Part - Concepts Operating System Linux Network How to Build a Cluster Carlos Henrique 5

6 Operating Systems Kernel Hardware Kernel System Calls Program How to Build a Cluster Carlos Henrique 6

7 Operating System: Structure Memory management; Disk and file systems; Networking; Security: Internal: Programs running locally; External: Network connections; Graphical user interface; Device Driver; Process management. Firewall How to Build a Cluster Carlos Henrique 7

8 Operating System: Process Process: A program in execution; An instance of a program running in a computer; The entity that can be assigned and executed on a processor (program code and a dataset); Identifier State Priority Program counter Memory pointers Context data I/O Status information Accounting information... How to Build a Cluster Carlos Henrique 8

9 Modern Unix System a.out coff elf File mappings Exec switch NFS FFS Device mappings Disk driver Anonymous mappings virtual memory framework block device switch Common facilities vnode/vfs interface Scheduler framework s5fs RFS Time sharing process Tape driver Streams System processes Network driver tty driver How to Build a Cluster Carlos Henrique 9

10 Why Linux is necessary? Linux is a free operating system; It was concepted by Linus Torvalds to attempt his necessities, without intentions to get money; The development style adopted is the collective help, where someone manages the colective efforts of a group to improve the system; Millions of people are gratuitously contributing to develop Linux, for pleasure or to have a better operating system. How to Build a Cluster Carlos Henrique 10

11 Operating System: Linux It is a multiprocessor and multiuser system, which means that many users have the possibility to execute many processes into the same computer using Linux in the same time; Free and open source software, basing on the GPL. Furthermore, most of the computational tools for this operational system are free too; It has some different Kernel to provide better solutions depending on the application; It is one of the best operational system for network and HPC applications; Free tutorial, manual and help on the Internet. How to Build a Cluster Carlos Henrique 11

12 Networking Application: Computer Cluster 0 1 How to Build a Cluster Carlos Henrique 12

13 Network: Services, Protocols and Layers Protocols: are related to the packets sent between peer entities on different machines; Services: is a set of primitives (operations) that a layer provides to the layer above it. In other words, is related to the interface between layers. Machine 1 Layer k+1 Service provided by layer k Protocol Layer k Machine 2 Layer k+1 Layer k Layer k-1 Layer k-1 How to Build a Cluster Carlos Henrique 13

14 Network: OSI Reference Model layers 7 Application Application protocol Application APDU Interface 6 Presentation Presentation protocol Presentation PPDU 5 Session Session protocol Session SPDU 4 Transport Transport protocol Communication subnet boundary Transport TPDU Internet subnet protocol 3 Network Network Network Network Packet 2 Data link Data link Data link Data link Data link 1 Physical Physical Physical Physical Bit How to Build a Cluster Carlos Henrique 14

15 Network: Open Systems Interconnection (ISO-OSI) Reference Model Deals with connecting open system, which are open for communication with other systems; OSI model has seven layers, based on: 1. A layer should be created where a different abstraction is needed; 2. Each layer should perform a well-defined function; 3. Each layer function should be defined in order to provide an international standardization protocol; 4. The layer boundaries should be chosen to minimize the information flow across the interfaces; 5. The number of layer should be large enough that distinct functions and small enough that the architecture does not become widely. How to Build a Cluster Carlos Henrique 15

16 2 nd Part Installing Fedora RedHat How to Build a Cluster Carlos Henrique 16

17 Choosing installation version i386 ppc Intel x86-compatible processors, including Intel Pentium and Pentium- MMX, Pentium Pro, Pentium-II, Pentium-III, Celeron, Pentium 4, Xeon, Core Duo, and Centrino/Centrino Duo; VIA C3/C3-m and Eden/Eden-N; and AMD Athlon, AthlonXP, Duron, AthlonMP, Sempron, and Turion PowerPC processors, such as those found in Apple Power Macintosh, G3, G4, and G5, and IBM pseries systems x86_64 64-bit AMD processors such as Athlon64/FX/X2, Turion64, Opteron; and Intel 64-bit processors such as EM64T How to Build a Cluster Carlos Henrique 17

18 Pre-installation How to Build a Cluster Carlos Henrique 18

19 Installation: System properties How to Build a Cluster Carlos Henrique 19

20 Installation: Operating System Resource How to Build a Cluster Carlos Henrique 20

21 3 rd Part Cluster Configuring Linux Network SSH NFS Installing MPI Libraries How to Build a Cluster Carlos Henrique 21

22 TCP: Transport Control Protocol Transport Layer at OSI Model It is connection oriented, which means that all problems no solved in IP level will be solved here; Some problems that must be solved by TCP: Lost of packages or destructions occurs during the transmission; Package expedition out of order or duplicated. How to Build a Cluster Carlos Henrique 22

23 TCP: Transport Control Protocol The TCP specify the data package format and patterns recognized in reliable changes between two computers, like the procedures that were applied in computers to ensure the data communication; Procedures to be considered: Distinguishing between multiples destinies in a determined machine; Error recovering, like some lost packages or duplicated. How to Build a Cluster Carlos Henrique 23

24 TCP: Transport Control Protocol Available multiple programs runs in one machines communicate; Multiplex the TCP input traffic between the applications programs; Uses numerical port to identify the last destination in a machine. Each port is identified by a low integer number; Example: specifies the TCP port number 25, in the machine with IP address How to Build a Cluster Carlos Henrique 24

25 IP: Internet Protocol Network Configuration - Structure How to Build a Cluster Carlos Henrique 25

26 Linux Network Configuration: shell Mode Identify network device in use; Open the file: /etc/sysconfig/network-scripts/ifcfg-ethx DEVICE=ethx BOOTPROTO=none ONBOOT=yes TYPE=Ethernet IPADDR= NETWORK= BROADCAST= NETMASK= GATEWAY= DNS1= # Identify the device # Type of boot (none, static ou dhcp) # Starting on machine boot # Type of network # IP address of the machine at intranet # Network Masquerade # Range of IP address in the network # Defines the subnet # Package destination to network exit # Import the DNS specified on /etc/resolv.conf How to Build a Cluster Carlos Henrique 26

27 Shell Script Network Address Translation modprobe ip_conntrack modprobe ip_conntrack_ftp modprobe ip_tables modprobe ipt_state modprobe iptable_nat modprobe ip_nat_ftp modprobe ipt_limit echo 1 > /proc/sys/net/ipv4/ip_forward echo 1 > /proc/sys/net/ipv4/tcp_syncookies echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts echo 1 > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 1 >$f done for f in /proc/sys/net/ipv4/conf/*/accept_redirects; do echo 0 >$f done for f in /proc/sys/net/ipv4/conf/*/send_redirects; do echo 0 >$f done for f in /proc/sys/net/ipv4/conf/*/accept_source_route; do echo 0 >$f done iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE How to Build a Cluster Carlos Henrique 27

28 Main Network Services Requested FTP/SFTP (File Transfer Protocol/SSH File Transfer Protocol): to share and access files in different machines or networks; SSH: Secure Shell: remote access control; NFS (Network File System): sharing files in a network How to Build a Cluster Carlos Henrique 28

29 SSH: Secure Shell O SSH has been defined on IANA beyond the RFC 4250; A free and open-source available is OpenSSH, which supports the versions 1.3, 1.5 e 2.0 of the SSH protocol; Shell command: ssh cepofslave The first time that a connection for a different is required, the following message is exhibit: The authenticity of host cepofslave can t be established. DSA key fingerprint is 95:90:3a:3a:bc:f3:9a:9b:01:5d:b3:07:38:e2:11:0c. Are you sure you want to continue connecting (yes/no)? How to Build a Cluster Carlos Henrique 29

30 NFS: Network File System It is a model to structure data file distribution in some servers, mounting a virtual directory.; Making possible to users to centralize their folders in a server; These tools is important in High Performance Computing structures that access disc. Computer cluster is an evident example. How to Build a Cluster Carlos Henrique 30

31 NFS: Network File System To allow client access in NFS servers the following Daemons are necessary: nfsd: to receive NFS clients requests; mountd: daemon to mount NFS partitions and to execute nfsd requests; portmap: allows to NFS clients dicover which NFS server port is using. How to Build a Cluster Carlos Henrique 31

32 NFS: Network File System A client can connect a shared data file, he needs of permission to do it. For this reason, is important to certify if he is allowed in the file /etc/exports; In the /etc/exports each line represents one file system to be exported. A remote system can be specify just only one time by the file system, and only one standard input can be defined. IP range identification /home/cluster / (rw,no_root_squash) Identifying IPs /home/cluster (rw,no_root_squash) /home/cluster (rw,no_root_squash) How to Build a Cluster Carlos Henrique 32

33 Configure Cluster: Server NFS Step 0 Step 1 Step 3 Step 4 Creating one user called cepof adduser cepof passwd cepof /etc/exports /home/cepof (rw, no_root_squash) /home/cepof (rw, no_root_squash) /home/cepof (rw, no_root_squash) /etc/hosts cepofslave cepofslave cepofslave3 /etc/hosts.equiv cepofslave1 cepofslave2 cepofslave3 How to Build a Cluster Carlos Henrique 33

34 Configure Cluster: Server Step 5 Step 6 Step 7 Step 8 Step 9 Step 10 Reboot NFS service service nfs restart Change to cepof user su cepof Creating the cryptographic key to access ssh-keygen -t dsa Creating authorization file cat.ssh/id_dsa.pub > authorized_keys Permission of the directory.ssh chmod 700.ssh File access permission chmod 644.ssh/authorized_keys How to Build a Cluster Carlos Henrique 34

35 Configure Cluster: Slave nodes Step 0 Creating cepof user adduser cepof Passwd cepof Step 1 /etc/hosts cepofslave cepofslave cepofslave3 Step 2 /etc/fstab :/home/cluster /home/cluster nfs exec,dev,suid,rw 1 1 How to Build a Cluster Carlos Henrique 35

36 Configure Cluster: Slave nodes Step 3 Step 4 Step 5 Step 6 Step 7 Step 8 FTP connection on the Server sftp cepof@ Getting the public key get.ssh/id_dsa.pub Ending connection exit Creating the authorization key file cat.ssh/id_dsa.pub > authorized_keys Permissions of the.ssh directory chmod 700.ssh Permissions of the authorization file chmod 644.ssh/authorized_keys How to Build a Cluster Carlos Henrique 36

37 Example: Hello World #include <unistd.h> #include <stdio.h> #include "mpi.h" int main(int argc, char ** argv) { int my_rank; int p; int source; int dest; int silen = 128; int gherr; char hname[128]; char message[800]; MPI_Status status; int tag = 50; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &my_rank); MPI_Comm_size(MPI_COMM_WORLD, &p); gherr = gethostname( hname, silen); if (my_rank!= 0) { sprintf(message, "Greetings from process %d on %s!", my_rank, hname); dest = 0; MPI_Send(message, strlen (message)+1, MPI_CHAR, dest, tag, MPI_COMM_WORLD); } else { printf ("Messages received by process %d on %s.\n\n", my_rank, hname); for (source = 1; source < p; source++) { MPI_Recv(message, 800, MPI_CHAR, source, tag, MPI_COMM_WORLD, &status); printf("%s\n", message); } } MPI_Finalize(); return 0; } How to Build a Cluster Carlos Henrique 37

38 Parallel Process Schematic Process Process 0 0 (Mater (Mater Node) Node) Process Process 1 1 Process Process 2 2 Process Process 3 3 How to Build a Cluster Carlos Henrique 38

39 Creating boot configuration file vi lamhosts Add every IP or hosts of the cluster machines lamboot v lamhosts mpicc hello.c o hello mpirun C hello LAM 7.1.1/MPI 2 C++/ROMIO - Indiana University n-1<13229> ssi:boot:base:linear: booting n0 ( ) n-1<13229> ssi:boot:base:linear: booting n1 ( ) n-1<13229> ssi:boot:base:linear: booting n2 ( ) n-1<13229> ssi:boot:base:linear: booting n3 ( ) n-1<13229> ssi:boot:base:linear: booting n4 ( ) n-1<13229> ssi:boot:base:linear: booting n5 ( ) n-1<13229> ssi:boot:base:linear: booting n6 ( ) n-1<13229> ssi:boot:base:linear: booting n7 ( ) n-1<13229> ssi:boot:base:linear: booting n8 ( ) n-1<13229> ssi:boot:base:linear: booting n9 ( ) n-1<13229> ssi:boot:base:linear: booting n10 ( ) n-1<13229> ssi:boot:base:linear: booting n11 ( ) n-1<13229> ssi:boot:base:linear: booting n12 ( ) n-1<13229> ssi:boot:base:linear: finished How to Build a Cluster Carlos Henrique 39

40 4 th Part Applications of HPC Electromagnetic Applications How to Build a Cluster Carlos Henrique 40

41 Cluster Application: Human Head SAR Simulation Slice 1 Slice 2 Slice 3 Air Skin Bone Blood Brain Muscle Temporal Muscle y x Slice 4 Slice 5 Ω Ω1 Ω2 Ω 3 How to Build a Cluster Carlos Henrique 41

42 Cluster Application: Human Head SAR Simulation SAR calculus considering energy and mass; SAR = σ E 2ρ i 2 = 2 J i [ mw 2ρσ / g ] Applying the Poynting vector SAR = t W m = t W V [ mw / g] Local average of the SAR in terms of the simulation time (N max ) SAR = σ 1 2ρ N max [ ] 2 2 E x + E y [ mw / g] How to Build a Cluster Carlos Henrique 42

43 Cluster Application: Human Head SAR Simulation Performance Time (seconds) Tempo Despendido (segundos) Tempo Despendido pela Simulação SAR na Cabeça Humana Time of the Human Head SAR Simulation (dimensions variations) Quantidade de Processos number of process 200x x x x x100 Efficiency of the Human Head SAR Parallel Simulation Eficiência da Simulação SAR na Cabeça Humana Percentual de Eficiência Efficiency 1,2 1 0,8 0,6 0,4 0, x x x x x1000 Dimensions Dimensões dos of the Domínios computational Computacionais domain 1 Processo 2 Processos 3 Processos 4 Processos 5 Processos How to Build a Cluster Carlos Henrique 43

44 Cluster Application: SAR Simulation varying the time steps SAR Simulation Eficiência Efficiency da in Simulação the Human Head SAR na Cabeça (400x400) Humana (400x400) Eficiência efficiency 1,2 1 0,8 0,6 0,4 0, Processo 1 process 2 Processos 2 processes 3 Processos 3 processes 4 Processos 4 processes 5 Processos 5 processes Quantidade Number de Passos of time de steps Tempo How to Build a Cluster Carlos Henrique 44

45 Cluster Application: SAR Simulation Slice 1 Slice 2 Slice 3 Slice 4 Slice 5 How to Build a Cluster Carlos Henrique 45

46 Cluster Application: Metamaterial simulations Metamaterial Tempo das Simulações Simulation Metamaterial Time 2D Time (seconds) Time(seconds) Tempo Despendido (segundos) number of process Quantidade de Processos 3000x323 AEL 1 AEL2 AEL3 AEL Eficiência do Processamento Paralelo Metamaterial Simulation Time AEL ,2 1 efficiency Eficiência 0,8 0,6 0,4 0, x323 Efficiency = T T sequential parallel number of process Quantidade de Processos How to Build a Cluster Carlos Henrique 46

47 CePOF Cluster 0 1 Public Key - DSA Internet Router Gateway Firewall Intranet How to Build a Cluster Carlos Henrique 47

48 CePOF Cluster: New Cluster Paradigm 15 nodes: 2 AMD Opteron 246 processors 4 GB of memory RAM 73 GB of HD SCSI Switch 3COM 2824 (Gigabit Network) Patch Cord Kat. 6 No break APC SURT 10000, GREE Air-Conditioner with Btus S.O. Virtual S.O. Virtual Hw. Virtual Hw. Virtual Virtual Machine Windows 2003 Server A2210 (AMD Opteron) How to Build a Cluster Carlos Henrique 48

49 Conclusions Computer Cluster allows high performance computing (HPC) with low costs; Free software and microcomputer reused (Beowulf cluster); Complex structure to configure, to use and to keep; Choose better Operational System distribution in order to improve the computational performance; Making possible to simulate and analyze new electromagnetic structures and many other applications. How to Build a Cluster Carlos Henrique 49

50 References A. S. Tanenbaum, Redes de Computadores 3ª. Edição, Editora Campus, C. A. F. Oliveira, Cluster Beowulf- uma solução de baixo custo, Revista CienteFico, Ano IV, V. I, Salvador, C. H. S. Santos, H. E. Hernández-Figueroa, Método FDTD 2D Paralelo para Simulações de Estruturas Metamateriais, 12º Simpósio Brasileiro de Microondas e Optoeletrônica (SBMO) e 7º Congresso Brasileiro de Eletromagnetismo (CBMAG), Belo Horizonte, C. H. S. Santos, L. A. Ambrosio, H. E. Hernández Figueroa, The Benefits of Heterogeneous Beowulf Cluster on the Human Head SAR Simulation, International Microwave & Optoelectronics Conference (IMOC), IEEE-SBMO, Brasília (2005) C. H. S. Santos, Computação Paralela Aplicada a Problemas Eletromagnéticos Utilizando o Método FDTD, dissertação de mestrado, Faculdade de Engenharia Elétrica e de Computação (FEEC), Universidade Estadual de Campinas (Unicamp), Campinas, J. J. Dongarra, et al, Numerical Linear Algebra for High-Performance Computers, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, M. A. F. Batista, L. N. F. Guimarães, Algoritmos Genéticos em Ambientes Paralelos, V Worcap INPE, São José dos Campos, M. Pitanga. Construindo Supercomputadores em Linux Segunda Edição, Editora Brasport, São Paulo, N. S. Simões, S. F. Souza, L. Muniz, F. Fardim Junior, A. F. Souza, Reis N. C. Junior, A. M. P. Valli, L. Catabriga, Instalação e Configuração de Clusters de Estações de Trabalho: Experiência do Laboratório de Computação de Alto Desempenho do Departamento de Informática da UFES, IV Workshop em Sistemas Computacionais de Alto Desempenho - WSCAD'2003, São Paulo, P. S. Pacheco, Parallel Programming with MPI, Morgan Kaufmann Publishers, San Francisco, P. Veríssimo, L. Rodrigues, Distributed Systems for System Architects, Kluwer Academic Publishers, Massachusetts, Introdução ao MPI, Centro Nacional de Processamento de Alto Desempenho-SP, (acessado em: 18/04/2007). S. Humphries, Field Solutions on Computer, CRC Press LLC, Florida, Curso de MPI, Centro Nacional de Processamento de Alto Desempenho-NE. (acessado em: 18/04/2007). (acessado em: 18/04/2007). H. Zima, B. Chapman, Supercompilers for Parallel and Vector Computers, ACM Press, New York, How to Build a Cluster Carlos Henrique 50

51 Thank you! Carlos Henrique da Silva Santos Leonardo André Ambrosio Hugo Enrique Hernández Figueroa How to Build a Cluster Carlos Henrique 51

52 Network: Data Communication Client process Client machine (1) Connect request (2) ACK Server machine (3) Request data System calls (4) Reply Server process Operating system Kernel Protocol stack Drivers (5) Disconnect (6) Disconnect Kernel Protocol stack Drivers How to Build a Cluster Carlos Henrique 52