Virtual Networking with z/vm Guest LANs and the z/vm Virtual Switch

Virtual Networking with z/vm Guest LANs and the z/vm Virtual Switch Alan Altmark, IBM z/vm Development, Endicott, NY

Note References to IBM products, programs, or services do not imply that IBM intends to make these available in all countries in which IBM operates. Any reference to an IBM product, program, or service is not intended to state or imply that only IBM's product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe on any of the intellectual property rights of IBM may be used instead. The evaluation and verification of operation in conjunction with other products, except those expressly designed by IBM, are the responsibility of the user. The following terms are trademarks of the International Business Machines Corporation in the United States or other countries or both: IBM IBM logo eserver zseries z/vm z/os DB2 Other company, product, and service names may be trademarks or service marks of others. Copyright 2003, 2004 by International Business Machines Corporation 2

Topics Guest LANs Virtual Network Interface Card Virtual Switch What features are supported in what releases What s new in z/vm 5.1 3

4

Multi-DMZ Network app app app db internet A DMZ (demilitarized zone) is a subnet that insulates critical network components (servers) from the rest of the network 5

Multi-DMZ Network on zseries app app app db zseries internet 6

Multi-DMZ Network with Guest LANs LPAR 1 LPAR 2 Guest LAN z/vm z/os DB2 app app app Guest LAN PR/SM HiperSockets = Firewall Router Ethernet LAN 7

Guest LAN vs. Virtual Switch Guest LAN Virtual Switch Guest Guest Guest Guest Guest Ethernet LAN Virtual router is required Different subnets External router awareness Guest-managed failover No virtual router Same subnets Transparent bridge CP-managed failover 8

z/vm Guest LAN A simulated LAN Ethernet: IPv4 and IPv6 HiperSockets: IPv4 LAN #1 Unicast, Multicast, and Broadcast No built-in connection to outside network Guest Guest Guest Guest Guest As many as you want LAN #2 Owned by system or individual user Is not a device - it is a system object Guest Guest Created in SYSTEM CONFIG, directory, or by CP DEFINE LAN command 9

Why Guest LAN instead of Dedicated Hardware? Dedicated network connections may be best for some environments: When intense network activity is expected When external connectivity is required z/vm Guest LAN may be better for other environments: When network hardware is limited When multiple nodes are guests in the same z/vm host image When network activity must be isolated from primary network (e.g. test environments, student labs, application server access to database servers) 10

Guest LAN Attributes Name Owner Type Permission Maximum number of connections Maximum frame size Accounting Some attributes can be changed after the LAN is defined 11

LAN Name and Owner The LAN name is a simple 1-8 character token The LAN owner is a VM user ID or SYSTEM (name, owner) is unique within the system Needed to connect (COUPLE) a NIC to the LAN Notes about LAN ownership: The LAN belongs to the system, not to the owner A Class G LAN owner can modify the LAN access list A Class G LAN owner can delete the LAN A Class B user can create, modify, or detach any LAN 12

System vs. User Guest LANs System owned by SYSTEM User owned by a specific z/vm user Created by SYSTEM CONFIG Class B DEFINE LAN Created by SYSTEM CONFIG Class B DEFINE LAN with OWNERID option Class G DEFINE LAN Modify or delete by Class B SET LAN or DETACH LAN Modify or delete by Class B or Class G owner SET LAN or DETACH LAN 13

HiperSockets vs. QDIO LAN TYPE HIPERsockets QDIO HiperSockets Synchronous Low latency Slightly smaller path length in CP (less CP time) OSA-Express in QDIO mode Asynchronous Higher latency than HiperSockets Higher CPU cost 14

Unrestricted vs. Restricted LANs Unrestricted Any user can connect (couple) to this LAN Hint: CP QUERY LAN can show you who is connected Restricted Only users in the access list can connect (couple) to this LAN LAN owner uses CP SET LAN to GRANT or REVOKE access CP QUERY LAN can show you the current access list CP QUERY LAN can show you who is connected 15

LAN MAXCONN MAXCONN INFinite nnnn Represents the maximum number of simultaneous connections permitted for this LAN Decimal number 1-1024 sets a specific limit INFINITE means no limit is defined for this LAN When the MAXCONN limit is reached, subsequent COUPLE commands issued by adapter (NIC) owners will fail 16

LAN MFS MFS 16K 24K 40K 64K Simulates CHPID OS=value specification in IOCDS for HiperSockets (TYPE=IQD) chpids Does not apply to QDIO Largest MTU specification = (MFS - 8K) Hint: If LAN is isolated, use large MFS and large MTU If LAN has external gateway, use MFS 16K and match external MTU (e.g. 1492) Jumbo frame (MTU 8992) gateway needs 24K MFS 17

LAN ACCOUNTING ACCOUNTING ON OFF Accounting ON Accounting is enabled for adapters connected to this LAN Directory options determine which guests: NetAccounting enables general network accounting NetRouter enables network accounting as a router Hint: Both LAN and USER must be set to create accounting records Accounting OFF Accounting is disabled for adapters connected to this LAN 18

Persistent vs. Transient LAN Persistent / Transient is inferred from other attributes Any LAN owned by user SYSTEM is persistent Any LAN created by SYSTEM CONFIG is persistent All other LANs are transient A persistent LAN must be explicitly deleted by CP DETACH LAN A transient LAN is automatically deleted when the last user uncouples from the LAN 19

Setting Guest LAN defaults and limits Set global VM LAN attributes in the SYSTEM CONFIG file: VMLAN LIMit PERSistent INFinite maxcount VMLAN LIMit TRANSient INFinite maxcount VMLAN ACNT ACCOUNTing SYSTEM ON OFF VMLAN ACNT ACCOUNTing USER ON OFF VMLAN MACPREFIX 020000-02FFFF VMLAN MACIDRANGE SYSTEM x-y [USER a-b] New for z/vm 5.1 Limit of 0 prevents dynamic definition unless limit is raised 20

Setting defaults and limits Modify global guest LAN attributes with SET VMLAN: CP SET VMLAN LIMit PERSistent INFinite maxcount CP SET VMLAN LIMit TRANSient INFinite maxcount CP SET VMLAN ACNT ACCOUNTing SYSTEM ON OFF CP SET VMLAN ACNT ACCOUNTing USER ON OFF Use CP QUERY VMLAN to see current values No LAN is deleted by this command if you set limit < current (it just prevents creation of new LAN until current < limit) Use SET VMLAN TRANSIENT LIMIT 0 to eliminate ability for Class G user to create his or her own guest LAN 21

Create a Guest LAN at system IPL Automated with SYSTEM CONFIG file statements: DEFINE LAN name [OWNERid ownerid] [TYPE HIPERsockets QDIO] [MAXCONN INFinite nnnn] [MFS 16K 24K 40K 64K] [ACCOUNTing ON OFF] [UNRESTricted RESTricted] [GRANT userlist] Examples: DEFINE LAN HIPER1 DEFINE LAN DELTA TYPE QDIO DEFINE LAN QDIO5 OWNER TCPMAINT TYPE QDIO 22

Grant Guest LAN permission at IPL Specify after DEFINE LAN statement in SYSTEM CONFIG to add users to access list MODIFY LAN name [OWNERid ownerid] [GRANT userid] Example: DEFINE LAN HIPER1 OWNER SYSTEM RESTRICTED MODIFY LAN HIPER1 OWNER SYSTEM GRANT LNX01 MODIFY LAN HIPER1 OWNER SYSTEM GRANT LNX02 23

Create a Guest LAN dynamically Interactive with CP DEFINE LAN commands: CP DEFINE LAN name [OWNERid ownerid] [TYPE HIPERsockets QDIO] [MAXCONN INFinite nnnn] [MFS 16K 24K 40K 64K] [UNRESTricted RESTricted] [ACCOUNTing ON OFF] Examples: CP DEFINE LAN HIPER1 OWNER SYSTEM RESTRICTED CP DEFINE LAN DELTA TYPE QDIO CP DEFINE LAN QDIO5 OWNER TCPMAINT TYPE QDIO 24

Modify a Guest LAN dynamically Modified by CP SET LAN commands: CP SET LAN name [OWNERid ownerid] [ACCOUNTing ON OFF] [GRANT userid] [REVOKE userid] Examples: CP SET LAN HIPER1 OWNER SYSTEM ACCOUNTING OFF CP SET LAN CSC201 OWNER TCPMAINT GRANT LNX01 25

26

Virtual Network Interface Card (NIC) A simulated network adapter OSA-Express QDIO HiperSockets Virtual Machine 3 or more devices per NIC More than 3 to simulate port sharing on 2ndlevel system Provides access to Guest LAN or Virtual Switch Guest LANs Created by directory or CP DEFINE NIC command 27

Virtual NIC - User Directory May be automated with USER DIRECT file: SPECIAL vdev {HIPERs QDIO} [devs [owner name]] NICDEF vdev [TYPE HIPERS QDIO] [LAN owner name] [CHPID xx] [MAC xxyyzz] Example: SPECIAL 1100 QDIO 3 SYSTEM SWITCH1 or NICDEF 1100 LAN SYSTEM SWITCH1 28

Virtual NIC - CP Command May be interactive with CP DEFINE NIC and COUPLE commands: CP DEFINE NIC vdev [[TYPE] HIPERsockets QDIO] [DEVices devs] [CHPID xx] CP COUPLE vdev [TO] owner name Example: CP DEFINE NIC 1200 TYPE QDIO CP COUPLE 1200 TO SYSTEM CSC201 29

NIC Virtual Device Address base virtual device address where this NIC is installed in your virtual I/O configuration A block of contiguous devices addresses is allocated to this NIC, beginning with vdev (see DEVICES parameter) One I/O subchannel ID is allocated to EACH I/O device beginning with the first available subchannel One virtual CHPID (Channel Path ID) is allocated for this virtual NIC 30

NIC DEVICES parameter DEVICES devs Specifies the (decimal) number of I/O devices created as part of this NIC, starting with the specified vdev The default (and minimum) is 3 devices: Read-Control Write-Control Data 31

NIC TYPE parameter TYPE HIPERsockets or QDIO Must connect to LAN of the same type 32

NIC CHPID parameter CHPID xx Specifies the Channel Path ID number (in hex) to use for this NIC Default is any available unused real CHPID number Needed when guest manages CHPID numbers (e.g. z/os) Notes for z/os configuration: This is a virtual CHPID number (not a real hardware CHPID) It must be a CHPID number that is NOT in use by hardware Hint: An easy way to find out what CHPID is available is: LOGON the guest and DEFINE NIC xxxx HIPER CP QUERY VIRTUAL xxxx to get SCHIB (ssss) CP DISPLAY SCHIB ssss to see the CHPID Code CHPID in MVS IOCDS 33

34

What s a switch anyway? T 1 1 2 1 1 2 2 2 3 1 3 1 It s a box that you plug cables in to create a LAN Cable/DSL multi-port and wireless switches for home are simplified versions of commercial switches (less configurable) It has its own IP address for management purposes 35

z/vm Virtual Switch A special-purpose Guest LAN guest guest guest Ethernet IPv4 Built-in 802.1q bridge to outside network Same subnet as OSA connection CP Each Virtual Switch has up to 3 separate OSA-Express connections associated with it Created in SYSTEM CONFIG or by CP DEFINE VSWITCH command Router AIX 36

Virtual Switch Attributes Name Associated OSAs (maximum 3) A controlling virtual machine (VM TCP/IP stack server) Controller not involved in data transfer Do not ATTACH or DEDICATE User needs IUCV *VSWITCH authorization User needs VSWITCH CONTROLLER statement in PROFILE TCPIP Similar to Guest LAN Owner SYSTEM Type QDIO Persistent Restricted 37

Create a Virtual Switch at system IPL Automated with SYSTEM CONFIG file statements: DEFINE VSWITCH name [RDEV NONE cuu [cuu [cuu]] ] [CONNECT DISCONNECT] [CONTROLLER * userid] [NONROUTER PRIROUTER] Example: DEFINE VSWITCH SWITCH12 RDEV 1E00 1F04 CONNECT 38

Modify a Virtual Switch at IPL Specify after DEFINE LAN statement in SYSTEM CONFIG to add users to access list MODIFY VSWITCH name GRANT userid [VLAN vid1 vid2 ] Example: MODIFY VSWITCH SWITCH12 GRANT LNX01 VLAN 1 3 8 z/vm 4.4 supports VLAN ANY, but don t use it 39

IEEE VLAN support guest guest guest guest vswitch CP VLAN creates multiple logical LAN segments on a single physical LAN segment Multiple VLANs are assigned to OSA port on switch CP ensures correct traffic flow Switch 40

Multi-DMZ Network on zseries - Reloaded app app app db zseries internet 41

Multi-DMZ Network on zseries with outboard firewall app app app db internet 42

Multi-DMZ Network with VSWITCH (A) LPAR 1 LPAR 2 app z/vm app app z/os DB2 VSWITCH 1 VSWITCH 2 43

Multi-DMZ Network with VSWITCH (B) LPAR 1 LPAR 2 app app z/vm app z/os DB2 VSWITCH To internet With 1 VSWITCH, 3 VLANs, and a multi-domain firewall 44

45

Network Configuration In general, configure a Guest LAN network like any other network Subnet routing Use the VSWITCH whenever possible Minimize number of VSWITCHes; exploit IEEE VLAN if you can By having virtual and real configurations be the same, you can easily test network configuration before deployment with real hardware 46

Built-in Diagnostics CP QUERY VMLAN to get global VM LAN information (e.g. limits) to find out if service has been applied CP QUERY LAN ACTIVE to find out which users are coupled to find out which IP addresses are active CP QUERY NIC DETAILS to find out if your adapter is coupled to find out if your adapter is initialized to find out if your IP addresses have been registered to find out (z/vm 4.3.0) how many bytes/packets sent/received 47

Support Summary for z/vm 4.2.0 HiperSockets NIC / LAN HiperSockets IPv4 unicast data transfer Required Service: Apply CP PTF UM30225 (APAR VM62938) Apply TCPIP PTF UQ61461 (APAR PQ51738) Apply TCPIP PTF UQ65226 (APAR PQ60093) Use current qeth drivers for your Linux 2.4.x distribution 48

Support Summary for z/vm 4.3.0 HiperSockets IPv4 multicast accounting data OSA-Express QDIO IPv4 unicast, multicast, broadcast accounting data Required Service: Apply CP PTF UM30359 (APAR VM63085) Apply CP PTF UM30743 (APAR VM63261) (Prereqs: VM63091, VM63132, and VM63172) 49

Support Summary for z/vm 4.4.0 Virtual Switch IPv4 Ethernet Requires OSA-Express QDIO DEFINE VSWITCH and SET VSWITCH used to establish and modify virtual switch settings Virtual OSA-Express QDIO IPv6 Virtual HiperSockets Broadcast 50

Coming in z/vm 5.1 ESM control for all guest LANs and VSWITCHes, including VLAN ID control RACF: Class VMLAN, Profile owner.lanname or owner.lanname.vid All LANs and VSwitches are restricted Layer 2 (MAC) communications Fulfillment of Statement of Direction All types of traffic, not just IP Virtual NIC MAC appears on network VMLAN updates to allow specification of ranges used for automatic and static MAC address assignments Better VSWITCH stall detection, error reporting, and error recovery. 51

Coming in z./vm 5.1 IEEE 802.1q compliance changes VLAN ANY is gone VSWITCH can be defined as VLAN-aware (or not). Default is not. When a NIC couples to a VLAN-aware VSWITCH, it will be assigned a PORTTYPE attribute ACCESS: VLAN tags not given to or accepted from guest TRUNK: VLAN tags are given to and expected from guest Default PORTTYPE comes from DEFINE VSWITCH Can be overridden by MODIFY VSWITCH GRANT Some configurations require migration effort 52

References Publications: Links: z/vm CP Planning and Administration SC24-6043 z/vm CP Command and Utility Reference SC24-6008 z/vm TCP/IP Level 430 Planning and Customization SC24-6019 VSE/ESA Version 2 Release 7.0 Release Guide SC33-6713 Linux for S/390 SG24-4987 Linux on zseries and S/390: ISP/ASP Solutions SG24-6299 zseries HiperSockets SG24-6816 Linux on IBM eserver zseries and S/390: TCP/IP Broadcast on z/vm Guest LAN (Form REDP3596) http://www.ibm.com/s390/linux/ http://www.linuxvm.org/ 53

Contact Information By e-mail: Alan_Altmark@us.ibm.com In person: USA 607.429.3323 On the Web: Mailing lists: http://ibm.com/vm/devpages/altmarka IBMTCP-L@vm.marist.edu VMESA-L@listserv.uark.edu LINUX-390@vm.marist.edu http://ibm.com/vm/techinfo/listserv.html 60

Thanks for Listening! 61