Top 10 Things You Always Wanted to Know About Automatic Storage Management But Were Afraid to Ask

Transcription

1

2 <Insert Picture Here> Top 10 Things You Always Wanted to Know About Automatic Storage Management But Were Afraid to Ask Nitin Vengurlekar RAC/ASM Engineering Grid Product Strategy

3 Agenda ASM Overview 2 minute tour Top Ten Questions ASM Architecture ASM Configuration & Performance Backups ACFS Migration 3 rd Party Software Summary <Insert Picture Here>

4 ASM Architecture - Review Automatic Storage Management (ASM) instance Instance that manages the diskgroup metadata Disk Groups Logical grouping of disks Determines file mirroring options ASM Disks LUNs presented to ASM ASM Files Files that are stored in ASM disk groups are called ASM files, this includes database files

5 ASM Architecture A Brief Overview Oracle DB Instance Non RAC Database ASM Instance Server Pool of Storage Disk Group Disk Group

6 ASM Process Architecture A Brief Overview RAC or Non RAC Databases Oracle DB Instance Oracle DB Instance Oracle DB Instance Oracle DB Instance Oracle DB Instance ASM Instance ASM Instance ASM Instance ASM Instance ASM Instance Clustered Servers Database Database Database Clustered Pool of Storage Disk Group Disk Group

7 <Insert Picture Here> Top 10 ASM Questions ASM Architecture

8 Top 10 ASM Questions Q. What init.ora parameters does a user need to configure for ASM instances? A. The default parameter settings work perfectly for ASM. The only parameters needed for 11g ASM: PROCESSES ASM_DISKSTRING* ASM_DISKGROUPS* INSTANCE_TYPE*

9 Top 10 ASM Questions Q. How does the database interact with the ASM instance and how do I make ASM go faster? A. ASM is not in the I/O path so ASM does not impede the database file access. Since the RDBMS instance is performing raw I/O, the I/O is as fast as possible.

10 RDBMS and ASM Instance Interaction DATABASE Server (1A) OPEN ASM (1) Database opens file (1B) Extent Map (2) Database Reads file (3) Database Creates file (2A) READ (2B) I/O Completes (3D) Initializes file (3A) CREATE (3C) Extent Map (3D) Commit (3B) Allocates file Operating System

11 Top 10 ASM Questions Q. Do I need to define the RDBMS FILESYSTEMIO_OPTIONS parameter when I use ASM? A. No. The RDBMS does I/O directly to the raw disk devices, the FILESYSTEMIO_OPTIONS parameter is only for filesystems.

12 Top 10 ASM Questions Q. Don t I lose all the advanced filesystem features when I move to ASM; e.g., directio, write coalescing, and prefetch? A. Yes, but that s okay. Most of the filesystem features mentioned, though good for general file data performance, interfere and fractionalize the benefits inherently provided by the database; e.g., DBWR & LGWR does write coalescing, and user processes do sequential pre-fetches All IO capable processes do un-buffered IO (raw IO) because of ASM

13 Top 10 ASM Questions Q. This is cool that ASM can now store Vote and OCR files. But how does CSS and CRS startup in this configuration? A. It just does There are two keys processes and a lot of crafty coding to get this to work in the correct startup sequence.

14 Top 10 ASM Questions ASMCMD> pwd +DATA/rst-cluster/OCRFILE ASMCMD> ls -l Type Redund Striped Time Sys Name OCRFILE UNPROT COARSE JUN 25 11:00:00 Y REGISTRY ASMCMD> lsdg State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED EXTERN N Y DATA/

15 CSS-CRS-ASM Instance Architecture CSS 1. Get ASM disk string 2. Do ASM disk discovery to locate vote disk 3. Once Vote disk located, startup CSS ASM DATA DG SPFILE Vote OCR ASM Disk ASM Disk GPnP 4. ASM mounts DATA dg 5. CRS open OCR in well known ASM dg location OHASD CRS Agents starts GPnP, CSS and ASM

16 <Insert Picture Here> Top 10 ASM Questions ASM Configuration

17 Top 10 ASM Questions Q. Do I need 11gR2 Grid Infrastructure to use ASM A. Yes. ASM is now part of Grid Infrastructure, which includes, Clusterware, ASM and ACFS. So you ll to Install GI to use ASM In 11gR2 there are two options for install GI for Standalone Server (aka Oracle Restart) and GI for Clusterware

18 Top 10 ASM Questions Q. We have a 16 TB database. I m curious about the number of disk groups we should use; e.g. 1 large disk group, a couple of disk groups, or otherwise? What about a database consolidation scenario A. For VLDBs you will probably end up with different storage tiers; e.g with some of our large customers they have Tier1 (RAID10 FC), Tier2 (RAID5 FC), Tier3 (SATA), etc. Each one of these is mapped to a diskgroup.

19 Top 10 ASM Questions Q. What is the best LUN size for ASM A. There is no best size! In most cases the storage team will dictate based on their standardized LUN size. The ASM admin merely has to communicate the ASM Best Practices and application characteristics to storage folks : Need equally sized / performance LUNs Minimum of 4 LUNs The capacity requirement The workload characteristic (random r/w, sequential r/w) & any response time SLA Using this info, and their standards, the storage folks should build a nice LUN group set for you.

20 Top 10 ASM Questions Q. In 11gR2 can my RDBMS and ASM instances run different versions? A. Yes. But since ASM is now part of GI stack, it must be at the highest version. Keep in mind, there s two components of compatibility: Software compatibility Diskgroup compatibility attributes: compatible.asm compatible.rdbms

21 Disk Group Compatibility Example Need to have compatible.asm set to for OCR/Vote files and SPFILE in ASM ACFS also needs compatible.asm Advance compatible.asm ALTER DISKGROUP data SET ATTRIBUTE compatible.asm =

22 Top 10 ASM Questions Q. Where do I run my database listener from; i.e., ASM HOME or DB HOME? A. For 11gR2, the SCAN listener is run from GI Home, and database listener from DB HOME. For pre-11gr2, it is recommended to run the listener from the ASM HOME. This is particularly important for RAC env, since the listener is a node-level resource. In this config, you can create additional [user] listeners from the database homes as needed.

23 <Insert Picture Here> Top 10 ASM Questions Backups

24 Top 10 ASM Questions Q. How do I backup my ASM instance? A. Not applicable! ASM has no files to backup

25 Top 10 ASM Questions Q. When should I use RMAN and when should I use ASMCMD copy? A. RMAN is the recommended and most complete and flexible method to backup and transport database files in ASM. ASMCMD copy is good for copying single files Supports all Oracle file types In some cases. can be used to instantiate a Data Guard environment Does not update the controlfile Does not create OMF files

26 ASMCMD Copy ASMCMD> ls +fra/dumpsets/expdp_5_5.dat ASMCMD> cp expdp_5_5.dat pdp_5_5.dat source +fra/dumpsets/expdp_5_5.dat target +DATA/dumpsets/expdp_5_5.dat copying file(s)... file, +DATA/dumpsets/expdp_5_5.dat, copy committed.

27 <Insert Picture Here> Top 10 ASM Questions Migration

28 Top 10 ASM Questions Q. We are migrating to a new storage array. How do I move my ASM database from storage A to storage B? A. Given that the new and old storage are both visible to ASM, simply add the new disks to the ASM disk group and drop the old disks. ASM rebalance will migrate data online. For pre-11gr2, See Note , which covers how to move OCR/Voting disks to the new storage array

29 Top 10 ASM Questions ASM_SQL> alter diskgroup DATA drop disk data_legacy1, data_legacy2, data_legacy3 add disk /dev/sddb1, /dev/sddc1, /dev/sddd1 ;

30 ASM Rebalancing Automatic online rebalance whenever storage configuration changes Only move data proportional to storage added No need for manual I/O tuning Disk Group DATA (legacy disks)

31 ASM Rebalancing Automatic online rebalance whenever storage configuration changes Online migration to new storage Disk Group DATA

32 ASM Rebalancing Automatic online rebalance whenever storage configuration changes Online migration to new storage Disk Group DATA

33 ASM Rebalancing Automatic online rebalance whenever storage configuration changes Online migration to new storage Disk Group DATA

34 ASM Rebalancing Automatic online rebalance whenever storage configuration changes Online migration to new storage Disk Group DATA (new disks)

35 Top 10 ASM Questions Q. Is it possible to unplug an ASM disk group from one platform and plug into a server on another platform (for example, from Solaris to Linux)? A. No. Cross-platform disk group migration not supported. To move datafiles between endian-ness platforms, you need to use XTTS, Datapump or Streams.

36 <Insert Picture Here> Top 10 ASM Questions ACFS

37 Extending ASM to Manage ALL Data Database RAC Application 3 rd Party FS Automatic Storage Management (ASM) ACFS Snapshot ASM Instance Managing Oracle DB Files ASM Cluster & Single Node File System (ACFS) Dynamic Volume Manager ASM Disk Group ASM Files DB DB Datafiles DB Datafiles DB Datafiles Datafiles OCR OCR & Voting Voting files files Dynamic Volumes ACFS FS Oracle Bin 3 rd Party FS 46

38 What is ASM Cluster File System (ACFS) General purpose scalable file system Journaling, extent based Single node and cluster POSIX, X/OPEN file system solution for UNIX/Linux Windows file system solution for Windows platforms Accessible through NAS protocols (NFS, CIFS) Leverages ASM technology Integrated with Oracle Clusterware for cluster support Multi OS platform (Linux and Windows at initial release) Integrated with Oracle system mgt tools Oracle installation and configuration Enterprise Manager and ASM Storage mgt tools Native OS File System Management tools 47

39 ACFS Features Provides filesystem snapshots (FCOW) File system integrity and fast recovery via ACFS metadata checksums and journaling. ACFS designed as a peer to peer, multi-node, shared file system model and delivers coherent data access ACFS file system is installed as a dynamically loadable OS VFS driver Starting with RHEL5, Redhat now supports a 'white list' - kernel APIs which they commit they will not change in updates or patches. APIs used by ACFS-ADVM were added to their 'white list'. Customers should be able to install an update or patch to the kernel and our drivers should not be impacted 48

40 Top 10 ASM Questions Q. Is ACFS supported on other platforms besides Linux A. Yes. Other platforms are forthcoming

41 Top 10 ASM Questions Q. Can ACFS be used to store database datafiles? What about archive logs? A. No. Currently we will not support database file to be stored in ACFS. This is due to performance reasons. Though you can do this in test/q&a environments where performance is not essential

42 Top 10 ASM Questions Q. Can I use ACFS to store BFILE data or other nondatabase related data A. Yes. ACFS is POSIX compliant filesystem, and thus can store any file data type (besides database files )

43 Top 10 ASM Questions Q. Will ACFS support other Data services, such advanced cloning, replication, de-dupe, etc.. A. Yes. ACFS Replication will be introduced in the next patchset release. Other advanced features are part of the roadmap.

44 <Insert Picture Here> Top 10 ASM Questions 3 rd Party Software

45 Top 10 ASM Questions Q. How does ASM work with multipathing software? A. It works great! Multipathing software is at a layer lower than ASM, and thus is transparent. You may need to adjust ASM_DISKSTRING to specify only the path to the multipathing pseudo devices.

46 ASM/RDBMS /dev/<pseudo> Multipath driver /dev/sda1 /dev/sdb1 IO cloud Controller Controller Cache Disk

47 Top 10 ASM Questions Q. Is ASM constantly rebalancing to manage hot spots? A. No No Nope!!

48 I/O Distribution IOPS Average IOPS per disk during OLTP w orkload Total Disk FG1: - cciss/c0d2 FG1: - cciss/c0d3 FG1: - cciss/c0d4 FG1: - cciss/c0d5 FG1: - cciss/c0d6 FG2: - cciss/c0d2 FG2: - cciss/c0d3 FG2: - cciss/c0d4 FG2: - cciss/c0d5 FG2: - cciss/c0d6 FG3: - cciss/c0d2 FG3: - cciss/c0d3 FG3: - cciss/c0d4 FG3: - cciss/c0d5 FG3: - cciss/c0d6 FG4: - cciss/c0d2 FG4: - cciss/c0d3 FG4: - cciss/c0d4 FG4: - cciss/c0d5 FG4: - cciss/c0d6 ASM spreads file extents evenly across all disks in disk group Since ASM distributes extents evenly, there are no hot spots

49 Summary: ASM requires very few parameters to run ASM based databases inherently leverage raw disk performance No additional database parameters needed to support ASM Mixed ASM-database version support Facilitates online storage changes RMAN recommended for backing up ASM based databases Spreads I/O evenly across all disks to maximize performance and eliminates hot spots

50 ASM Collateral and Content ASM 11g New Features ASM Best Practices ASM vendor papers ASM-RAC Customer Case Studies

51 <Insert Picture Here> Top 10 ASM Questions Extra credit questions

52 Top 10 ASM Questions Q. Is ASMLIB required on Linux systems and are there any benefits to using it? A. ASMLIB is not required to run ASM, but it is certainly recommended. ASMLIB has following benefits: Simplified disk discovery Persistent disk names Efficient use of system resources

53 Top 10 ASM Questions Q. Is it possible to do rolling upgrades on ASMLIB in a RAC configuration A. ASMLIB is independent of Oracle Clusterware and Oracle Database, and thus can be upgraded on its own

54