Top 10 Tricks and Tools of an Oracle EPM Administrator Effective EPM Administration

Transcription

1 Oct 2014 Top 10 Tricks and Tools of an Oracle EPM Administrator Effective EPM Administration Nicholas King Database Administrator

2 Purpose and background Agenda: s & Tricks for... Improving uptime by building a troubleshooting toolkit Automation / working effectively / less toil Security / auditing Nicholas King 6 years as Unix sysadmin and software development 8 years Hyperion infrastructure consulting =14 years administering complex enterprise systems

3 1: Manual Troubleshooting Focus on Uptime

4 1 Too many logs Search for *.log on one server Effective troubleshooting depends on getting useful error messages from logs. Too Many Logs! Types of Logs Domains: WebLogic Logs user_projects\domains\epmsystem\servers Diagnostics: Application Logs user_projects\epmsystem1\diagnostics\logs Windows Event Viewer HFM Error Log Viewer FDM Outbox

5 1 Demo Demo: Manual Troubleshooting

6 2: Log Analysis Utility Find Problems Fast

7 Using the Log Analysis Utility 2 What if you could scan hundreds of logs at once? Step 1: Search folder < 60 min old loganalysis.bat -system -tmin 60 -d search_directory -o output_filename Step 2: Search both domains and diagnostics loganalysis.bat... -d d:\oracle\middleware\user_projects\ domains -o domains_report loganalysis.bat... -d d:\oracle\middleware\\user_projects\epmsystem1\ diagnostics -o diagnostics_report Step 3: Run against multiple servers using UNC path loganalysis.bat... \\svr1\d$\oracle \Middleware\user_projects\epmsystem1 \diagnostics -o node1_diagnostics

8 Log Analysis Utility on multiple servers start cmd /C "loganalysis.bat -system -tmin 60 -d \\hsssvr\d$\oracle\middleware\user_projects\domains -o web1_domains " start cmd /C "loganalysis.bat -system -tmin 60 -d \\hsssvr\d$\oracle\middleware\user_projects\epmsystem1\diagnostics -o web1_diagnostics " start cmd /C "loganalysis.bat -system -tmin 60 -d \\hfmsvr\d$\oracle\middleware\user_projects\domains -o hfm1_domains " start cmd /C "loganalysis.bat -system -tmin 60 -d \\hfmsvr\d$\oracle\middleware\user_projects\epmsystem1\diagnostics -o hfm1_diagnostics " Server 1 Server 2... Server x Server y Server z 2

9 Log Analysis Utility output Each browser tab contains a different report hss_diagnostics, hss_domains ra_diagnostics, ra_domains hfm_diagnostics, hfm_domains 2

10 2 Demo Demo: Log Analysis Utility

11 2 Log Analysis Utility comparison Advantages Targeted search for just errors No experience necessary Simple, first pass at log analysis Hundreds of files scanned in seconds Easy to read HTML report Detailed messages can lead to root cause Disadvantages Scans Oracle log format only Still need logs to get context about activities before and after error message

12 3: Validate Utility Tiny Testing Tool

13 Introduction to Validate Utility What does it do? A custom test suite with hundreds of intricate tests Invocation: user_projects\epmsystem1\bin\validate.bat 3

14 Validate Utility internals 3 user_projects\epmsystem1\config\validation\ \vtconfig.xml hat_tmp\hatinput.vbs > diagnostics\logs\validation\hat.output ----STARTING VALIDATION SCRIPTS---LOGGING IN HFM... CREATING APPLICATION... OPENING APPLICATION... LOADING SECURITY... LOADING METADATA...

15 3 Validate Utility comparison Advantages Specialized test suite Intricate tests - much more than just a connectivity test HTML based output is easy to read Pinpoint detailed internal errors Disadvantages Errors can be missed if they do not fail a specific test Failed tests may not identify the root cause Must be run on each node

16 4: LCM Automation Duct Tape for Quick Repairs

17 Lifecycle Management automation 4 Nightly Automated LCM Exports to Date Stamped Folder (versioning) Partial restore without downtime Financial Reports Shared Services Security Internal HSS registry Application artifacts - planning forms, EPMA hierarchy...etc How to Invoke Location: D:\Oracle\Middleware\user_projects\epmsystem1\bin Utility.bat XML_File Simple: only need an XML file Every LCM export in SS contains a template import.xml and export.xml. Just add username, password to XML file.

18 4 Demo Demo: Versioned Financial Reports

19 5: Copy Provisioning Establish Security Fast

20 5 Copy provisioning introduction Clone/mirror Shared Services security from one app to another FMAPP2 FMAPP3 user1: Consol Admin user2: Post Journals Provisioning Why Copy Provisioning? Cloning security for Essbase cubes Quickly apply security to a test app Migrating security from an old app to newer version Pushing/Staging large scale security changes

21 Sync security example Example: Clone FMTEST2 Security to FMTEST3 FMTEST3 is not provisioned Initial State: Provisioning Report for FMTEST2 Provisioning Report for FMTEST3 5

22 Example: Syncing security Shared Services Application Groups Source: FMTEST2 FMTEST3 security established in a few mouse clicks 5

23 6: Password Changes Keep the System Secure

24 Database accounts in Config Tool 6 Steps to Change Typical DB Accounts 1. Change password for product in the Config Tool 2. Redeploy the application server 2. Change directly in WebLogic Home>Summary of Security Realms>Summary of JDBC Data Sources> eas_datasource

25 7: Working with Oracle Support How Detailed SRs Save You Time

26 General Oracle Support tips 7 Adding Details in the SR Saves You Time Incomplete info will result in lost hours/days Provide overview, context, step by step process, screenshots Reproduce the problem - prove within a shadow of a doubt Simply the problem - can you reproduce with a sample app, basic report, etc. Flip side - how can Oracle reproduce the problem outside of your env? Ask for a WebEx - demonstrate the problem first hand

27 8: Oracle SR Tools Tools Required for Working with SRs

28 Creating a SR - deployment report 8 Deployment report - Layout of environment Location: user_projects\epmsystem1\bin Invocation: epmsys_registry report deployment Output: diagnostics\reports\deployment_report.html Database Connections Product / Server Layout

29 Creating a SR - versions and patch levels 8 Opatch lsinventory - Exact versions of all products on server Location: D:\oracle\Middleware\EPMSystem11R1\OPatch Invocation: opatch lsinventory -oh D:\Oracle\Middleware\EPMSystem11R1 -jre D:\Oracle\Middleware\jdk160_29 Output: EPMSystem11R1\cfgtoollogs\opatch\lsinv Patch : applied on Mon Oct 29 16:14:16 PDT 2012 Patch : applied on Mon Oct 29 15:51:44 PDT 2012 Patch : applied on Mon Oct 29 15:07:44 PDT 2012 support.oracle.com Patch Search

30 Creating a SR - more tools 8 Zip logs - Collect all relevant logs on system Invocation: ziplogs Output: diagnostics\ziplogs\epm_logs_hss-node_ zip Registry report - HTML view of internal EPM HSS Registry Invocation:epmsys_registry (no arguments) Output: diagnostics\reports\registry.html Validate (shown earlier) Invocation: validate Output: diagnostics\reports\instance_report*.html Script Locations: user_projects\epmsystem1\bin

31 9: Auto Restart Failed Services Hands Free Problem Solving

32 Restart failed Windows services automatically 9 Error causes service stop/crash Not frequent, but may save some headaches Avoid repeated restarts Custom settings reset every time you rerun config tool Command line sc \\localhost failure HyS9FRReports actions= restart/60000/restart/60000/""/60000 reset= 0

33 10: Exploring Audit Identifying System Usage

34 Exploring Audit 10 Goal: View the number of logins v. time per product from the audit logs Enable Audit in Shared Services: Administration - Configure Auditing

35 Audit tables: basic query Basic query for Essbase logins v. time To Do: Add unified time series for graphing 10

36 Audit tables: building a useful query (1) Create uniform time sequence for graphing 10

37 Audit tables: building a useful query (2) Join time series with original query 10

38 Audit tables: building a useful query (3) Results show uniform time with number of logins 10

39 10 Visualizing the results Logins vs. Time by Product Essbase Workspace

40 10 Recap s and Tricks: Manual Troubleshooting Log Analysis Utility Validate Utility LCM Automation Copy Provisioning Password Changes Working with Oracle Support Oracle SR Tools Auto restart failed services Exploring Audit Objectives: Uptime Less toil / Effective EPM Administration Security

41 Thank You! Q&A Nicholas King