The importance of Performance Testing in a SAGE ERP implementation. Richard Bishop

Transcription

1 The importance of Performance Testing in a SAGE ERP implementation Richard Bishop

2 Introduction Who are we? Trust IV Testing consultancy, specialising in automated non-functional testing since 2005 Richard Bishop, Trust IV Ltd IT consultant with 20 years experience Specialising in performance engineering / testing HP specialist, UK leader of Vivit (HP software user group in UK) Colleagues Mixture of consultants and contract resources Primarily HP LoadRunner specialists Skills in multiple test tools, platforms etc. On customer sites and working remotely

3 Non-Functional Testing What on earth is NFT? Non-functional, automated testing specialists What on earth is NFT? In a nutshell. Usability, reliability and scalability. Compatibility testing Compliance testing Security Testing Backup and Disaster Recovery Testing Load Testing Performance Testing Scalability Testing Stress Testing

4 Project Background University Hospital Birmingham Sage 500 to Sage 1000 migration Concerns re: scalability Needed performance test to determine maximum user load Test to include a single user journey, simulating a product requisition Objective was to increase user load until system failure

5 The arguments against testing* *(non-functional software testing) The vendor/developer has already tested this so we don't need to NFT not required if functional testing and UAT is OK If it's slow we can throw kit at it Testing is too expensive / time consuming It s too difficult.

6 Vendor/developer already tested Do schools let children mark their own homework?

7 But it works for one user. NFT not required if functional testing and UAT is OK Single user DB Server Load balancer IIS servers Forms Server

8 But it works for one user. What about single server failures? Single user Multi user DB Server DB Server Load balancer Load balancer IIS servers Forms Server IIS servers Forms Server

9 But it works for one user. What about network bottlenecks? Single user Multi user DB Server DB Server Load balancer Load balancer IIS servers Forms Server IIS servers Forms Server

10 But it works for one user. What about database bottlenecks? Single user Multi user DB Server DB Server Load balancer Load balancer IIS servers Forms Server IIS servers Forms Server

11 Tin is cheap if it's slow we can "throw kit at it"

12 Testing is expensive Sometimes true, but not testing can be more expensive

13 Testing is expensive Sometimes true, but not testing can be more expensive The money spent with Trust IV was the best money spent on the whole project University Hospital Birmingham

14 Our standard test approach POC Analysis Scripting Performance tests Low vol. tests

15 Digging deeper SAGE 1000 uses two communications protocols POC Had to convert displayed human readable text to legacy formats to allow SAGE 1000 to interpret our simulated user input

16 Digging deeper Complex test data requirements POC Scripting SUBMIT POST " "Comms&" " CS3SessionID " IDENTIFIER 136 BODY "MfcISAPICommand=Comms& CS3SessionID \x00\... <Some data removed for brevity>.."\x00\xed\x00\r\x00f\x00o\x00r\x00c\x00e\x00p\x00s\x00\x00\x00\x00\x00\x0 1\x04\x00\r\x00\x00\x00\x00\x01\x1d\x00\r\x00".."\

17 What we did POC Scripting Initial tests showed problems with scalability just simulating logins 20 user load

18 Initial Tests User experience Low vol. tests Target = 230 users Adding 1 user every 10 seconds Response times > 20 seconds Site failed with less than 20 users

19 Next steps Reconfiguration & re-test Analysis Performance tests We made recommendations to improve performance We asked Datel to check: Heap size Application pool size Timeout values Datel reconfigured application server: Altered TCP/IP timeout values and keep alives Set lifetime session limit to 30 minutes Registry changes Encrypted login credentials within the application

20 Re-test Analysis Performance tests Observations Despite load balancer problem Next steps - retest Response times consistent No degradation over time

21 Conclusion Analysis What have we learnt? You probably need to test Reduced response times for SAGE 1000 login from > 20 seconds (and timeouts) to 3 s Application worked, just not our particular configuration Testing needn t be expensive Thanks to UHB for the endorsement Don t trust vendors (or developers) to test

22