About me - Joel Montvelisky

Transcription

1

2 About me - Joel Montvelisky PractiTest Co Founder & Prod. Architect QA Instructor & Consultant Mercury Interactive - QA Manager (retired ) TD, QC, WR, QTP, etc ITCB (IL) Advisory Board QABlog.practitest.com

3 Today s Agenda SaaS - Software as a Service Things we Can Stop Testing Non-trivial Stuff to Think About & TEST Leveraging Live Feedback into the Testing Lifecycle Open Season for Questions

4 What is SaaS?

5 Software as a Service Hosted Platforms Total Control No Maintenance No Client Installations Customers buys each month keep them happy Users don t care it is SaaS, it needs to be better than the alternative

6 SaaS QA Management End-to-End QA Management (Reqs Tests Bugs) Enterprise Level SaaS Solution - Methodology - Customizations & Flexibility Global platform - Communication - Languages - Available 24x7 Worldwide Intuitive & Simple Agile Dev, RoR, on Amazon EC2

7 What does SaaS mean to the Testing Process?

8 Things we can STOP Testing Client or Server installations Multiplatform backend support Multiple version upgrades Backwards compatibility Savings of approx. 1/3 of the Traditional Enterprise Application Testing Cycle

9 The regular testing scenarios Functionality - Manual scripts - Exploratory sessions - Checklists - End-2-End Scenarios - Selenium - Cucumber & RSpec Agile Sprints & Updates of 3 4 weeks Each sprint s cycle is based on its content Load & Stress Multi-platform & Multi-browser support I18N English, Swedish, German, Chinese Simplified, Hebrew

10 Non-trivial stuff to think about & TEST 1. Remote accessibility & usage Staging Servers in USA - Amazon EC2 Development & Testing teams in Israel (& WW collaborators) Globally distributed monitoring services (

11 Non-trivial stuff to think about & TEST 2. Security Application-related: Cross-site scripting SQL injections HTTP header injections etc Infrastructure-based: Secured communication Backups & storage policies Controlled Access to the site & servers

12 Non-trivial stuff to think about & TEST 3. Live updates & deployments (3 to 4 weeks) No concrete separation between Dev & IT Minimal service disruption Dressed rehearsals (including rollbacks!) Automated sanity & manual verification Patches & hot-fixes (extraordinary occasions ) Prove of Concepts & Beta Releases QA & Testing as a facilitator for Balance: Stable & Professional, yet Flexible Environment

13 Non-trivial stuff to think about & TEST 4. Disaster recovery procedures Two Main scenarios: (a) System down to be brought up quickly Configure machines Install & deploy software Restore data Define a policy & schedule regular drills to ensure you can achieve it! (b) Rollback to last known stable data (with or without data restore)

14 Leveraging Real Data into the Testing Process

15 Levering live feedback into the Testing Process (1) Internally developed monitoring tools - Check usage patterns - Validate need & usability assumptions No need to continue guessing!!! - Analyze patterns and create realistic & prioritized testing scenarios

16 Levering live feedback into the Testing Process (2) Site Monitoring systems Real-time usage to monitor traffic Allow users to contact the company with questions or issues Pingdom ( Up to date response times for real projects and data. Proactive alerts when things start slowing down

17 Levering live feedback into the Testing Process (3) Proactive mail notifications on issues ( Real-time notifications on issues on the system. Information about the issue including system parameters that allow you to do an initial trouble-shoot of the issue

18 Levering live feedback into the Testing Process (4) Ask users for Real Projects No better way to test than using real data Most users will trust you not to abuse their information in exchange for assurance of working software Since the data sits in your system it is easier to copy projects to your testing environment Need to be careful with who has access to the data; making sure your testing environment is as secured as your production environment

19 Wrapping up

20 Wrapping up SaaS is only the delivery model, customers will not compromise on Functionality, Stability or Response Time Since the system is hosted you can save all tests related to backend installation and support You need to pay more attention to stuff like accessibility, security, deployments & DRPs The lines between the R&D and IT organizations in SaaS are blurry SaaS application allows better access to real data to be leveraged to improve the effectiveness of our testing operations.

21 Open Season for questions!

22 Special thanks to Anne-Marie Charrett