FIMA/UBICOM Case Sandvik Test automation Petri Hakomäki / Competence Center Control Systems 24.08.2011
Content of Presentation Background Some words about history of SW testing in. Testing challenges Visio about how we want to develop testing Test automation today What we have currently in use How we use it Learnings so far
Background Business environment SW based control systems are developed for different types of mining machines R&D projects vary from customer specific modifications to new machine generations Project organizations are very different Need for testing is different in each project More and more intelligence is built-in to the machines Automated drilling sequences Advanced configuration and diagnostic features Data collection, monitoring and reporting Safety is the most important feature Safety related testing has a key role
Background Testing challenges In the past testing has been executed mainly in the final product Major failures/product delays due to SW problems Advanced Control System SW projects are using incremental development cycle Testing activities are increased amount of needed regression testing is getting higher Challenge is to get systematic testing to all development levels Control System SW testing is a new thing in many sites All projects handle their testing activities individually No common testing strategy Testing methods and techniques are project specific
Background Visio of what we wanted to achieve Well defined testing strategy and test process Test process is supported by practical & proven toolset for all phases Test automation is needed Speed up the development cycle Free available testing resources to do complex testing that is needed anyhow More emphasis on quality Test results should be used to measure the quality of the product Certain quality level must be achieved before entering the next development step
Test automation today What have been done Define Control System Software Test Process Test process was defined to be guideline for new product projects Tools to manage and follow-up testing. Developing of test setup for Control System SW functionality testing SIMATE project HIL environment Simulated machine functionalities Used in manual testing Developing tool set for automated testing of Control System SW HIL environment as base system Python used in testing script writing Tool for automating GUI related testing
Test automation today Example of used testing environment Display Autom. IF HIL environment includes all programmable target HWs IO controlled with IO cards and simulation SW Mainly Creanex systems PLC PLC PLC Test PC Other CANopen nodes simulated with software running in Windows PC Engine model Sensors, actuators, EPM valves etc. Motor Model Virtual Valves and Sensors Creanex Environment model Automatization IF Automatization Interface used to have controlling interface to Contorl System Read/write signals in Control System level Read/write control system signals used in Display
Test automation today Test automation tools SITA, Sandvik Intelligent Test Automation system Python based A collection of ready-made, reusable automation blocks for accessing (automatic testing) various platform services and handling signals and the test environment (as e.g. Creanex simulation) in general Three different GUI views for running pre-defined CLI commands (local or SSH), constructing test cases and executing those GUI test automation tool Based on Tdriver (Nokia made open source tool for Qt-testing) Tool development project ended in June PolarionTCRunner: Tool for Polarion stored test case execution Implemented in TIKOSU project for Sandvik Reads test case one by one according given search parameters from Polarion Runs automated test script Returns test result to Polarion
Command line helper
SITA GUI (TC executing)
SITA helper (TC construction)
View from GUI test automation tool
PolarionTCrunner
Test automation today Where we use it Test automation is currently used for machine level testing Pilot project for verifying usability of GUI test automation tool is starting during 2H/2011 Test automation is mainly used for regression testing. Test automation is not used for new feature testing.
Test automation today Learnigs so far Test automation is good tool for reducing work effort for regression testing. E.g. in platform testing need of regression is rising by each new release This would cause need of new resources to enable enough testing for reaching needed quality level. An example from real product project: Machine control part of regression testing is mainly automated, ca. 600 test cases Manually executing would take ca. 1-2 weeks from one person - automated testing takes ca. 5-6 hours New features and functionalities needs to be tested manually first Test automation is not reasonable to do without this phase or needs very good specification and tools set and controlled way of working What to automate and what not needs to be evaluated before making decision about taking test automation as part of project Building test automation system needs quite high amount of resources both personnel and HW an investment that management needs to support
Any questions? Thank You!