The Future of Test Automation

Transcription

1 2nd edition Annual conference and exhibition Thursday June 21st 2012 World Trade Center (WTC) Rotterdam The Future of Test Automation CONFERENCE ORGANIZATION 30 Experts in One Day! Chairman Prof. Dr. Arie van Deursen Professor in software engineering, head of the Software Engineering Research Group, Delft University of Technology Co Host (U.S.) Dion Johnson Advisor, Automated Testing Institute FOUNDING PARTNER PARTNERS Keynote Walter Belgers Partner & Principal Security Consultant, Madison Gurkha Keynote (U.S.) Scott Barber CTO, PerfTestPlus, Inc. Best practices, workshops & Q/A sessions by: Derk Jan de Grood, Valori Michiel Vroon, Quaboo Ron Rouhof, SQS Nederland Martin Gijsen (independent) Clemens Reijnen, Sogeti Zeeger Lubsen, Software Improvement Group Alexandra Schladebeck, BREDEX GmbH Maurice Siteur, Capgemini Machiel van der Bijl, Axini Anko Tijman, Ordina Bryan Bakker, Sioux Embedded Systems John Kronenberg, Bartosz ICT Pepijn van de Vorst, Ordina ICT M&C BU Testing Bernd Beersma & Adrian Canlon, Squerist Rix Groenboom, Parasoft Netherlands Marc van Lint, IBM The Netherlands Ayal Zylberman, QualiTest Jan de Coster, Micro Focus E.J. Smit & R. Kwinkelenberg, TRICENTIS Gerard van der Pol, Microsoft Marcel Diepenbroek, TrendIC Hans Beverwijk, Remain Tim Prijn & Hubert ten Hove, Info Support SPONSORS To view the program and tweets on your mobile phone scan the QR code. More information and registration at: EXHIBITORS KZA BV RANOREX SQS NEDERLAND B.V. VX COMPANY WITH COOPERATION OF AUTOMATED TESTING INSTITUTE

2 The Future of Test Automation The fundamental software engineering challenge is dealing with change. To address that challenge, we need test automation. If you are interested in test automation, you will be well aware of the benefits. You can run your tests faster, as often as you want, and you can run more of them. You can provide more variation, by exercising test cases with larger and different data sets or configuration options. Most importantly, you can exercise your test suites on a daily basis through a continuous integration server, allowing you to catch bugs as soon as they are introduced. You may, however, also have experienced some of the darker sides of test automation. The required automation infrastructure does not come for free, and neither do the test cases. The cost of fragile test suites that need to be adjusted whenever a change is made can be substantial, canceling their benefits. Last but not least, automated test suites not covering the right functionality will give a false sense of safety. Balancing these pros and cons requires insight and experience. Sharing such insight and experience is exactly the purpose of this year's "Test Automation Day". In line with the day's theme, "The Future of Test Automation", a number of experts will present their methods, tools, techniques, and experience from the trenches. Furthermore, you will be there, challenging the speakers, and sharing your experience in questions and discussions. The future of test automation is bright, but many engineering and organizational challenges remain to be solved. I hope and I expect that the Test Automation Day will contribute to that, and will help you to become even more effective in your day to day test automation activities. Chairman Prof. Dr. Arie van Deursen Professor in software engineering, head of the Software Engineering Research Group, Delft University of Technology Co-Host Dion Johnson Advisor, Automated Testing Institute Test Automation Day 2012 After a very successful first edition of Test Automation Day, we are happy to announce the 2012 edition on June 21st in Rotterdam, the Netherlands! A top-level conference with even more international keynotes, expert-insights and lessons learned to this year s theme: The Future of Test Automation. Program committee An inspiring program of Test Automation Day is guaranteed by the independent program committee. Members of the committee are (from left to right): Arie van Deursen; Professor in software engineering, head of the Software Engineering Research Group, Delft University of Technology Scott Barber; CTO, PerfTestPlus, Inc. Pieter Koopman; Researcher, University of Nijmegen, Managing Director, LaQuSo Edwin van Loon; Senior quality consultant, Valid 2012 edition In cooperation with the independent program committee, CKC Seminars has put together: Keynote sessions with gurus as Arie van Deursen, Dion Johnson (US), Scott Barber (US), and Walter Belgers 22 sessions in which business cases and workshops are presented by national & international thought leaders and experts An exclusive lunch tutorial with Scott Barber A Future Tool Lab at the Exposition And a Meet & Greet dinner! Target audience Test Automation Day Test Automation Day mainly aims at: IT Management; CIOs, CTOs, IT Managers. IT Professionals; for example Engineers, Analysts, Developers, Specialists, Test Coordinators, Integrators and Test Managers, Software and IT quality professionals. Business Management; for example Software providers, Project Managers, seconding parties and consultants. Visit Test Automation Day and: Meet world s leading experts; Get the latest insights in emerging technologies, proven practices and trends; Network with more than 250 IT colleagues! Be there, for only 195,-! Register now and ensure your place at the event: Tickets are limited! Participation fee is only 195,- (excl. VAT) for subscribers or members of magazines, newsletters, portals and branch associations which are cooperating with Test Automation Day (find them at the registration form). Regular fee is 295,-. We hope to see your participation in this exciting conference! Register now! Tickets are limited!

3 PROGRAM Track 1 Best practices in Test Automation Track 2 Trends & Approaches Track 3 Agile Track 4 Open Source Tooling / Solutions Track 5 Concepts in Test Automation Track 6 Question/Answers Functional testing Test related tooling Tutorial Registrations Kick off by CKC Seminars Kick off by Chairman Arie van Deursen Scott Barber: The Future is Cool, but Does it Add Value? Walter Belgers: Security Testing (Keynote speaker for Squerist) BREAK Zaal 1 Zaal 2 Zaal 3 Zaal 4 Zaal 5 Zaal 6 Rix Groenboom Jan de Coster Bernd Beersma & Adrian Canlon Tim Prijn & Hubert ten Hove Ernst Jan Smit & Remco Kwinkelenberg Application Virtualization in Practise: Do your web and mobile websites How Agile Testing demands Picture driven Automated Graphical Methodical TestCase-Design with Modelling of the GBA-V work properly in mobile devices? Automated Regression Testing Testing with Sikuli Linear Q Speaker for Speaker for Speaker for Speaker for Speaker for Parasoft Netherlands B.V. Micro Focus Squerist Info Support B.V. Tricentis NL BV CHANGE ROOM Bryan Bakker Clemens Reijnen John Kronenberg & Pepijn vd Vorst Alexandra Schladebec Gerard van der Pol Trackleader Automated testing of an X-Ray Agile Application Lifecycle Testing of processes within an The Eclipse Jubula Project: From Manual to Automated Testing Maurice Siteur medical device Management - agile environment using Moving away from Capture-Repla Speaker for with Martin Gijsen Getting testing done in a sprint Fitnesse and Selenium Microsoft and Machiel vd Bijl Tutorial LUNCH BREAK Scott Barber What automation advancement is right for us? Introduction afternoon program by Arie van Deursen Will be published at our website CHANGE ROOMS Ayal Zylberman Marc van Lint Anko Tijman Marcel Diepenbroek Hans Beverwijk Test Automation Solutions for Develop and Deliver: Collaboration in Test Automation Meet Marcel's Mother Boosting collaboration and Philips Medical Systems The Integrated Role of Testing Speaker for empowering people: Robust Speaker for Speaker for TrendIC Automated Testing for Everyone! QualiTest IBM Speaker for Remain BV CHANGE ROOM Michiel Vroon & Ron Rouhof Derk-Jan de Grood Zeeger Lubsen Martin Gijsen Machiel vd Bijl Trackleader Your ticket towards outsourcing Interactive discussion; Test the tester: Automated tests that survive Model-based testing in action Maurice Siteur of test automation Test Automation: A door to open maintainable Test Automation continuous change with Derk-Jan de Grood or to lock tightly? Pepijn vd Vorst and John Kronenberg BREAK Closing keynote by Dion Johnson with his vision of "The Future of Test Automation" and a retrospective view of the Conference DRINKS Meet & Greet Dinner for delegates, presenters, sponsors KEYNOTES Kick off by Chairman van Deursen Professor Arie van Deursen will introduce you to the central theme of the second edition: The Future of Test Automation. During his work as a professor and director of a research group, he combines research with collaboration in industrial practice. In this presentation he will do the same by sharing the newest technological developments, the pro and cons, and developments in practice The Future is Cool, but Does it Add Value? DevOps, clouds, gestures and artificial intelligence are driving the near Future of Test Automation. While I love cool technological advances, a voice in my head keeps saying The ability to do something doesn t mean we should. Join Scott Barber to explore the question of Just cool? Or Valuable? Security Testing Functionality means: the application does what it should do. Security means: the application does not do, what it should not do. This is often not tested for. In the presentation, we will look at the problems this introduces in web applications, how to test for them and how to fix or prevent them The Future of Test Automation New developments in test automation will be followed by more changes to come. As practitioners it s important to understand the current trajectory of these changes so that we can help guide them appropriately. It s also important to understand these changes so that we are prepared to effectively operate in the new ways of doing business. This presentation acknowledges these points and invites you to take a peek into the Future of Test Automation. Prof. dr. Arie van Deursen, professor in software engineering, head of the Software Engineering Research Group, Delft University of Technology Mr. Scott Barber, CTO, PerfTestPlus, Inc. Speaker for Squerist Mr. Walter Belgers, Partner & Principal Security Consultant, Madison Gurkha Technologies Mr. Dion Johnson Advisor, Automated Testing Institute TRACK 1 Best Practices in Test Automation Application Virtualization in Practise: Modelling of the GBA-V Modern application landscapes are becoming more interconnected which make them complex to test. Based on the test environment of the Gemeentelijk Basisadministratie (GBA-V), we demonstrate how Application Behavior Virtualization reduces this complexity by removing external dependencies. We discuss the process (Capture, Model, Consume) and share the lessons learned. Automated Testing of an X-Ray medical device Case study of test automation of medical x-ray device on system level Hardware interfaces used to interact with system resulting in low probe effect Scope of test automation focuses on reliability testing Incremental development of test framework to deliver fast results ROI calculation to convince management of approach Test Automation Solutions for Philips Medical Systems Just a few minutes away from Philips labs, QualiTest Test Center is located. This Onshore Test Center provides Philips with outsourced test automation solutions provided by QualiTest testing experts. These high end automation solutions from such a close location to Philips, allow better communication, reduced management attention and higher efficiency. In this presentation, Ayal Zylberman will take you along their solutions and procedures. Your tickets towards outsourcing of Test Automation This best practice presentation is based on the experiences of test automation of the webservices at AirFrance/KLM, a workpackage that is being delivered from the Competence Center Test Management to the Altea DC. We show opportunities, challenges and pitfalls of Outsourcing of test automation. There will be an overview on Entry criteria and KPI s important for the external service providers that deliver the test automation capacity. Mr. Rix Groenboom, Solution Architect, Parasoft Netherlands B.V. Mr. Bryan Bakker, Test Expert, Sioux Embedded Systems B.V Mr. Ayal Zylberman, President, QualiTest Mr. Ron Rouhof, Director of Operations, SQS Mr. Michiel Vroon, Director, Quaboo b.v.

4 TRACK 2 Trends & Approaches Do your web and mobile websites work properly in mobile devices? Every day enterprises offer more mobile services. Due to smaller screen sizes and different input methods (touch screen), many Web applications look different on a mobile device. Learn about customer experiences using an easy, intuitive way to test mobile applications through both emulators and mobile devices. Agile Application Lifecycle Management - Getting testing done in a sprint Agile teams find it hard to get the testing effort in sync with the other development activities. Not only development tests are executed during sprints, all other kind of testing activities are part of done. This session will give guidance how Microsoft Visual Studio ALM tools can support agile teams. How to run sprints and get testing done in a sprint. Develop and Deliver: The Integrated Role of Testing A very first challenge development teams face in today's dynamic product environment is the integration of the various disciplines of software development and delivery. We tell here how one company's application of test development benefited from being an integral part of these integrations, in this case, using IBM Rational products. Test Automation: A door to open or to lock tightly? What are the conditions that must be fulfilled when you want to do test automation? Derk-Jan will tell about experiences with different groups, their perspective and the problems they want to solve. This introductory talk is the Launchpad for a discussion on: preconditions for TA, Business Case Content and how we can assist an organization in their choice of TA. Mr. Jan de Coster, Subject Matter Expert, Micro Focus Mr. Clemens Reijnen, Management Consultant, Sogeti Netherlands, Microsoft MVP, specializing in Application Lifecycle Management Mr. Marc van Lint, Rational Consultant, IBM The Netherlands Mr. Derk-Jan de Grood, Test Expert, Valori TRACK 3 Agile How Agile Testing demands Automated Regression Testing In this session, the presenters will demonstrate the positive link between Agile Projects and Automated Regression Testing. Squerist has developed a model that respects the Agile Manifesto while at the same time proposes a solution for some of the most common problems within Agile Projects. During this demonstration, customer Simon van Zessen CJIB will share his experiences with Agile Testing. Testing of processes within an agile environment using Fitnesse and Selenium How we have used Fitnesse, Selenium and a stub framework for testing processes Testing the GUI and the technical layer Specific points of attention regarding our Agile environment Identified pitfalls Has test automation brought us what we expected from it? Collaboration in Test Automation Test automation seems to be a single person task. But Teamwork is essential in delivering quality. The challenges for TA experts are: 1) How to make your team feel responsible 2) How to involve Maintenance dept 3) How about customer involvement? In this session I will focus on the collaboration aspects in test automation. Test the tester: maintainable test automation With modern and widely used Agile development methods such as SCRUM, which are known for their short delivery cycles, the need for short and frequent test cycles increases. Test automation is then a necessary step. This session explores the need for building maintainable test scripts and the lessons that can be learned from current development experience. Mr. Bernd Beersma, Competence Lead Test Automation, Squerist Mr. Adrian Canlon, Certified Agile Test Trainer, Squerist Mr. Simon van Zessen, Test Navigator, CJIB (Centraal Justitieel Incassobureau) Mr. John Kronenberg, Test Consultant, Bartosz ICT Mr. Pepijn van de Vorst, Test Consultant, Ordina ICT M&C - BU Testing Mr. Anko Tijman, Sr. Agile coach, consultant & trainer, Ordina Mr. Zeeger Lubsen, Management Consultant, Software Improvement Group TRACK 4 Open Source tooling / solutions TRACK 4 Functional testing Picture driven automated graphical testing with Sikuli The goal of this hands-on session is to become acquainted with Sikuli; an open-source graphical tool for automated testing. With use of a real life case currently performed at the Dutch railways we will show a practical use of Sikuli. A laptop (Windows, MAC, Linux) is needed for this workshop. Please bring your own laptop. The Eclipse Jubula Project: Moving away from Capture-Replay Jubula is an Eclipse open source project for automated GUI testing. The approach used is different to many other tools: there is no "capture-replay" and tests are not written in code. Jubula focuses on testing from the customer perspective and on best practices like reusability and readability. This talk introduces the concepts behind Jubula and a demo of the tool. Meet Marcel s Mother In this session, Marcel will challenge you to use a different methodology for Test Automation. Without programming, any (!) chain of applications can be tested. Step by step, just like Marcel explains any operation on a computer to his mother. This makes CitraTest FT successful in automating regression and acceptance tests- even in difficult environments. Automated Tests that survive continuous change Maintenance has killed too many automated testing efforts. Use the available techniques to continue reaping the benefits of your hard work. Use the features of your test tool well and add some if necessary to achieve low maintenance automated tests. Mr. Tim Prijn, Competence Center Java lead, Info Support Mr. Hubert ten Hove, Developer/Designer, Info Support Mrs. Alexandra Schladebeck, Consultant Automated Testing and Test Processes, BREDEX GmbH Ir. Marcel Diepenbroek, Business Development, TrendIC BV Mr. Martin Gijsen, independent test automation Architect TRACK 5 Concepts in Test Automation Methodical TestCase-Design with Linear Q Linear Q gets its name from its key feature: a unique TestCase Design method. TRICENTIS has developed the Linear Expansion method on the basis of established methodical concepts (creation of equivalence classes and boundary values as well as combinatorial principles) for designing a highly efficient set of test cases. This workshop takes you through the process of using Linear Expansion for creating a set of test cases and shows how they can be executed in an automated way. From Manual to Automated Testing In this workshop we will introduce you to the key concepts of test automation and the tools that are available within the Visual Studio Toolset. These concepts will include Fast forward for Manual Testing, an Introduction to Coded UI Tests and how to maintain these Tests. We will also give you a brief overview of the Microsoft Test platform and all its capabilities. TRACK 5 Test related tooling Boosting collaboration and empowering people: Robust Automated Testing for Everyone! Original Software bridges the gaps in application quality management by offering an integrated, easy-to-use solution for every stakeholder in the software quality process. In this session Hans Beverwijk will take you through the process, and show you how it incorporates functional test automation, manual and database testing, and acceptance testing in a successful way. Model-based testing in action In this workshop we will apply Model-based testing, the next generation test automation technique, to a protocol of one of our clients. We will show how it is possible to automate the entire test-process based on this model in order to achieve high coverage of the functionality of the system under test (including bad weather behaviour). Mr. Ernst Jan Smit, Sales Manager Benelux, TRICENTIS Technology & Consulting NL Mr. Remco Kwinkelenberg, Consultant, TRICENTIS Technology & Consulting NL Mr. Gerard van der Pol, Developer Consultant, Microsoft B.V. Mr. Hans Beverwijk, Consultant, Remain Software Dr. Ir. Machiel van der Bijl, Founder, Axini B.V.

5 2nd edition Annual conference and exhibition Thursday June 21st 2012 World Trade Center (WTC) Rotterdam Our Keynotes Prof. Dr. A. (Arie) van Deursen Arie van Deursen is Professor in software engineering at Delft University of Technology. Van Deursen is director of a Software Engineering Research Group. With his team, he for example verifies software architecture, testing of Ajax applications, agile testing, social media for software developers and end-user programming. Arie van Deursen combines his knowledge with practise. Follow professor van Deursen at twitter.com/avandeursen. D. (Dion) Johnson Dion Johnson has 15+ years of experience providing IT consulting and training services for several organizations including Lockheed Martin, Unisys, Northrup Grumman, AOL, Hewlett Packard and more. He s presented at numerous industry conferences including the TestKIT Conference for which he is also the chairperson, and published articles in various IT publications, including the Automated Software Testing Magazine. As a technical board member for the Automated Testing Institute, he is a contributor to the Test Automation Body of Knowledge (TABOK) program. W. (Walter) Belgers Mr. Belgers is partner and Principal Security Consultant at Madison Gurkha, a company specialized in technical IT security. He has conducted many security tests on (web)applications, systems and networks, has given trainings and workshops, performed social engineering attacks and more. He likes to share his knowledge with others. His life revolves around security, but occasionally he has time to read, sail or drift in an old BMW. S. (Scott) Barber Many view Scott Barber as the world s most prominent thought-leader in software system performance testing. He has contributed to three books, composed over 100 articles, delivered keynote addresses on five continents, served as the Executive Director of the Association for Software Testing, and co-founded the Workshop of Performance and Reliability and is currently a Director of the Computer Measurement Group. Today, Scott continues his work related to the delivery of world-class system performance and is leading a project he refers to as Business-Centric Assessment of Software Systems. SOCIAL MEDIA Follow Test Automation Day on Twitter with the hashtag #TADNL Keep informed about the latest developments regarding our conference and become a member of our Linkedin Group Test Automation Day and attend the event. PRACTICAL INFORMATION Date and place The 2nd edition of Test Automation Day will be organized on Thursday June 21st 2012 in World Trade Center, Rotterdam (The Netherlands). World Trade Center (WTC) Beursplein AA Rotterdam Participation fee 2012 Visitors of Test Automation Day will pay 295,- (excl. VAT) per person. Members of magazines, newsletters, portals and branche associations which are cooperating with Test Automation Day, will receive a discount of 100,-. After you have received the invoice, payment is required within 14 days. Coffee, tea, lunch and documentation are included. Registration Registration is possible at our website: Two weeks before the conference, you will receive an official confirmation with directions by . If you have not heard from us within 24 hours after your registration, please contact us. Unable to attend? If you are unable to attend, you may ask a colleague to replace you. Provided is the registration confirmation (send by CKC Seminars). Cancellation is possible until 2 weeks before the conference and needs to occur by writing. Cancellation up to 2 weeks before the conference comes with 50,- (excl. VAT) administration costs. In case of cancellation within 2 weeks before the conference, the complete amount will be charged. Conference Organization Esther Pieterse Project Manager esther@ckc-seminars.nl Chiona Nanlohy Project Assistant chiona@ckc-seminars.nl P.O. Box 984, 5600 AZ Eindhoven Telephone: +31(0)

6 founding partner partners Squerist is a leading Dutch consultancy company, specialized in Quality Assurance, Software Testing, Test Automation and Process Management. We focus on quality improvement, cost reduction and closing the gap between Internal Process and IT. Our professionals in Test Automation are qualified in using, implementing and giving training in various test tooling (commercially and open source) for Automated Testing, Performance Testing and Security Testing. We have several partners for Test Automation, which are named on our website. media IBM Parasoft Netherlands B.V. Agile Record NGI Micro Focus QualiTest Computable Software Testing Club sponsors Info Support B.V. TrendIC GIA Softwaretesten.net Microsoft Tricentis NL BV Informatie Testevents.com Remain BV KIVI NIRIA Testing Experience exhibitors NAF, Nederlands Architectuur Forum KZA BV SQS Nederland B.V. with cooperation of Ranorex Ranorex VX Company Automated Testing Institute Innovate IT Thursday June 14th, World Trade Center, Rotterdam Other ICT events for 2012 Landelijk Architectuur Congres (LAC) November 28th & 29th, NBC, Nieuwegein