2011/09/12 Satoshi Masuda, Director, ASTER Challenges in Japanese Software Testing Industry and Test Automation Association of Software Test Engineering
Contents 1. Introduction 2. Facts 3. Problems and Concerns 4. Challenges 5. Test Automation & Agile Testing 6. Conclusion 2
1. Introduction Self Introduction Mr. Satoshi Masuda Manager of Testing Services, IBM Japan Ltd., Director Association of Software Testing Engineers (ASTER), Satoshi Masuda has been involved in Information Technology for 20 years. During this time he has worked for enterprise application software development and software engineering. Now he is leading Testing Services for the customer in Japan. He also has been participated NPO ASTER in Japan since 2005. He contributed for establishing Asian Software Testing Alliance (ASTA) among Asian countries. 3
1. Introduction Session objectives 1. To allow attendees to see bigger picture about testing industry in Japan. 2. To share perspectives for anyone who wants to introduce test automation into the team/organization and the ones who wants to improve effectiveness of test automation 4
2. Facts Atmospheres in Japanese economy Lost decades since the collapse of bubble economy Billion US$ GDP USA Japan China 5 Source: IMF - World Economic Outlook(Apr, 2011)
2. Facts Atmospheres in Japanese industry Number of workers not Grow Workers Getting Old million 120.0 100.0 6 80.0 60.0 40.0 20.0 0.0 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 Age 60-50 40 30 20 Source: Ministry of Internal affair Japan
2. Facts Pessimistic atmosphere in Japan But, Software Engineers have high motivation Take a look Software Industry.. 7
2. Facts in Information Services Industry Information Services Industry Facts Industry population Industry structure Vendor Structure Contract Structure Embedded Systems, Enterprise Systems Out Source 8
2. Facts in Information Services Industry Total number of information services Employee and Revenue Total Revenue (Trillion JPY) Total Revenue Total number of Emp Total number of Emp (Ten Thousands) 9 Source: Japan Information Services Agency, 2011 From Ministry of Economy, Trade and Industry
2. Facts in Information Services Industry Types of Information Services Types of Industry Other Information Services Internet Services Information Services Industrial System Operation Custom Software Development Others Financial, Insurance Data center services Other SW Dev SoftwareProduct Distribution Public Information Communication 10 Source: Japan Information Services Agency, 2011 From Ministry of Economy, Trade and Industry
2. Facts in Information Services Industry Structure of software industry Enterprise software VS. Embedded software Enterprise system e.g. Banking system SCM system, etc. Embedded system e.g. Video recorder Car navigation system, etc. 11
2. Facts in Information Services Industry Structure of Information Services Vendor Multiple Vendor Structure Customer Sub Vendor Primary Vendor Sub Vendor Hitachi, NEC, Fujitsu, NTT Data, IBM Japan etc. Sub Vendor Sub Sub Vendor Sub Sub Vendor 12
2. Facts in Information Services Industry Off shore Out Sourcing (Million JPY) Total Amount of Off shoring (in 288Companies) 1.5 Billion US$ = (1US$ = 80 JPY) 13 Source: IT Human Resource White Paper 2010 Information-Technology Promotion Agency Japan
2. Facts in Information Services Industry Off shore Out Sourcing Percentage of offshore country China India Vietnam Philippine US,Canada South Korea Others 14 Source: IT Human Resource White Paper 2010 Information-Technology Promotion Agency Japan
2. Facts in Software Testing Software Testing Market Perspective 15 In enterprise, market exists over testing and development In embedded, some market exists on testing Testing Developm ent Enterprise Software market Embedded Software market market market
3. Problems and Concerns 1. Quality Improvements There were Many troubles 1. Mega Bank system troubles Performance trouble, when huge number of transactions came http://www.asahi.com/english/tky201103180212.html 16
3. Problems and Concerns 1. Quality Improvements There were Many troubles 2. Railway Ticket gate system trouble Programming error Ticket gate glitch hits over 600 Tokyo rail stations. TOKYO, Oct. 12 Kyodo Around 4,400 automated ticket gates at 662 railway and subway stations in the Tokyo area failed to operate properly early Friday morning when they were turned on for the first train runs of the day, prompting operators to give passengers free access at almost all stations by around 11 a.m. to avoid rush hour confusion, the operators said, adding that train runs had not been disrupted. The operators said the extensive malfunction is suspected of having been caused by a glitch in the programming that links gates and computer servers which handle the data for IC fare cards -- Suica for JR East passengers and Pasmo for other metropolitan subway and rail operators. All the affected gates were built by Nippon Signal Co. http://www.thefreelibrary.com/5th+ld%3a+ticket+gate+glitch+hits+over+600+tokyo+rail+stations.-a0172104413 17
3. Problems and Concerns 2. Cost Reduction Testing Costs Costs = Unit Price * Person Month Unit price reduction by outsourcing Person Month reduction by Standardization, Simplify, Tools (include productivity up) 18
3. Problems and Concerns 3. Support business agility Businesses move faster, software should move more In order to move fast Innovation? Cloud Process? Agile Process. Scrum Human Resources? Skills up 19
4. Challenges in Software Testing What do Japanese challenge in Software Testing? Japan Symposium on Software Testing (JaSST) Biggest Testing Symposium in Japan http://jasst.jp/ By researching software testing event, What Japanese doing: 1. Innovation (Testing Technique) 2. Education 3. Test Automation 20
4. Challenges in Software Testing Challenges in testing area for the Problems and Concerns Problems and concers 1.Quality Improvement 2.Costs reduction 3.Support business agility Challenges in testing 1.Innovation in in Testing Technique Methodology 2. 2. Education 3.Test Automation Test Automation Agile Testing 21
4. Challenges in Software Testing 1. Innovations in software testing area Mapping software testing technique for us to select appropriate ones Testing Technique Map Requirements Black Box Ad hoc testing Exploratory Testin Alternative Use Case Random testing Use Case α βtesting System Business Scenario Requirements Single function Orthogonal Array, Pair-wise, Testing testing HAYST technique Specified Risk Base Testing Testing by Statistical data Specify Dynamic testing Defect by results ofstatic testing hypothesis Specification Equivalence partitioning Boundary Analysis Decision Table Cause Result graph Case Flow Diagram State change testing Covered Analyze Specula Error guessing Timing testing Model Chcking 22 Design White Box Source: Koichi Akiyama, Fuji Xerox
4. Challenges in Software Testing 1. Innovations in software testing area Developing Software Testing Methodologies to get testing efficiently and effectively. A example of methodologies 1. Identify test objectives by using Mind Map 2. Break down into details 3. Grouping the objectives 23 Source http://jasst.jp/archives/jasst11e/pdf/a4-3.pdf Suzuki, SMG
4. Challenges in Software Testing 2. Education Develop software testing skills standard for education. Test.SSF (Test Skill Standard Framework) Test.SSF consists of 1. Layers of software engineering 2. Break down into detail skills Test.SSF 24 Source http://jasst.jp/archives/jasst10e/pdf/a2.pdf, Suzuki, ASTER
Challenges in Software Testing Test Automation Association of Software Test Engineering
Challenges in Software Testing 5. Test Automation Test Automation is a key solution for the problems Approach to test automation 1. Develop Test Automation Plan 2. Prepare Test Automation 3. Execute Automated Test 4. Maintain Test Automation 26
Challenges in Software Testing 5. Test Automation Two types test automation 1. Test execution automation Test case execution, compare results 2. Test management automation Progress management, Defect management, etc. 27
Challenges in Software Testing 5. Test Automation Which test to be automated? Types of Tests Non-Functional Functional Test 28 Which to be tested? Which can be automated? Examples Auto Function test Regression Test Interface Test Performance Test Security Test Recovery test should be tested can be tested Unit Test Auto Auto Integratio n test Auto Auto System test Levels of Tests Auto Auto Auto Auto System Integratio n Test Acceptan ce Test Operabili ty Test Auto
Challenges in Software Testing 5. Test Automation Test Automation Steps Analyze Test Cases Design Test Automatio n Develop Automatio n Scripts Execute Automate d Scripts Maintain Test Scripts Test Cases Test execution automation Analyze Test cases Repeatable test cases Automated test cases Repeatable Test Cases Automated Test Cases 29
Challenges in Software Testing 5. Test Automation Design and Maintain Test Scripts for Capture & Reply Tools Reusable Test Scripts Test Scripts captured by V.1.0 Test Scripts run for V2.0? Version 1.0 Version 2.0 Tester Sees Tool Sees 30
Challenges in Software Testing 5. Test Automation Design and Maintain Test Scripts for Capture & Reply Tools For Reusable Test Scripts 1. Minimize influences from the change of test object by componentized scripts Capture in one script 1. Login 2. Select 3. Input 4. Close 1. Logon 2. Select 3. Input 4. Close Componentized script Main Call 1 Call 2 Call 3 Call 4 1. Login 2. select 3. Input 4. Close Main Call 1 Call 2 Call 3 Call 4 1. Logon 2. select 3. Input 4. Close 31
Challenges in Software Testing 5. Test Automation - Agile Testing Agile process is getting popular. Test Automation is a key factor in Agile process Agile Testing, (Lisa Crispin, Janet Gregory) Agile Testing, (Lisa Crispin, Janet Gregory) Agile Test = Test in Agile team 32 Agile Testing translated into Japanese (Satoshi Masuda)
Challenges in Software Testing 5. Test Automation - Agile Testing The Agile Manifesto a statement of values Individuals and interactions over Process and tools Working software Customer collaboration Responding to change over over over Comprehensive documentation Contract negotiation Following a plan 33 Source: www.agilemanifesto.org
Challenges in Software Testing 5. Test Automation - Agile Testing Agile Test = Test in Agile team Functional Team Agile Team Programmer Business Analyst Programmer Business Analyst Gate Keeper Tester Gate Keeper from Functional team to Agile Team Tester Whole Team Approach 34
Challenges in Software Testing 5. Test Automation - Agile Testing Agile Testing Quadrant Agile Testing Quadrant is category for Agile Testing User Requirements points of view Support to get Team Efficiency, Quality up Critique points of view 35 Programming, Non-functional points of view * Agile Testing P.98 Figure6.1
Challenges in Software Testing 5. Test Automation - Agile Testing Agile Testing Quadrant Which can be automated? Unit Tests Functional Tests Test Data Generation Load Tests Continuous Integration Which should NOT be automated? Usability Tests Exploratory Tests Test Automation 36
Challenges in Software Testing 5. Test Automation - Agile Testing Quadrant 1 Tool - xunit Agile Testing is based on Unit Test Unit tests are automated by xunit tools, for example, JUnit, djunit and etc. Unit Test is automated in Build Automation. Example Eclipse and JUnit/djUnit Report Generation 1. Coding test scripts on IDE 2. Unit Test is automated in Build Automation 37 3. Feed Back Reports from JUnit or djunit
Challenges in Software Testing 5. Test Automation - Agile Testing Quadrant 2 Tool - Fit/FitNesse Test Automation in Business-facing Supporting team. By using FitNesse, we can describe test cases, data and check results on Wiki. Business Analyst Programmer Develop Check Check テスト 記 述 ファイル(html) テスト 結 果 ファイル(html) Input FitNesse: Wiki with Fit By using FitNesse, we can describe test cases, data and check results on Wiki. Outoput Fit Runner Test Automation in Build Cycle Test Execution Fixtu re System / Program 38 開 発
Challenges in Software Testing 5. Test Automation - Agile Testing Quadrant 4 Tool Performance & Load testing Security testing ility testing. Usability, Scalability, etc. By using tools, test can be automated Vendor tools Open source tools 39
6. Conclusion What we discussed: 1. Atmosphere in Japan 2. Software Industry structure Vendor and Contract Structure Embedded Systems, Enterprise Systems Out Source 3. Problems and Concerns 4. Challenges Innovation Education 5. Test Automation Agile Testing 40
תוד ה Hindi Russian Hebrew Arabic Grazie Italian Traditional Chinese Thai Simplified Chinese Thank You English Gracias Obrigado Brazilian Portuguese Spanish Danke German Merci French Tamil Japanese Korean 41