Recruiting, motivating and energizing superior test engineers

Transcription

1 Recruiting, motivating and energizing superior test engineers Jeffrey Feldstein Cisco Systems 1

2 Agenda Introduce myself Common Quality Problems Classic Test Engineers (Manual & Scripting) Ideal Test Engineer is a Software Engineer Developer s perception of testers Organizational Support Requirements 8 reasons Test is better for a Software Engineer Alternative Approaches 2

3 Agenda (con t) Retention Tips Common issues Conclusion Questions and Answers 3

4 Introduce Myself Software Career: 25 years Software Test: 7 years Test Automation: 6 years Current position at Cisco: Manage a group of 65 test engineers located in San Jose, India (2 sites) and Netanya, Israel, testing a next generation Network Management Platform (Active Network Abstraction) 4

5 Who talked me into testing? Come Back! Avoided Live testing Santa at all cost Barbara! 1997 Left Cisco to stay in Santa Barbara Will you test? 5

6 What are some common quality problems? 6

7 Common Quality Problems Bugs found too late Regression bugs introduced Insufficient Performance Poor scaling Poor response to stress Poor long-term stability (soak performance) 7

8 Classic Testers Manual Testers Come from the problem domain or business background Often are experts in the product May understand the customer and application very well Can get very good at designing test cases Can be creative around ways of killing the project Not necessarily software engineers or computer scientists 8

9 Scripters The machine mimics the human Picked up a book on Perl or other scripting language Record/Playback on a GUI tool Scripting a GUI tool 9

10 Scripting Issues Often does what a human can do, only more exacting with less imagination. Sometimes faster than a human, sometimes not Not Scalable Little capability for data driven Error detection and recovery is difficult Scale is difficult Reuse is difficult 10

11 Ideal Test Engineer is a software engineer Ensures the system/application/product is designed for testability Ensures proper (developer performed) unit testing Designs and builds test automation that is at least as sophisticated as the system under test Can simulate the customer environment 11

12 What stereotypes to developers carry about test engineering? 12

13 Developer s perception of test engineering Testers are dumb Testing is boring, manual and repetitive Testing is not creative No innovation takes place in testing Test is a necessary evil standing between my brilliance and the user Testing is not a career Test engineers are pencil-pushing quality/process geeks Test conferences are never fun 13

14 Organizational Support Requirements Test Engineers must: Be paid equally to developers Have a career track within the company comparable to developers Have an equal say with all other organizations making product decisions Cannot be treated as a service organization to development Be independent of the development organization 14

15 Organizational Support Requirements (con t) Test Engineers must: Be co-located with the development engineers Viewed as development peers by upper management Not be the sole group in charge of Assuring Quality 15

16 Software Engineer Software testing software Designs sophisticated software to perform not only functional tests but also develops advance software that stresses the system. 16

17 Test is Better Recruiting point 1 Sophisticated Software You still design and develop highly engineered software, sophisticated, error resilient. It's only the purpose of the software that is different. Instead of creating a small slice of the application you are developing a complete system to verify it s functionality, scalability, reliability, etc. 17

18 Test is Better Recruiting point 2 You decide what to build In today s market-driven software business most software developers are building systems that need to meet a set of requirements published by a marketing or product management team. New features and capabilities must first be sold to the product managers before it can be put into a product. In test, you show your boss how the software will increase productivity or quality then you build your system. 18

19 Test is Better Recruiting point 3 Small Teams Since teams are smaller, you tend to work on a bigger chunk of a project (or maybe a whole project) instead of just a small slice. In addition you tend to have a better perception of the big picture. 19

20 Test is Better Recruiting point 4 Creativity Since developers are typically building what a product manager specifies, their creativity is usually focused around implementation. Test engineers can be creative around not only how something is implemented but on what they build as well. Two common areas for creativity is in the tooling required for testing and new creative ways of testing (i.e. MBT & Automatic Test Generation) 20

21 Test is Better Recruiting point 5 You get to break your buddy's code. 21

22 Test is Better Recruiting point 6 Innovation Test automation, especially highly-engineered automation, is a much newer (and therefore less crowded field) than pure software development, which allows for more room for innovation. 22

23 Test is Better Recruiting point 7 Customer Interaction & Business knowledge The big picture view leads to more customer interaction, through support and beta test contacts. This is often rewarding in itself but also can lead to a better understanding of the business you are in and therefore positions you for future business leadership 23

24 Test is Better Recruiting point 8 Did I mention you get to break your buddy's code? 24

25 Recruiting Issues You will not win over every engineer with this argument Some salesmanship is required But: Developers don t win everybody either. 25

26 Alternate Approach #1: Interns Often ecstatic to get their first real job Economical Highly motivated High energy Can bring new, un-orthodox solutions Upon graduation, convert to full time May find they enjoy developing automation 26

27 Alternate Approach #2: Test first, develop later Works best at entry level Sell the position as gaining an overall understanding of the product and product quality. Still get to program After a specified time, they can move to development if they wish We have found that enough people like test after months that they do not wish to move 27

28 Retention Tips The leader must enjoy testing and their work Make time for training activities Problem Domain Software Engineering skills Business Skills Management/Leadership 28

29 Retention Tips (con t) Involve test engineering very early in the product life If development is researching or training a new technology or architecture, always include test engineers. 29

30 Common Issues I don t get the respect I deserve from the developers I am often not invited to development meetings Development wants me to perform the xxx boring task 30

31 Conclusion The best test engineers are software engineers Organizationally, elevate test engineering to the same level as software engineering Test engineering should be it s own career path It s OK to move talent between test engineering and software engineering 31

32 /mbt 32