How To Write Unit Tests In A Continuous Integration

Transcription

1 Continuous Integration

2 1. It works on my machine.

3

4

5 Risk 1 Lack of Deployable Software

6

7

8 Risk 2 Lack of project visibility

9 2011 CTG, Inc. 9

10 2011 CTG, Inc. 10

11 Risk 3 Low quality software

12

13

14 Risk 4 Late discovery of defects

15 All these risks are related to Old School Development Practices

16 Old School Development Write Code Developer Write Code Write Code Builds sources Developer Lead Developer Commit Code Get latest sources Version Control System

17 Manual Effort Slow Feedback Debugging Dependencies

18

19 Continuous Integration 2011 CTG, Inc. 19

20 Build software better, faster, cheaper

21 Development practice where team members integrate their work daily verified by automated builds using different testing types

22 Development practice where team members integrate their work daily verified by automated builds using different testing types

23 3 building blocks of continuous integration

24 1. Version Control System

25 </CODE> </CODE> </CODE>

26 2. Automated Builds

27 </CODE> </CODE> </CODE>

28 3. Team Agreement

29 </CODE> </CODE> </CODE>

30 7 Principles of Continuous Integration

31 1. Commit code frequently

32 Check in at least once a day Avoid the 5PM check in mess Small Steps are beter than one big leap Check in after each task

33 But, we can check in as often as we want

34 Continuous Integration without testing = Continuous Compilation

35 1. Commit code frequently 2. Write automated unit tests

36

37

38

39 </CODE> </CODE> </CODE>

40 </CODE> </CODE> </CODE>

41 </CODE> </CODE> </CODE>

42 1. Commit code frequently 2. Write automated unit tests 3. Run private builds

43 </CODE> </CODE> </CODE>

44 </CODE> </CODE> </CODE>

45 </CODE> </CODE> </CODE>

46 1. Commit code frequently 2. Write automated unit tests 3. Run private builds 4. Don t commit broken code

47 </CODE> </CODE> </CODE>

48 </CODE> </CODE> </CODE>

49 </CODE> </CODE> </CODE>

50 1. Commit code frequently 2. Write automated unit tests 3. Run private builds 4. Don t commit broken code 5. Fix broken build immediately

51 You Break It (The Build) You Buy It (Round of Beer) You Fix It (Quickly)

52 1. Commit code frequently 2. Write automated unit tests 3. Run private builds 4. Don t commit broken code 5. Fix broken build immediately 6. Avoid getting broken code

53

54 1. Commit code frequently 2. Write automated unit tests 3. Run private builds 4. Don t commit broken code 5. Fix broken build immediately 6. Avoid getting broken code 7. All tests and inspections should pass

55 </CODE> </CODE> </CODE>

56 </CODE> </CODE> </CODE> <CODE ANALYSIS>

57 Development practice where team members integrate their work daily verified by automated builds using different testing types

58 one team

59 one team Developers

60 one team Testers Developers

61 Development practice where team members integrate their work daily verified by automated builds using different testing types

62 in CI speed matters

63 UI Test Execution Time Service/API Unit

64 Code Quality

65

66

67

68

69

70 Unit Testing

71 1 Unit 99,5% reliable 1 system = 50 Units 99,5% 50 = 78% reliable

72

73 Let s take a look at a Unit Test

74 A (very) simple class

75 Some Unit Tests

76 About Unit Testing Verify behavior of small elements in a software system, most often a single class Occasionally more than 1 class in case of tightly coupled classes No connections to DB, filesystem, Code Coverage per test is generally limited Should be very fast Crucial Component of CI, unit tests run in every build CI works well with TDD

77 Where in CI? Developer Feedback Mechanism Commit Code + Unit Tests Polls for changes CI Server VCS Build Server Builds sources Unit Tests

78 Build the code RIGHT Build the RIGHT code 2013 CTG, Inc. 78

79 About Unit Integration/Acceptance Testing Also a white box test May cross architectural boundaries More code coverage per test Longer running duration than Unit Tests Doesn t run in every build

80 Where in CI? Developer Send Feedback Feedback Mechanism Polls for changes CI Server Commit Build (Lightweight) Builds sources Unit Tests Commit Code + Unit Tests VCS If Commit Build Succesful Secondary Build (Heavyweight) Unit Integration Tests

81 Some Namedropping JUnit NUnit MS Tests TestNG DBUnit

82 Unit Performance Testing

83 JUnitPerf example

84 What to unit performance test?

85 2013 CTG, Inc. 85

86 2013 CTG, Inc. 86

87 Where in CI? Developer Send Feedback Feedback Mechanism Commit Code + Unit Tests VCS Polls for changes CI Server If Commit Build Succesful Commit Build (Lightweight) Builds sources Unit Tests Secondary Build (Heavyweight) Unit Integration Tests Unit Performance Tests

88 Some Namedropping JUnitPerf JUnitBench ContiPerf NUnitPerf

89 API/Service Testing

90 FitNesse

91 Simple Example

92 Run Test Case Red

93 Fixture Red

94 The actual Code Red

95 Run Test Case Red

96 Fix Code Red

97 Run Test Case Green

98 Refactor Code Refactor

99 More advanced example

100 SoapUI

101 Jmeter

102 Some Namedropping FitNesse Cucumber Concordion SOAPUI JMeter HP ServiceTest

103 Where in CI? Developer Send Feedback Feedback Mechanism Commit Code + Unit Tests VCS Polls for changes CI Server Commit Build (Lightweight) Builds sources Unit Tests Secondary Build (Heavyweight) Unit Integration Tests Unit Performance Tests Service/API Tests

104 GUI Testing

105 We re not done yet! GUI Automated Testing Selenium HP Unified Functional Testing - FASTBoX MS Coded UI Testing E2E Performance Testing JMeter HP LoadRunner MS Visual Studio RadView WebLoad Dynamic Security Testing HP WebInspect HP QAInspect

106 Where in CI? Developer Send Feedback Feedback Mechanism Commit Code + Unit Tests VCS Polls for changes CI Server Nightly Build GUI Automated E2E Performance Dynamic Security Commit Build (Lightweight) Builds sources Unit Tests Secondary Build (Heavyweight) Unit Integration Tests Unit Performance Tests Service/API Tests

107 Mapping Tests on Build Types

108 Builds Strategy Lightweight Commit Build Unit Tests Secondary Build (Heavyweight) Unit Integration Unit Performance API/Service Test Automated UI Smoke Tests Nightly Build Automated UI Tests Static Testing Static Application Security Testing Dynamic Application Security Testing Release Build Automated UI Tests Manual Tests

109 Manual Testing & CI

110

111

112 HP ALM: Application Lifecycle Intelligence

113 HP ALM: Build Reports

114 MS Test Manager: Recommended Tests

115 one team Testers Developers

116 one team Testers Operations Developers

117

118

119

120 120

121 Deployment pipeline

122 Reduce project Risks

123 Reduce repetitive manual processes

124 Generate deployable software

125 Remember our project risks?

126 Risk 1 Lack of Deployable Software

127

128

129 Risk 1 Lack of Deployable Software Risk 2 Lack of Project Visibility

130 Heatmap with LoC (Size), and Code Coverage (Color)

131

132 Risk 1 Lack of Deployable Software Risk 2 Lack of Project Visibility Risk 3 Low Structural Quality

133 Violations per category + trend

134 Level of Code Duplication

135 Risk 1 Lack of Deployable Software Risk 2 Lack of Project Visibility Risk 3 Low Structural Quality Risk 4 Late Discovery of Defects

136 Builds Strategy Lightweight Commit Build Unit Tests Secondary Build (Heavyweight) Unit Integration Unit Performance API/Service Test UI Smoke Tests Non-Functional Build (Nightly/Weekly) E2E Performance Testing Static Testing Static Application Security Testing Dynamic Security Testing Release Build (2 weekly) Manual Testing UI Tests

137 Risk 1 Lack of Deployable Software Risk 2 Lack of Project Visibility Risk 3 Low Structural Quality Risk 4 Late Discovery of Defects

138 2013 CTG, Inc. 139

139 CI is an attitude NOT a tool