Performance Testing Crash Course

Transcription

1 Performance Testing Crash Course Dustin

2 The performance of your application affects your business more than you might think. Top engineering organizations think of performance not as a nice-to-have, but as a crucial feature of their product. Those organizations understand that performance has a direct impact on user experience and, ultimately, their bottom line. Unfortunately, most engineering teams do not regularly test the performance and scalability of their infrastructure. Join this session to find out about the latest performance testing tools and why your team should add performance testing to your agile development process. 2

3 Dustin Whittle San Francisco, California, USA Technologist, Traveler, Pilot, Skier, Diver, Sailor, Golfer 3

4 What I have worked on Developer Consultant & Developer 4

5 Why does performance matter? 5

6 Microsoft found that Bing searches that were 2 seconds slower resulted in a 4.3% drop in revenue per user 6

7 When Mozilla shaved 2.2 seconds off their landing page, Firefox downloads increased 15.4% 7

8 Making Barack Obama s website 60% faster increased donation conversions by 14% 8

9 Amazon and Walmart increase revenue 1% for every 100ms of improvement 9

10 10

11 Performance directly impacts the bottom line 11

12 12

13 HealthCare.gov HealthCare.gov 13

14 Treat performance as a feature! 14

15 AN EXAMPLE SPRING APP TO TEST PERFORMANCE 15

16 16

17 17

18 GIT CLONE CLOUDFOUNDRY- SAMPLES/HELLO- SPRING- CLOUD 18

19 MVN PACKAGE 19

20 20

21 21

22 CF PUSH 22

23 DEMO.CFAPPS.IO/ 23

24 Tools of the trade for performance testing 24

25 Understand your baseline performance 25

26 Static vs Hello World vs Application 26

27 Apache Bench 27

28 ab -c 1 -t 10 -k 28

29 Benchmarking acmedemoapp.com (be patient) Finished 187 requests Server Software: Nginx/1.6 Server Hostname: acmedemoapp.com Server Port: 80 Document Path: / Document Length: 5217 bytes Concurrency Level: 1 Time taken for tests: seconds Complete requests: 187 Failed requests: 0 Keep-Alive requests: 187 Total transferred: bytes HTML transferred: bytes Requests per second: [#/sec] (mean) Time per request: [ms] (mean) 29

30 ab -c 10 -t 10 -k 30

31 Benchmarking acmedemoapp.com (be patient) Finished 659 requests Server Software: Nginx/1.6 Server Hostname: acmedemoapp.com Server Port: 80 Document Path: / Document Length: 5217 bytes Concurrency Level: 10 Time taken for tests: seconds Complete requests: 659 Failed requests: 0 Keep-Alive requests: 659 Total transferred: bytes HTML transferred: bytes Requests per second: [#/sec] (mean) Time per request: [ms] (mean) 31

32 Siege 32

33 siege -c 10 -b -t 10S 33

34 ** SIEGE ** Preparing 10 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 623 hits Availability: % Elapsed time: 9.57 secs Data transferred: 3.10 MB Response time: 0.15 secs Transaction rate: trans/sec Throughput: 0.32 MB/sec Concurrency: 9.91 Successful transactions: 623 Failed transactions: 0 Longest transaction: 0.30 Shortest transaction:

35 Crawl the entire app to discover all urls 35

36 sproxy -o./urls.txt 36

37 SPROXY v1.02 listening on port appending HTTP requests to:./urls.txt...default connection timeout: 120 seconds 37

38 wget -r -l 0 -t 1 --spider -w 1 -e "http_proxy = " sort -u -o urls.txt urls.txt 38

39

40 Benchmark traffic across all unique urls with siege 40

41 siege -v -c 50 -i -t 3M -f urls.txt -d 10

42 ** SIEGE ** Preparing 10 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 623 hits Availability: % Elapsed time: 9.57 secs Data transferred: 3.10 MB Response time: 0.15 secs Transaction rate: trans/sec Throughput: 0.32 MB/sec Concurrency: 9.91 Successful transactions: 623 Failed transactions: 0 Longest transaction: 0.30 Shortest transaction:

43 Apache JMeter 43

44 44

45 45

46 Multi-Mechanize is an open source framework for performance and load testing 46

47 pip install multi-mechanize 47

48 multimech-newproject demo 48

49 import requests class Transaction(object): def run(self): r = requests.get( r.raw.read() 49

50 import mechanize import time class Transaction(object): def run(self): br = mechanize.browser() br.set_handle_robots(false) start_timer = time.time() resp = br.open( resp.read() latency = time.time() - start_timer self.custom_timers['homepage'] = latency start_timer = time.time() resp = br.open( resp.read() latency = time.time() - start_timer self.custom_timers['cart'] = latency assert (resp.code == 200) 50

51 [global] run_time = 10 rampup = 5 51

52 multimech-run demo 52

53 53

54 What about when you need more than one machine? 54

55 Who lives in the cloud? 55

56 Bees with Machine Guns 56

57 A utility for arming (creating) many bees (micro EC2 instances) to attack (load test) targets (web applications) 57

58 pip install beeswithmachineguns 58

59 # ~/.boto [Credentials] aws_access_key_id=xxx aws_secret_access_key=xxx [Boto] ec2_region_name = us-west-2 ec2_region_endpoint = ec2.us-west-2.amazonaws.com 59

60 bees up -s 2 -g default -z us-west-2b -i ami-bc05898c -k appdynamics-dustinwhittle-aws-uswest-2 -l ec2-user 60

61 Connecting to the hive. Attempting to call up 2 bees. Waiting for bees to load their machine guns Bee i c is ready for the attack. Bee i d is ready for the attack. The swarm has assembled 2 bees. 61

62 bees report 62

63 Read 2 bees from the roster. Bee i c: Bee i d:

64 bees attack -n c 50 -u 64

65 Read 2 bees from the roster. Connecting to the hive. Assembling bees. Each of 2 bees will fire 500 rounds, 25 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. Offensive complete. Complete requests: 1000 Requests per second: [#/sec] (mean) Time per request: [ms] (mean) 50% response time: [ms] (mean) 90% response time: [ms] (mean) Mission Assessment: Target crushed bee offensive. The swarm is awaiting new orders. 65

66 bees attack -n c u 66

67 Read 2 bees from the roster. Connecting to the hive. Assembling bees. Each of 2 bees will fire rounds, 500 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. Offensive complete. Complete requests: Requests per second: [#/sec] (mean) Time per request: [ms] (mean) 50% response time: [ms] (mean) 90% response time: [ms] (mean) Mission Assessment: Target crushed bee offensive. The swarm is awaiting new orders. 67

68 Read 2 bees from the roster. Connecting to the hive. Assembling bees. Each of 2 bees will fire rounds, 500 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. Bee 0 is joining the swarm. Bee 1 is joining the swarm. Bee 0 is firing his machine gun. Bang bang! Bee 0 lost sight of the target (connection timed out). Bee 1 lost sight of the target (connection timed out). Offensive complete. Target timed out without fully responding to 2 bees. No bees completed the mission. Apparently your bees are peace-loving hippies. The swarm is awaiting new orders. 68

69 bees down 69

70 Read 2 bees from the roster. Connecting to the hive. Calling off the swarm. Stood down 2 bees. 70

71 locust.io 71

72 PIP INSTALL LOCUSTIO 72

73 73

74 74

75 LOCUST HOST= 75

76 76

77 77

78 There are many load testing tools.. Gatling.io Wrk Vegeta Tsung 78

79 What about the client side? 79

80 In modern web applications more latency comes from the clientside than the server-side. 80

81 Google PageSpeed 81

82 Google PageSpeed Insights 82

83 83

84 84

85 WBench

86 gem install wbench 86

87 wbench 87

88

89 Automate clientside performance testing with Grunt

90 Use Bower (for dependencies), Grunt/Gulp (for automation), and Yeoman (for bootstrapping) (checkout jhipster too) 90

91 How many people understand exactly how fast their site runs in production? 91

92 Track performance in development and production 92

93 Instrument everything = code, databases, caches, queues, third party services, and infrastructure. 93

94 Statsd + Graphite + Grafana 94

95 95

96 Track performance of end users 96

97 97

98 webpagetest.org 98

99 99

100 100

101 101

102 SiteSpeed.io 102

103 103

104 104

105 105

106 106

107 107

108 108

109 Load testing services from the cloud 109

110 110

111 111

112 112

113 Test for failures NetFlix Simian Army + Chaos Monkey What happens if you lose a caching layer? What happens if dependencies slow down? 113

114 Best Practices Treat performance as a feature Capacity plan and load test the server- side Optimize and performance test the client- side Understand your starting point Instrument everything Monitor performance in development and production Measure the difference of every change Automate performance testing in your build and deployment process Understand how failures impact performance 114

115 Integrate automated performance testing into continuous integration for server-side and client-side 115

116 Understand the performance implications of every deployment and package upgrade 116

117 Monitor end user experience from end to end in production 117

118 Questions? 118

119 Find these slides on SpeakerDeck 119