Break It Before You Buy It!

Transcription

1 Break It Before You Buy It! Test Driven Development and Continuous Integration Chris Hartjes -- CodeMash

2

3 Huge Sports Nerd

4

5 TL;DR

6 The Problem Build us a new fantasy games platform. Oh yeah, you re the only one who will be doing the work. Testing? Can t we just click around on the pages and tell you what we find?

7

8 The Solution Use a framework with a commitment to testing so writing tests will be easier Automate of as much stuff as I can Do some pragmatic TDD Create a great feedback loop

9 Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes a failing automated test case that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new code to acceptable standards. Kent Beck, who is credited with having developed or 'rediscovered' the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.

10 In software engineering, continuous integration (CI) implements continuous processes of applying quality control small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.

11 Someone did a study... (summary of contents of study)

12 15% - 35% 40% - 90%

13 Pay now, or pay later, but YOU WILL PAY

14 Why do I use tests?

15 Make sure I m going in the right direction!

16 Our general process for rails upgrades is pretty consistent. First make the app load in the limited sense, avoiding occurrences of LoadError whilst starting rake. Focus on the minimum number of changes needed to get rake running, then unit, functional and integration tests. The goal is to have tests pass, not to convert to the newest API or experiment with new Rails 3 stuff. You only add what must be added to reach the goal.

17 I am not a Testing Zealot! (Simon The TDD Zealot)

18 I prefer to test things that are complicated 100% test coverage is nice but not practical I m starting to believe in BDD over TDD

19 Behavior driven development (or BDD) is an agile software development technique that encourages collaboration between developers, QA and non-technical or business participants in a software project. It was originally named in 2003 by Dan North as a response to Test Driven Development, including Acceptance Test or Customer Test Driven Development practices as found in Extreme Programming. It has evolved over the last few years.

20 BDD focuses on obtaining a clear understanding of desired software behaviour through discussion with stakeholders. It extends TDD by writing test cases in a natural language that non-programmers can read. Behavior-driven developers use their native language in combination with the ubiquitous language of domain driven design to describe the purpose and benefit of their code. This allows the developers to focus on why the code should be created, rather than the technical details, and minimizes translation between the technical language in which the code is written and the domain language spoken by the business, users, stakeholders, project management, etc.

21 Lettuce -- BDD using Python

22 BDD using PHPUnit <?php require_once 'PHPUnit/Extensions/Story/TestCase.php'; require_once 'BowlingGame.php'; class BowlingGameSpec extends PHPUnit_Extensions_Story_TestCase { /** */ public function scoreforonespareis16() { $this->given('new game') ->when('player rolls', 5) ->and('player rolls', 5) ->and('player rolls', 3) ->then('score should be', 16); } }?> //...

23 Your goal is to build the best application possible with the fewest bugs making it out into the real world

24 Automation is your secret weapon

25

26

27 TDD/BDD and CI add automation to your software development workflow.

28 What I Used PHP Zend Framework Python py.test + PHPUnit

29 public function teststatsbydaterange() { $expectedresults = unserialize(file_get_contents(test_path. '/fixtures/ rfs-1.txt')); $params = array( 'league' => 'testhockey', 'startdate' => ' ', 'enddate' => ' ' ); $this->rfs->getbydaterange($params); $results = $this->rfs->sortbyfantasypoints(); $this->asserttrue(count($results) > 0, 'Checking for results'); $this->assertequals($results, $expectedresults, 'Got expected result set'); $this->asserttrue(isset($results[0]['goals']), 'Checking for goals'); $this->asserttrue(isset($results[0]['assists']), 'Checking for assists'); }

30

31 import py import football_scoring def test_football_empty_data_set(): expected_points = 0 test_data = dict() assert expected_points == football_scoring.qb_points(test_data) assert expected_points == football_scoring.rb_points(test_data) assert expected_points == football_scoring.rec_points(test_data) assert expected_points == football_scoring.k_points(test_data) assert expected_points == football_scoring.def_points(test_data) def test_football_qb_points(): test_data = { 'league': 'football', 'yards': 300, 'interceptions': 1, 'touchdowns': 1, 'two_point_conversions': 1 } expected_points = 15 assert expected_points == football_scoring.qb_points(test_data)

32

33 So how can you test your PHP code and your Python code at the same time?

34

35 OR

36 Do you also want to do this every time the code changes?

37 What is the bare minimum required for continuous integration?

38 Hooks into your VCS

39 Automated running of your tests!

40 TELL ME WHAT WENT WRONG!

41 Buildbot (I highly recommend you read Jacob Kaplan-Moss series on his experiences using Buildbot for Django testing) PROS: It s Python. Infinitely flexible. CONS: It s Python. Infinitely flexible.

42 Cruise Control PROS: pioneer in CI solutions field, easy to find help CON: Unfriendly configuration, GUI shows it s age

43 Hudson (soon to be renamed Jenkins) PROS: Very easy configuration, awesome plugin ecosystem CON: Forces some conventions, writing plugins looks painful for Javaphobes

44 Can you guess which one I picked?

45 No screenshots of Hudson, that s just boring

46 Get yourself a server. Doesn t have to be powerful.

47 Configure it to be as close to your production setup as possible Web server Interpreter or VM Database server Support libraries Testing tools Test data

48 Configure your version control system to talk to Hudson every time you commit code.

49 Plugins I Use, YMMV Hudson Clover (for code coverage reports) Create Job Advanced (tells you who triggered the job) Hudson Extension (better support)

50 More Plugins I Use Subversion plugin xunit Plugin (to support PHPUnit reports)

51 Write a shell script to automatically run your tests for you

52 Brush Up On Your Shell Scripts mkdir /var/www/games-hudson/${build_id} cd ${WORKSPACE}/games /usr/local/zend/bin/php doctrine-cli migrate cd ${WORKSPACE}/games/tests /usr/local/zend/bin/phpunit --log-junit=${workspace}/build/logs/phpunit-results.xml cd ${WORKSPACE}/games/scripts /usr/bin/py.test --junitxml=${workspace}/build/logs/pytest-xmlrunner.xml cp -R /var/lib/hudson/jobs/${job_name}/workspace/games/* /var/www/games-hudson/${build_id} chmod -R 777 /var/www/games-hudson/${build_id}/public/images rm -rf /var/www/games-hudson/current ln -sf /var/www/games-hudson/${build_id} /var/www/games-hudson/current

53 Configure Hudson to send s every time you do a build

54 What Hudson Sends Me Games-Corpdraft - Build # Successful: Check console output at sekretserver.xmlteam.com:9999/job/games- Corpdraft/635/ to view the results. Changes for Build #635 [Chris Hartjes] Fixed date range for week 17

55 What Hudson Shows Me

56 What Hudson Shows Me Started by remote host Reverting svn+ssh://leafs.xmlteam.com/opt/repos/games/trunk Updating svn+ssh://leafs.xmlteam.com/opt/repos/games/trunk U application/configs/application.ini At revision 819 No s were triggered. [workspace] $ /bin/sh -xe /tmp/hudson sh + mkdir /var/www/games-hudson/ _ cd /var/lib/hudson/jobs/games-corpdraft/workspace/games + /usr/local/zend/bin/php doctrine-cli migrate <snipped weirdness from running migrations> + cd /var/lib/hudson/jobs/games-corpdraft/workspace/games/tests + /usr/local/zend/bin/phpunit --log-junit=/var/lib/hudson/jobs/games-corpdraft/workspace/build/logs/phpunit-results.xml PHPUnit by Sebastian Bergmann.... Time: 25 seconds, Memory: 32.25Mb [30;42m[2KOK (36 tests, 119 assertions) [0m[2K+ cd /var/lib/hudson/jobs/games-corpdraft/workspace/games/scripts

57 What Hudson Shows Me + /usr/bin/py.test --junitxml=/var/lib/hudson/jobs/games-corpdraft/workspace/build/logs/ pytest-xmlrunner.xml ============================= test session starts ============================== platform linux2 -- Python pytest test path 1: /var/lib/hudson/jobs/games-corpdraft/workspace/games/scripts test_baseball_scoring.py... test_football_scoring.py... generated xml file: /var/lib/hudson/jobs/games-corpdraft/workspace/build/logs/pytestxmlrunner.xml ========================== 11 passed in 0.07 seconds =========================== + cp -R /var/lib/hudson/jobs/games-corpdraft/workspace/games/application /var/lib/hudson/ jobs/games-corpdraft/workspace/games/build.xml /var/lib/hudson/jobs/games-corpdraft/ workspace/games/doctrine-cli /var/lib/hudson/jobs/games-corpdraft/workspace/games/ doctrine_generator.php /var/lib/hudson/jobs/games-corpdraft/workspace/games/imports /var/ lib/hudson/jobs/games-corpdraft/workspace/games/library /var/lib/hudson/jobs/games- Corpdraft/workspace/games/public /var/lib/hudson/jobs/games-corpdraft/workspace/games/ scripts /var/lib/hudson/jobs/games-corpdraft/workspace/games/tests /var/lib/hudson/jobs/ Games-Corpdraft/workspace/games/tmp /var/www/games-hudson/ _ chmod -R 777 /var/www/games-hudson/ _ /public/images + rm -rf /var/www/games-hudson/current + ln -sf /var/www/games-hudson/ _ /var/www/games-hudson/current Recording test results [xunit] [INFO] - Starting to record. [xunit] [INFO] - Processing PHPUnit-3.4 (default) [xunit] [INFO] - [PHPUnit-3.4 (default)] - 1 test report file(s) were found with the pattern 'build/logs/*-results.xml' relative to '/var/lib/hudson/jobs/games-corpdraft/workspace' for the testing framework 'PHPUnit-3.4 (default)'. [xunit] [INFO] - Converting '/var/lib/hudson/jobs/games-corpdraft/workspace/build/logs/ phpunit-results.xml'. [xunit] [INFO] - Stopping recording. was triggered for: Success Sending for trigger: Success Sending to: sekret@xmlteam.com Finished: SUCCESS

58 What Hudson Shows Me When I Break Something Games-Corpdraft - Build # Failure: Check console output at Games-Corpdraft/611/ to view the results. Changes for Build #611 [Chris Hartjes] Add category to league definition

59 What Hudson Shows Me When I Break Something

60

61

62 Your VCS strategies might have to change

63 Your deployment strategies might have to change.

64 The people testing your application might have to change

65 So why are we doing all this work to automate the process?

66 Remember This? 15% - 35% 40% - 90%

67

68 Questions? Comments? Hate TDD?

69 Thanks! Blog: Slides: