CSE 70: Software Development Pipeline Build Process, XML, Repositories

Transcription

1 CSE 70: Software Development Pipeline Build Process, XML, Repositories Ingolf Krueger Department of Computer Science & Engineering University of California, San Diego La Jolla, CA , USA California Institute for Telecommunications and Information Technologies La Jolla, CA , USA

2 Learning Goals for Today Ingolf Krueger, 2010 CSE 2

3 Learning Goals Be able to identify key elements of the software development pipeline Understand a basic Java program and how it is compiled/run Be able to change a given program to introduce functions and additional classes that can evolve separately Understand the need for a build tool See Ant used to automate compilation and other tasks Be able to write basic Ant files yourself Understand the challenges of maintaining a code base in an individual and team development setting Understand the need for and basic principles of version control/ configuration management Ingolf Krueger, 2010 CSE 3

4 Key elements of the software development pipeline Ingolf Krueger, 2010 CSE 4

5 Software Development Pipeline: Requirements System Ingolf Krueger, 2010 CSE 5

6 Stage 0: Individual, using an IDE Ingolf Krueger, 2010 CSE 6

7 Stage 1: Individual, using IDE, Test and Build Tools Ingolf Krueger, 2010 CSE 7

8 Stage 2: Individual, w/ide, Local Build/Test Tools and Version Control Ingolf Krueger, 2010 CSE 8

9 Stage 3: Small Team w/ide, Test, Build, Version Control Ingolf Krueger, 2010 CSE 9

10 Stage 4: Small to Medium Teams w/automated Build Process Ingolf Krueger, 2010 CSE 10

11 Stage 4: Small to Medium Teams w/automated Build Process We ll cover all of these over the next few lectures and labs Ingolf Krueger, 2010 CSE 11

12 Stage 4: Small to Medium Teams w/automated Build Process Today s focus: Build Tools & Basics of Version Control Ingolf Krueger, 2010 CSE 12

13 Stage 0 to Stage 1 Why? Ingolf Krueger, 2010 CSE 13

14 Write a program that formats and outputs a given String Hello! H-e-l-l-o-!-! Ingolf Krueger, 2010 CSE 14

15 We want to change the Formatter and the main program separately: Introduce a StringFormatter class Ingolf Krueger, 2010 CSE 15

16 1 st attempt: introduce a dedicated class! package today;! StringFormatter.java! public class StringFormatter {! StringFormatter(){};! public String formatstring(string in) {! char[] cv = new char[2*in.length()];! for(int i = 0; i < cv.length; i+=2) {! cv[i] = in.charat(i/2);! cv[i+1] = - ;! }! return new String(cv);! }! }! Ingolf Krueger, 2010 CSE 16

17 1 st attempt: introduce a dedicated class! package today;! Printer.java! public class Printer {! public static void main(string[] args) {! StringFormatter sf = new StringFormatter();! String out = sf.formatstring("hello!");! System.out.println("formatted: "+out);! }! }! Ingolf Krueger, 2010 CSE 17

18 Looks good! Let s get it compiled! By now we have TWO Java source files: Printer.java! StringFormatter.java! In what order do we need to compile them? Ingolf Krueger, 2010 CSE 18

19 Java Compile Command (command line) 1 javac -d bin -cp bin src3/stringformatter.java! 2 javac -d bin -cp bin src3/printer.java! Ingolf Krueger, 2010 CSE 19

20 Introduce symbolic names for special characters Ingolf Krueger, 2010 CSE 20

21 2 nd attempt: add a class for symbolic names! package today;! CharConstantHolder.java! public abstract class CharConstantHolder {! }! public static char SPACE = ' ';! public static char DASH = '-';! public static char STAR = '*';!...! public class StringFormatter {!...! }! cv[i] = in.charat(i/2);! cv[i+1] = CharConstantHolder.STAR;!...! StringFormatter.java! Ingolf Krueger, 2010 CSE 21

22 Looks good! Let s get it compiled! By now we have THREE Java source files: Printer.java! depends on 3 CharConstantHolder.java! depends on 1 StringFormatter.java! 2 In what order do we need to compile them? Ingolf Krueger, 2010 CSE 22

23 Java Compile Command (command line) 1 javac -d bin -cp bin src4/charconstantholder.java! 2 javac -d bin -cp bin src4/stringformatter.java! javac -d bin -cp bin src4/printer.java! 3 Ingolf Krueger, 2010 CSE 23

24 Can we avoid manual recompiles and automate other tasks as well? Yes! Enter: Ant Ingolf Krueger, 2010 CSE 24

25 What is Ant? Apache Ant is an open source Java-based Build tool Works across platforms written in Java, input: XML Interlude: XML Ingolf Krueger, 2010 CSE 25

26 extensible Markup Language (XML) XML is a set of rules/guidelines for describing structured data in plain text XML supports creation of domain-specific languages that add meaning to data Start tag Example! <project>! <name> week2 </name>! <basedir>. </basedir>! <default> compile </default>! </project>! End tag Ingolf Krueger, 2010 CSE 26

27 XML: Tags and Elements XML is a set of rules/guidelines for describing structured data in plain text XML supports creation of domain-specific languages that add meaning to data Start tag Example! <project>! <name> week2 </name>! <basedir>. </basedir>! <default> compile </default>! </project>! Element/Data End tag Ingolf Krueger, 2010 CSE 27

28 XML Example XML is a set of rules/guidelines for describing structured data in plain text XML supports creation of domain-specific languages that add meaning to data <project>! <name> week2 </name>! <basedir>. </basedir>! <default> compile </default>! </project>! Example! Ingolf Krueger, 2010 CSE 28

29 XML Example XML is a set of rules/guidelines for describing structured data in plain text XML supports creation of domain-specific languages that add meaning to data <project>! <name> week2 </name>! <basedir>. </basedir>! <default> compile </default>! </project>! Example! Ingolf Krueger, 2010 CSE 29

30 In short: tags add meaning to data Ingolf Krueger, 2010 CSE 30

31 Writing Data Structures in XML vs. Java <project>! <name> week2 </name>! <basedir>. </basedir>! <default> compile </default>! </project>! Example.xml! class Project {! private:! Name name;! Basedir basedir;! Default default;!! }! vs. class Name {! private:! String s;!! }! class Default {! private:! String d;!! }! class Basedir {! private:! String bd;!! }! Ingolf Krueger, 2010 CSE 31

32 <project>! <name> week2 </name>! <basedir>. </basedir>! <default> compile </default>! </project>! Alternative Styles <project name = week2 basedir =. default = compile >! </project>! vs. <project name = week2 basedir =. default = compile />! vs. Attribute Ingolf Krueger, 2010 CSE 32

33 Example Write an XML file that holds Customer information using suitable tags and data entries: The customer s first and last name, their phone number, and their address! Example customer: Bilbo Baggins, , Bilbo.Baggins@lor.com Ingolf Krueger, 2010 CSE 33

34 One Solution Customer.xml! <customer>! <phone> </phone>! <first_name> Bilbo </first_name>! < > </ >! <last_name> Baggins </last_name>! </customer>! Ingolf Krueger, 2010 CSE 34

35 XML Pros and Cons XML files can be self-documenting via well-chosen tags Easily extendible Broadly applicable and applied communication standards B2B data exchange For small files: all you need is a text editor Tags/tag syntax add complexity to the file XML files can become unwieldy; special editing software can mitigate this Performance loss Ingolf Krueger, 2010 CSE 35

36 Back to Ant and its build files Ingolf Krueger, 2010 CSE 36

37 What is Ant? Apache Ant is an open source Java-based Build tool Works across platforms written in Java, input: XML Basic idea: 1. Read in a project specification (the build.xml file), containing Targets: A set of tasks you want to automate Each target can depend on multiple other targets Tasks (something you want to get done automatically), examples: Compile a set of Java files Create, delete, move a file/directory Create a jar file a zipped archive of your source 2. Determine dependencies among targets and within tasks 3. Execute all required tasks following the order determined in 2. Ingolf Krueger, 2010 CSE 37

38 Remember.java.class other input build.xml Ant output output files files other side-effects create/delete files create/delete dirs. program execution Ingolf Krueger, 2010 CSE 38

39 Ant Build File: Project Base directory, relative to location of build.xml build.xml! <project name = "week2" basedir = "." default = "compile">!!<target name = "compile">!!!<javac srcdir = "src4"!!!! destdir = "bin"/>!!</target>! </project>! Default target Ingolf Krueger, 2010 CSE 39

40 Ant Build File: Target </project>! build.xml! <project name = "week2" basedir = "." default = "compile">!!<target name = "compile">!!!<javac srcdir = "src4"! Target specification: one set of tasks you want to!!! destdir = "bin"/>! automate!</target>! Ingolf Krueger, 2010 CSE 40

41 Ant Build File: Task build.xml! <project name = "week2" basedir = "." default = "compile">!!<target name = "compile">! Task specification:!!<javac srcdir = "src4"! compile all.java files in src4 and!!! destdir = "bin"/>! put the corresponding.class files!</target>! into bin </project>! To execute the target named compile, type ant compile! Ingolf Krueger, 2010 CSE 41

42 Add a target that cleans out the bin directory Ingolf Krueger, 2010 CSE 42

43 Ant Build File: Task <project name = "week2" basedir = "." default = "compile">!!<target name = "compile">!!!<javac srcdir = "src4"!!!! destdir = "bin"/>!!</target>! <target name = "clean">! <delete dir= "bin"/>! </target>! </project>! build.xml! Task specification: remove the directory bin and all of its contents!!! To execute the target named clean, type ant clean! Ingolf Krueger, 2010 CSE 43

44 Add a target that creates the bin directory if it does not already exist Ingolf Krueger, 2010 CSE 44

45 Ant Build File: Task build.xml! <project name = "week2" basedir = "." default = "compile">!!<target name = "compile >!!!<javac srcdir = "src4"!!!! destdir = "bin"/>!!</target>! <target name = "clean">! <delete dir= "bin"/>! </target>! <target name = "init">! <mkdir dir = "bin"/>! </target>! </project>! Task specification: create the directory bin if it doesn t exist already To execute the target named init, type ant init! Ingolf Krueger, 2010 CSE 45

46 Add dependencies between compile and init targets Ingolf Krueger, 2010 CSE 46

47 Ant Build File: Task build.xml! <project name = "week2" basedir = "." default = "compile">!!<target name = "compile depends = init >!!!<javac srcdir = "src4"!!!! destdir = "bin"/>!!</target>! <target name = "clean">! <delete dir= "bin"/>! </target>! <target name = "init">! <mkdir dir = "bin"/>! </target>! </project>! What happens now if you enter the following ant clean; ant compile! Ingolf Krueger, 2010 CSE 47

48 Version Control (a.k.a Configuration Management) with subversion Ingolf Krueger, 2010 CSE 48

49 Stage 1 to Stage 2 Why? Ingolf Krueger, 2010 CSE 49

50 Scenario What can go wrong with your source files even if you are the only developer, and you are developing locally on your machine? Ingolf Krueger, 2010 CSE 50

51 Problems for Individual Non-Version Controlled Software If you save your work over what you had before, any prior work is gone forever Some help from backup mode of IDEs/Editors, but only a crutch Some help from physical backups, but how timely are they? No ability to revert back to a prior version of a particular file (the one without the error!) Never happened to you? It will trust me! Ingolf Krueger, 2010 CSE 51

52 Problems for Individual Non-Version Controlled Software How do you freeze one version for deployment and continue to work on a new version? Crucial if you need to deliver software at some point Many ad-hoc solutions that quickly fall apart due to lack of tool support What about a hard-disk failure? Are you protected? How often do you back up your development machine? What if you are working on a desktop at school/the office, and want to continue to work at your home desktop? the code? Carry a USB-stick? Do you have a good naming scheme? Ingolf Krueger, 2010 CSE 52

53 Scenario Assume now, you are developing in a small team. The team uses a regularly backed-up file-sharing mechanism (such as a shared network drive). What can go wrong? F.java Alice Bob Ingolf Krueger, 2010 CSE 53

54 Problems for Team Non-Version Controlled Software The last one who saves wins Blame game: You wrote over my changes! All my work is lost! How do you coordinate who can save a file at what time? How do you coordinate with 5 or 10 or 100 or 1000 developers on the team? Work gets artificially sequentialized I ll wait before I can touch that file. Who knows what will happen? Could potentially work on separate parts of a file, but merge by hand is error-prone Still no ability to revert back to a prior version of a particular file Ingolf Krueger, 2010 CSE 54

55 Problems for Team Non-Version Controlled Software Still difficult to freeze one version for deployment and continue to work on a new version Need configuration manager Many ad-hoc solutions that quickly fall apart due to lack of tool support What if you are working on a desktop at school/the office, and want to continue to work at your home desktop but have no access to the file share? Now it gets downright dangerous to carry the code home with you: the scope for conflicts/artificial sequentialization grows rapidly How do you merge back your changes into the shared file? Ingolf Krueger, 2010 CSE 55

56 Solution: Version Management with Dedicated Repository Introduce a Version Management Repository that holds the master copy of all artifacts (code, tests, build files, graphics, ) of your project Ingolf Krueger, 2010 CSE 56

57 Solution: Version Management with Dedicated Repository If Alice needs access to the code, she checks out a local copy check out Alice Ingolf Krueger, 2010 CSE 57

58 Solution: Version Management with Dedicated Repository If Bob also needs access to the code, he also checks out a local copy check out Alice Bob Ingolf Krueger, 2010 CSE 58

59 Solution: Version Management with Dedicated Repository Bob and Alice individually make changes to their own local copy Alice Bob Ingolf Krueger, 2010 CSE 59

60 Solution: Version Management with Dedicated Repository When Alice is done with her edits, she commits her local copy to the repository. Her changes are automatically merged into the master copy of the file in the repository. commit Alice Bob Ingolf Krueger, 2010 CSE 60

61 Scenario Now Bob is done with his edits, he also wants to commit. What can happen?? commit Alice Bob Ingolf Krueger, 2010 CSE 61

62 Solution: Version Management with Dedicated Repository Scenario 1: Alice and Bob have edited in different areas of the file No conflict! Bob s changes will be merged into the master copy. commit succeds Alice Bob Ingolf Krueger, 2010 CSE 62

63 Solution: Version Management with Dedicated Repository Scenario 2: Alice and Bob have edited in the same areas of the file Conflict! Bob s changes will NOT be merged into the master copy. commit fails Alice Bob Ingolf Krueger, 2010 CSE 63

64 Solution: Version Management with Dedicated Repository To resolve the conflict, Bob first updates his local copy. update Alice Bob Ingolf Krueger, 2010 CSE 64

65 Solution: Version Management with Dedicated Repository He gets notifications of the conflicts inside the affected files. He makes decisions to resolve the conflicts. update Alice Bob Ingolf Krueger, 2010 CSE 65

66 Solution: Version Management with Dedicated Repository Once he has resolved all conflicts, Bob commits again. This time he succeeds! commit succeeds Alice Bob Ingolf Krueger, 2010 CSE 66

67 Solution: Version Management with Dedicated Repository Alice updates at the beginning of her next work increment. check out Alice Bob Ingolf Krueger, 2010 CSE 67

68 SVN Workflow: Always Update BEFORE Starting Work Ingolf Krueger, 2010 CSE 68

69 SVN Workflow: Do something! Ingolf Krueger, 2010 CSE 69

70 SVN Workflow: Update BEFORE EACH Commit Ingolf Krueger, 2010 CSE 70

71 SVN Workflow: Resolve Conflicts Ingolf Krueger, 2010 CSE 71

72 SVN Workflow: Commit Ingolf Krueger, 2010 CSE 72

73 SVN Workflow Ingolf Krueger, 2010 CSE 73

74 Let s revisit these steps with subversion commands Ingolf Krueger, 2010 CSE 74

75 Solution: Version Management with Dedicated Repository Create subversion repository 1> ls ~/java/week2/svenx! branches/ tags/ trunk/! 1> ls ~/java/week2/svenx/trunk! F.java G.java! Alice 1> svnadmin create /var/svn/cse70-labs! Ingolf Krueger, 2010 CSE 75

76 Solution: Version Management with Dedicated Repository A new repository, named svn has been created (can link to it via Apache web server) svn Alice Ingolf Krueger, 2010 CSE 76

77 Solution: Version Management with Dedicated Repository Now Alice imports her existing folder (and its files) into the repository 1> svn import ~/java/week2/svenx \ import Alice Ingolf Krueger, 2010 CSE 77

78 Solution: Version Management with Dedicated Repository Now Alice needs access to check out her fresh local copy to synchronize with the repository 1> svn checkout \ check out Alice Ingolf Krueger, 2010 CSE 78

79 Solution: Version Management with Dedicated Repository Bob also checks out a local copy into his local wow folder 1> svn checkout check out Alice Bob Ingolf Krueger, 2010 CSE 79

80 Solution: Version Management with Dedicated Repository Bob and Alice individually make changes to their own local copy./trunk:!./trunk:! Alice Bob Ingolf Krueger, 2010 CSE 80

81 Solution: Version Management with Dedicated Repository Now Alice commits (publishes her edited files) 1> svn commit F.java! commit Alice Bob Ingolf Krueger, 2010 CSE 81

82 Solution: Version Management with Dedicated Repository Scenario 2: Bob updates Conflict! 1> svn update! C F.java! Updated to revision 233.! update conflict Alice Bob Ingolf Krueger, 2010 CSE 82

83 Bob looks at the file to find and resolve the conflict package today;! public class StringFormatter {! StringFormatter(){};! public String formatstring(string in) {! char[] cv = new char[2*in.length()];!!!for(int i = 0; i < cv.length; i+=2) {! I better chat!!!cv[i] = in.charat(i/2);! with Alice <<<<<<<.mine! about this!!!cv[i+1] = CharConstantHolder.STAR;! =======!!!!cv[i] = CharConstantHolder.STAR;! >>>>>>>.r233! }!!!}! return new String(cv);! }! Ingolf Krueger, 2010 CSE 83

84 Bob makes decisions to resolve the conflicts package today;! public class StringFormatter {! StringFormatter(){};! public String formatstring(string in) {! char[] cv = new char[2*in.length()];!!!for(int i = 0; i < cv.length; i+=2) {!!!!cv[i] = in.charat(i/2);!!!!cv[i+1] = CharConstantHolder.STAR;!!!}! return new String(cv);! }! }! Looks better already Ingolf Krueger, 2010 CSE 84

85 Solution: Version Management with Dedicated Repository Bob indicates that he has resolved the conflict 1> svn resolved F.java! Resolved conflicted state of F.java'! resolved Alice Bob Ingolf Krueger, 2010 CSE 85

86 Solution: Version Management with Dedicated Repository Once he has resolved all conflicts, Bob commits again. This time he succeeds! 1> svn commit F.java! commit succeeds Alice Bob Ingolf Krueger, 2010 CSE 86

87 Solution: Version Management with Dedicated Repository Alice updates at the beginning of her next work increment and the cycle repeats. 1> svn update! check out Alice Bob Ingolf Krueger, 2010 CSE 87

88 What else can you do with subversion? Ingolf Krueger, 2010 CSE 88

89 Create Branches Other Features of subversion Isolate changes to separate lines of development Example: one branch is the currently deployed version for maintenance, another branch is the current development version Can merge branches, e.g. Create Tags Similar to a branch Ingolf Krueger, 2010 CSE 89

90 What have you learned today? Ingolf Krueger, 2010 CSE 90

91 Learning Goals Be able to identify key elements of the software development pipeline Understand a basic Java program and how it is compiled/run Be able to change a given program to introduce functions and additional classes that can evolve separately Understand the need for a build tool See Ant used to automate compilation and other tasks Be able to write basic Ant files yourself Understand the challenges of maintaining a code base in an individual and team development setting Understand the need for and basic principles of version control/ configuration management Ingolf Krueger, 2010 CSE 91

92 Stage 2: Individual, w/ide, Local Build/Test Tools and Version Control Ingolf Krueger, 2010 CSE 92

93 Stage 4: Small to Medium Teams w/automated Build Process Ingolf Krueger, 2010 CSE 93