Building Programs with Make. What does it mean to Build a Program?

Transcription

1 Building Programs with Make John Pormann, Ph.D. What does it mean to Build a Program? Often, a single executable program is compiled and linked from multiple source-code files and multiple libraries We could type: % gcc -O -o prog.exe file1.c file2.c file3.c file4.c -llib1 -llib2 But this would be: A lot to type each time (and typos will lead to errors) Inefficient -- regardless of what files may have changed, all 4 *.c files will be re-compiled each time What if file3.c calls functions in file4.c (that are changing)? What if file1.c cannot be compiled with -O? What if we want to split file2.c into file2a.c and file2b.c? What if we don t control file4.c, we have to copy it from someone else? 1

2 Program-File Dependencies We can think of the final program as being dependent on the source code files and libraries file4.c file2.c file3.c lib1.a file1.c lib2.a prog.exe If a file changes, the dependent pieces must be re-compiled Dependencies, cont d Digging deeper, we know that C-code is compiled to Obj-code; then the Obj-code is linked (with libraries) into the EXE-code file4.c file2.c file3.c file4.o file1.c file2.o file3.o lib1.a file1.o lib2.a prog.exe 2

3 Efficiency Now, if file1.c changes, we see that file1.o must be rebuilt (recompiled) And because file1.o changes, prog.exe must also be rebuilt... but (existing) file2.o, file3.o, and file4.o are reused in the linkstage for prog.exe MUCH FASTER!! We can also do an efficient rebuild with the more complex relationships (file3/file4)... but we have to be careful to explain what the relationship is Including Header Files Global header files can be problematic file3n4.h file4.c file2.c file3.c file4.o file1.c file2.o file3.o lib1.a file1.o lib2.a prog.exe 3

4 The Make Program Unix/Linux include a utility called make that can help build your program, given a set of dependency criteria You may have used this with other open-source projects %./configure % make configure is another tool that can be used to create a makefile for different computers/architectures/operating systems make will build the library or executable You will probably see a lot of gcc commands fly by Make Program, cont d By default, make will look for a file called makefile or Makefile in the current directory You can also specify another filename with -f myprog.mk The makefile will define rules and dependencies rules allow for default operations on some class of file-types E.g. defines basic compilation steps for *.c to be converted to *.o dependencies identify which files have to be re-built if other files are touched; and what command is used to build them You can also include other makefiles in the current one Useful for OS-specific customization Also allows you to define and use environment variables 4

5 Makefile Example # comment lines # can be added for clarity prog.exe: file1.o file2.o file3.o file4.o By default, builds the first target in the makefile gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 file1.o: file1.c gcc -o file1.o -c file1.c file2.o: file2.c gcc -o file2.o -c file2.c gcc -o file3.o -c file3.c file4.o: file4.c gcc -o file4.o -c file4.c Note: the action lines MUST start with a tab In this example, no defaults are used, no simplifications are made This level of detail can be tedious if your program uses 10 s or 100 s of files Makefile Example, cont d To build the program, we just type: % make You ll see a lot of gcc lines go by: % make gcc -o file1.o -c file1.c gcc -o file2.o -c file2.c gcc -o file3.o -c file3.c gcc -o file4.o -c file4.c gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 That s it, your program is ready to run What if you edit file2.c?? % make gcc -o file2.o -c file2.c gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 5

6 A Quick Warning Make is reasonably intelligent It will not build files that it does not think need to be built It looks at the Unix file-access times to determine if a given.c and.o file are the same But make only knows what the OS tells it!! File-access times can sometimes be wrong when you scp/sftp a file from another system It is easy to forget about header files in your dependency lists If a function changes arguments, some compiled files may be using the old arguments, and some use the new arguments If you re seeing strange behavior, try rm *.o and start over This is often put in the makefile as a target named clean Multiple Targets in a Makefile A single makefile can hold commands for multiple target outputs E.g. prog1.exe and prog2.exe use lots of similar functions and files, so they can be both put into a single makefile #... prog1.exe: file1.o file2.o $(CC) $(LFLAGS) -o prog1.exe file1.o file2.o prog2.exe: file1.o file3.o $(CC) $(LFLAGS) -o prog2.exe file1.o file3.o #... To identify which one to use: % make prog1.exe % make prog2.exe 6

7 Multiple Targets, cont d It is often useful to add Phony targets which only group related build items #....PHONY: all #... all: prog1.exe prog2.exe # really nothing to do for all, # just build prog1 and prog2 prog1.exe: file1.o file2.o $(CC) $(LFLAGS) -o prog1.exe file1.o file2.o prog2.exe: file1.o file3.o $(CC) $(LFLAGS) -o prog2.exe file1.o file3.o clean: rm *.o prog1.exe prog2.exe % make clean #... % make prog1.exe % make prog2.exe % make % make all More Complex Commands If you need to do more work for a given build-target, you can put them on subsequent, tabbed lines: # basic makefile # file3 gets no optimization flag # and we ll print the diff relative to # some previous copy of the file diff file3.c file3_orig.c $(CC) $(DEF) -o file3.o -c file3.c # otherwise, compilation will be optimized $(CC) $(CFLAGS) -o $@ -c $< 7

8 Compiler Arguments/Flags What about compiler options/flags/arguments? If needed, you can always include them, as usual, in the gcc lines for each individual file: # comment lines # can be added for clarity prog.exe: file1.o file2.o file3.o file4.o gcc -O -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 file1.o: file1.c gcc -O -D_MY_C_DEFINE -o file1.o -c file1.c # file2 has lots of small functions, let s inline them for speed file2.o: file2.c gcc -O -finline-functions -o file2.o -c file2.c # file3 will crash with -O gcc -o file3.o -c file3.c file4.o: file4.c gcc -O -o file4.o -c file4.c Again, for lots of files this can get tedious. Simplifying the Makefile IF all (or most) of your files are compiled the same way, you can define a default.c.o rule I.e. how should make convert a *.c file to a *.o file? # basic makefile prog.exe: file1.o file2.o file3.o file4.o gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 gcc -o $@ -c $< In the.c.o rule, we use $@ for the name of the target.o file, and $< for the name of the input file You can define.f.o for Fortran,.cpp.o or.cc.o for C++ You can even create your own suffixes if needed 8

9 Simplifying, the Makefile cont d IF a dependency line exists for a specific file, then that line is used and NOT the default rule So you can define a base compilation, then override when needed # basic makefile prog.exe: file1.o file2.o file3.o file4.o gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 # file3 gets no optimization flag gcc -o file3.o -c file3.c # otherwise, compilation will be optimized gcc -O -o $@ -c $< Running make : Note, we ve now added -O to ALL default compilations! % make -f makefile2 gcc -O -o file1.o -c file1.c gcc -O -o file2.o -c file2.c gcc -o file3.o -c file3.c gcc -O -o file4.o -c file4.c gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 Common Options/Flags If you use the same set of compiler flags a lot, you can declare them in a variable and re-use them through the variable name This makes it easy to change one line and recompile everything for debug or production code # basic makefile CFLAGS = -O -D_MY_C_DEFINE LFLAGS = -O prog.exe: file1.o file2.o file3.o file4.o gcc $(LFLAGS) -o prog.exe file1.o file2.o file3.o file4.o \ -llib1 -llib2 # file3 gets no optimization flag gcc -o file3.o -c file3.c Continues on next line # otherwise, compilation will be optimized gcc $(CFLAGS) -o $@ -c $< 9

10 Common Options/Flags, cont d To change to debug mode, we only modify the top lines of the makefile: # basic makefile CFLAGS = -g -D_MY_C_DEFINE LFLAGS = -g prog.exe: file1.o file2.o file3.o file4.o gcc $(LFLAGS) -o prog.exe file1.o file2.o file3.o file4.o \ -llib1 -llib2 # file3 gets no optimization flag gcc -o file3.o -c file3.c # otherwise, compilation will be optimized gcc $(CFLAGS) -o $@ -c $< Note that make will NOT realize that you need to recompile everything now (for the -g to take effect everywhere) A Usual Makefile # basic makefile CC = gcc OPT = -O DEF = -D_MY_C_DEFINE CFLAGS = $(OPT) $(DEF) LFLAGS = -O OBJS = file1.o file2.o file3.o file4.o LIBS = -llib1 -llib2 prog.exe: $(OBJS) $(CC) $(LFLAGS) -o prog.exe $(OBJS) $(LIBS) # file3 gets no optimization flag $(CC) $(DEF) -o file3.o -c file3.c # otherwise, compilation will be optimized $(CC) $(CFLAGS) -o $@ -c $< 10

11 A Usual Makefile, cont d Why use $(OPT) and $(DEF) and $(CFLAGS)?... CFLAGS = -O -D_MY_C_DEFINE LFLAGS = -O... # file3 gets no optimization flag $(CC) -D_MY_C_DEFINE -o file3.o -c file3.c # otherwise, compilation will be optimized $(CC) $(CFLAGS) -o $@ -c $< What if -D_MY_C_DEFINE changes? We d have to change it in two location in the makefile... will you really remember to do that? In previous makefile: Everything above OBJS = is flexible Everything below OBJS = is fixed Including other Makefiles An makefile may include other makefiles or makefile-fragments # basic makefile include./debug.mk #include./optimize.mk OBJS = file1.o file2.o file3.o file4.o LIBS = -llib1 -llib2 prog.exe: $(OBJS) $(CC) $(LFLAGS) -o prog.exe $(OBJS) $(LIBS) # file3 gets no optimization flag $(CC) -o file3.o -c file3.c # otherwise, compilation will be optimized $(CC) $(CFLAGS) -o $@ -c $< # debug.mk CC = gcc CFLAGS = -g -D_VERBOSE LFLAGS = -g # optimize.mk CC = gcc CFLAGS = -O -D_FAST_SOLVER LFLAGS = -O 11

12 Cross-Platform Builds make can actually handle cross-platform builds fairly smoothly Although you need some help (from the OS or the user) to pick a good set of options # header for gcc on Linux CC = gcc CFLAGS = -O -D_MY_C_DEFINE LFLAGS = -O # header for cc on SGI CC = cc CFLAGS = -fast -D_MY_OS=IRIX LFLAGS = -fast # header for Intel compilers CC = icc CFLAGS = -O3 -D_MY_OS=LINUX LFLAGS = -O3 It is often easy to compartmentalize the changes Cross-Platform Builds, cont d # basic makefile # OS-specific stuff (CC, CFLAGS) defined in # separate file include./os_stuff.mk OBJS = file1.o file2.o file3.o file4.o LIBS = -llib1 -llib2 YOU have to set os_stuff.mk properly (or copy from one of several pre-defined files) prog.exe: $(OBJS) $(CC) $(LFLAGS) -o prog.exe $(OBJS) $(LIBS) # file3 gets no optimization flag $(CC) -o file3.o -c file3.c # otherwise, compilation will be optimized $(CC) $(CFLAGS) -o $@ -c $< 12

13 Recursive Builds You can also have make enter a subdirectory and do a make Useful for projects that require libraries that require libraries... # top-level makefile prog.exe: ( cd lib1 ; make ) ( cd lib2 ; make ) $(CC) -o prog.exe lib1/lib1.a lib2/lib2.a Why the parens (...)? If you cd into a subdirectory, you would have to remember to cd out And what if one of the commands inside that subdirectory do a cd and forget to cd..? The parens execute the commands, then return to the original directory Usually separate commands by semi-colons Recursive Builds, cont d topdir # topdir makefile all: ( cd lib1; make ) ( cd lib2; make ) lib1 # lib1 makefile lib1: lib11.o lib12.o $(CC) -o lib1.a lib11.o lib12.o $(CC) -o $@ -c $< lib2 # lib2 makefile lib2: lib21.o lib22.o $(CC) -o lib2.a lib21.o lib22.o $(CC) -o $@ -c $< 13

14 Putting It All Together How to do multiple targets, multiple subdirectories, cross-platform program building? Top-level directory Makefile - has targets like all and clean, but doesn t really do the work buildenv.mk - includes OS-specific variables, copied from one of several build_linux.mk, build_sgi.mk, build_intel.mk, etc. buildtype.mk - includes production vs. debug variables, copied from one of build_opt.mk or build_debug.mk Component-specific subdirectories Makefile - has targets for program-components, files, etc. Each one includes../buildenv.mk and../buildtype.mk For sub-sub-dirs, include../../buildenv.mk, etc. Putting It All Together, cont d topdir buildenv.mk buildtype.mk lib1 # build_linux.mk CC = gcc OPT = -O DEBUG = -g # build_intel.mk CC = icc OPT = -fast DEBUG = -g # build_debug.mk CFLAGS = $(DEBUG) LFLAGS = $(DEBUG) # build_opt.mk CFLAGS = $(OPT) LFLAGS = $(OPT) # topdir makefile include./buildenv.mk include./buildtype.mk all: ( cd lib1; make ) ( cd lib2; make ) clean: ( cd lib1; make clean ) ( cd lib2; make clean ) rm *.o lib2 # lib1 makefile include../buildenv.mk include../buildtype.mk lib1: lib11.o lib12.o $(CC) $(LFLAGS) -o lib1.a lib11.o lib12.o clean: rm *.o $(CC) $(CFLAGS) -o $@ -c $< 14

15 Suffix Rules Make supports a lot of standard compilations as suffix rules.c.o.cpp.o.f.o But this is somewhat OS- or version-specific You can create your own using the special variable.suffixes.suffixes:.f77.java.class....f77.o: $(F77) -o $@ -c $<.java.class: $(JAVAC) -o $@ -c $< Other Makefile Tricks Adding sign to the beginning of a command hides the command # NOTE: this is not a.phony target!! all: prog1.exe All done with build Without make would print the line echo... then execute the line, which would print the... Adding a - sign to the beginning of a command ignores errors # remove old files clean: -rm prog.exe *.o Without the -, make might see an error from rm if prog.exe did not already exist 15

16 Non-Programming Uses for Make While make is particularly useful for building programs, it really is just a dependency/rule processing engine You can use it to automate many other tasks Perl-documentation to Man page conversion:.suffixes:.pod.3.html MAN_FILES = abc.3 def.3 ghi.3 HTML_FILES = abc.html def.html ghi.html final_man: $(MAN_FILES) echo DONE with MAN final_html: $(HTML_FILES) echo DONE with HTML.pod.3: pod2man $<.pod.html: pod2html $< 16