C programs in (address) space and (run-)time. Systems Programming. C, assembler, and machine code. Where is my data and why do I have to know?

Transcription

1 3 4 C programs in (address) space and (run-)time Systems Programming 02. C Programs in Space and Time Alexander Holupirek Database and Information Systems Group Department of Computer & Information Science University of Konstanz Summer Term 2008 Where is my data and why do I have to know? C is closely related to the machine. Before talking about pointers, storage allocation etc. some background knowledge about address space, (virtual) memory and its allocation during program execution comes in handy Knowledge about the memory layout of a program is quite helpful when debugging Knowledge about what is happening inside the machine on program execution is fundamental, to both, debugging programs and, in first place, writing clean code 1 C, assembler, and machine code 2 Repetition Computer Architecture Storage Classes From Source Code To Executable Code Construction of an Executable Relocation Process C-Quellcode int a, b; a = b * b; Intel ia32-assembler-quellcode mov 0x403030,%eax imul 0x403030,%eax mov %eax,0x Maschinenbefehle bzw. Prozessorinstruktionen ausführbarer Binärcode (hexadezimal dargestellt) 4012ee a1 4012ef f f f f3 0f 4012f4 af 4012f f f f f fa a3 4012fb fc fd fe 00 Adresse Inhalt (je 1 Byte)

2 7 8 C, assembler, and machine code Address Space C-Quellcode Ausführbarer Binärcode Assembler-Quellcode int a=4, b; int main(void) { if (a>5) b=1; else b=0; a liegt auf Adresse 0x b liegt auf Adresse 0x804958c Speicheradresse Speicherinhalt (=Maschinenbefehl) : 83 3d cmpl $0x5,0x b: 7e 0c jle d: c7 05 8c movl $0x1,0x804958c : : eb 0a jmp : c7 05 8c movl $0x0,0x804958c : : c9... Zahlenwerte in Binär- und Assemblercode sind alle hexadezimal zu verstehen Tiefstmögliche Adresse (»Speicherbeginn«) Startadresse des Datenblocks Letzte Byteadresse des Datenblocks Adresse des ersten Byte nach dem Datenblock Adressen einzelner Byte Höchstmögliche Adresse (»Speicherende«) Speicheradressen 0 0x x f 0x x x max. Speicherinhalte Datenblock 0x56 0xfc 16 Byte Größe des Datenblocks 5 6 Byte Ordering Alignment Rules Goal: Optimal Performance n Adressraum 0 max. Adr. Daten (4 Byte): Big-Endian-System Adr. n n+1 n+2 n+3 Inhalt d3 d2 d1 d0 MSB LSB d3 d2 d1 d0 MSB LSB Little-Endian-System Adr. n n+1 n+2 n+3 Inhalt Mit der Adresse n wird auf die 4 Byte großen Daten im Programm zugegriffen MSB = Most Significant Byte (höchstwertiges Byte) LSB = Least Significant Byte (niedrigstwertiges Byte) d0 d1 d2 d3 LSB MSB Determine the address locations for variables and instructions Great impact on compiler, assembler, linker tools Daten- Langwort (misaligned) Adressraum Adressen (hexadezimal) 0x35 0x36 0x37 0x38 Datenbus Adressoffsets (Byteadressen) +0 0x x x x37 0x38 0x39 0x3a 0x3b 1. Zugriff 2. Zugriff Langwortgrenzen auf dem Bus Langwortgrenzen (ohne Rest durch 4 teilbar) im Adressraum

3 Alignment Rules (cont.) For derived types 16 (constructed from the basic types) alignment rules apply to each single component: Repetition Computer Architecture struct artikel {char name[5]; int anzahl; double preis;; alignment(1) alignment(4) Storage Classes From Source Code To Executable Code Construction of an Executable Alignment rules may be influenced through compiler directives (-malign-int aligns variables on 32-bit boundaries producing code that runs somewhat faster on processors with 32-bit busses at the expense of memory) Relocation Process 16 arrays, functions, pointers, structures, unions (we will discuss them later) 9 10 Storage Classes Automatic Storage Class Placement of data in memory depends on storage class An object, such as a variable, is a location in storage, and its interpretation depends on two main attributes: its storage class and its type The storage class determines the lifetime of the storage associated with the identified object The types determines the meaning of the values found in the identified object. In C we have two storage classes: automatic and static Storage class specifiers (auto, extern, register, static) together with the context of an object s declaration, specify its storage class Automatic Objects auto and register give the declared objects automatic storage class, and may be used only within functions They are local to a block 17, discarded on exit from the block Declarations within a block create automatic objects if no storage class specification is mentioned or auto is used Initialization of automatic objects is performed each time the block is entered at the top (if a jump into the block is executed the initializations are not performed) Objects declared register are automatic, and are (if possible) stored in fast registers of the machine For register the address operator & is not allowed 17 aka compound statement, such as the body of a function 11 12

4 15 16 Static Storage Class Storage Class and Sections Static Objects May be local to a block or external to all blocks In both cases, they retain their values across exit from and reentry to functions and blocks Within a block, static objects are declared with static Objects declared outside of all blocks (at the same level as function definitions) are always static On the outer level, the keyword static makes them local to a particular translation unit (internal linkage) They are global to an entire program by omitting an explicit storage class, or by using extern (external linkage) Intermediate Summary A program executed does not only use storage for its instructions, but additionally needs space for, e.g., variables Variables may be temporary, dynamically allocated, or static (i.e., permanent in terms of storage allocation), initialized or uninitialized, declared as constant (const) and thus read-only Placement of data in memory depends on its storage class During the translation process the compiler uses sections to divide the address space into logical units Details vary with operating systems and compiler used Typical Program Organisation Program Sections A typical program divides naturally in sections Code machine instructions, should be unmodifiable, size is known after compilation, does not change (.text) Data static data initialized (.data) /uninitialized (.bbs) constant address in memory permanent life time dynamic data stack or heap storage space not known volatile life time.text.data.bss Adressraum PROM oder RAM schreibgeschützt RAM RAM PROM: Programmable Read Only Memory (im Betrieb nicht beschreibbarer Speicherbaustein) RAM: Random Access Memory (Speicher mit wahlfreiem Zugriff)

5 0Adressen 0Adressen Virtual Memory and Segments A Program in Memory Virtual Memory Whenever a process is created, the kernel provides a chunk of physical memory which can be located anywhere Through the magic of virtual memory (VM), the process believes it has all the memory on the computer Typically the VM space is laid out in a similar manner: static data dynamic data 0 Code, Konstanten initialisierte Daten nicht initialisierte Daten Heap aus ausführbarer Datei geladen bei Prozessstart bereitgestellt und mit 0 initialisiert (gelöscht) bei Prozessstart bereitgestellt, für dynamische Speicherallozierung, wächst dem Stapel entgegen Text Segment (.text) Initialized Data Segment (.data) Uninitialized Data Segment (.bss) Adressen Stack bei Prozessstart bereitgestellt, wächst zu tieferen Adressen (bzw. zu höheren Adr.; prozessorabhängig) The Stack The Heap Different Memory Layouts Memory Segments Programmstartadresse (A) Lösung auf PC (ia32) Stack Code, Konstanten initialisierte Daten nicht initialisierte Daten Heap (B) Stack umgekehrt wachsend Code, Konstanten initialisierte Daten nicht initialisierte Daten Stack Heap Text Segment The text segment contains the actual code (including constants) to be executed. It s usually sharable, so multiple instances of a program can share the text segment to lower memory requirements. This segment is usually marked read-only so a program can t modify its own instructions. Initialized Data Segment This segment contains global variables which are initialized by the programmer. Uninitialized Data Segment Also named.bss (block started by symbol) which was an operator used by an old assembler. This segment contains uninitialized global variables. All variables in this segment are initialized to 0 or NULL pointers before the program begins to execute.

6 23 24 Memory Segments (cont.) Variable Placement and Life Time (Code) int a; static int b; The Stack The stack is a collection of stack frames which we will discuss later. When a new frame needs to be added (as a result of a newly called function), the stack grows downward. The Heap Dynamic memory, where storage can be (de-)allocated via C s free(3)/malloc(3). The C library also gets dynamic memory for its own personal workspace from the heap as well. As more memory is requested on the fly, the heap grows upward. void func ( void ) { char c; static int d; int main ( void ) { int e; int *pi = ( int *) malloc ( sizeof ( int )); func (); func (); free (pi ); return (0); Variable Placement and Life Time (Code) Variable Placement and Life Time (Diagram) int a; /* Permanent life time */ static int b; /* dito, but reduced scope */ void func ( void ) { char c; /* only for the life time of func () */ /* but 2x; visible only in func () */ static int d; /* i m unique, exist once at a stable */ /* address, visible only in func () */ int main ( void ) { int e; /* life time of main () */ int *pi = ( int *) malloc ( sizeof ( int )); /* newborn */ func (); func (); free ( pi ); /* RIP, pi points to an invalid address */ return (0); PC(t=0) PC(t=x) Adresse 0 pi SP(t=x) SP(t=0) max. 1. Instruktion 2. Instruktion 3. Instruktion 4. Instruktion... a b d int c pi e Code Daten Halde (Heap) Stapel (Stack) t=0: Programmausführung wird gestartet, d.h., Ausführungsumgebung ist bereits initialisiert t=x: beliebiger Zeitpunkt während der Programmausführung

7 27 28 Variable Placement Repetition Computer Architecture Variables (outside a function) Globally declared variables go to the Uninitialized Data Segment if they are not initialized, to Initialized Data Segment otherwise. Necessary for the OS to decide if storage has to be loaded with initialization data from the executable binary. Variables (inside a function) Implicit assumption of auto, go to The Stack. Declared as static, see above. Constants (const) Text Segment Function Parameters Are pushed on The Stack or stored in registers. If pointers are passed, data is elsewhere. Storage Classes From Source Code To Executable Code Construction of an Executable Relocation Process From source code to executable code Translation steps using gcc(1) Translation Steps (multi-phase compilation) Compilation HLL source code to assembler source code Assembly Assembler source code to object code Linking Object code to executable code Quellcode C/C++ Eingabe- *.c/*.cc/*.cpp dateien Assembler-Quellcode *.s Objektdatei, Bibliotheksdatei *.o/*.a Compilers and assemblers create object files containing the generated binary code and data for a source file. Linkers combine multiple object files into one, loaders take object files and load them into memory. Goal: An executable binary file (a.out) From high-level language (HLL) source code to executable code, i.e., concrete processor instructions in combination with data. Präprozessor Compiler Assembler Binder Ausgabedateien *.i/*.ii *.s *.o a.out Vorverarbeiteter Assembler-Quellcode Objektdatei Ausführbare Datei C/C++-Quellcode (ungebunden) (= Objektdatei, ladbar)

8 31 32 File suffixes and their meaning Creation of an executable file For any given input file, the file name suffix determines what kind of compilation is done (see gcc(1)) for more details and suffixes: suffix compilation step.c C source code which must be preprocessed.i C source code which should not be preprocessed.h Header file to be turned into a precompiled header.s Assembler code.o An object file to be fed straight into linking Object/Library Files ld (Filename).c Kompilieren gcc (Filename).s Assemblieren gas (Filename).o = Operation = Kommando = Eingang oder Ausgang Binden a.out The C Preprocessor File Inclusion A control line of the form # include filename The C preprocessor performs... Inclusion of named files Macro Substitution Conditional Compilation causes the replacement of that line by the entire contents of the file filename. Note The characters in the name filename must not include > or \n, and the effect is undefined if it contains any of ",, \, or /*. Location The named file is searched for in a sequence of implementationdependent places (often starting in /usr/include).

9 35 36 Macro Substitution Macro Substitution (cont.) A control line of the form # define identifier token - sequence causes the preprocessor to replace subsequent instances of the identifier with the given sequence of tokens. Example # define EXIT_ FAILURE 1 # define EXIT_ SUCCESS 0 # define S_IRWXU /* RWX mask for owner */ # define S_IRUSR /* R for owner */ # define S_IWUSR /* W for owner */ # define S_IXUSR /* X for owner */ A control line of the form # define identifier ( identifier - list ) token - sequence where there is no space between the first identifier and the (, is a macro definition with parameters given by the identifier list. Example # define S_ISDIR ( m) (( m & ) == ) /* directory */ # define S_ISCHR ( m) (( m & ) == ) /* char sp. */ # define S_ISBLK ( m) (( m & ) == ) /* block sp.*/ # define S_ISREG ( m) (( m & ) == ) /* regular */ # define S_ISFIFO ( m) (( m & ) == ) /* fifo */ Macro Substitution (cont.) 33 Conditional Inclusion 34 A control line of the form # undef identifier causes the identifier s preprocessor definition to be forgotten. It is not erroneous to apply #undef to an unknown identifier. Example /* * Some header files may define an abs macro. * If defined, undef it to prevent a syntax error * and issue a warning. * # warning is a pragma ( implementation - dependent action ) */ # ifdef abs # undef abs # warning abs macro collides with abs () prototype, undefining # endif Parts of a program may be compiled conditionally Example # ifndef NULL # ifdef GNUG # define NULL null # else # define NULL 0L # endif # endif

10 39 40 Predefined Names Compilation Several identifiers are predefined, and expand to produce special information. They, and also the preprocessor expression operator defined, may not be undefined or redefined. evtl. temporäre Dateien Text LINE FILE DATE TIME STDC A decimal constant containing the current source line number A string literal containing the name of the file being compiled A string literal containing the data of compilation Mmm dd yyyy A string literal containing the data of compilation hh:mm:ss The constant 1. It is intended that this identifier be defined to be 1 only in standard-conforming implementations Text HLL-Quellcode Kompilation Compiler Assembler-Quellcode Text Übersetzungsliste mit Fehlermeldungen Assembly Linking evtl. temporäre Dateien Text Assembler- Quellcode Assemblierung Assembler Objektformat Maschinencode und Zusatzinformationen Text Übersetzungsliste mit Fehlermeldungen und Symboltabelle Objektformat Maschinencode und Zusatzinfo. Objektformat Maschinencode und Zusatzinfo. Bibliotheksobjektformat Maschinencode und Zusatzinfo. library search evtl. temporäre Dateien Binden Binder (Linker) Binärcode od. Objektformat Absoluter Code oder relozierbarer Code mit Zusatzinfo. Text Link Map (Adressraumbenutzung), Symbolliste

11 43 44 Program Section In Virtual Memory Repetition Computer Architecture Storage Classes 0 Nach Kompilation Sektion.text (Code): 0 Nach Bindung Adressraum From Source Code To Executable Code Construction of an Executable Relocation Process 0x x xffffffff Alle Sektionen sind im Adress- raum»absolut«platziert xx Sektion.data (init. Daten) 0 yy Jede Sektion beginnt bei Adr. 0, Sektionen sind»logische. Adressräume«des Compilers Linking an Executable Binary Relocation Records OBJ1 OBJ2 OBJ3.text1.text2.bss2.data1.text3.data3.bss3.bss1 Eingabedaten: ungebundene Objektdateien Bindung (linking).text: Code.data: initialisierte Variablen.bss: nicht initialisierte Variablen Once sections are placed subsequently, relocation can start Executable code contains embedded addresses Static data, function calls, jump targets On relocation those have to be changed inside the code Without a relocation table this is not possible OBJtotal.text1.text2.text3.data1.data3.bss1.bss2.bss3 Verarbeitungsresultat: ausführbare Datei (gebunden, reloziert) A relocation record holds the relative address of a symbol (name of a variable, a function etc.) Each object code (compiled seperately) starts at address 0 Linking them together involves centralization of sections relocation of adresses RELOCATION RECORDS FOR [. text ]: OFFSET TYPE VALUE a R_386_32 b R_386_32 a R_386_32 b

12 Source File: compile.c Analysis of Object Files (compile.o) int a = 1; /* Global variable, initialized ->. data */ int b; /* Global variable, uninitialized ->. bss */ int main ( void ) { static int c; /* Local, static variable ->. bss */ b = 5; c = b + a + 16; return c; Compile a relocatable object file cc -c compile.c (creates compile.o) Linking an executable binary (one-step compilation) cc compile.c -o compile $ file compile. o ELF 32 - bit LSB relocatable, Intel 80386, version 1, not stripped $ objdump -x compile. o compile. o: file format elf32 - i386 compile.o architecture : i386, flags 0 x : HAS_RELOC, HAS_SYMS start address 0 x Sections : Idx Name Size VMA LMA File off Algn 0. text a **2 CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE 1. data **2 CONTENTS, ALLOC, LOAD, DATA 2. bss **2 ALLOC 3. rodata **0 CONTENTS, ALLOC, LOAD, READONLY, DATA Object File: compile.o (cont.) SYMBOL TABLE : l df * ABS * compile. c l d. text l d. data l d. bss l O. bss c l d. rodata g O. data a g F. text a main O * COM * b RELOCATION RECORDS FOR [. text ]: OFFSET TYPE VALUE a R_386_32 b R_386_32 a R_386_32 b R_386_32. bss R_386_32. bss c R_386_32. rodata compile. o: file format elf32 - i386 Disassembly of section. text : <main >: 0: 55 push % ebp 1: 89 e5 mov % esp,% ebp 3: 83 ec 18 sub $0x18,% esp 6: 83 e4 f0 and $0xfffffff0,% esp 9: b mov $0x0,% eax e: 29 c4 sub % eax,% esp 10: a mov 0x0,% eax 15: e8 mov % eax,0 xffffffe8 (% ebp ) 18: c movl $0x5,0 x0 1f: : a mov 0x0,% eax 27: add 0x0,% eax 2d: 83 c0 10 add $0x10,% eax 30: a mov % eax,0 x0 35: a mov 0x0,% eax 3a: 8b 55 e8 mov 0 xffffffe8 (% ebp ),% edx 3d: 3b cmp 0x0,% edx 43: je 58 < main +0 x58 > 45: 83 ec 08 sub $0x8,% esp 48: ff 75 e8 pushl 0 xffffffe8 (% ebp ) 4b: push $0x0 50: e8 fc ff ff ff call 51 < main +0 x51 > 55: 83 c4 10 add $0x10,% esp 58: c9 leave 59: c3 ret 48 46

13 compile. o: file format elf32 - i386 Disassembly of section. text : <main >: int b; /* Global variable, uninitialized ->. bss */ int main ( void ) { 0: 55 push % ebp... 6 more lines... 15: e8 mov % eax,0 xffffffe8 (% ebp ) static int c; /* Local, static variable ->. bss */ b = 5; 18: c movl $0x5,0 x0 1f: c = b + a + 16; 22: a mov 0x0,% eax 27: add 0x0,% eax 2d: 83 c0 10 add $0x10,% eax 30: a mov % eax,0 x0 return c; 35: a mov 0x0,% eax more lines... Executable Binary File: compile compile : file format elf32 - i386 compile architecture : i386, flags 0 x : EXEC_P, HAS_SYMS, D_PAGED start address 0 x1c Sections : Idx Name Size VMA LMA File off Algn text c c **2 CONTENTS, ALLOC, LOAD, READONLY, CODE data c c **2 CONTENTS, ALLOC, LOAD, DATA bss c c **5 ALLOC SYMBOL TABLE : 3 c l O. bss c.0 3 c g O. bss b 1 c0005c0 g F. text a main 3 c g O. data a c0005c0 <main >: int b; /* Global variable, uninitialized ->. bss */ int main ( void ) { 1 c0005c0 : 55 push % ebp 1 c0005c1 : 89 e5 mov %esp,% ebp 1 c0005c3 : 83 ec 18 sub $0x18,% esp 1 c0005c6 : 83 e4 f0 and $0xfffffff0,% esp 1 c0005c9 : b mov $0x0,% eax 1 c0005ce : 29 c4 sub %eax,% esp 1 c0005d0 : a c mov 0 x3c003100,% eax 1 c0005d5 : e8 mov %eax,0 xffffffe8 (% ebp ) static int c; /* Local, static variable ->. bss */ Repetition Computer Architecture Storage Classes From Source Code To Executable Code b = 5; 1 c0005d8 : c c 05 movl $0x5,0 x3c c0005df : c = b + a + 16; 1 c0005e2 : a c mov 0 x3c001018,% eax 1 c0005e7 : c add 0 x3c003280,% eax 1 c0005ed : 83 c0 10 add $0x10,% eax 1 c0005f0 : a c mov % eax,0 x3c return c; 1 c0005f5 : a c mov 0 x3c003140,% eax 51 Construction of an Executable Relocation Process 52

14 55 Relocation Of An Assembler Instruction Relocation Of An Assembler Instruction (cont.) During the linking process relocated addresses are injected in the code, for example the assignment b = 5; Before relocation ( relocatable compile.o ): 18: c movl $0x5,0 x0 1 c0005d8 : c c 05 movl $0x5,0 x3c After relocation ( executable compile ): The proper address for b can be found in the symbol table. SYMBOL TABLE : ( compile ) 3 c g O. bss b The symbol table for compile yields 3c for variable b? How to find the right places in the machine code to perform the substitutions? Linker has relocation record (relative address) of b RELOCATION RECORDS FOR [. text ]: ( compile. o) a R_386_32 b Linker has absolute address of main from symbol table SYMBOL TABLE : ( compile ) 3 c g O. bss b 1 c0005c0 g F. text a main Relocation Of An Assembler Instruction (cont.) Putting it all together: RELOCATION RECORDS FOR [. text ]: ( compile. o) a R_386_32 b ( relative offset ) SYMBOL TABLE : ( compile ) 3 c g O. bss b ( abs. address of b) 1 c0005c0 g F. text a main ( abs. address of main ) Computing the address where substitution must be performed: 1 c0005c a = 1 c0005da 18: c movl $0x5,0 x0 1 c0005d8 : c c 05 movl $0x5,0 x3c003280