Beginner's guide to linkers

Editor's Notes

• #15: [Text Segment] Code Segment，包含程式碼以及可以執行的指令 放在Memory中low level address的位置，避免被stack/heap overwrite 此區塊通常是shareable，比如說同時存在相同的Process時，只會共用一份text Read-only，避免程式自己修改自己的Instruction [Initialized Data Segment] 包含由Programmer自己定義的Global Variable/Static Variable 可以再細分為Read-Only和Read-Write Segment (global) int a = 1; //Read-Write const int a = 1; //Read-Only [Uninitialized Data Segment] A.K.A BSS segment 包含所有宣告為global/static的變數中，沒有任何初始值，或是初始值為0的變數 static int i; //BSS segment static int j = 0; //BSS segment [Stack] 存放"Automatic Variables"，比方說function內的local variable 記錄function的堆疊狀況，比如caller的address，讓function結束後可以順利return。每個function都有自己的堆疊空間，不同function的變數不會互相干擾 [Heap] 可配置的動態記憶體空間，通常使用malloc/realloc/free函式 p1 = (char *)malloc(10); 根據以上的程式碼，會在Heap分配一塊10byte的空間。但是p1仍然是在Stack中，存放剛剛分配出來的空間的位置。 當Heap跟Stack的pointer collision，就會 Out Of Memory All threads share a common Heap.每個thread都有自己的stack，但是Heap是共用的 process 1. Text Segment: A text segment , also known as a code segment or simply as text, is one of the sections of a program in an object file or in memory, which contains executable instructions. As a memory region, a text segment may be placed below the heap or stack in order to prevent heaps and stack overflows from overwriting it. Usually, the text segment is sharable so that only a single copy needs to be in memory for frequently executed programs, such as text editors, the C compiler, the shells, and so on. Also, the text segment is often read-only, to prevent a program from accidentally modifying its instructions. 2. Initialized Data Segment: Initialized data segment, usually called simply the Data Segment. A data segment is a portion of virtual address space of a program, which contains the global variables and static variables that are initialized by the programmer. Note that, data segment is not read-only, since the values of the variables can be altered at run time. This segment can be further classified into initialized read-only area and initialized read-write area. For instance the global string defined by char s[] = “hello world” in C and a C statement like int debug=1 outside the main (i.e. global) would be stored in initialized read-write area. And a global C statement like const char* string = “hello world” makes the string literal “hello world” to be stored in initialized read-only area and the character pointer variable string in initialized read-write area. Ex: static int i = 10 will be stored in data segment and global int i = 10 will also be stored in data segment 3. Uninitialized Data Segment: Uninitialized data segment, often called the “bss” segment, named after an ancient assembler operator that stood for “block started by symbol.” Data in this segment is initialized by the kernel to arithmetic 0 before the program starts executing uninitialized data starts at the end of the data segment and contains all global variables and static variables that are initialized to zero or do not have explicit initialization in source code. For instance a variable declared static int i; would be contained in the BSS segment. For instance a global variable declared int j; would be contained in the BSS segment. 4. Stack: The stack area traditionally adjoined the heap area and grew the opposite direction; when the stack pointer met the heap pointer, free memory was exhausted. (With modern large address spaces and virtual memory techniques they may be placed almost anywhere, but they still typically grow opposite directions.) The stack area contains the program stack, a LIFO structure, typically located in the higher parts of memory. On the standard PC x86 computer architecture it grows toward address zero; on some other architectures it grows the opposite direction. A “stack pointer” register tracks the top of the stack; it is adjusted each time a value is “pushed” onto the stack. The set of values pushed for one function call is termed a “stack frame”; A stack frame consists at minimum of a return address. Stack, where automatic variables are stored, along with information that is saved each time a function is called. Each time a function is called, the address of where to return to and certain information about the caller’s environment, such as some of the machine registers, are saved on the stack. The newly called function then allocates room on the stack for its automatic and temporary variables. This is how recursive functions in C can work. Each time a recursive function calls itself, a new stack frame is used, so one set of variables doesn’t interfere with the variables from another instance of the function. 5. Heap: Heap is the segment where dynamic memory allocation usually takes place. The heap area begins at the end of the BSS segment and grows to larger addresses from there.The Heap area is managed by malloc, realloc, and free, which may use the brk and sbrk system calls to adjust its size (note that the use of brk/sbrk and a single “heap area” is not required to fulfill the contract of malloc/realloc/free; they may also be implemented using mmap to reserve potentially non-contiguous regions of virtual memory into the process’ virtual address space). The Heap area is shared by all shared libraries and dynamically loaded modules in a process.
• #16: （未談及的技術問題：本節不涉及鏈接器「重定位(relocation)」這一重要特性的介紹。不同的程序大小也不同，因此，當動態庫在不同程序中使用時，將被映射成不同的地址空間，也就是說庫中所有的函數和變量在不同的程序中有不同的地址。如果所有訪問該地址之處，都使用相對地址（如「向後偏移1020字節」）而不是絕對地址（固定的某個地址值，如 0x102218BF），那這也不是個事兒，可現在我們要考慮的問題在於，現實並不總這麼盡如人意，當這種情況出現時，所有絕對地址都必須加上一個合適的偏移量——這就是重定位的概念。由於這一概念對C/C++程序員來說幾乎是完全透明的，並且鏈接中報的錯誤也幾乎不可能由重定位問題導致，因此下文將不會對此贅述。）
• #22: 共享庫之所以使用更大的鏈接粒度是因為現代操作系統已經相當聰明了，當你想用靜態庫的時候，他為了節省一些硬盤空間，就採用小粒度的鏈接方式，但對於共享庫來說，不同的程序運行時共用同一個代碼段（但並不共同數據段和 bss 段，因為畢竟不同的程序使用不同的內存空間）。為了做到這一點，必須對整個庫的內容進行一次性映射，這樣才能保證庫內部的符號集中保存在一片連續的空間裡——否則，如果某個進程導入了 a.o 和 c.o， 另一個進程導入的是 b.o 和 c.o，那麼就沒什麼共同點可以供操作系統利用了。 The reason for this larger granularity is because modern operating systems are clever enough that you can save more than just the duplicate disk space that happens with static libraries; different running processes that use the same shared library can also share the code segment (but not the data/bss segments—two different processes could be in different places for their strtok after all). In order to do this, the whole library has to be mapped in one go, so that the internal references all line up to the same places—if one process pulled in a.o and c.o and another pulled in b.o and c.o, there wouldn't be any commonality for the OS to leverage.
• #26: 在 C++ 中，構造過程所需完成的操作遠比「拷貝定值」複雜得多：在程序開始正常運行之前，類層次體系中各種構造函數裡的代碼都必須提前執行。 為了處理好這一切，編譯器在每一個C++文件的目標文件中都保存了一些額外信息，例如，保存了某個文件所需的構造函數列表。在鏈接階段，鏈接器把所有列表合成一張大表，通過一次次掃瞄該表來調用每個全局對象對應的構造函數。 The sane way 1.即將每個實例函數代碼展開，每個目標文件中都會包含它所調用的所有模板函數的代碼 2. max(int,int) and max(double,double)。 3.目標函數將這兩個函數的定義標記成「弱符號」（weak symbos） 這表示當鏈接器最終生成可執行程序時，將只留下所有重複定義的其中之一，剩餘的定義都將棄之不用 每個目標文件都將佔用更多的磁盤空間。 The SUN way Solaris 系統中的 C++ 編譯器所使用的方法 它不會在目標文件中包含任何跟模板相關的代碼，只將這些符號標記成「未定義」。等到了鏈接階段，鏈接器將所有模板實例化函數對應的未定義符號收集在一起，然後為它們生成相應的機器碼。 節省每個目標文件所佔的空間大小， 慢
• #27: 這個例子中的C++文件調用了兩種類型的 max(int,int) 和 max(double,double)，而對於另一個 C++ 文件，可能會調用該模板的其他實例化函數：比如max(float,float)，甚至還有可能是更複雜的 max(MyFloatingPointClass,MyFloatingPointClass)。 模板的每一個實例化函數執行時使用的都是不同的機器碼，因此在程序的鏈接階段，編譯器和鏈接器需要確保程序調用的每個模板實例函數都擴展出相應類型的程序代碼（但對於未被調用的其他模板實例函數而言，不會有任何多餘的代碼生成，這樣可以避免程序代碼過度膨脹）。 那麼編譯器和鏈接器是如何做到這一切換呢？一般來說，有兩種實現方案：一種是將每個實例函數代碼展開，另一種是將實例化操作延遲到鏈接階段（我喜歡將這兩種方法分別稱作「普通方法」（the sane way）和 「Sun方法」（the sane way）（譯註：之所以取這個名字，是因為Solaris系統下的編譯器採用這樣的方法，而Solaris是當年Sun公司旗下最著名的操作系統。））。 對於第一種方法，即將每個實例函數代碼展開，每個目標文件中都會包含它所調用的所有模板函數的代碼，以上文的 C++ 文件為例，目標文件內容如下：