SlideShare a Scribd company logo
Chapter 4. The Build System
The goal of the previous chapter was to get you up and running as quickly as pos-
sible with custom AOSP development. There’s nothing precluding you from clos-
ing this book at this point and starting to dig in and modify your AOSP tree to fit
your needs. All you need to do to test your modifications is to rebuild the AOSP,
start the emulator again, and, if need be, shell back into it using ADB. If you want
to maximize your efforts, however, you’ll likely want some insight into Android’s
build system.
Despite its modularity, Android’s build system is fairly complex and doesn’t re-
semble any of the mainstream build systems out there; none that are used for
most open source projects, at least. Specifically, it uses make in a fairly unconven-
tional way and doesn’t provide any sort of menuconfig-based configuration (or
equivalent for that matter). Android very much has its own build paradigm that
takes some time to get used to. So grab yourself a good coffee or two—things are
about to get serious.
WARNING
Like the rest of the AOSP, the build system is a moving target. So while the following infor-
mation should remain valid for a long time, you should be on the lookout for changes in
the AOSP version you’re using.
Comparison with Other Build Systems
Before I start explaining how Android’s build system works, allow me to begin by
emphasizing how it differs from what you might already know. First and fore-
most, unlike most make-based build systems, the Android build system doesn’t
rely on recursive makefiles. Unlike the Linux kernel, for instance, there isn’t a
top-level makefile that will recursively invoke subdirectories’ makefiles. Instead,
there is a script that explores all directories and subdirectories until it finds an
Android.mk file, whereupon it stops and doesn’t explore the subdirectories under-
neath that file’s location—unless the Android.mk found instructs the build system
otherwise. Note that Android doesn’t rely on makefiles called Makefile. Instead,
it’s the Android.mk files that specify how the local “module” is built.
WARNING
Android build “modules” have nothing to do with kernel “modules.” Within the context of
Android’s build system, a “module” is any component of the AOSP that needs to be built.
This might be a binary, an app package, a library, etc., and it might have to be built for the
target or the host, but it’s still a “module” with regards to the build system.
HOW MANY BUILD MODULES?
Just to give you an idea of how many modules can be built by the AOSP, try running this
command in your tree:
This will look for all Android.mk files and count how many there are. In 2.3.7/Gingerbread
there are 1,143 and in 4.2/Jelly Bean, 2,037.
$ find . -name Android.mk | wc -l
Another Android specificity is the way the build system is configured. While most
of us are used to systems based on kernel-style menuconfig or GNU autotools (i.e.,
autoconf, automake, etc.), Android relies on a set of variables that are either set
dynamically as part of the shell’s environment by way of envsetup.sh and lunch or
are defined statically ahead of time in a buildspec.mk file. Also—always seeming
to be a surprise to newcomers—the level of configurability made possible by
Android’s build system is fairly limited. So while you can specify the properties of
the target for which you want the AOSP to be built and, to a certain extent, which
apps should be included by default in the resulting AOSP, there is no way for you
to enable or disable most features, as is possible à la menuconfig. You can’t, for in-
stance, decide that you don’t want power management support or that you don’t
want the Location Service to start by default.
Also, the build system doesn’t generate object files or any sort of intermediate out-
put within the same location as the source files. You won’t find the .o files along-
side their .c source files within the source tree, for instance. In fact, none of the
existing AOSP directories are used in any of the output. Instead, the build system
creates an out/ directory where it stores everything it generates. Hence, a make
clean is very much the same thing as an rm -rf out/. In other words, removing the
out/ directory wipes out anything that was built.
The last thing to say about the build system before we start exploring it in more
detail is that it’s heavily tied to GNU make. And, more to the point, version 3.81;
even the newer 3.82 won’t work with many AOSP versions without patching. The
build system in fact heavily relies on many GNU make-specific features such as
the define , include , and ifndef directives.
SOME BACKGROUND ON THE DESIGN OF ANDROID’S BUILD SYSTEM
If you would like to get more insight into the design choices that were made when
Android’s build system was put together, have a look at the build/core/build-system.html file
in the AOSP. It’s dated May 2006 and seems to have been the document that went around
within the Android dev team to get consensus on a rework of the build system. Some of the
information and the hypothesis are out of date or have been obsoleted, but most of the
nuggets of the current build system are there. In general, I’ve found that the further back
the document was created by the Android dev team, the more insightful it is regarding raw
motivations and technical background. Newer documents tend to be “cleaned up” and ab-
stract, if they exist at all.
If you want to understand the technical underpinnings of why Android’s build system
doesn’t use recursive make, have a look at the paper entitled “Recursive Make
Considered Harmful” by Peter Miller in AUUGN Journal of AUUG Inc., 19(1), pp. 14−25.
The paper explores the issues surrounding the use of recursive makefiles and explains a
different approach involving the use of a single global makefile for building the entire
project based on module-provided .mk files, which is exactly what Android does.
Architecture
As illustrated in Figure 4-1, the entry point to making sense of the build system is
the main.mk file found in the build/core/ directory, which is invoked through the
top-level makefile, as we saw earlier. The build/core/ directory actually contains
the bulk of the build system, and we’ll cover key files from there. Again, remem-
ber that Android’s build system pulls everything into a single makefile; it isn’t re-
cursive. Hence, each .mk file you see eventually becomes part of a single huge
makefile that contains the rules for building all the pieces in the system.
Figure 4-1. Android’s build system
WHY DOES MAKE HANG?
Every time you type make, you witness the aggregation of the .mk files into a single set
through what might seem like an annoying build artifact: The build system prints out the
build configuration and seems to hang for quite some time without printing anything to
the screen. After these long moments of screen silence, it actually starts proceeding again
and builds every part of the AOSP, at which point you see regular output to your screen as
you’d expect from any regular build system. Anyone who’s built the AOSP has wondered
what in the world the build system is doing during that time. What it’s doing is incorporat-
ing every Android.mk file it can find in the AOSP.
If you want to see this in action, edit build/core/main.mk and replace this line:
with this:
The next time you type make, you’ll actually see what’s happening:
Configuration
One of the first things the build system does is pull in the build configuration
through the inclusion of config.mk. The build can be configured either by the use
include $(subdir_makefiles)
$(foreach subdir_makefile, $(subdir_makefiles), 
$(info Including $(subdir_makefile)) 
$(eval include $(subdir_makefile)) 
)
subdir_makefile :=
$ make -j16
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.3.4
TARGET_PRODUCT=generic
...
============================================
Including ./bionic/Android.mk
Including ./development/samples/Snake/Android.mk
Including ./libcore/Android.mk
Including ./external/elfutils/Android.mk
Including ./packages/apps/Camera/Android.mk
Including ./device/htc/passion-common/Android.mk
...
of the envsetup.sh and lunch commands or by providing a buildspec.mk file at the
top-level directory. In either case, some of the following variables need to be set.
TARGET_PRODUCT
Android flavor to be built. Each recipe can, for instance, include a different
set of apps or locales or build different parts of the tree. Have a look at the
various single product .mk files included by the AndroidProducts.mk files in
build/target/product/, device/samsung/crespo/, and device/htc/passion/ for ex-
amples in 2.3/Gingerbread. In case of 4.2/Jelly Bean, look at
device/asus/grouper/ and device/samsung/amgnuro/ instead of Crespo and
Passion. Values include the following:
generic
The “vanilla” kind, the most basic build of the AOSP parts you can
have.
full
The “all dressed” kind, with most apps and the major locales enabled.
full_crespo
Same as full but for Crespo (Samsung Nexus S).
full_grouper
Same as full but for Grouper (Asus Nexus 7).
sim
Android simulator (see The Simulator: A Piece of Android’s
History). Even though this is available in 2.3/Gingerbread, this target
has since been removed and isn’t in 4.2/Jelly Bean.
sdk
The SDK; includes a vast number of locales.
TARGET_BUILD_VARIANT
Selects which modules to install. Each module is supposed to have a
LOCAL_MODULE_TAGS variable set in its Android.mk to at least one of the fol-
lowing: user , debug , eng , tests , optional , or samples . By selecting
the variant, you will tell the build system which module subsets should be
included—the only exception to this is packages (i.e., modules that generate
.apk files) for which these rules don’t apply. Specifically:
eng
Includes all modules tagged as user , debug , or eng .
userdebug
Includes both modules tagged as user and debug .
user
Includes only modules tagged as user .
TARGET_BUILD_TYPE
Dictates whether or not special build flags are used or DEBUG variables are
defined in the code. The possible values here are either release or debug .
Most notably, the frameworks/base/Android.mk file chooses between either
frameworks/base/core/config/debug or frameworks/base/core/config/ndebug,
depending on whether or not this variable is set to debug . The former
causes the ConfigBuildFlags.DEBUG Java constant to be set to true ,
whereas the latter causes it to be set to false . Some code in parts of the
system services, for instance, is conditional on DEBUG . Typically,
TARGET_BUILD_TYPE is set to release .
TARGET_TOOLS_PREFIX
[18]
By default, the build system will use one of the cross-development
toolchains shipped with it underneath the prebuilt/ directory — prebuilts/ as
of 4.2/Jelly Bean. However, if you’d like it to use another toolchain, you can
set this value to point to its location.
OUT_DIR
By default, the build system will put all build output into the out/ directory.
You can use this variable to provide an alternate output directory.
BUILD_ENV_SEQUENCE_NUMBER
If you use the template build/buildspec.mk.default to create your own
buildspec.mk file, this value will be properly set. However, if you create a
buildspec.mk with an older AOSP release and try to use it in a future AOSP
release that contains important changes to its build system and, hence, a
different value, this variable will act as a safety net. It will cause the build
system to inform you that your buildspec.mk file doesn’t match your build
system.
THE SIMULATOR: A PIECE OF ANDROID’S HISTORY
If you go back to the menu printed by 2.3/Gingerbread’s lunch in Building Android, you’ll
notice an entry called simulator . In fact you’ll find references to the simulator at a num-
ber of locations in 2.3/Gingerbread, including quite a few Android.mk files and subdirecto-
ries in the tree. The most important thing you need to know about the simulator is that it
has nothing to do with the emulator. They are two completely different things.
That said, the simulator appears to be a remnant of the Android team’s early work to cre-
ate Android. Since at the time they didn’t even have Android running in QEMU, they used
their desktop OSes and the LD_PRELOAD mechanism to simulate an Android device, hence
the term “simulator.” It appears that they stopped using it as soon as running Android on
QEMU became possible. It continued being in the AOSP up until 4.0/Ice-Cream Sandwich,
though, and was potentially useful for building parts of the AOSP for development and test-
ing on developer workstations. 4.2/Jelly Bean, for instance, doesn’t have a simulator target.
The presence of the simulator build target in 2.3/Gingerbread and before didn’t mean that
you could run the AOSP on your desktop. In fact you couldn’t, if only because you needed a
kernel that had Binder included and you would’ve needed to be using Bionic instead of
your system’s default C library. But, if you wanted to run parts of what’s built from the
AOSP on your desktop, this product target allowed you to do so.
In 2.3/Gingerbread, various parts of the code build very differently if the target is the simu-
lator. When browsing the code, for example, you’ll sometimes find conditional builds
around the HAVE_ANDROID_OS C macro, which is only defined when compiling for the sim-
ulator. The code that talks to the Binder is one of these. If HAVE_ANDROID_OS is not defined,
that code will return an error to its caller instead of trying to actually talk to the Binder
driver.
For the full story behind the simulator, have a look at Android developer Andrew
McFadden’s response to a post entitled “Android Simulator Environment” on the an-
droid-porting mailing list in April 2009.
In addition to selecting which parts of the AOSP to build and which options to
build them with, the build system also needs to know about the target it’s building
for. This is provided through a BoardConfig.mk file, which will specify things such
as the command line to be provided to the kernel, the base address at which the
kernel should be loaded, or the instruction set version most appropriate for the
board’s CPU ( TARGET_ARCH_VARIANT ). Have a look at build/target/board/ for a set
of per-target directories that each contain a BoardConfig.mk file. Also have a look
at the various device/*/ TARGET_DEVICE /BoardConfig.mk files included in the AOSP.
The latter are much richer than the former because they contain a lot more hard-
ware-specific information. The device name (i.e., TARGET_DEVICE ) is derived from
the PRODUCT_DEVICE specified in the product .mk file provided for the
TARGET_PRODUCT set in the configuration. In 2.3/Gingerbread, for example,
device/samsung/crespo/AndroidProducts.mk includes
device/samsung/crespo/full_crespo.mk, which sets PRODUCT_DEVICE to crespo .
Hence, the build system looks for a BoardConfig.mk in device/*/crespo/, and there
happens to be one at that location. The same goes on in 4.2/Jelly Bean for the
PRODUCT_DEVICE set in device/asus/grouper/full_grouper.mk to grouper , thereby
pointing the build system to device/*/grouper/BoardConfig.mk.
The final piece of the puzzle with regard to configuration is the CPU-specific op-
tions used to build Android. For ARM, those are contained in
build/core/combo/arch/arm/armv*.mk, with TARGET_ARCH_VARIANT determining
the actual file to use. Each file lists CPU-specific cross-compiler and cross-linker
flags used for building C/C++ files. They also contain a number of
ARCH_ARM_HAVE_* variables that enable others parts of the AOSP to build code
conditionally based on whether a given ARM feature is found in the target’s CPU.
envsetup.sh
Now that you understand the kinds of configuration input the build system needs,
we can discuss the role of envsetup.sh in more detail. As its name implies,
envsetup.sh actually is for setting up a build environment for Android. It does
only part of the job, though. Mainly, it defines a series of shell commands that are
useful to any sort of AOSP work:
In 4.2/Jelly Bean, hmm has replaced help, and the command set made available to
you has been expanded:
$ cd ~/android/aosp-2.3.x
$ . build/envsetup.sh
$ help
Invoke ". build/envsetup.sh" from your shell to add the following functions to
your environment:
- croot: Changes directory to the top of the tree.
- m: Makes from the top of the tree.
- mm: Builds all of the modules in the current directory.
- mmm: Builds all of the modules in the supplied directories.
- cgrep: Greps on all local C/C++ files.
- jgrep: Greps on all local Java files.
- resgrep: Greps on all local res/*.xml files.
- godir: Go to the directory containing a file.
Look at the source to view more functions. The complete list is:
add_lunch_combo cgrep check_product check_variant choosecombo chooseproduct choo
setype choosevariant cproj croot findmakefile gdbclient get_abs_build_var getbug
reports get_build_var getprebuilt gettop godir help isviewserverstarted jgrep lu
nch m mm mmm pgrep pid printconfig print_lunch_menu resgrep runhat runtest set_j
ava_home setpaths set_sequence_number set_stuff_for_environment settitle smokete
st startviewserver stopviewserver systemstack tapas tracedmdump
$ cd ~/android/aosp-4.2
$ . build/envsetup.sh
$ hmm
You’ll likely find the croot and godir commands quite useful for traversing the
tree. Some parts of it are quite deep, given the use of Java and its requirement
that packages be stored in directory trees bearing the same hierarchy as each sub-
part of the corresponding fully qualified package name. For instance, a file part of
the com.foo.bar package must be stored under the com/foo/bar/ directory.
Hence, it’s not rare to find yourself 7 to 10 directories underneath the AOSP’s top-
level directory, and it rapidly becomes tedious to type something like cd ../../../ ... to
return to an upper part of the tree.
Invoke ". build/envsetup.sh" from your shell to add the following functions to y
our environment:
- lunch: lunch <product_name>-<build_variant>
- tapas: tapas [<App1> <App2> ...] [arm|x86|mips] [eng|userdebug|user]
- croot: Changes directory to the top of the tree.
- m: Makes from the top of the tree.
- mm: Builds all of the modules in the current directory.
- mmm: Builds all of the modules in the supplied directories.
- cgrep: Greps on all local C/C++ files.
- jgrep: Greps on all local Java files.
- resgrep: Greps on all local res/*.xml files.
- godir: Go to the directory containing a file.
Look at the source to view more functions. The complete list is:
addcompletions add_lunch_combo cgrep check_product check_variant choosecombo cho
oseproduct choosetype choosevariant cproj croot findmakefile gdbclient get_abs_b
uild_var getbugreports get_build_var getlastscreenshot getprebuilt getscreenshot
path getsdcardpath gettargetarch gettop godir hmm isviewserverstarted jgrep key_
back key_home key_menu lunch _lunch m mm mmm pid printconfig print_lunch_menu re
sgrep runhat runtest set_java_home setpaths set_sequence_number set_stuff_for_en
vironment settitle smoketest startviewserver stopviewserver systemstack tapas tr
acedmdump
m and mm are also quite useful since they allow you to, respectively, build from
the top level regardless of where you are or just build the modules found in the
current directory. For example, if you made a modification to the Launcher and
are in packages/apps/Launcher2, you can rebuild just that module by typing mm
instead of cd’ing back to the top level and typing make. Note that mm doesn’t re-
build the entire tree and, therefore, won’t regenerate AOSP images even if a de-
pendent module has changed. m will do that, though. Still, mm can be useful to
test whether your local changes break the build or not until you’re ready to regen-
erate the full AOSP.
Although the online help doesn’t mention lunch, it is one of the commands de-
fined by envsetup.sh. When you run lunch without any parameters, it shows you a
list of potential choices. This is the list from 2.3/Gingerbread:
This is the list from 4.2/Jelly Bean:
$ lunch
You're building on Linux
Lunch menu... pick a combo:
1. generic-eng
2. simulator
3. full_passion-userdebug
4. full_crespo4g-userdebug
5. full_crespo-userdebug
Which would you like? [generic-eng]
$ lunch
These choices are not static. Most depend on what’s in the AOSP at the time
envsetup.sh runs. They’re in fact individually added using the
add_lunch_combo() function that the script defines. In 2.3/Gingerbread, for in-
stance, envsetup.sh adds generic-eng and simulator by default:
You're building on Linux
Lunch menu... pick a combo:
1. full-eng
2. full_x86-eng
3. vbox_x86-eng
4. full_mips-eng
5. full_grouper-userdebug
6. full_tilapia-userdebug
7. mini_armv7a_neon-userdebug
8. mini_armv7a-userdebug
9. mini_mips-userdebug
10. mini_x86-userdebug
11. full_mako-userdebug
12. full_maguro-userdebug
13. full_manta-userdebug
14. full_toroplus-userdebug
15. full_toro-userdebug
16. full_panda-userdebug
Which would you like? [full-eng]
# add the default one here
add_lunch_combo generic-eng
# if we're on linux, add the simulator. There is a special case
# in lunch to deal with the simulator
if [ "$(uname)" = "Linux" ] ; then
In 4.2/Jelly Bean, simulator is no longer a valid target and envsetup.sh does this
instead:
envsetup.sh also includes all the vendor-supplied scripts it can find. Here’s how it’s
done in 2.3/Gingerbread:
Here’s how it’s done in 4.2/Jelly Bean:
add_lunch_combo simulator
fi
# add the default one here
add_lunch_combo full-eng
add_lunch_combo full_x86-eng
add_lunch_combo vbox_x86-eng
add_lunch_combo full_mips-eng
# Execute the contents of any vendorsetup.sh files we can find.
for f in `/bin/ls vendor/*/vendorsetup.sh vendor/*/build/vendorsetup.sh device/*
/*/vendorsetup.sh 2> /dev/null`
do
echo "including $f"
. $f
done
unset f
# Execute the contents of any vendorsetup.sh files we can find.
for f in `/bin/ls vendor/*/vendorsetup.sh vendor/*/*/vendorsetup.sh device/*/*/v
endorsetup.sh 2> /dev/null`
do
In 2.3/Gingerbread the device/samsung/crespo/vendorsetup.sh file, for instance,
does this:
Similarly, in 4.2/Jelly Bean the device/asus/grouper/vendorsetup.sh file does this:
So that’s how you end up with the menu we saw earlier. Note that the menu asks
you to choose a combo. Essentially, this is a combination of a TARGET_PRODUCT
and TARGET_BUILD_VARIANT , with the exception of the simulator in
2.3/Gingerbread. The menu provides the default combinations, but the others re-
main valid and can be passed to lunch as parameters on the command line. In
2.3/Gingerbread, for instance, you can do something like this:
echo "including $f"
. $f
done
unset f
add_lunch_combo full_crespo-userdebug
add_lunch_combo full_grouper-userdebug
$ lunch generic-user
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.3.4
TARGET_PRODUCT=generic
TARGET_BUILD_VARIANT=user
TARGET_SIMULATOR=false
Once lunch has finished running for a generic-eng combo, it will set up environ-
ment variables described in Table 4-1 in your current shell to provide the build
system with the required configuration information.
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=GINGERBREAD
============================================
$ lunch full_crespo-eng
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.3.4
TARGET_PRODUCT=full_crespo
TARGET_BUILD_VARIANT=eng
TARGET_SIMULATOR=false
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=GINGERBREAD
============================================
Table 4-1. Environment variables set by lunch (in no particular order) for the default build
target (i.e., generic-eng) in 2.3/Gingerbread
Variable Value
PATH $ANDROID_JAVA_TOOLCHAIN:$PATH:$ANDROID_BUILD_PATHS
ANDROID_EABI_TOOLCHAIN aosp-root /prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin
ANDROID_TOOLCHAIN $ANDROID_EABI_TOOLCHAIN
ANDROID_QTOOLS aosp-root /development/emulator/qtools
ANDROID_BUILD_PATHS aosp-root /out/host/linux-
x86:$ANDROID_TOOLCHAIN:$ANDROID_QTOOLS:$ANDROID_TOOLCHAIN:$ANDROID_EABI_TOOLC
ANDROID_BUILD_TOP aosp-root
ANDROID_JAVA_TOOLCHAIN $JAVA_HOME/bin
ANDROID_PRODUCT_OUT aosp-root /out/target/product/generic
OUT ANDROID_PRODUCT_OUT
BUILD_ENV_SEQUENCE_NUMBER 10
OPROFILE_EVENTS_DIR aosp-root /prebuilt/linux-x86/oprofile
TARGET_BUILD_TYPE release
Variable Value
TARGET_PRODUCT generic
TARGET_BUILD_VARIANT eng
TARGET_BUILD_APPS empty
TARGET_SIMULATOR false
PROMPT_COMMAND "033]0;[${TARGET_PRODUCT}-${TARGET_BUILD_VARIANT}] ${USER}@${HOSTNAME}:
${PWD}007"
JAVA_HOME /usr/lib/jvm/java-6-sun
USING CCACHE
If you’ve already done any AOSP building while reading these pages, you’ve noticed how
long the process is. Obviously, unless you can construct yourself a bleeding-edge build
farm, any sort of speedup on your current hardware would be greatly appreciated. As a
sign that the Android development team might itself also feel the pain of the rather long
builds, they’ve added support for ccache . ccache stands for Compiler Cache and is part
of the Samba Project. It’s a mechanism that caches the object files generated by the com-
piler based on the preprocessor’s output. Hence, if under two separate builds the
preprocessor’s output is identical, use of ccache will result in the second build not actu-
ally using the compiler to build the file. Instead, the cached object file will be copied to the
destination where the compiler’s output would have been.
To enable the use of ccache , all you need to do is make sure that the USE_CCACHE environ-
ment variable is set to 1 before you start your build:
You won’t gain any acceleration the first time you run, since the cache will be empty at that
time. Every other time you build from scratch, though, the cache will help accelerate the
build process. The only downside is that ccache is for C/C++ files only. Hence, it can’t accel-
erate the build of any Java file, I must add sadly. In 2.3/Gingerbread, there are about 15,000
C/C++ files and 18,000 Java files in the AOSP. Those numbers are 27,000 and 29,000 in
4.2/Jelly Bean. So, while the cache isn’t a panacea, it’s better than nothing.
If you’d like to learn more about ccache , have a look at the article titled “Improve collab-
orative build times with ccache” by Martin Brown on IBM’s developerWorks site. The ar-
ticle also explores the use of distcc, which allows you to distribute builds over several ma-
chines, so you can pool your team’s workstation caches together.
For all its benefits, some developers have reported weird errors in some cases when using
ccache. For instance, I ran into such issues while maintaining my own AOSP fork. First, I
got a version of the AOSP on my workstation and built it, creating a warm cache. I then
$ export USE_CCACHE=1
proceeded to upload that tree to http://github.com. Finally, I did a repo sync on the tree I
had just uploaded but from another directory on my workstation than the original one up-
loaded. Using diff to compare both trees showed both trees were identical. Yet, the original
built fine with the warm cache while the second continued to fail building until the cache
was erased.
Of course, if you get tired of always typing build/envsetup.sh and lunch, all you
need to do is copy the build/buildspec.mk.default into the top-level directory, re-
name it to buildspec.mk, and edit it to match the configuration that would have
otherwise been set by running those commands. The file already contains all the
variables you need to provide; it’s just a matter of uncommenting the correspond-
ing lines and setting the values appropriately. Once you’ve done that, all you have
to do is go to the AOSP’s directory and invoke make directly. You can skip
envsetup.sh and lunch.
Function Definitions
Because the build system is fairly large—there are more than 40 .mk files in
build/core/ alone—there are benefits in being able to reuse as much code as possi-
ble. This is why the build system defines a large number of functions in the
definitions.mk file. That file is actually the largest one in the build system at about
60KB, with about 140 functions on about 1,800 lines of makefile code in
2.3/Gingerbread. It’s still the largest file in the build system in 4.2/Jelly Bean at
about 73KB, 170 functions, and about 2,100 lines of makefile code. Functions offer
a variety of operations, including file lookup (e.g., all-makefiles-under and
all-c-files-under ), transformation (e.g., transform-c-to-o and transform-
java-to-classes.jar ), copying (e.g., copy-file-to-target ), and utility (e.g.,
my-dir .)
Not only are these functions used throughout the rest of the build system’s compo-
nents, acting as its core library, but they’re sometimes also directly used in mod-
ules’ Android.mk files. Here’s an example snippet from the Calculator app’s
Android.mk:
Although thoroughly describing definitions.mk is outside the scope of this book, it
should be fairly easy for you to explore it on your own. If nothing else, most of the
functions in it are preceded with a comment explaining what they do. Here’s an
example from 2.3/Gingerbread:
Main Make Recipes
At this point you might be wondering where any of the goodies are actually gen-
erated. How are the various images such as RAM disk generated or how is the SDK
put together, for example? Well, I hope you won’t hold a grudge, but I’ve been
keeping the best for last. So without further ado, have a look at the Makefile in
LOCAL_SRC_FILES := $(call all-java-files-under, src)
###########################################################
## Find all of the java files under the named directories.
## Meant to be used like:
## SRC_FILES := $(call all-java-files-under,src tests)
###########################################################
define all-java-files-under
$(patsubst ./%,%, 
$(shell cd $(LOCAL_PATH) ; 
find $(1) -name "*.java" -and -not -name ".*") 
)
endef
build/core/ (not the top-level one). The file starts with an innocuous-looking
comment:
But don’t be fooled. This is where some of the best meat is. Here’s the snippet that
takes care of generating the RAM disk, for example, in 2.3/Gingerbread:
And here’s the snippet that creates the certs packages for checking over-the-air
(OTA) updates in the same AOSP version:
# Put some miscellaneous rules here
# -----------------------------------------------------------------
# the ramdisk
INTERNAL_RAMDISK_FILES := $(filter $(TARGET_ROOT_OUT)/%, 
$(ALL_PREBUILT) 
$(ALL_COPIED_HEADERS) 
$(ALL_GENERATED_SOURCES) 
$(ALL_DEFAULT_INSTALLED_MODULES))
BUILT_RAMDISK_TARGET := $(PRODUCT_OUT)/ramdisk.img
# We just build this directly to the install location.
INSTALLED_RAMDISK_TARGET := $(BUILT_RAMDISK_TARGET)
$(INSTALLED_RAMDISK_TARGET): $(MKBOOTFS) $(INTERNAL_RAMDISK_FILES) | $(MINIGZIP)
$(call pretty,"Target ram disk: $@")
$(hide) $(MKBOOTFS) $(TARGET_ROOT_OUT) | $(MINIGZIP) > $@
# -----------------------------------------------------------------
# Build a keystore with the authorized keys in it, used to verify the
# authenticity of downloaded OTA packages.
#
Obviously there’s a lot more than I can fit here, but have a look at Makefile for in-
formation on how any of the following are created:
Properties (including the target’s /default.prop and /system/build.prop).
RAM disk.
Boot image (combining the RAM disk and a kernel image).
NOTICE files: These are files required by the AOSP’s use of the Apache
Software License (ASL). Have a look at the ASL for more information about
NOTICE files.
OTA keystore.
Recovery image.
System image (the target’s /system directory).
Data partition image (the target’s /data directory).
OTA update package.
SDK.
Nevertheless, some things aren’t in this file:
# This rule adds to ALL_DEFAULT_INSTALLED_MODULES, so it needs to come
# before the rules that use that variable to build the image.
ALL_DEFAULT_INSTALLED_MODULES += $(TARGET_OUT_ETC)/security/otacerts.zip
$(TARGET_OUT_ETC)/security/otacerts.zip: KEY_CERT_PAIR :=
$(DEFAULT_KEY_CERT_PAIR)
$(TARGET_OUT_ETC)/security/otacerts.zip: $(addsuffix .x509.pem,
$(DEFAULT_KEY_CERT_PAIR))
$(hide) rm -f $@
$(hide) mkdir -p $(dir $@)
$(hide) zip -qj $@ $<
.PHONY: otacerts
otacerts: $(TARGET_OUT_ETC)/security/otacerts.zip
Kernel images
Don’t look for any rule to build these. There is no kernel part of the official
AOSP releases—some of the third-party projects listed in Appendix E, how-
ever, actually do package kernel sources directly into the AOSPs they dis-
tribute. Instead, you need to find an Androidized kernel for your target,
build it separately from the AOSP, and feed it to the AOSP. You can find a
few examples of this in the devices in the device/ directory. In
2.3/Gingerbread, for example, device/samsung/crespo/ includes a kernel im-
age (file called kernel) and a loadable module for the Crespo’s WiFi
(bcm4329.ko file). Both of these are built outside the AOSP and copied in bi-
nary form into the tree for inclusion with the rest of the build.
NDK
While the code to build the NDK is in the AOSP, it’s entirely separate from
the AOSP’s build system in build/. Instead, the NDK’s build system is in
ndk/build/. We’ll discuss how to build the NDK shortly.
CTS
The rules for building the CTS are in build/core/tasks/cts.mk.
Cleaning
As I mentioned earlier, a make clean is very much the equivalent of wiping out
the out/ directory. The clean target itself is defined in main.mk. There are, how-
ever, other cleanup targets. Most notably, installclean , which is defined in
cleanbuild.mk, is automatically invoked whenever you change TARGET_PRODUCT ,
TARGET_BUILD_VARIANT or PRODUCT_LOCALES . For instance, if I had first built
2.3/Gingerbread for the generic-eng combo and then used lunch to switch the
combo to full-eng , the next time I started make, some of the build output would
be automatically pruned using installclean :
In contrast to clean , installclean doesn’t wipe out the entirety of out/. Instead,
it only nukes the parts that need rebuilding given the combo configuration
change. There’s also a clobber target which is essentially the same thing as a
clean .
Module Build Templates
What I just described is the build system’s architecture and the mechanics of its
core components. Having read that, you should have a much better idea of how
Android is built from a top-down perspective. Very little of that, however, perme-
ates down to the level of AOSP modules’ Android.mk files. The system has in fact
$ make -j16
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.3.4
TARGET_PRODUCT=full
TARGET_BUILD_VARIANT=eng
...
============================================
*** Build configuration changed: "generic-eng-{mdpi,nodpi}" -> "full-eng-{en_US,
en_GB,fr_FR,it_IT,de_DE,es_ES,mdpi,nodpi}"
*** Forcing "make installclean"...
*** rm -rf out/target/product/generic/data/* out/target/product/generic/data-qem
u/* out/target/product/generic/userdata-qemu.img out/host/linux-x86/obj/NOTICE_F
ILES out/host/linux-x86/sdk out/target/product/generic/*.img out/target/product/
generic/*.txt out/target/product/generic/*.xlb out/target/product/generic/*.zip
out/target/product/generic/data out/target/product/generic/obj/APPS out/target/p
roduct/generic/obj/NOTICE_FILES out/target/product/generic/obj/PACKAGING out/tar
get/product/generic/recovery out/target/product/generic/root out/target/product/
generic/system out/target/product/generic/dex_bootjars out/target/product/generi
c/obj/JAVA_LIBRARIES
*** Done with the cleaning, now starting the real build.
been architected so that module build recipes are pretty much independent from
the build system’s internals. Instead, build templates are provided so that module
authors can get their modules built appropriately. Each template is tailored for a
specific type of module, and module authors can use a set of documented vari-
ables, all prefixed by LOCAL_ , to modulate the templates’ behavior and output. Of
course, the templates and underlying support files (mainly base_rules.mk) closely
interact with the rest of the build system to deal properly with each module’s
build output. But that’s invisible to the module’s author.
The templates are themselves found in the same location as the rest of the build
system in build/core/. Android.mk gets access to them through the include direc-
tive. Here’s an example:
As you can see, Android.mk files don’t actually include the .mk templates by name.
Instead, they include a variable that is set to the corresponding .mk file. Table 4-2
provides the full list of available module templates.
include $(BUILD_PACKAGE)
Table 4-2. Module build templates list
Variable Template
What It
Builds
Most Notable
Use
BUILD_EXECUTABLE executable.mk Target
binaries
Native
commands and
daemons
BUILD_HOST_EXECUTABLE host_executable.mk Host
binaries
Development
tools
BUILD_RAW_EXECUTABLE raw_executable.mk Target
binaries
that run on
bare metal
Code in the
bootloader/
directory
BUILD_JAVA_LIBRARY java_library.mk Target Java
libaries
Apache
Harmony and
Android
Framework
BUILD_STATIC_JAVA_LIBRARY static_java_library.mk Target
static Java
libraries
N/A, few
modules use this
BUILD_HOST_JAVA_LIBRARY host_java_library.mk Host Java
libraries
Development
tools
Variable Template
What It
Builds
Most Notable
Use
BUILD_SHARED_LIBRARY shared_library.mk Target
shared
libraries
A vast number of
modules,
including many
in external/ and
frameworks/base/
BUILD_STATIC_LIBRARY static_library.mk Target
static
libraries
A vast number of
modules,
including many
in external/
BUILD_HOST_SHARED_LIBRARY host_shared_library.mk Host
shared
libraries
Development
tools
BUILD_HOST_STATIC_LIBRARY host_static_library.mk Host static
libraries
Development
tools
BUILD_RAW_STATIC_LIBRARY raw_static_library.mk Target
static
libraries
that run on
bare metal
Code in
bootloader/
BUILD_PREBUILT prebuilt.mk Copies
prebuilt
Configuration
files and binaries
Variable Template
What It
Builds
Most Notable
Use
target files
BUILD_HOST_PREBUILT host_prebuilt.mk Copies
prebuilt
host files
Tools in prebuilt/
and
configuration
files
BUILD_MULTI_PREBUILT multi_prebuilt.mk Copies
prebuilt
modules of
multiple
but known
types, like
Java
libraries or
executables
Rarely used
BUILD_PACKAGE package.mk Built-in
AOSP apps
(i.e.,
anything
that ends
up being
an .apk)
All apps in the
AOSP
BUILD_KEY_CHAR_MAP key_char_map.mk Device
character
All device
character maps
Variable Template
What It
Builds
Most Notable
Use
maps in AOSP
These build templates allow Android.mk files to be usually fairly lightweight:
Tells the build template where the current module is located.
Clears all previously set LOCAL_* variables that might have been set
for other modules.
Sets various LOCAL_* variables to module-specific values.
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_VARIABLE_1 := value_1
LOCAL_VARIABLE_2 := value_2
...
include $(BUILD_MODULE_TYPE)
Invokes the build template that corresponds to the current module’s
type.
NOTE
Note that CLEAR_VARS , which is provided by clear_vars.mk, is very impor-
tant. Recall that the build system includes all Android.mk into what amounts
to a single huge makefile. Including CLEAR_VARS ensures that the LOCAL_*
values set for modules preceding yours are zeroed out by the time your
Android.mk is included. Also, a single Android.mk can describe multiple mod-
ules one after the other. Hence, CLEAR_VARS ensures that previous module
recipes don’t pollute subsequent ones.
Here’s the Service Manager’s Android.mk in 2.3/Gingerbread, for instance
(frameworks/base/cmds/servicemanager/):
And here’s the one from 2.3/Gingerbread’s Desk Clock app
(packages/app/DeskClock/):
[19]
[20]
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_SHARED_LIBRARIES := liblog
LOCAL_SRC_FILES := service_manager.c binder.c
LOCAL_MODULE := servicemanager
ifeq ($(BOARD_USE_LVMX),true)
LOCAL_CFLAGS += -DLVMX
endif
include $(BUILD_EXECUTABLE)
[21]
As you can see, essentially the same structure is used in both modules, even
though they provide very different input and result in very different output.
Notice also the last line from the Desk Clock’s Android.mk, which basically in-
cludes all subdirectories’ Android.mk files. As I said earlier, the build system looks
for the first makefile in a hierarchy and doesn’t look in any subdirectories under-
neath the directory where one was found, hence the need to manually invoke
those. Obviously, the code here just goes out and looks for all makefiles under-
neath. However, some parts of the AOSP either explicitly list subdirectories or
conditionally select them based on configuration.
The documentation at http://source.android.com used to provide an exhaustive
list of all the LOCAL_* variables with their meaning and use. Unfortunately, at the
time of this writing, this list is no longer available. The build/core/build-
system.html file, however, contains an earlier version of that list, and you should
refer to that one until up-to-date lists become available again. Here are some of
the most frequently encountered LOCAL_* variables:
LOCAL_PATH
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES := $(call all-java-files-under, src)
LOCAL_PACKAGE_NAME := DeskClock
LOCAL_OVERRIDES_PACKAGES := AlarmClock
LOCAL_SDK_VERSION := current
include $(BUILD_PACKAGE)
include $(call all-makefiles-under,$(LOCAL_PATH))
The path of the current module’s sources, typically provided by invoking
$(call my-dir) .
LOCAL_MODULE
The name to attribute to this module’s build output. The actual filename or
output and its location will depend on the build template you include. If
this is set to foo , for example, and you build an executable, then the final
executable will be a command called foo and it will be put in the target’s
/system/bin/. If LOCAL_MODULE is set to libfoo and you include
BUILD_SHARED_LIBRARY instead of BUILD_EXECUTABLE , the build system
will generate libfoo.so and put it in /system/lib/.
Note that the name you provide here must be unique for the particular
module class (i.e., build template type) you are building. There can’t be two
libfoo.so libraries, for instance. It’s expected that the module name will
have to be globally unique (i.e., across all module classes) at some point in
the future.
LOCAL_SRC_FILES
The source files used to build the module. You may provide those by using
one of the build system’s defined functions, as the Desk Clock uses all-
java-files-under , or you may list the files explicitly, as the Service
Manager does.
LOCAL_PACKAGE_NAME
Unlike all other modules, apps use this variable instead of LOCAL_MODULE to
provide their names, as you can witness by comparing the two Android.mk
files shown earlier.
LOCAL_SHARED_LIBRARIES
Use this to list all the libraries your module depends on. As mentioned ear-
lier, the Service Manager’s dependency on liblog is specified using this
variable.
LOCAL_MODULE_TAGS
As I mentioned earlier, this allows you to control under which
TARGET_BUILD_VARIANT this module is built. Usually, this should just be set
to optional .
LOCAL_MODULE_PATH
Use this to override the default install location for the type of module
you’re building.
A good way to find out about more LOCAL_* variables is to look at existing
Android.mk files in the AOSP. Also, clear_vars.mk contains the full list of variables
that are cleared. So while it doesn’t give you the meaning of each, it certainly lists
them all.
Also, in addition to the cleaning targets that affect the AOSP globally, each module
can define its own cleaning rules by providing a CleanSpec.mk, much like modules
provide Android.mk files. Unlike the latter, though, the former aren’t required. By
default, the build system has cleaning rules for each type of module. But you can
specify your own rules in a CleanSpec.mk in case your module’s build does some-
thing the build system doesn’t generate by default and, therefore, wouldn’t typi-
cally know how to clean up.
Output
Now that we’ve looked at how the build system works and how module build tem-
plates are used by modules, let’s look at the output it creates in out/. At a fairly
high level, the build output operates in three stages and in two modes, one for the
host and one for the target:
1. Intermediates are generated using the module sources. These intermedi-
ates’ format and location depend on the module’s sources. They may be .o
files for C/C++ code, for example, or .jar files for Java-based code.
2. Intermediates are used by the build system to create actual binaries and
packages: taking .o files, for example, and linking them into an actual
binary.
3. The binaries and packages are assembled together into the final output re-
quested of the build system. Binaries, for instance, are copied into directo-
ries containing the root and /system filesystems, and images of those filesys-
tems are generated for use on the actual device.
out/ is mainly separated into two directories, reflecting its operating modes: host/
and target/. In each directory, you will find a couple of obj/ directories that con-
tain the various intermediates generated during the build. Most of these are
stored in subdirectories named like the one that the BUILD_* macros presented
earlier and serve a specific complementary purpose during the build system’s
operation:
EXECUTABLES/
JAVA_LIBRARIES/
SHARED_LIBRARIES/
STATIC_LIBRARIES/
APPS/
DATA/
ETC/
KEYCHARS/
PACKAGING/
NOTICE_FILES/
include/
lib/
The directory you’ll likely be most interested in is
out/target/product/ PRODUCT_DEVICE /. That’s where the output images will be lo-
cated for the PRODUCT_DEVICE defined in the corresponding product
configuration’s .mk. Table 4-3 explains the content of that directory.
Table 4-3. Product output
Entry Description
android-info.txt Contains the code name for the board for
which this product is configured
clean_steps.mk Contains a list of steps that must be executed
to clean the tree, as provided in CleanSpec.mk
files by calling the add-clean-step function
data/ The target’s /data directory
installed-files.txt A list of all the files installed in data/ and
system/ directories
obj/ The target product’s intermediaries
previous_build_config.mk The last build target; will be used on the next
make to check if the config has changed,
thereby forcing an installclean
ramdisk.img The RAM disk image generated based on the
content of the root/ directory
root/ The content of the target’s root filesystem
symbols/ Unstripped versions of the binaries put in the
root filesystem and /system directory
Entry Description
system/ The target’s /system directory
system.img The /system image, based on the content of
the system/ directory
userdata.img The /data image, based on the content of the
data/ directory
Have a look back at Chapter 2 for a refresher on the root filesystem, /system, and
/data. Essentially, though, when the kernel boots, it will mount the RAM disk im-
age and execute the /init found inside. That binary, in turn, will run the /init.rc
script that will mount both the /system and /data images at their respective loca-
tions. We’ll come back to the root filesystem layout and the system’s operation at
boot time in Chapter 6.
Build Recipes
With the build system’s architecture and functioning in mind, let’s take a look at
some of the most common, and some slightly uncommon, build recipes. We’ll only
lightly touch on using the results of each recipe, but you should have enough in-
formation to get started.
The Default droid Build
Earlier, we went through a number of plain make commands but never really ex-
plained the default target. When you run plain make, it’s as if you had typed:[22]
droid is in fact the default target as defined in main.mk. You don’t usually need
to specify this target manually. I’m providing it here for completeness, so you
know it exists.
Seeing the Build Commands
When you build the AOSP, you’ll notice that it doesn’t actually show you the com-
mands it’s running. Instead, it prints out only a summary of each step it’s at. If you
want to see everything it does, like the gcc command lines for example, add the
showcommands target to the command line:
$ make droid
$ make showcommands
...
host Java: apicheck (out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/c
lasses)
for f in ; do if [ ! -f $f ]; then echo Missing file $f; exit 1; fi; unzip -qo $
f -d out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes; (cd ou
t/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes && rm -rf META-I
NF); done
javac -J-Xmx512M -target 1.5 -Xmaxerrs 9999999 -encoding ascii -g -extdirs ""
-d out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes @out/host
/common/obj/JAVA_LIBRARIES/apicheck_intermediates/java-source-list-uniq || ( rm
-rf out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes ; exit 41
)
rm -f out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/java-source-list
rm -f out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/java-source-list
-uniq
jar -cfm out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/javalib.jar b
uild/tools/apicheck/src/MANIFEST.mf -C out/host/common/obj/JAVA_LIBRARIES/apich
eck_intermediates/classes .
Header: out/host/linux-x86/obj/include/libexpat/expat.h
Illustrating what I explained in the previous section, this is the same as:
As you’ll rapidly notice when using this, it generates a lot of output and is there-
fore hard to follow. You may, however, want to save the standard output and stan-
dard error into files if you’d like to analyze the actual commands used to build the
AOSP:
You can also do something like this to merge all output into a single file:
Some also report that they prefer using the nohup command instead:
cp -f external/expat/lib/expat.h out/host/linux-x86/obj/include/libexpat/expat.h
Header: out/host/linux-x86/obj/include/libexpat/expat_external.h
cp -f external/expat/lib/expat_external.h out/host/linux-x86/obj/include/libexpa
t/expat_external.h
Header: out/target/product/generic/obj/include/libexpat/expat.h
cp -f external/expat/lib/expat.h out/target/product/generic/obj/include/libexpat
/expat.h
...
$ make droid showcommands
$ make showcommands > aosp-build-stdout 2> aosp-build-stderr
$ make showcommands 2>&1 | tell build.log
$ nohup make showcommands
Building the SDK for Linux and Mac OS
The official Android SDK is available at http://developer.android.com. You can,
however, build your own SDK using the AOSP if, for instance, you extended the
core APIs to expose new functionality and would like to distribute the result to de-
velopers so they can benefit from your new APIs. To do so, you’ll need to select a
special combo:
Once this is done, the SDK will be in out/host/linux-x86/sdk/ when built on Linux
and in out/host/darwin-x86/sdk/ when built on a Mac. There will be two copies,
one a ZIP file, much like the one distributed at http://developer.android.com,
and one uncompressed and ready to use.
Assuming you had already configured Eclipse for Android development using the
instructions at http://developer.android.com, you’ll need to carry out two addi-
tional steps to use your newly built SDK. First, you’ll need to tell Eclipse the loca-
tion of the new SDK. To do so, go to Window→Preferences→Android, enter the
path to the new SDK in the SDK Location box, and click OK. Also, for reasons that
aren’t entirely clear to me at the time of this writing, you also need to go to
Window→Android SDK Manager, deselect all the items that might be selected ex-
cept the first two under Tools, and then click “Install 2 packages...” Once that is
done, you’ll be able to create new projects using the new SDK and access any new
APIs you expose in it. If you don’t do that second step, you’ll be able to create new
Android projects, but none of them will resolve Java libraries properly and will,
therefore, never build.
$ . build/envsetup.sh
$ lunch sdk-eng
$ make sdk
Building the SDK for Windows
The instructions for building the SDK for Windows are slightly different from
Linux and Mac OS:
The resulting output will be in out/host/windows/sdk/.
Building the CTS
If you want to build the CTS, you don’t need to use envsetup.sh or lunch. You can
go right ahead and type:
$ . build/envsetup.sh
$ lunch sdk-eng
$ make win_sdk
$ make cts
...
Generating test description for package android.sax
Generating test description for package android.performance
Generating test description for package android.graphics
Generating test description for package android.database
Generating test description for package android.text
Generating test description for package android.webkit
Generating test description for package android.gesture
Generating test plan CTS
Generating test plan Android
Generating test plan Java
Generating test plan VM
Generating test plan Signature
Generating test plan RefApp
Generating test plan Performance
The cts command includes its own online help. Here’s the corresponding sample
output from 2.3/Gingerbread:
Generating test plan AppSecurity
Package CTS: out/host/linux-x86/cts/android-cts.zip
Install: out/host/linux-x86/bin/adb
$ cd out/host/linux-x86/bin/
$ ./cts
Listening for transport dt_socket at address: 1337
Android CTS version 2.3_r3
$ cts_host > help
Usage: command options
Available commands and options:
Host:
help: show this message
exit: exit cts command line
Plan:
ls --plan: list available plans
ls --plan plan_name: list contents of the plan with specified name
add --plan plan_name: add a new plan with specified name
add --derivedplan plan_name -s/--session session_id -r/--result result_type:
derive a plan from the given session
rm --plan plan_name/all: remove a plan or all plans from repository
start --plan test_plan_name: run a test plan
start --plan test_plan_name -d/--device device_ID: run a test plan using the
specified device
start --plan test_plan_name -t/--test test_name: run a specific test
...
$ cts_host > ls --plan
List of plans (8 in total):
Signature
RefApp
Once you have a target up and running, such as the emulator, you can launch the
test suite and it will use adb to run tests on the target:
Building the NDK
As I had mentioned earlier, the NDK has its own separate build system, with its
own setup and help system, which you can invoke like this:
VM
Performance
AppSecurity
Android
Java
CTS
$ ./cts start --plan CTS
Listening for transport dt_socket at address: 1337
Android CTS version 2.3_r3
Device(emulator-5554) connected
cts_host > start test plan CTS
CTS_INFO >>> Checking API...
CTS_INFO >>> This might take several minutes, please be patient...
...
$ cd ndk/build/tools
$ export ANDROID_NDK_ROOT= aosp-root /ndk
$ ./make-release --help
Usage: make-release.sh [options]
Valid options (defaults are in brackets):
When you are ready to build the NDK, you can invoke make-release as follows,
and witness its rather emphatic warning:
--help Print this help.
--verbose Enable verbose mode.
--release=name Specify release name [20110921]
--prefix=name Specify package prefix [android-ndk]
--development=path Path to development/ndk directory [/home/karim/
opersys-dev/android/aosp-2.3.4/development/ndk]
--out-dir=path Path to output directory [/tmp/ndk-release]
--force Force build (do not ask initial question) [no]
--incremental Enable incremental packaging (debug only). [no]
--darwin-ssh=hostname Specify Darwin hostname to ssh to for the build.
--systems=list List of host systems to build for [linux-x86]
--toolchain-src-dir=path Use toolchain sources from path
$ ./make-release
IMPORTANT WARNING !!
This script is used to generate an NDK release package from scratch
for the following host platforms: linux-x86
This process is EXTREMELY LONG and may take SEVERAL HOURS on a dual-core
machine. If you plan to do that often, please read docs/DEVELOPMENT.TXT
that provides instructions on how to do that more easily.
Are you sure you want to do that [y/N]
y
Downloading toolchain sources...
...
Updating the API
The build systems has safeguards in case you modify the AOSP’s core API. If you
do, the build will fail by default with a warning such as this:
As the error message suggests, to get the build to continue, you’ll need to do some-
thing like this:
******************************
You have tried to change the API from what has been previously approved.
To make these errors go away, you have two choices:
1) You can add "@hide" javadoc comments to the methods, etc. listed in the
errors above.
2) You can update current.xml by executing the following command:
make update-api
To submit the revised current.xml to the main Android repository,
you will need approval.
******************************
make: *** [out/target/common/obj/PACKAGING/checkapi-current-timestamp] Error 38
make: *** Waiting for unfinished jobs....
$ make update-api
...
Install: out/host/linux-x86/framework/apicheck.jar
Install: out/host/linux-x86/framework/clearsilver.jar
Install: out/host/linux-x86/framework/droiddoc.jar
Install: out/host/linux-x86/lib/libneo_util.so
The next time you start make, you won’t get any more errors regarding API
changes. Obviously at this point you’re no longer compatible with the official APIs
and are therefore unlikely to be able to get certified as an “Android” device by
Google.
Building a Single Module
Up to now, we’ve looked at building the entire tree. You can also build individual
modules. Here’s how you can ask the build system to build the Launcher2 module
(i.e., the Home screen):
Install: out/host/linux-x86/lib/libneo_cs.so
Install: out/host/linux-x86/lib/libneo_cgi.so
Install: out/host/linux-x86/lib/libclearsilver-jni.so
Copying: out/target/common/obj/JAVA_LIBRARIES/core_intermediates/emma_out/lib/cl
asses-jarjar.jar
Install: out/host/linux-x86/framework/dx.jar
Install: out/host/linux-x86/bin/dx
Install: out/host/linux-x86/bin/aapt
Copying: out/target/common/obj/JAVA_LIBRARIES/bouncycastle_intermediates/emma_ou
t/lib/classes-jarjar.jar
Copying: out/target/common/obj/JAVA_LIBRARIES/ext_intermediates/emma_out/lib/cla
sses-jarjar.jar
Install: out/host/linux-x86/bin/aidl
Copying: out/target/common/obj/JAVA_LIBRARIES/core-junit_intermediates/emma_out/
lib/classes-jarjar.jar
Copying: out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/emma_out/l
ib/classes-jarjar.jar
Copying current.xml
$ make Launcher2
You can also clean modules individually:
If you’d like to force the build system to regenerate the system image to include
your updated module, you can add the snod target to the command line:
Building Out of Tree
If you’d ever like to build code against the AOSP and its Bionic library but don’t
want to incorporate that into the AOSP, you can use a makefile such as the follow-
ing to get the job done:
$ make clean-Launcher2
$ make Launcher2 snod
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.3.4
TARGET_PRODUCT=generic
...
target Package: Launcher2 (out/target/product/generic/obj/APPS/Launcher2_interme
diates/package.apk)
'out/target/common/obj/APPS/Launcher2_intermediates//classes.dex' as 'classes.d
ex'...
Install: out/target/product/generic/system/app/Launcher2.apk
Install: out/host/linux-x86/bin/mkyaffs2image
make snod: ignoring dependencies
Target system fs image: out/target/product/generic/system.img
[23]
# Paths and settings
TARGET_PRODUCT = generic
ANDROID_ROOT = /home/karim/android/aosp-2.3.x
BIONIC_LIBC = $(ANDROID_ROOT)/bionic/libc
PRODUCT_OUT = $(ANDROID_ROOT)/out/target/product/$(TARGET_PRODUCT)
CROSS_COMPILE = 
$(ANDROID_ROOT)/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-
# Tool names
AS = $(CROSS_COMPILE)as
AR = $(CROSS_COMPILE)ar
CC = $(CROSS_COMPILE)gcc
CPP = $(CC) -E
LD = $(CROSS_COMPILE)ld
NM = $(CROSS_COMPILE)nm
OBJCOPY = $(CROSS_COMPILE)objcopy
OBJDUMP = $(CROSS_COMPILE)objdump
RANLIB = $(CROSS_COMPILE)ranlib
READELF = $(CROSS_COMPILE)readelf
SIZE = $(CROSS_COMPILE)size
STRINGS = $(CROSS_COMPILE)strings
STRIP = $(CROSS_COMPILE)strip
export AS AR CC CPP LD NM OBJCOPY OBJDUMP RANLIB READELF 
SIZE STRINGS STRIP
# Build settings
CFLAGS = -O2 -Wall -fno-short-enums
HEADER_OPS = -I$(BIONIC_LIBC)/arch-arm/include 
-I$(BIONIC_LIBC)/kernel/common 
-I$(BIONIC_LIBC)/kernel/arch-arm
LDFLAGS = -nostdlib -Wl,-dynamic-linker,/system/bin/linker 
$(PRODUCT_OUT)/obj/lib/crtbegin_dynamic.o 
$(PRODUCT_OUT)/obj/lib/crtend_android.o 
-L$(PRODUCT_OUT)/obj/lib -lc -ldl
# Installation variables
EXEC_NAME = example-app
INSTALL = install
In this case, you don’t need to care about either envsetup.sh or lunch. You can just
go ahead and type the magic incantation:
Obviously this won’t add your binary to any of the images generated by the AOSP.
Even the install target here will be of value only if you’re mounting the target’s
INSTALL_DIR = $(PRODUCT_OUT)/system/bin
# Files needed for the build
OBJS = example-app.o
# Make rules
all: example-app
.c.o:
$(CC) $(CFLAGS) $(HEADER_OPS) -c $<
example-app: ${OBJS}
$(CC) -o $(EXEC_NAME) ${OBJS} $(LDFLAGS)
install: example-app
test -d $(INSTALL_DIR) || $(INSTALL) -d -m 755 $(INSTALL_DIR)
$(INSTALL) -m 755 $(EXEC_NAME) $(INSTALL_DIR)
clean:
rm -f *.o $(EXEC_NAME) core
distclean:
rm -f *~
rm -f *.o $(EXEC_NAME) core
$ make
filesystem off NFS, and that’s valuable only during debugging, which is what this
makefile is assumed to be useful for. To an extent, it could also be argued that us-
ing such a makefile is actually counterproductive, since it’s far more complicated
than the equivalent Android.mk that would result if this code were added as a
module part of the AOSP.
Still, this kind of hack can have its uses. Under certain circumstances, for in-
stance, it might make sense to modify the conventional build system used by a
rather large codebase to build that project against the AOSP yet outside of it; the
alternative being to copy the project into the AOSP and create Android.mk files to
reproduce the mechanics of its original conventional build system, which might
turn out to be a substantial endeavor in and of itself.
Building Recursively, In-Tree
You can, if you really want to, hack yourself a makefile to build within the AOSP a
component that is based on recursive makefiles instead of trying to reproduce the
same functionality using Android.mk files, as was suggested in the last section.
Several of the AOSP forks mentioned in Appendix E, for instance, include the ker-
nel sources at the top level of the AOSP and modify the AOSP’s main makefile to
invoke the kernel’s existing build system.
Here’s another example where an Android.mk was created by Linaro’s Bernhard
Rosenkränzer in order to build ffmpeg—which relies on a GNU autotools-like
script—using its original build files:
include $(CLEAR_VARS)
FFMPEG_TCDIR := $(realpath $(shell dirname $(TARGET_TOOLS_PREFIX)))
FFMPEG_TCPREFIX := $(shell basename $(TARGET_TOOLS_PREFIX))
# FIXME remove -fno-strict-aliasing once the aliasing violations are fixed
FFMPEG_COMPILER_FLAGS = $(subst -I ,-I../../,$(subst -include 
system/core/include/arch/linux-arm/AndroidConfig.h,,$(subst -include
build/core/combo/include/arch/linux-arm/AndroidConfig.h,, 
$(TARGET_GLOBAL_CFLAGS)))) -fno-strict-aliasing -Wno-error=address 
-Wno-error=format-security
ifneq ($(strip $(SHOW_COMMANDS)),)
FF_VERBOSE="V=1"
endif
.PHONY: ffmpeg
droidcore: ffmpeg
systemtarball: ffmpeg
REALTOP=$(realpath $(TOP))
ffmpeg: x264 $(PRODUCT_OUT)/obj/STATIC_LIBRARIES/libvpx_intermediates/libvpx.a
mkdir -p $(PRODUCT_OUT)/obj/ffmpeg
cd $(PRODUCT_OUT)/obj/ffmpeg && 
export PATH=$(FFMPEG_TCDIR):$(PATH) && 
$(REALTOP)/external/ffmpeg/configure 
--arch=arm 
--target-os=linux 
--prefix=/system 
--bindir=/system/bin 
--libdir=/system/lib 
--enable-shared 
--enable-gpl 
--disable-avdevice 
--enable-runtime-cpudetect 
--disable-libvpx 
--enable-libx264 
--enable-cross-compile 
--cross-prefix=$(FFMPEG_TCPREFIX) 
--extra-ldflags="-nostdlib -Wl,-dynamic-linker, 
/system/bin/linker,-z,muldefs$(shell if test $(PRODUCT_SDK_VERSION) -lt 16;
Basic AOSP Hacks
You most likely bought this book with one thing in mind: to hack the AOSP to fit
your needs. Over the next few pages, we’ll start looking into some of the most ob-
vious hacks you’ll likely want to try. Of course we’re only setting the stage here
with the parts that pertain to the build system, which is where you’ll likely want
to start anyway.
NOTE
While the following explanations are based on 2.3/Gingerbread, they’ll work
just the same on 4.2/Jelly Bean, and likely many versions after that one, too.
The fact is, these mechanisms have been constant for quite some time. Still,
where relevant, changes in 4.2/Jelly Bean are highlighted.
then echo -n ',-T$(REALTOP)/$(BUILD_SYSTEM)/armelf.x'; fi),-z,nocopyreloc, 
--no-undefined -L$(REALTOP)/$(TARGET_OUT_STATIC_LIBRARIES) 
-L$(REALTOP)/$(PRODUCT_OUT)/system/lib 
-L$(REALTOP)/$(PRODUCT_OUT)/obj/STATIC_LIBRARIES/libvpx_intermediates -ldl -lc" 
--extra-cflags="$(FFMPEG_COMPILER_FLAGS) 
-I$(REALTOP)/bionic/libc/include -I$(REALTOP)/bionic/libc/kernel/common 
-I$(REALTOP)/bionic/libc/kernel/arch-arm 
-I$(REALTOP)/bionic/libc/arch-arm/include -I$(REALTOP)/bionic/libm/include 
-I$(REALTOP)/external/libvpx -I$(REALTOP)/external/x264" 
--extra-libs="-lgcc" && 
$(MAKE) 
TARGET_CRTBEGIN_DYNAMIC_O=$(REALTOP)/$(TARGET_CRTBEGIN_DYNAMIC_O) 
TARGET_CRTEND_O=$(REALTOP)/$(TARGET_CRTEND_O) $(FF_VERBOSE) && 
$(MAKE) install DESTDIR=$(REALTOP)/$(PRODUCT_OUT)
Adding a Device
Adding a custom device is most likely one of the topmost items (if not the top-
most) on your list of reasons for reading this book. I’m about to show you how to
do just that, so you’ll likely want to bookmark this section. Of course I’m actually
only showing you the build aspects of the work. There are a lot more steps in-
volved in porting Android to new hardware. Still, adding the new device to the
build system will definitely be one of the first things you do. Fortunately, doing
that is relatively straightforward.
For the purposes of the current exercise, assume you work for a company called
ACME and that you’re tasked with delivering its latest gizmo: the CoyotePad, in-
tended to be the best platform for playing all bird games. Let’s get started by cre-
ating an entry for our new device in device/:
The first thing we’ll need in here is an AndroidProducts.mk file to describe the
various AOSP products that could be built for the CoyotePad:
While we could describe several products (see
build/target/product/AndroidProducts.mk for an example), the typical case is to
specify just one, as in this case, and it’s described in full_coyotepad.mk:
$ cd ~/android/aosp-2.3.x
$ . build/envsetup.sh
$ mkdir -p device/acme/coyotepad
$ cd device/acme/coyotepad
PRODUCT_MAKEFILES := 
$(LOCAL_DIR)/full_coyotepad.mk
It’s worth taking a closer look at this makefile. First, we’re using the inherit-
product function to tell the build system to pull in other product descriptions as
the basis of ours. This allows us to build on other people’s work and not have to
specify from scratch every bit and piece of the AOSP that we’d like to include.
languages_full.mk will pull in a vast number of locales, and full.mk will make sure
we get the same set of modules as if we had built using the full-eng combo.
With regard to the other variables:
DEVICE_PACKAGE_OVERLAYS
Allows us to specify a directory that will form the basis of an overlay that
will be applied onto the AOSP’s sources, thereby allowing us to substitute
default package resources with device-specific resources. You’ll find this
useful if you’d like to set custom layouts or colors for Launcher2 or other
apps, for instance. We’ll look at how to use this in the next section.
PRODUCT_PACKAGES
$(call inherit-product, $(SRC_TARGET_DIR)/product/languages_full.mk)
# If you're using 4.2/Jelly Bean, use full_base.mk instead of full.mk
$(call inherit-product, $(SRC_TARGET_DIR)/product/full.mk)
DEVICE_PACKAGE_OVERLAYS :=
PRODUCT_PACKAGES +=
PRODUCT_COPY_FILES +=
PRODUCT_NAME := full_coyotepad
PRODUCT_DEVICE := coyotepad
PRODUCT_MODEL := Full Android on CoyotePad, meep-meep
Allows us to specify packages we’d like to have this product include in addi-
tion to those specified in the products we’re already inheriting from. If you
have custom apps, binaries, or libraries located within
device/acme/coyotepad/, for instance, you’ll want to add them here so that
they are included in the final images generated. Notice the use of the +=
sign. It allows us to append to the existing values in the variable instead of
substituting its content.
PRODUCT_COPY_FILES
Allows us to list specific files we’d like to see copied to the target’s filesys-
tem and the location where they need to be copied. Each source/destination
pair is colon-separated, and pairs are space-separated among themselves.
This is useful for configuration files and prebuilt binaries such as firmware
images or kernel modules.
PRODUCT_NAME
The TARGET_PRODUCT , which you can set either by selecting a lunch combo
or passing it as part of the combo parameter to lunch, as in:
PRODUCT_DEVICE
The name of the actual finished product shipped to the customer.
TARGET_DEVICE derives from this variable. PRODUCT_DEVICE has to match
an entry in device/acme/, since that’s where the build looks for the corre-
sponding BoardConfig.mk. In this case, the variable is the same as the name
of the directory we’re already in.
PRODUCT_MODEL
$ lunch full_coyotepad-eng
The name of this product as provided in the “Model number” in the “About
the phone” section of the settings. This variable actually gets stored as the
ro.product.model global property accessible on the device.
Version 4.2/Jelly Bean also includes a PRODUCT_BRAND that is typically set to
Android. The value of this variable is then available as the ro.product.brand
global property. The latter is used by some parts of the stack that take action
based on the device’s vendor.
Now that we’ve described the product, we must also provide some information
regarding the board the device is using through a BoardConfig.mk file:
This is a very skinny BoardConfig.mk and ensures that we actually build success-
fully. For a real-life version of that file, have a look at
device/samsung/crespo/BoardConfigCommon.mk in 2.3/Gingerbread, and also at
device/asus/grouper/BoardConfigCommon.mk in 4.2/Jelly Bean.
You’ll also need to provide a conventional Android.mk in order to build all the
modules that you might have included in this device’s directory:
TARGET_NO_KERNEL := true
TARGET_NO_BOOTLOADER := true
TARGET_CPU_ABI := armeabi
BOARD_USES_GENERIC_AUDIO := true
USE_CAMERA_STUB := true
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
ifneq ($(filter coyotepad,$(TARGET_DEVICE)),)
It’s in fact the preferred modus operandi to put all device-specific apps, binaries,
and libraries within the device’s directory instead of globally within the rest of
the AOSP. If you do add modules here, don’t forget to also add them to
PRODUCT_PACKAGES as I explained earlier. If you just put them here and provide
them valid Android.mk files, they’ll build, but they won’t be in the final images.
If you have several products sharing the same set of packages, you may want to
create a device/acme/common/ directory containing the shared packages. You can
see an example of this in 4.2/Jelly Bean’s device/generic/ directory. In that same
version, you can also check how device/samsung/maguro/device.mk inherits from
device/samsung/tuna/device.mk for an example of how one device can be based on
another device.
Lastly, let’s close the loop by making the device we just added visible to
envsetup.sh and lunch. To do so, you’ll need to add a vendorsetup.sh in your
device’s directory:
You also need to make sure that it’s executable if it’s to be operational:
We can now go back to the AOSP’s root and take our brand-new ACME CoyotePad
for a runchase:
include $(call all-makefiles-under,$(LOCAL_PATH))
endif
add_lunch_combo full_coyotepad-eng
$ chmod 755 vendorsetup.sh
$ croot
$ . build/envsetup.sh
$ lunch
You're building on Linux
Lunch menu... pick a combo:
1. generic-eng
2. simulator
3. full_coyotepad-eng
4. full_passion-userdebug
5. full_crespo4g-userdebug
6. full_crespo-userdebug
Which would you like? [generic-eng] 3
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.3.4
TARGET_PRODUCT=full_coyotepad
TARGET_BUILD_VARIANT=eng
TARGET_SIMULATOR=false
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=GINGERBREAD
============================================
$ make -j16
As you can see, the AOSP now recognizes our new device and prints the informa-
tion correspondingly. When the build is done, we’ll also have the same type of
output provided in any other AOSP build, except that it will be a product-specific
directory:
Also, have a look at the build.prop file in system/. It contains various global prop-
erties that will be available at runtime on the target and that relate to our configu-
ration and build:
$ ls -al out/target/product/coyotepad/
total 89356
drwxr-xr-x 7 karim karim 4096 2011-09-21 19:20 .
drwxr-xr-x 4 karim karim 4096 2011-09-21 19:08 ..
-rw-r--r-- 1 karim karim 7 2011-09-21 19:10 android-info.txt
-rw-r--r-- 1 karim karim 4021 2011-09-21 19:41 clean_steps.mk
drwxr-xr-x 3 karim karim 4096 2011-09-21 19:11 data
-rw-r--r-- 1 karim karim 20366 2011-09-21 19:20 installed-files.txt
drwxr-xr-x 14 karim karim 4096 2011-09-21 19:20 obj
-rw-r--r-- 1 karim karim 327 2011-09-21 19:41 previous_build_config.mk
-rw-r--r-- 1 karim karim 2649750 2011-09-21 19:43 ramdisk.img
drwxr-xr-x 11 karim karim 4096 2011-09-21 19:43 root
drwxr-xr-x 5 karim karim 4096 2011-09-21 19:19 symbols
drwxr-xr-x 12 karim karim 4096 2011-09-21 19:19 system
-rw------- 1 karim karim 87280512 2011-09-21 19:20 system.img
-rw------- 1 karim karim 1505856 2011-09-21 19:14 userdata.img
# begin build properties
# autogenerated by buildinfo.sh
ro.build.id=GINGERBREAD
ro.build.display.id=full_coyotepad-eng 2.3.4 GINGERBREAD eng.karim.20110921.1908
49 test-keys
ro.build.version.incremental=eng.karim.20110921.190849
ro.build.version.sdk=10
ro.build.version.codename=REL
ro.build.version.release=2.3.4
ro.build.date=Wed Sep 21 19:10:04 EDT 2011
ro.build.date.utc=1316646604
ro.build.type=eng
ro.build.user=karim
ro.build.host=w520
ro.build.tags=test-keys
ro.product.model=Full Android on CoyotePad, meep-meep
ro.product.brand=generic
ro.product.name=full_coyotepad
ro.product.device=coyotepad
ro.product.board=
ro.product.cpu.abi=armeabi
ro.product.manufacturer=unknown
ro.product.locale.language=en
ro.product.locale.region=US
ro.wifi.channels=
ro.board.platform=
# ro.build.product is obsolete; use ro.product.device
ro.build.product=coyotepad
# Do not try to parse ro.build.description or .fingerprint
ro.build.description=full_coyotepad-eng 2.3.4 GINGERBREAD eng.karim.20110921.190
849 test-keys
ro.build.fingerprint=generic/full_coyotepad/coyotepad:2.3.4/GINGERBREAD/eng.kari
m.20110921.190849:eng/test-keys
# end build properties
...
WARNING
You may want to carefully vet the default properties before using the build on a real de-
vice. Some developers have encountered some severe issues due to default values. In both
2.3/Gingerbread and 4.2/Jelly Bean, for instance, ro.com.android.dataroaming is set to
true in some builds. Hence, if you’re doing development on a device connected to a live
cell network, changing the value to false might save you some money.
As you can imagine, there’s a lot more to be done here to make sure the AOSP
runs on our hardware. But the preceding steps give us the starting point.
However, by isolating the board-specific changes in a single directory, this config-
uration will simplify adding support for the CoyotePad to the next version of the
AOSP that gets released. Indeed, it’ll just be a matter of copying the corresponding
directory to the new AOSP’s device/ directory and adjusting the code therein to use
the new APIs.
Adding an App
Adding an app to your board is relatively straightforward. As a starter, try creat-
ing a HelloWorld! app with Eclipse and the default SDK; all new Android projects
in Eclipse are a HelloWorld! by default. Then copy that app from the Eclipse
workspace to its destination:
You’ll then have to create an Android.mk file in aosp-
root /device/acme/coyotepad/HelloWorld/ to build that app:
$ cp -a ~/workspace/HelloWorld ~/android/aosp-2.3.x/device/acme/coyotepad/
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
Given that we’re tagging this module as optional , it won’t be included by default
in the AOSP build. To include it, you’ll need to add it to the PRODUCT_PACKAGES
listed in the CoyotePad’s full_coyotepad.mk.
If, instead of adding your app for your board only, you would like to add a default
app globally to all products generated by the AOSP alongside the existing stock
apps, you’ll need to put it in packages/apps/ instead of your board’s directory.
You’ll also need to modify one of the built-in .mk files, such as aosp-
root/build/target/product/core.mk, to have your app built by default. This is not
recommended, though, as it’s not very portable since it will require you to make
this modification to every new AOSP release. As I stated earlier, it’s best to keep
your custom modifications in device/acme/coyotepad/ in as much as possible.
Adding an App Overlay
Sometimes you don’t actually want to add an app but would rather modify exist-
ing ones included by default in the AOSP. That’s what app overlays are for.
Overlays are a mechanism included in the AOSP to allow device manufacturers to
change the resources provided (such as for apps), without actually modifying the
original resources included in the AOSP. To use this capability, you must create an
overlay tree and tell the build system about it. The easiest location for an overlay
is within a device-specific directory such as the one we created in the previous
section:
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES := $(call all-java-files-under, src)
LOCAL_PACKAGE_NAME := HelloWorld
include $(BUILD_PACKAGE)
To tell the build system to take this overlay into account, we need to modify our
full_coyotepad.mk such that:
At this point, though, our overlay isn’t doing much. Let’s say we want to modify
some of Launcher2’s default strings. We could then do something like this:
You can then trim your local strings.xml to override only those strings that you
need. Most importantly, your device will have a Launcher2 that has your custom
strings, but the default Launcher2 will still have its original strings. So if someone
relies on the same AOSP sources you’re using to build for another product, they’ll
still get the original strings. You can, of course, replace most resources like this, in-
cluding images and XML files. So long as you put the files in the same hierarchy as
they are found in the AOSP but within device/acme/coyotepad/overlay/, they’ll be
taken into account by the build system.
$ cd device/acme/coyotepad/
$ mkdir overlay
DEVICE_PACKAGE_OVERLAYS := device/acme/coyotepad/overlay
$ mkdir -p overlay/packages/apps/Launcher2/res/values
$ cp aosp-root /packages/apps/Launcher2/res/values/strings.xml 
> overlay/packages/apps/Launcher2/res/values/
WARNING
Overlays can be used only for resources. You can’t overlay source code. If you want to cus-
tomize parts of Android’s internals, for instance, you’ll still have to make those modifica-
tions in situ. There’s no way, currently at least, to isolate those changes to your board.
Adding a Native Tool or Daemon
Like the example above of adding an app for your board, you can add your cus-
tom native tools and daemons as subdirectories of device/acme/coyotepad/.
Obviously, you’ll need to provide an Android.mk in the directory containing the
code to build that module:
As in the app’s case, you’ll also need to make sure hello-world is part of the
CoyotePad’s PRODUCT_PACKAGES .
If you intend to add your binary globally for all product builds instead of just lo-
cally to your board, you need to know that there are a number of locations in the
tree where native tools and daemons are located. Here are the most important
ones:
system/core/ and system/
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := hello-world
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES := hello-world.cpp
LOCAL_SHARED_LIBRARIES := liblog
include $(BUILD_EXECUTABLE)
Custom Android binaries that are meant to be used outside the Android
Framework or are standalone pieces.
frameworks/base/cmds/
Binaries that are tightly coupled to the Android Framework. This is where
the Service Manager and installd are found, for example.
external/
Binaries that are generated by an external project that is imported into the
AOSP. strace, for instance, is here.
Having identified from the list above where the code generating your binary
should go, you’ll also need to add it as part of one of the global .mk files such as
aosp-root/build/target/product/core.mk. As I said above, however, such global addi-
tions are not recommended since they can’t be transferred as easily to newer
AOSP versions.
Adding a Native Library
Like apps and binaries, you can also add native libraries for your board.
Assuming, as above, that the sources to build the library are in a subdirectory of
device/acme/coyotepad/, you’ll need an Android.mk to build your library:
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := libmylib
LOCAL_MODULE_TAGS := optional
LOCAL_PRELINK_MODULE := false
LOCAL_SRC_FILES := $(call all-c-files-under,.)
include $(BUILD_SHARED_LIBRARY)
NOTE
Note that LOCAL_PRELINK_MODULE has been removed and is no longer neces-
sary as of 4.0/Ice-Cream Sandwich.
To use this library, you must add it to the libraries listed by the Android.mk file of
whichever binaries depend on it:
You’ll also likely need to add relevant headers to an include/ directory located in
about the same location as you put your library, so that the code that needs to link
against your library can find those headers, such as
device/acme/coyotepad/include/.
Should you want to make your library apply globally to all AOSP builds, not just
your device, you’ll need a little bit more information regarding the various loca-
tions where libraries are typically found in the tree. First, you should know that,
unlike binaries, a lot of libraries are used within a single module but nowhere
else. Hence, these libraries will typically be placed within that module’s code and
not in the usual locations where libraries used systemwide are found. The latter
are typically in the following locations:
system/core/
Libraries used by many parts of the system, including some outside the
Android Framework. This is where liblog is, for instance.
frameworks/base/libs/
Libraries intimately tied to the framework. This is where libbinder is.
LOCAL_SHARED_LIBRARIES := libmylib
frameworks/native/libs/
In 4.2/Jelly Bean, many libraries that were in frameworks/base/libs/ in
2.3/Gingerbread have been moved out and into frameworks/native/libs/.
external/
Libraries generated by external projects imported into the AOSP. OpenSSL’s
libssl is here.
Similarly, instead of using a CoyotePad-specific include directory, you’d use a
global directory such as system/core/include/ or frameworks/base/include/ or, in
4.2/Jelly Bean, frameworks/base/include/. Again, as stated earlier, you should care-
fully review whether such global additions are truly required, as they’ll represent
additional work when you try to port for your device to the next version of
Android.
LIBRARY PRELINKING
If you look closely at the example Android.mk we provide for the library, you’ll notice a
LOCAL_PRELINK_MODULE variable. To reduce the time it takes to load libraries, Android ver-
sions up to 2.3/Gingerbread used to prelink most of their libraries. Prelinking is done by
specifying ahead of time the address location where the library will be loaded instead of
letting it be figured out at runtime. The file where the addresses are specified in
2.3/Gingerbread is build/core/prelink-linux-arm.map, and the tool that does the mapping is
called apriori. It contains entries such as these:
If you want to add a custom native library to 2.3/Gingerbread, you need to either add it to
the list of libraries in prelink-linux-arm.map or set the LOCAL_PRELINK_MODULE to false .
The build will fail if you forget to do one of these.
# core system libraries
libdl.so 0xAFF00000 # [<64K]
libc.so 0xAFD00000 # [~2M]
libstdc++.so 0xAFC00000 # [<64K]
libm.so 0xAFB00000 # [~1M]
liblog.so 0xAFA00000 # [<64K]
libcutils.so 0xAF900000 # [~1M]
libthread_db.so 0xAF800000 # [<64K]
libz.so 0xAF700000 # [~1M]
libevent.so 0xAF600000 # [???]
libssl.so 0xAF400000 # [~2M]
...
# assorted system libraries
libsqlite.so 0xA8B00000 # [~2M]
libexpat.so 0xA8A00000 # [~1M]
libwebcore.so 0xA8300000 # [~7M]
libbinder.so 0xA8200000 # [~1M]
libutils.so 0xA8100000 # [~1M]
libcameraservice.so 0xA8000000 # [~1M]
libhardware.so 0xA7F00000 # [<64K]
libhardware_legacy.so 0xA7E00000 # [~1M]
...
Library prelinking was dropped starting in 4.0/Ice-Cream Sandwich.
If you do not provide a value, defaults will be used. For instance, all apps are set to op‐
tional by default. Also, some modules are part of GRANDFATHERED_USER_MODULES in
user_tags.mk. No LOCAL_MODULE_TAGS need be specified for those; they’re always included.
This file contains a set list of variables starting with the string LOCAL_ . If a variable
isn’t specifically listed in this file, it won’t be taken into account by CLEAR_VARS .
This version is cleaned up a little (removed commented code, for instance) and slightly
reformatted.
Also slightly modified to remove white space and comments.
This assumes you had already run envsetup.sh and lunch.
This makefile is inspired by a blog post by Row Boat developer Amit Pundir and is
based on the example makefile provided in Chapter 4 of Building Embedded Linux Systems,
2nd ed. (O’Reilly).
[18]
[19]
[20]
[21]
[22]
[23]
4. The Build System _ Embedded Android.pdf

More Related Content

PDF
Aosp+
PDF
dylibencapsulation
PDF
Getting started-with-zend-framework
PPT
Drupal - Introduction to Drupal Creating Modules
PDF
You can now use PVS-Studio with Visual Studio absent; just give it the prepro...
PDF
Makefile
PDF
Drupal theming training
Aosp+
dylibencapsulation
Getting started-with-zend-framework
Drupal - Introduction to Drupal Creating Modules
You can now use PVS-Studio with Visual Studio absent; just give it the prepro...
Makefile
Drupal theming training

Similar to 4. The Build System _ Embedded Android.pdf (20)

PPT
Migraine Drupal - syncing your staging and live sites
PPTX
Improving Drupal Performances
PDF
ASP.NET Unit-2.pdf
PDF
Building aosp
PDF
Building resuable and customizable Vue components
DOCX
Vipul divyanshu mahout_documentation
PPTX
Basic iOS Training with SWIFT - Part 1
PPT
ODP
Drupal Multi-Site Setup
PPTX
Get the best out of Bootstrap with Bootstrap4XPages - Engage 2014
PDF
Dependent things dependency management for apple sw - slideshare
PDF
Android open source project build system phi innovations - android summit 2015
DOC
Exploit Frameworks
PPT
SynapseIndia drupal presentation on drupal best practices
PDF
The dependency inversion principle
PDF
Composer tools and frameworks for drupal.ppt
PDF
Composer tools and frameworks for Drupal
PDF
Trying to Sell PVS-Studio to Google, or New Bugs in Chromium
PPTX
Android presentation - Gradle ++
Migraine Drupal - syncing your staging and live sites
Improving Drupal Performances
ASP.NET Unit-2.pdf
Building aosp
Building resuable and customizable Vue components
Vipul divyanshu mahout_documentation
Basic iOS Training with SWIFT - Part 1
Drupal Multi-Site Setup
Get the best out of Bootstrap with Bootstrap4XPages - Engage 2014
Dependent things dependency management for apple sw - slideshare
Android open source project build system phi innovations - android summit 2015
Exploit Frameworks
SynapseIndia drupal presentation on drupal best practices
The dependency inversion principle
Composer tools and frameworks for drupal.ppt
Composer tools and frameworks for Drupal
Trying to Sell PVS-Studio to Google, or New Bugs in Chromium
Android presentation - Gradle ++
Ad

More from VishalKumarJha10 (6)

PDF
Hilman-Runtime-Power management linux .pdf
PDF
kocialkowski-overview-linux-userspace-graphics-stack.pdf
PDF
Android memory analysis Debug slides.pdf
PDF
Linux Scheduler Latest_ viresh Kumar.pdf
PDF
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
PDF
breaking_dependencies_the_solid_principles__klaus_iglberger__cppcon_2020.pdf
Hilman-Runtime-Power management linux .pdf
kocialkowski-overview-linux-userspace-graphics-stack.pdf
Android memory analysis Debug slides.pdf
Linux Scheduler Latest_ viresh Kumar.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
breaking_dependencies_the_solid_principles__klaus_iglberger__cppcon_2020.pdf
Ad

Recently uploaded (20)

PDF
Wondershare Filmora 15 Crack With Activation Key [2025
PPTX
history of c programming in notes for students .pptx
PDF
How to Choose the Right IT Partner for Your Business in Malaysia
PPTX
ManageIQ - Sprint 268 Review - Slide Deck
PDF
Adobe Illustrator 28.6 Crack My Vision of Vector Design
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 41
PPTX
ISO 45001 Occupational Health and Safety Management System
PDF
Odoo Companies in India – Driving Business Transformation.pdf
PPTX
Online Work Permit System for Fast Permit Processing
PPTX
Odoo POS Development Services by CandidRoot Solutions
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
PDF
medical staffing services at VALiNTRY
PPTX
ai tools demonstartion for schools and inter college
PDF
AI in Product Development-omnex systems
PPTX
Introduction to Artificial Intelligence
PDF
System and Network Administraation Chapter 3
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
PDF
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
PDF
top salesforce developer skills in 2025.pdf
Wondershare Filmora 15 Crack With Activation Key [2025
history of c programming in notes for students .pptx
How to Choose the Right IT Partner for Your Business in Malaysia
ManageIQ - Sprint 268 Review - Slide Deck
Adobe Illustrator 28.6 Crack My Vision of Vector Design
Internet Downloader Manager (IDM) Crack 6.42 Build 41
ISO 45001 Occupational Health and Safety Management System
Odoo Companies in India – Driving Business Transformation.pdf
Online Work Permit System for Fast Permit Processing
Odoo POS Development Services by CandidRoot Solutions
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
medical staffing services at VALiNTRY
ai tools demonstartion for schools and inter college
AI in Product Development-omnex systems
Introduction to Artificial Intelligence
System and Network Administraation Chapter 3
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
VVF-Customer-Presentation2025-Ver1.9.pptx
top salesforce developer skills in 2025.pdf

4. The Build System _ Embedded Android.pdf

  • 1. Chapter 4. The Build System The goal of the previous chapter was to get you up and running as quickly as pos- sible with custom AOSP development. There’s nothing precluding you from clos- ing this book at this point and starting to dig in and modify your AOSP tree to fit your needs. All you need to do to test your modifications is to rebuild the AOSP, start the emulator again, and, if need be, shell back into it using ADB. If you want to maximize your efforts, however, you’ll likely want some insight into Android’s build system. Despite its modularity, Android’s build system is fairly complex and doesn’t re- semble any of the mainstream build systems out there; none that are used for most open source projects, at least. Specifically, it uses make in a fairly unconven- tional way and doesn’t provide any sort of menuconfig-based configuration (or equivalent for that matter). Android very much has its own build paradigm that takes some time to get used to. So grab yourself a good coffee or two—things are about to get serious. WARNING Like the rest of the AOSP, the build system is a moving target. So while the following infor- mation should remain valid for a long time, you should be on the lookout for changes in the AOSP version you’re using.
  • 2. Comparison with Other Build Systems Before I start explaining how Android’s build system works, allow me to begin by emphasizing how it differs from what you might already know. First and fore- most, unlike most make-based build systems, the Android build system doesn’t rely on recursive makefiles. Unlike the Linux kernel, for instance, there isn’t a top-level makefile that will recursively invoke subdirectories’ makefiles. Instead, there is a script that explores all directories and subdirectories until it finds an Android.mk file, whereupon it stops and doesn’t explore the subdirectories under- neath that file’s location—unless the Android.mk found instructs the build system otherwise. Note that Android doesn’t rely on makefiles called Makefile. Instead, it’s the Android.mk files that specify how the local “module” is built. WARNING Android build “modules” have nothing to do with kernel “modules.” Within the context of Android’s build system, a “module” is any component of the AOSP that needs to be built. This might be a binary, an app package, a library, etc., and it might have to be built for the target or the host, but it’s still a “module” with regards to the build system. HOW MANY BUILD MODULES? Just to give you an idea of how many modules can be built by the AOSP, try running this command in your tree: This will look for all Android.mk files and count how many there are. In 2.3.7/Gingerbread there are 1,143 and in 4.2/Jelly Bean, 2,037. $ find . -name Android.mk | wc -l
  • 3. Another Android specificity is the way the build system is configured. While most of us are used to systems based on kernel-style menuconfig or GNU autotools (i.e., autoconf, automake, etc.), Android relies on a set of variables that are either set dynamically as part of the shell’s environment by way of envsetup.sh and lunch or are defined statically ahead of time in a buildspec.mk file. Also—always seeming to be a surprise to newcomers—the level of configurability made possible by Android’s build system is fairly limited. So while you can specify the properties of the target for which you want the AOSP to be built and, to a certain extent, which apps should be included by default in the resulting AOSP, there is no way for you to enable or disable most features, as is possible à la menuconfig. You can’t, for in- stance, decide that you don’t want power management support or that you don’t want the Location Service to start by default. Also, the build system doesn’t generate object files or any sort of intermediate out- put within the same location as the source files. You won’t find the .o files along- side their .c source files within the source tree, for instance. In fact, none of the existing AOSP directories are used in any of the output. Instead, the build system creates an out/ directory where it stores everything it generates. Hence, a make clean is very much the same thing as an rm -rf out/. In other words, removing the out/ directory wipes out anything that was built. The last thing to say about the build system before we start exploring it in more detail is that it’s heavily tied to GNU make. And, more to the point, version 3.81; even the newer 3.82 won’t work with many AOSP versions without patching. The build system in fact heavily relies on many GNU make-specific features such as the define , include , and ifndef directives.
  • 4. SOME BACKGROUND ON THE DESIGN OF ANDROID’S BUILD SYSTEM If you would like to get more insight into the design choices that were made when Android’s build system was put together, have a look at the build/core/build-system.html file in the AOSP. It’s dated May 2006 and seems to have been the document that went around within the Android dev team to get consensus on a rework of the build system. Some of the information and the hypothesis are out of date or have been obsoleted, but most of the nuggets of the current build system are there. In general, I’ve found that the further back the document was created by the Android dev team, the more insightful it is regarding raw motivations and technical background. Newer documents tend to be “cleaned up” and ab- stract, if they exist at all. If you want to understand the technical underpinnings of why Android’s build system doesn’t use recursive make, have a look at the paper entitled “Recursive Make Considered Harmful” by Peter Miller in AUUGN Journal of AUUG Inc., 19(1), pp. 14−25. The paper explores the issues surrounding the use of recursive makefiles and explains a different approach involving the use of a single global makefile for building the entire project based on module-provided .mk files, which is exactly what Android does. Architecture As illustrated in Figure 4-1, the entry point to making sense of the build system is the main.mk file found in the build/core/ directory, which is invoked through the top-level makefile, as we saw earlier. The build/core/ directory actually contains the bulk of the build system, and we’ll cover key files from there. Again, remem- ber that Android’s build system pulls everything into a single makefile; it isn’t re- cursive. Hence, each .mk file you see eventually becomes part of a single huge makefile that contains the rules for building all the pieces in the system.
  • 5. Figure 4-1. Android’s build system WHY DOES MAKE HANG? Every time you type make, you witness the aggregation of the .mk files into a single set through what might seem like an annoying build artifact: The build system prints out the build configuration and seems to hang for quite some time without printing anything to the screen. After these long moments of screen silence, it actually starts proceeding again and builds every part of the AOSP, at which point you see regular output to your screen as you’d expect from any regular build system. Anyone who’s built the AOSP has wondered what in the world the build system is doing during that time. What it’s doing is incorporat- ing every Android.mk file it can find in the AOSP. If you want to see this in action, edit build/core/main.mk and replace this line:
  • 6. with this: The next time you type make, you’ll actually see what’s happening: Configuration One of the first things the build system does is pull in the build configuration through the inclusion of config.mk. The build can be configured either by the use include $(subdir_makefiles) $(foreach subdir_makefile, $(subdir_makefiles), $(info Including $(subdir_makefile)) $(eval include $(subdir_makefile)) ) subdir_makefile := $ make -j16 ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=2.3.4 TARGET_PRODUCT=generic ... ============================================ Including ./bionic/Android.mk Including ./development/samples/Snake/Android.mk Including ./libcore/Android.mk Including ./external/elfutils/Android.mk Including ./packages/apps/Camera/Android.mk Including ./device/htc/passion-common/Android.mk ...
  • 7. of the envsetup.sh and lunch commands or by providing a buildspec.mk file at the top-level directory. In either case, some of the following variables need to be set. TARGET_PRODUCT Android flavor to be built. Each recipe can, for instance, include a different set of apps or locales or build different parts of the tree. Have a look at the various single product .mk files included by the AndroidProducts.mk files in build/target/product/, device/samsung/crespo/, and device/htc/passion/ for ex- amples in 2.3/Gingerbread. In case of 4.2/Jelly Bean, look at device/asus/grouper/ and device/samsung/amgnuro/ instead of Crespo and Passion. Values include the following: generic The “vanilla” kind, the most basic build of the AOSP parts you can have. full The “all dressed” kind, with most apps and the major locales enabled. full_crespo Same as full but for Crespo (Samsung Nexus S). full_grouper Same as full but for Grouper (Asus Nexus 7). sim Android simulator (see The Simulator: A Piece of Android’s History). Even though this is available in 2.3/Gingerbread, this target has since been removed and isn’t in 4.2/Jelly Bean. sdk
  • 8. The SDK; includes a vast number of locales. TARGET_BUILD_VARIANT Selects which modules to install. Each module is supposed to have a LOCAL_MODULE_TAGS variable set in its Android.mk to at least one of the fol- lowing: user , debug , eng , tests , optional , or samples . By selecting the variant, you will tell the build system which module subsets should be included—the only exception to this is packages (i.e., modules that generate .apk files) for which these rules don’t apply. Specifically: eng Includes all modules tagged as user , debug , or eng . userdebug Includes both modules tagged as user and debug . user Includes only modules tagged as user . TARGET_BUILD_TYPE Dictates whether or not special build flags are used or DEBUG variables are defined in the code. The possible values here are either release or debug . Most notably, the frameworks/base/Android.mk file chooses between either frameworks/base/core/config/debug or frameworks/base/core/config/ndebug, depending on whether or not this variable is set to debug . The former causes the ConfigBuildFlags.DEBUG Java constant to be set to true , whereas the latter causes it to be set to false . Some code in parts of the system services, for instance, is conditional on DEBUG . Typically, TARGET_BUILD_TYPE is set to release . TARGET_TOOLS_PREFIX [18]
  • 9. By default, the build system will use one of the cross-development toolchains shipped with it underneath the prebuilt/ directory — prebuilts/ as of 4.2/Jelly Bean. However, if you’d like it to use another toolchain, you can set this value to point to its location. OUT_DIR By default, the build system will put all build output into the out/ directory. You can use this variable to provide an alternate output directory. BUILD_ENV_SEQUENCE_NUMBER If you use the template build/buildspec.mk.default to create your own buildspec.mk file, this value will be properly set. However, if you create a buildspec.mk with an older AOSP release and try to use it in a future AOSP release that contains important changes to its build system and, hence, a different value, this variable will act as a safety net. It will cause the build system to inform you that your buildspec.mk file doesn’t match your build system.
  • 10. THE SIMULATOR: A PIECE OF ANDROID’S HISTORY If you go back to the menu printed by 2.3/Gingerbread’s lunch in Building Android, you’ll notice an entry called simulator . In fact you’ll find references to the simulator at a num- ber of locations in 2.3/Gingerbread, including quite a few Android.mk files and subdirecto- ries in the tree. The most important thing you need to know about the simulator is that it has nothing to do with the emulator. They are two completely different things. That said, the simulator appears to be a remnant of the Android team’s early work to cre- ate Android. Since at the time they didn’t even have Android running in QEMU, they used their desktop OSes and the LD_PRELOAD mechanism to simulate an Android device, hence the term “simulator.” It appears that they stopped using it as soon as running Android on QEMU became possible. It continued being in the AOSP up until 4.0/Ice-Cream Sandwich, though, and was potentially useful for building parts of the AOSP for development and test- ing on developer workstations. 4.2/Jelly Bean, for instance, doesn’t have a simulator target. The presence of the simulator build target in 2.3/Gingerbread and before didn’t mean that you could run the AOSP on your desktop. In fact you couldn’t, if only because you needed a kernel that had Binder included and you would’ve needed to be using Bionic instead of your system’s default C library. But, if you wanted to run parts of what’s built from the AOSP on your desktop, this product target allowed you to do so. In 2.3/Gingerbread, various parts of the code build very differently if the target is the simu- lator. When browsing the code, for example, you’ll sometimes find conditional builds around the HAVE_ANDROID_OS C macro, which is only defined when compiling for the sim- ulator. The code that talks to the Binder is one of these. If HAVE_ANDROID_OS is not defined, that code will return an error to its caller instead of trying to actually talk to the Binder driver. For the full story behind the simulator, have a look at Android developer Andrew McFadden’s response to a post entitled “Android Simulator Environment” on the an- droid-porting mailing list in April 2009.
  • 11. In addition to selecting which parts of the AOSP to build and which options to build them with, the build system also needs to know about the target it’s building for. This is provided through a BoardConfig.mk file, which will specify things such as the command line to be provided to the kernel, the base address at which the kernel should be loaded, or the instruction set version most appropriate for the board’s CPU ( TARGET_ARCH_VARIANT ). Have a look at build/target/board/ for a set of per-target directories that each contain a BoardConfig.mk file. Also have a look at the various device/*/ TARGET_DEVICE /BoardConfig.mk files included in the AOSP. The latter are much richer than the former because they contain a lot more hard- ware-specific information. The device name (i.e., TARGET_DEVICE ) is derived from the PRODUCT_DEVICE specified in the product .mk file provided for the TARGET_PRODUCT set in the configuration. In 2.3/Gingerbread, for example, device/samsung/crespo/AndroidProducts.mk includes device/samsung/crespo/full_crespo.mk, which sets PRODUCT_DEVICE to crespo . Hence, the build system looks for a BoardConfig.mk in device/*/crespo/, and there happens to be one at that location. The same goes on in 4.2/Jelly Bean for the PRODUCT_DEVICE set in device/asus/grouper/full_grouper.mk to grouper , thereby pointing the build system to device/*/grouper/BoardConfig.mk. The final piece of the puzzle with regard to configuration is the CPU-specific op- tions used to build Android. For ARM, those are contained in build/core/combo/arch/arm/armv*.mk, with TARGET_ARCH_VARIANT determining the actual file to use. Each file lists CPU-specific cross-compiler and cross-linker flags used for building C/C++ files. They also contain a number of ARCH_ARM_HAVE_* variables that enable others parts of the AOSP to build code conditionally based on whether a given ARM feature is found in the target’s CPU. envsetup.sh Now that you understand the kinds of configuration input the build system needs, we can discuss the role of envsetup.sh in more detail. As its name implies,
  • 12. envsetup.sh actually is for setting up a build environment for Android. It does only part of the job, though. Mainly, it defines a series of shell commands that are useful to any sort of AOSP work: In 4.2/Jelly Bean, hmm has replaced help, and the command set made available to you has been expanded: $ cd ~/android/aosp-2.3.x $ . build/envsetup.sh $ help Invoke ". build/envsetup.sh" from your shell to add the following functions to your environment: - croot: Changes directory to the top of the tree. - m: Makes from the top of the tree. - mm: Builds all of the modules in the current directory. - mmm: Builds all of the modules in the supplied directories. - cgrep: Greps on all local C/C++ files. - jgrep: Greps on all local Java files. - resgrep: Greps on all local res/*.xml files. - godir: Go to the directory containing a file. Look at the source to view more functions. The complete list is: add_lunch_combo cgrep check_product check_variant choosecombo chooseproduct choo setype choosevariant cproj croot findmakefile gdbclient get_abs_build_var getbug reports get_build_var getprebuilt gettop godir help isviewserverstarted jgrep lu nch m mm mmm pgrep pid printconfig print_lunch_menu resgrep runhat runtest set_j ava_home setpaths set_sequence_number set_stuff_for_environment settitle smokete st startviewserver stopviewserver systemstack tapas tracedmdump $ cd ~/android/aosp-4.2 $ . build/envsetup.sh $ hmm
  • 13. You’ll likely find the croot and godir commands quite useful for traversing the tree. Some parts of it are quite deep, given the use of Java and its requirement that packages be stored in directory trees bearing the same hierarchy as each sub- part of the corresponding fully qualified package name. For instance, a file part of the com.foo.bar package must be stored under the com/foo/bar/ directory. Hence, it’s not rare to find yourself 7 to 10 directories underneath the AOSP’s top- level directory, and it rapidly becomes tedious to type something like cd ../../../ ... to return to an upper part of the tree. Invoke ". build/envsetup.sh" from your shell to add the following functions to y our environment: - lunch: lunch <product_name>-<build_variant> - tapas: tapas [<App1> <App2> ...] [arm|x86|mips] [eng|userdebug|user] - croot: Changes directory to the top of the tree. - m: Makes from the top of the tree. - mm: Builds all of the modules in the current directory. - mmm: Builds all of the modules in the supplied directories. - cgrep: Greps on all local C/C++ files. - jgrep: Greps on all local Java files. - resgrep: Greps on all local res/*.xml files. - godir: Go to the directory containing a file. Look at the source to view more functions. The complete list is: addcompletions add_lunch_combo cgrep check_product check_variant choosecombo cho oseproduct choosetype choosevariant cproj croot findmakefile gdbclient get_abs_b uild_var getbugreports get_build_var getlastscreenshot getprebuilt getscreenshot path getsdcardpath gettargetarch gettop godir hmm isviewserverstarted jgrep key_ back key_home key_menu lunch _lunch m mm mmm pid printconfig print_lunch_menu re sgrep runhat runtest set_java_home setpaths set_sequence_number set_stuff_for_en vironment settitle smoketest startviewserver stopviewserver systemstack tapas tr acedmdump
  • 14. m and mm are also quite useful since they allow you to, respectively, build from the top level regardless of where you are or just build the modules found in the current directory. For example, if you made a modification to the Launcher and are in packages/apps/Launcher2, you can rebuild just that module by typing mm instead of cd’ing back to the top level and typing make. Note that mm doesn’t re- build the entire tree and, therefore, won’t regenerate AOSP images even if a de- pendent module has changed. m will do that, though. Still, mm can be useful to test whether your local changes break the build or not until you’re ready to regen- erate the full AOSP. Although the online help doesn’t mention lunch, it is one of the commands de- fined by envsetup.sh. When you run lunch without any parameters, it shows you a list of potential choices. This is the list from 2.3/Gingerbread: This is the list from 4.2/Jelly Bean: $ lunch You're building on Linux Lunch menu... pick a combo: 1. generic-eng 2. simulator 3. full_passion-userdebug 4. full_crespo4g-userdebug 5. full_crespo-userdebug Which would you like? [generic-eng] $ lunch
  • 15. These choices are not static. Most depend on what’s in the AOSP at the time envsetup.sh runs. They’re in fact individually added using the add_lunch_combo() function that the script defines. In 2.3/Gingerbread, for in- stance, envsetup.sh adds generic-eng and simulator by default: You're building on Linux Lunch menu... pick a combo: 1. full-eng 2. full_x86-eng 3. vbox_x86-eng 4. full_mips-eng 5. full_grouper-userdebug 6. full_tilapia-userdebug 7. mini_armv7a_neon-userdebug 8. mini_armv7a-userdebug 9. mini_mips-userdebug 10. mini_x86-userdebug 11. full_mako-userdebug 12. full_maguro-userdebug 13. full_manta-userdebug 14. full_toroplus-userdebug 15. full_toro-userdebug 16. full_panda-userdebug Which would you like? [full-eng] # add the default one here add_lunch_combo generic-eng # if we're on linux, add the simulator. There is a special case # in lunch to deal with the simulator if [ "$(uname)" = "Linux" ] ; then
  • 16. In 4.2/Jelly Bean, simulator is no longer a valid target and envsetup.sh does this instead: envsetup.sh also includes all the vendor-supplied scripts it can find. Here’s how it’s done in 2.3/Gingerbread: Here’s how it’s done in 4.2/Jelly Bean: add_lunch_combo simulator fi # add the default one here add_lunch_combo full-eng add_lunch_combo full_x86-eng add_lunch_combo vbox_x86-eng add_lunch_combo full_mips-eng # Execute the contents of any vendorsetup.sh files we can find. for f in `/bin/ls vendor/*/vendorsetup.sh vendor/*/build/vendorsetup.sh device/* /*/vendorsetup.sh 2> /dev/null` do echo "including $f" . $f done unset f # Execute the contents of any vendorsetup.sh files we can find. for f in `/bin/ls vendor/*/vendorsetup.sh vendor/*/*/vendorsetup.sh device/*/*/v endorsetup.sh 2> /dev/null` do
  • 17. In 2.3/Gingerbread the device/samsung/crespo/vendorsetup.sh file, for instance, does this: Similarly, in 4.2/Jelly Bean the device/asus/grouper/vendorsetup.sh file does this: So that’s how you end up with the menu we saw earlier. Note that the menu asks you to choose a combo. Essentially, this is a combination of a TARGET_PRODUCT and TARGET_BUILD_VARIANT , with the exception of the simulator in 2.3/Gingerbread. The menu provides the default combinations, but the others re- main valid and can be passed to lunch as parameters on the command line. In 2.3/Gingerbread, for instance, you can do something like this: echo "including $f" . $f done unset f add_lunch_combo full_crespo-userdebug add_lunch_combo full_grouper-userdebug $ lunch generic-user ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=2.3.4 TARGET_PRODUCT=generic TARGET_BUILD_VARIANT=user TARGET_SIMULATOR=false
  • 18. Once lunch has finished running for a generic-eng combo, it will set up environ- ment variables described in Table 4-1 in your current shell to provide the build system with the required configuration information. TARGET_BUILD_TYPE=release TARGET_BUILD_APPS= TARGET_ARCH=arm HOST_ARCH=x86 HOST_OS=linux HOST_BUILD_TYPE=release BUILD_ID=GINGERBREAD ============================================ $ lunch full_crespo-eng ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=2.3.4 TARGET_PRODUCT=full_crespo TARGET_BUILD_VARIANT=eng TARGET_SIMULATOR=false TARGET_BUILD_TYPE=release TARGET_BUILD_APPS= TARGET_ARCH=arm HOST_ARCH=x86 HOST_OS=linux HOST_BUILD_TYPE=release BUILD_ID=GINGERBREAD ============================================
  • 19. Table 4-1. Environment variables set by lunch (in no particular order) for the default build target (i.e., generic-eng) in 2.3/Gingerbread Variable Value PATH $ANDROID_JAVA_TOOLCHAIN:$PATH:$ANDROID_BUILD_PATHS ANDROID_EABI_TOOLCHAIN aosp-root /prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin ANDROID_TOOLCHAIN $ANDROID_EABI_TOOLCHAIN ANDROID_QTOOLS aosp-root /development/emulator/qtools ANDROID_BUILD_PATHS aosp-root /out/host/linux- x86:$ANDROID_TOOLCHAIN:$ANDROID_QTOOLS:$ANDROID_TOOLCHAIN:$ANDROID_EABI_TOOLC ANDROID_BUILD_TOP aosp-root ANDROID_JAVA_TOOLCHAIN $JAVA_HOME/bin ANDROID_PRODUCT_OUT aosp-root /out/target/product/generic OUT ANDROID_PRODUCT_OUT BUILD_ENV_SEQUENCE_NUMBER 10 OPROFILE_EVENTS_DIR aosp-root /prebuilt/linux-x86/oprofile TARGET_BUILD_TYPE release
  • 20. Variable Value TARGET_PRODUCT generic TARGET_BUILD_VARIANT eng TARGET_BUILD_APPS empty TARGET_SIMULATOR false PROMPT_COMMAND "033]0;[${TARGET_PRODUCT}-${TARGET_BUILD_VARIANT}] ${USER}@${HOSTNAME}: ${PWD}007" JAVA_HOME /usr/lib/jvm/java-6-sun
  • 21. USING CCACHE If you’ve already done any AOSP building while reading these pages, you’ve noticed how long the process is. Obviously, unless you can construct yourself a bleeding-edge build farm, any sort of speedup on your current hardware would be greatly appreciated. As a sign that the Android development team might itself also feel the pain of the rather long builds, they’ve added support for ccache . ccache stands for Compiler Cache and is part of the Samba Project. It’s a mechanism that caches the object files generated by the com- piler based on the preprocessor’s output. Hence, if under two separate builds the preprocessor’s output is identical, use of ccache will result in the second build not actu- ally using the compiler to build the file. Instead, the cached object file will be copied to the destination where the compiler’s output would have been. To enable the use of ccache , all you need to do is make sure that the USE_CCACHE environ- ment variable is set to 1 before you start your build: You won’t gain any acceleration the first time you run, since the cache will be empty at that time. Every other time you build from scratch, though, the cache will help accelerate the build process. The only downside is that ccache is for C/C++ files only. Hence, it can’t accel- erate the build of any Java file, I must add sadly. In 2.3/Gingerbread, there are about 15,000 C/C++ files and 18,000 Java files in the AOSP. Those numbers are 27,000 and 29,000 in 4.2/Jelly Bean. So, while the cache isn’t a panacea, it’s better than nothing. If you’d like to learn more about ccache , have a look at the article titled “Improve collab- orative build times with ccache” by Martin Brown on IBM’s developerWorks site. The ar- ticle also explores the use of distcc, which allows you to distribute builds over several ma- chines, so you can pool your team’s workstation caches together. For all its benefits, some developers have reported weird errors in some cases when using ccache. For instance, I ran into such issues while maintaining my own AOSP fork. First, I got a version of the AOSP on my workstation and built it, creating a warm cache. I then $ export USE_CCACHE=1
  • 22. proceeded to upload that tree to http://github.com. Finally, I did a repo sync on the tree I had just uploaded but from another directory on my workstation than the original one up- loaded. Using diff to compare both trees showed both trees were identical. Yet, the original built fine with the warm cache while the second continued to fail building until the cache was erased. Of course, if you get tired of always typing build/envsetup.sh and lunch, all you need to do is copy the build/buildspec.mk.default into the top-level directory, re- name it to buildspec.mk, and edit it to match the configuration that would have otherwise been set by running those commands. The file already contains all the variables you need to provide; it’s just a matter of uncommenting the correspond- ing lines and setting the values appropriately. Once you’ve done that, all you have to do is go to the AOSP’s directory and invoke make directly. You can skip envsetup.sh and lunch. Function Definitions Because the build system is fairly large—there are more than 40 .mk files in build/core/ alone—there are benefits in being able to reuse as much code as possi- ble. This is why the build system defines a large number of functions in the definitions.mk file. That file is actually the largest one in the build system at about 60KB, with about 140 functions on about 1,800 lines of makefile code in 2.3/Gingerbread. It’s still the largest file in the build system in 4.2/Jelly Bean at about 73KB, 170 functions, and about 2,100 lines of makefile code. Functions offer a variety of operations, including file lookup (e.g., all-makefiles-under and all-c-files-under ), transformation (e.g., transform-c-to-o and transform- java-to-classes.jar ), copying (e.g., copy-file-to-target ), and utility (e.g., my-dir .)
  • 23. Not only are these functions used throughout the rest of the build system’s compo- nents, acting as its core library, but they’re sometimes also directly used in mod- ules’ Android.mk files. Here’s an example snippet from the Calculator app’s Android.mk: Although thoroughly describing definitions.mk is outside the scope of this book, it should be fairly easy for you to explore it on your own. If nothing else, most of the functions in it are preceded with a comment explaining what they do. Here’s an example from 2.3/Gingerbread: Main Make Recipes At this point you might be wondering where any of the goodies are actually gen- erated. How are the various images such as RAM disk generated or how is the SDK put together, for example? Well, I hope you won’t hold a grudge, but I’ve been keeping the best for last. So without further ado, have a look at the Makefile in LOCAL_SRC_FILES := $(call all-java-files-under, src) ########################################################### ## Find all of the java files under the named directories. ## Meant to be used like: ## SRC_FILES := $(call all-java-files-under,src tests) ########################################################### define all-java-files-under $(patsubst ./%,%, $(shell cd $(LOCAL_PATH) ; find $(1) -name "*.java" -and -not -name ".*") ) endef
  • 24. build/core/ (not the top-level one). The file starts with an innocuous-looking comment: But don’t be fooled. This is where some of the best meat is. Here’s the snippet that takes care of generating the RAM disk, for example, in 2.3/Gingerbread: And here’s the snippet that creates the certs packages for checking over-the-air (OTA) updates in the same AOSP version: # Put some miscellaneous rules here # ----------------------------------------------------------------- # the ramdisk INTERNAL_RAMDISK_FILES := $(filter $(TARGET_ROOT_OUT)/%, $(ALL_PREBUILT) $(ALL_COPIED_HEADERS) $(ALL_GENERATED_SOURCES) $(ALL_DEFAULT_INSTALLED_MODULES)) BUILT_RAMDISK_TARGET := $(PRODUCT_OUT)/ramdisk.img # We just build this directly to the install location. INSTALLED_RAMDISK_TARGET := $(BUILT_RAMDISK_TARGET) $(INSTALLED_RAMDISK_TARGET): $(MKBOOTFS) $(INTERNAL_RAMDISK_FILES) | $(MINIGZIP) $(call pretty,"Target ram disk: $@") $(hide) $(MKBOOTFS) $(TARGET_ROOT_OUT) | $(MINIGZIP) > $@ # ----------------------------------------------------------------- # Build a keystore with the authorized keys in it, used to verify the # authenticity of downloaded OTA packages. #
  • 25. Obviously there’s a lot more than I can fit here, but have a look at Makefile for in- formation on how any of the following are created: Properties (including the target’s /default.prop and /system/build.prop). RAM disk. Boot image (combining the RAM disk and a kernel image). NOTICE files: These are files required by the AOSP’s use of the Apache Software License (ASL). Have a look at the ASL for more information about NOTICE files. OTA keystore. Recovery image. System image (the target’s /system directory). Data partition image (the target’s /data directory). OTA update package. SDK. Nevertheless, some things aren’t in this file: # This rule adds to ALL_DEFAULT_INSTALLED_MODULES, so it needs to come # before the rules that use that variable to build the image. ALL_DEFAULT_INSTALLED_MODULES += $(TARGET_OUT_ETC)/security/otacerts.zip $(TARGET_OUT_ETC)/security/otacerts.zip: KEY_CERT_PAIR := $(DEFAULT_KEY_CERT_PAIR) $(TARGET_OUT_ETC)/security/otacerts.zip: $(addsuffix .x509.pem, $(DEFAULT_KEY_CERT_PAIR)) $(hide) rm -f $@ $(hide) mkdir -p $(dir $@) $(hide) zip -qj $@ $< .PHONY: otacerts otacerts: $(TARGET_OUT_ETC)/security/otacerts.zip
  • 26. Kernel images Don’t look for any rule to build these. There is no kernel part of the official AOSP releases—some of the third-party projects listed in Appendix E, how- ever, actually do package kernel sources directly into the AOSPs they dis- tribute. Instead, you need to find an Androidized kernel for your target, build it separately from the AOSP, and feed it to the AOSP. You can find a few examples of this in the devices in the device/ directory. In 2.3/Gingerbread, for example, device/samsung/crespo/ includes a kernel im- age (file called kernel) and a loadable module for the Crespo’s WiFi (bcm4329.ko file). Both of these are built outside the AOSP and copied in bi- nary form into the tree for inclusion with the rest of the build. NDK While the code to build the NDK is in the AOSP, it’s entirely separate from the AOSP’s build system in build/. Instead, the NDK’s build system is in ndk/build/. We’ll discuss how to build the NDK shortly. CTS The rules for building the CTS are in build/core/tasks/cts.mk. Cleaning As I mentioned earlier, a make clean is very much the equivalent of wiping out the out/ directory. The clean target itself is defined in main.mk. There are, how- ever, other cleanup targets. Most notably, installclean , which is defined in cleanbuild.mk, is automatically invoked whenever you change TARGET_PRODUCT , TARGET_BUILD_VARIANT or PRODUCT_LOCALES . For instance, if I had first built 2.3/Gingerbread for the generic-eng combo and then used lunch to switch the combo to full-eng , the next time I started make, some of the build output would be automatically pruned using installclean :
  • 27. In contrast to clean , installclean doesn’t wipe out the entirety of out/. Instead, it only nukes the parts that need rebuilding given the combo configuration change. There’s also a clobber target which is essentially the same thing as a clean . Module Build Templates What I just described is the build system’s architecture and the mechanics of its core components. Having read that, you should have a much better idea of how Android is built from a top-down perspective. Very little of that, however, perme- ates down to the level of AOSP modules’ Android.mk files. The system has in fact $ make -j16 ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=2.3.4 TARGET_PRODUCT=full TARGET_BUILD_VARIANT=eng ... ============================================ *** Build configuration changed: "generic-eng-{mdpi,nodpi}" -> "full-eng-{en_US, en_GB,fr_FR,it_IT,de_DE,es_ES,mdpi,nodpi}" *** Forcing "make installclean"... *** rm -rf out/target/product/generic/data/* out/target/product/generic/data-qem u/* out/target/product/generic/userdata-qemu.img out/host/linux-x86/obj/NOTICE_F ILES out/host/linux-x86/sdk out/target/product/generic/*.img out/target/product/ generic/*.txt out/target/product/generic/*.xlb out/target/product/generic/*.zip out/target/product/generic/data out/target/product/generic/obj/APPS out/target/p roduct/generic/obj/NOTICE_FILES out/target/product/generic/obj/PACKAGING out/tar get/product/generic/recovery out/target/product/generic/root out/target/product/ generic/system out/target/product/generic/dex_bootjars out/target/product/generi c/obj/JAVA_LIBRARIES *** Done with the cleaning, now starting the real build.
  • 28. been architected so that module build recipes are pretty much independent from the build system’s internals. Instead, build templates are provided so that module authors can get their modules built appropriately. Each template is tailored for a specific type of module, and module authors can use a set of documented vari- ables, all prefixed by LOCAL_ , to modulate the templates’ behavior and output. Of course, the templates and underlying support files (mainly base_rules.mk) closely interact with the rest of the build system to deal properly with each module’s build output. But that’s invisible to the module’s author. The templates are themselves found in the same location as the rest of the build system in build/core/. Android.mk gets access to them through the include direc- tive. Here’s an example: As you can see, Android.mk files don’t actually include the .mk templates by name. Instead, they include a variable that is set to the corresponding .mk file. Table 4-2 provides the full list of available module templates. include $(BUILD_PACKAGE)
  • 29. Table 4-2. Module build templates list Variable Template What It Builds Most Notable Use BUILD_EXECUTABLE executable.mk Target binaries Native commands and daemons BUILD_HOST_EXECUTABLE host_executable.mk Host binaries Development tools BUILD_RAW_EXECUTABLE raw_executable.mk Target binaries that run on bare metal Code in the bootloader/ directory BUILD_JAVA_LIBRARY java_library.mk Target Java libaries Apache Harmony and Android Framework BUILD_STATIC_JAVA_LIBRARY static_java_library.mk Target static Java libraries N/A, few modules use this BUILD_HOST_JAVA_LIBRARY host_java_library.mk Host Java libraries Development tools
  • 30. Variable Template What It Builds Most Notable Use BUILD_SHARED_LIBRARY shared_library.mk Target shared libraries A vast number of modules, including many in external/ and frameworks/base/ BUILD_STATIC_LIBRARY static_library.mk Target static libraries A vast number of modules, including many in external/ BUILD_HOST_SHARED_LIBRARY host_shared_library.mk Host shared libraries Development tools BUILD_HOST_STATIC_LIBRARY host_static_library.mk Host static libraries Development tools BUILD_RAW_STATIC_LIBRARY raw_static_library.mk Target static libraries that run on bare metal Code in bootloader/ BUILD_PREBUILT prebuilt.mk Copies prebuilt Configuration files and binaries
  • 31. Variable Template What It Builds Most Notable Use target files BUILD_HOST_PREBUILT host_prebuilt.mk Copies prebuilt host files Tools in prebuilt/ and configuration files BUILD_MULTI_PREBUILT multi_prebuilt.mk Copies prebuilt modules of multiple but known types, like Java libraries or executables Rarely used BUILD_PACKAGE package.mk Built-in AOSP apps (i.e., anything that ends up being an .apk) All apps in the AOSP BUILD_KEY_CHAR_MAP key_char_map.mk Device character All device character maps
  • 32. Variable Template What It Builds Most Notable Use maps in AOSP These build templates allow Android.mk files to be usually fairly lightweight: Tells the build template where the current module is located. Clears all previously set LOCAL_* variables that might have been set for other modules. Sets various LOCAL_* variables to module-specific values. LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_VARIABLE_1 := value_1 LOCAL_VARIABLE_2 := value_2 ... include $(BUILD_MODULE_TYPE)
  • 33. Invokes the build template that corresponds to the current module’s type. NOTE Note that CLEAR_VARS , which is provided by clear_vars.mk, is very impor- tant. Recall that the build system includes all Android.mk into what amounts to a single huge makefile. Including CLEAR_VARS ensures that the LOCAL_* values set for modules preceding yours are zeroed out by the time your Android.mk is included. Also, a single Android.mk can describe multiple mod- ules one after the other. Hence, CLEAR_VARS ensures that previous module recipes don’t pollute subsequent ones. Here’s the Service Manager’s Android.mk in 2.3/Gingerbread, for instance (frameworks/base/cmds/servicemanager/): And here’s the one from 2.3/Gingerbread’s Desk Clock app (packages/app/DeskClock/): [19] [20] LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_SHARED_LIBRARIES := liblog LOCAL_SRC_FILES := service_manager.c binder.c LOCAL_MODULE := servicemanager ifeq ($(BOARD_USE_LVMX),true) LOCAL_CFLAGS += -DLVMX endif include $(BUILD_EXECUTABLE) [21]
  • 34. As you can see, essentially the same structure is used in both modules, even though they provide very different input and result in very different output. Notice also the last line from the Desk Clock’s Android.mk, which basically in- cludes all subdirectories’ Android.mk files. As I said earlier, the build system looks for the first makefile in a hierarchy and doesn’t look in any subdirectories under- neath the directory where one was found, hence the need to manually invoke those. Obviously, the code here just goes out and looks for all makefiles under- neath. However, some parts of the AOSP either explicitly list subdirectories or conditionally select them based on configuration. The documentation at http://source.android.com used to provide an exhaustive list of all the LOCAL_* variables with their meaning and use. Unfortunately, at the time of this writing, this list is no longer available. The build/core/build- system.html file, however, contains an earlier version of that list, and you should refer to that one until up-to-date lists become available again. Here are some of the most frequently encountered LOCAL_* variables: LOCAL_PATH LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE_TAGS := optional LOCAL_SRC_FILES := $(call all-java-files-under, src) LOCAL_PACKAGE_NAME := DeskClock LOCAL_OVERRIDES_PACKAGES := AlarmClock LOCAL_SDK_VERSION := current include $(BUILD_PACKAGE) include $(call all-makefiles-under,$(LOCAL_PATH))
  • 35. The path of the current module’s sources, typically provided by invoking $(call my-dir) . LOCAL_MODULE The name to attribute to this module’s build output. The actual filename or output and its location will depend on the build template you include. If this is set to foo , for example, and you build an executable, then the final executable will be a command called foo and it will be put in the target’s /system/bin/. If LOCAL_MODULE is set to libfoo and you include BUILD_SHARED_LIBRARY instead of BUILD_EXECUTABLE , the build system will generate libfoo.so and put it in /system/lib/. Note that the name you provide here must be unique for the particular module class (i.e., build template type) you are building. There can’t be two libfoo.so libraries, for instance. It’s expected that the module name will have to be globally unique (i.e., across all module classes) at some point in the future. LOCAL_SRC_FILES The source files used to build the module. You may provide those by using one of the build system’s defined functions, as the Desk Clock uses all- java-files-under , or you may list the files explicitly, as the Service Manager does. LOCAL_PACKAGE_NAME Unlike all other modules, apps use this variable instead of LOCAL_MODULE to provide their names, as you can witness by comparing the two Android.mk files shown earlier. LOCAL_SHARED_LIBRARIES
  • 36. Use this to list all the libraries your module depends on. As mentioned ear- lier, the Service Manager’s dependency on liblog is specified using this variable. LOCAL_MODULE_TAGS As I mentioned earlier, this allows you to control under which TARGET_BUILD_VARIANT this module is built. Usually, this should just be set to optional . LOCAL_MODULE_PATH Use this to override the default install location for the type of module you’re building. A good way to find out about more LOCAL_* variables is to look at existing Android.mk files in the AOSP. Also, clear_vars.mk contains the full list of variables that are cleared. So while it doesn’t give you the meaning of each, it certainly lists them all. Also, in addition to the cleaning targets that affect the AOSP globally, each module can define its own cleaning rules by providing a CleanSpec.mk, much like modules provide Android.mk files. Unlike the latter, though, the former aren’t required. By default, the build system has cleaning rules for each type of module. But you can specify your own rules in a CleanSpec.mk in case your module’s build does some- thing the build system doesn’t generate by default and, therefore, wouldn’t typi- cally know how to clean up. Output Now that we’ve looked at how the build system works and how module build tem- plates are used by modules, let’s look at the output it creates in out/. At a fairly high level, the build output operates in three stages and in two modes, one for the host and one for the target:
  • 37. 1. Intermediates are generated using the module sources. These intermedi- ates’ format and location depend on the module’s sources. They may be .o files for C/C++ code, for example, or .jar files for Java-based code. 2. Intermediates are used by the build system to create actual binaries and packages: taking .o files, for example, and linking them into an actual binary. 3. The binaries and packages are assembled together into the final output re- quested of the build system. Binaries, for instance, are copied into directo- ries containing the root and /system filesystems, and images of those filesys- tems are generated for use on the actual device. out/ is mainly separated into two directories, reflecting its operating modes: host/ and target/. In each directory, you will find a couple of obj/ directories that con- tain the various intermediates generated during the build. Most of these are stored in subdirectories named like the one that the BUILD_* macros presented earlier and serve a specific complementary purpose during the build system’s operation: EXECUTABLES/ JAVA_LIBRARIES/ SHARED_LIBRARIES/ STATIC_LIBRARIES/ APPS/ DATA/ ETC/ KEYCHARS/ PACKAGING/ NOTICE_FILES/ include/ lib/
  • 38. The directory you’ll likely be most interested in is out/target/product/ PRODUCT_DEVICE /. That’s where the output images will be lo- cated for the PRODUCT_DEVICE defined in the corresponding product configuration’s .mk. Table 4-3 explains the content of that directory.
  • 39. Table 4-3. Product output Entry Description android-info.txt Contains the code name for the board for which this product is configured clean_steps.mk Contains a list of steps that must be executed to clean the tree, as provided in CleanSpec.mk files by calling the add-clean-step function data/ The target’s /data directory installed-files.txt A list of all the files installed in data/ and system/ directories obj/ The target product’s intermediaries previous_build_config.mk The last build target; will be used on the next make to check if the config has changed, thereby forcing an installclean ramdisk.img The RAM disk image generated based on the content of the root/ directory root/ The content of the target’s root filesystem symbols/ Unstripped versions of the binaries put in the root filesystem and /system directory
  • 40. Entry Description system/ The target’s /system directory system.img The /system image, based on the content of the system/ directory userdata.img The /data image, based on the content of the data/ directory Have a look back at Chapter 2 for a refresher on the root filesystem, /system, and /data. Essentially, though, when the kernel boots, it will mount the RAM disk im- age and execute the /init found inside. That binary, in turn, will run the /init.rc script that will mount both the /system and /data images at their respective loca- tions. We’ll come back to the root filesystem layout and the system’s operation at boot time in Chapter 6. Build Recipes With the build system’s architecture and functioning in mind, let’s take a look at some of the most common, and some slightly uncommon, build recipes. We’ll only lightly touch on using the results of each recipe, but you should have enough in- formation to get started. The Default droid Build Earlier, we went through a number of plain make commands but never really ex- plained the default target. When you run plain make, it’s as if you had typed:[22]
  • 41. droid is in fact the default target as defined in main.mk. You don’t usually need to specify this target manually. I’m providing it here for completeness, so you know it exists. Seeing the Build Commands When you build the AOSP, you’ll notice that it doesn’t actually show you the com- mands it’s running. Instead, it prints out only a summary of each step it’s at. If you want to see everything it does, like the gcc command lines for example, add the showcommands target to the command line: $ make droid $ make showcommands ... host Java: apicheck (out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/c lasses) for f in ; do if [ ! -f $f ]; then echo Missing file $f; exit 1; fi; unzip -qo $ f -d out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes; (cd ou t/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes && rm -rf META-I NF); done javac -J-Xmx512M -target 1.5 -Xmaxerrs 9999999 -encoding ascii -g -extdirs "" -d out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes @out/host /common/obj/JAVA_LIBRARIES/apicheck_intermediates/java-source-list-uniq || ( rm -rf out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes ; exit 41 ) rm -f out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/java-source-list rm -f out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/java-source-list -uniq jar -cfm out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/javalib.jar b uild/tools/apicheck/src/MANIFEST.mf -C out/host/common/obj/JAVA_LIBRARIES/apich eck_intermediates/classes . Header: out/host/linux-x86/obj/include/libexpat/expat.h
  • 42. Illustrating what I explained in the previous section, this is the same as: As you’ll rapidly notice when using this, it generates a lot of output and is there- fore hard to follow. You may, however, want to save the standard output and stan- dard error into files if you’d like to analyze the actual commands used to build the AOSP: You can also do something like this to merge all output into a single file: Some also report that they prefer using the nohup command instead: cp -f external/expat/lib/expat.h out/host/linux-x86/obj/include/libexpat/expat.h Header: out/host/linux-x86/obj/include/libexpat/expat_external.h cp -f external/expat/lib/expat_external.h out/host/linux-x86/obj/include/libexpa t/expat_external.h Header: out/target/product/generic/obj/include/libexpat/expat.h cp -f external/expat/lib/expat.h out/target/product/generic/obj/include/libexpat /expat.h ... $ make droid showcommands $ make showcommands > aosp-build-stdout 2> aosp-build-stderr $ make showcommands 2>&1 | tell build.log $ nohup make showcommands
  • 43. Building the SDK for Linux and Mac OS The official Android SDK is available at http://developer.android.com. You can, however, build your own SDK using the AOSP if, for instance, you extended the core APIs to expose new functionality and would like to distribute the result to de- velopers so they can benefit from your new APIs. To do so, you’ll need to select a special combo: Once this is done, the SDK will be in out/host/linux-x86/sdk/ when built on Linux and in out/host/darwin-x86/sdk/ when built on a Mac. There will be two copies, one a ZIP file, much like the one distributed at http://developer.android.com, and one uncompressed and ready to use. Assuming you had already configured Eclipse for Android development using the instructions at http://developer.android.com, you’ll need to carry out two addi- tional steps to use your newly built SDK. First, you’ll need to tell Eclipse the loca- tion of the new SDK. To do so, go to Window→Preferences→Android, enter the path to the new SDK in the SDK Location box, and click OK. Also, for reasons that aren’t entirely clear to me at the time of this writing, you also need to go to Window→Android SDK Manager, deselect all the items that might be selected ex- cept the first two under Tools, and then click “Install 2 packages...” Once that is done, you’ll be able to create new projects using the new SDK and access any new APIs you expose in it. If you don’t do that second step, you’ll be able to create new Android projects, but none of them will resolve Java libraries properly and will, therefore, never build. $ . build/envsetup.sh $ lunch sdk-eng $ make sdk
  • 44. Building the SDK for Windows The instructions for building the SDK for Windows are slightly different from Linux and Mac OS: The resulting output will be in out/host/windows/sdk/. Building the CTS If you want to build the CTS, you don’t need to use envsetup.sh or lunch. You can go right ahead and type: $ . build/envsetup.sh $ lunch sdk-eng $ make win_sdk $ make cts ... Generating test description for package android.sax Generating test description for package android.performance Generating test description for package android.graphics Generating test description for package android.database Generating test description for package android.text Generating test description for package android.webkit Generating test description for package android.gesture Generating test plan CTS Generating test plan Android Generating test plan Java Generating test plan VM Generating test plan Signature Generating test plan RefApp Generating test plan Performance
  • 45. The cts command includes its own online help. Here’s the corresponding sample output from 2.3/Gingerbread: Generating test plan AppSecurity Package CTS: out/host/linux-x86/cts/android-cts.zip Install: out/host/linux-x86/bin/adb $ cd out/host/linux-x86/bin/ $ ./cts Listening for transport dt_socket at address: 1337 Android CTS version 2.3_r3 $ cts_host > help Usage: command options Available commands and options: Host: help: show this message exit: exit cts command line Plan: ls --plan: list available plans ls --plan plan_name: list contents of the plan with specified name add --plan plan_name: add a new plan with specified name add --derivedplan plan_name -s/--session session_id -r/--result result_type: derive a plan from the given session rm --plan plan_name/all: remove a plan or all plans from repository start --plan test_plan_name: run a test plan start --plan test_plan_name -d/--device device_ID: run a test plan using the specified device start --plan test_plan_name -t/--test test_name: run a specific test ... $ cts_host > ls --plan List of plans (8 in total): Signature RefApp
  • 46. Once you have a target up and running, such as the emulator, you can launch the test suite and it will use adb to run tests on the target: Building the NDK As I had mentioned earlier, the NDK has its own separate build system, with its own setup and help system, which you can invoke like this: VM Performance AppSecurity Android Java CTS $ ./cts start --plan CTS Listening for transport dt_socket at address: 1337 Android CTS version 2.3_r3 Device(emulator-5554) connected cts_host > start test plan CTS CTS_INFO >>> Checking API... CTS_INFO >>> This might take several minutes, please be patient... ... $ cd ndk/build/tools $ export ANDROID_NDK_ROOT= aosp-root /ndk $ ./make-release --help Usage: make-release.sh [options] Valid options (defaults are in brackets):
  • 47. When you are ready to build the NDK, you can invoke make-release as follows, and witness its rather emphatic warning: --help Print this help. --verbose Enable verbose mode. --release=name Specify release name [20110921] --prefix=name Specify package prefix [android-ndk] --development=path Path to development/ndk directory [/home/karim/ opersys-dev/android/aosp-2.3.4/development/ndk] --out-dir=path Path to output directory [/tmp/ndk-release] --force Force build (do not ask initial question) [no] --incremental Enable incremental packaging (debug only). [no] --darwin-ssh=hostname Specify Darwin hostname to ssh to for the build. --systems=list List of host systems to build for [linux-x86] --toolchain-src-dir=path Use toolchain sources from path $ ./make-release IMPORTANT WARNING !! This script is used to generate an NDK release package from scratch for the following host platforms: linux-x86 This process is EXTREMELY LONG and may take SEVERAL HOURS on a dual-core machine. If you plan to do that often, please read docs/DEVELOPMENT.TXT that provides instructions on how to do that more easily. Are you sure you want to do that [y/N] y Downloading toolchain sources... ...
  • 48. Updating the API The build systems has safeguards in case you modify the AOSP’s core API. If you do, the build will fail by default with a warning such as this: As the error message suggests, to get the build to continue, you’ll need to do some- thing like this: ****************************** You have tried to change the API from what has been previously approved. To make these errors go away, you have two choices: 1) You can add "@hide" javadoc comments to the methods, etc. listed in the errors above. 2) You can update current.xml by executing the following command: make update-api To submit the revised current.xml to the main Android repository, you will need approval. ****************************** make: *** [out/target/common/obj/PACKAGING/checkapi-current-timestamp] Error 38 make: *** Waiting for unfinished jobs.... $ make update-api ... Install: out/host/linux-x86/framework/apicheck.jar Install: out/host/linux-x86/framework/clearsilver.jar Install: out/host/linux-x86/framework/droiddoc.jar Install: out/host/linux-x86/lib/libneo_util.so
  • 49. The next time you start make, you won’t get any more errors regarding API changes. Obviously at this point you’re no longer compatible with the official APIs and are therefore unlikely to be able to get certified as an “Android” device by Google. Building a Single Module Up to now, we’ve looked at building the entire tree. You can also build individual modules. Here’s how you can ask the build system to build the Launcher2 module (i.e., the Home screen): Install: out/host/linux-x86/lib/libneo_cs.so Install: out/host/linux-x86/lib/libneo_cgi.so Install: out/host/linux-x86/lib/libclearsilver-jni.so Copying: out/target/common/obj/JAVA_LIBRARIES/core_intermediates/emma_out/lib/cl asses-jarjar.jar Install: out/host/linux-x86/framework/dx.jar Install: out/host/linux-x86/bin/dx Install: out/host/linux-x86/bin/aapt Copying: out/target/common/obj/JAVA_LIBRARIES/bouncycastle_intermediates/emma_ou t/lib/classes-jarjar.jar Copying: out/target/common/obj/JAVA_LIBRARIES/ext_intermediates/emma_out/lib/cla sses-jarjar.jar Install: out/host/linux-x86/bin/aidl Copying: out/target/common/obj/JAVA_LIBRARIES/core-junit_intermediates/emma_out/ lib/classes-jarjar.jar Copying: out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/emma_out/l ib/classes-jarjar.jar Copying current.xml $ make Launcher2
  • 50. You can also clean modules individually: If you’d like to force the build system to regenerate the system image to include your updated module, you can add the snod target to the command line: Building Out of Tree If you’d ever like to build code against the AOSP and its Bionic library but don’t want to incorporate that into the AOSP, you can use a makefile such as the follow- ing to get the job done: $ make clean-Launcher2 $ make Launcher2 snod ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=2.3.4 TARGET_PRODUCT=generic ... target Package: Launcher2 (out/target/product/generic/obj/APPS/Launcher2_interme diates/package.apk) 'out/target/common/obj/APPS/Launcher2_intermediates//classes.dex' as 'classes.d ex'... Install: out/target/product/generic/system/app/Launcher2.apk Install: out/host/linux-x86/bin/mkyaffs2image make snod: ignoring dependencies Target system fs image: out/target/product/generic/system.img [23] # Paths and settings TARGET_PRODUCT = generic ANDROID_ROOT = /home/karim/android/aosp-2.3.x BIONIC_LIBC = $(ANDROID_ROOT)/bionic/libc
  • 51. PRODUCT_OUT = $(ANDROID_ROOT)/out/target/product/$(TARGET_PRODUCT) CROSS_COMPILE = $(ANDROID_ROOT)/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi- # Tool names AS = $(CROSS_COMPILE)as AR = $(CROSS_COMPILE)ar CC = $(CROSS_COMPILE)gcc CPP = $(CC) -E LD = $(CROSS_COMPILE)ld NM = $(CROSS_COMPILE)nm OBJCOPY = $(CROSS_COMPILE)objcopy OBJDUMP = $(CROSS_COMPILE)objdump RANLIB = $(CROSS_COMPILE)ranlib READELF = $(CROSS_COMPILE)readelf SIZE = $(CROSS_COMPILE)size STRINGS = $(CROSS_COMPILE)strings STRIP = $(CROSS_COMPILE)strip export AS AR CC CPP LD NM OBJCOPY OBJDUMP RANLIB READELF SIZE STRINGS STRIP # Build settings CFLAGS = -O2 -Wall -fno-short-enums HEADER_OPS = -I$(BIONIC_LIBC)/arch-arm/include -I$(BIONIC_LIBC)/kernel/common -I$(BIONIC_LIBC)/kernel/arch-arm LDFLAGS = -nostdlib -Wl,-dynamic-linker,/system/bin/linker $(PRODUCT_OUT)/obj/lib/crtbegin_dynamic.o $(PRODUCT_OUT)/obj/lib/crtend_android.o -L$(PRODUCT_OUT)/obj/lib -lc -ldl # Installation variables EXEC_NAME = example-app INSTALL = install
  • 52. In this case, you don’t need to care about either envsetup.sh or lunch. You can just go ahead and type the magic incantation: Obviously this won’t add your binary to any of the images generated by the AOSP. Even the install target here will be of value only if you’re mounting the target’s INSTALL_DIR = $(PRODUCT_OUT)/system/bin # Files needed for the build OBJS = example-app.o # Make rules all: example-app .c.o: $(CC) $(CFLAGS) $(HEADER_OPS) -c $< example-app: ${OBJS} $(CC) -o $(EXEC_NAME) ${OBJS} $(LDFLAGS) install: example-app test -d $(INSTALL_DIR) || $(INSTALL) -d -m 755 $(INSTALL_DIR) $(INSTALL) -m 755 $(EXEC_NAME) $(INSTALL_DIR) clean: rm -f *.o $(EXEC_NAME) core distclean: rm -f *~ rm -f *.o $(EXEC_NAME) core $ make
  • 53. filesystem off NFS, and that’s valuable only during debugging, which is what this makefile is assumed to be useful for. To an extent, it could also be argued that us- ing such a makefile is actually counterproductive, since it’s far more complicated than the equivalent Android.mk that would result if this code were added as a module part of the AOSP. Still, this kind of hack can have its uses. Under certain circumstances, for in- stance, it might make sense to modify the conventional build system used by a rather large codebase to build that project against the AOSP yet outside of it; the alternative being to copy the project into the AOSP and create Android.mk files to reproduce the mechanics of its original conventional build system, which might turn out to be a substantial endeavor in and of itself. Building Recursively, In-Tree You can, if you really want to, hack yourself a makefile to build within the AOSP a component that is based on recursive makefiles instead of trying to reproduce the same functionality using Android.mk files, as was suggested in the last section. Several of the AOSP forks mentioned in Appendix E, for instance, include the ker- nel sources at the top level of the AOSP and modify the AOSP’s main makefile to invoke the kernel’s existing build system. Here’s another example where an Android.mk was created by Linaro’s Bernhard Rosenkränzer in order to build ffmpeg—which relies on a GNU autotools-like script—using its original build files: include $(CLEAR_VARS) FFMPEG_TCDIR := $(realpath $(shell dirname $(TARGET_TOOLS_PREFIX))) FFMPEG_TCPREFIX := $(shell basename $(TARGET_TOOLS_PREFIX)) # FIXME remove -fno-strict-aliasing once the aliasing violations are fixed FFMPEG_COMPILER_FLAGS = $(subst -I ,-I../../,$(subst -include system/core/include/arch/linux-arm/AndroidConfig.h,,$(subst -include
  • 54. build/core/combo/include/arch/linux-arm/AndroidConfig.h,, $(TARGET_GLOBAL_CFLAGS)))) -fno-strict-aliasing -Wno-error=address -Wno-error=format-security ifneq ($(strip $(SHOW_COMMANDS)),) FF_VERBOSE="V=1" endif .PHONY: ffmpeg droidcore: ffmpeg systemtarball: ffmpeg REALTOP=$(realpath $(TOP)) ffmpeg: x264 $(PRODUCT_OUT)/obj/STATIC_LIBRARIES/libvpx_intermediates/libvpx.a mkdir -p $(PRODUCT_OUT)/obj/ffmpeg cd $(PRODUCT_OUT)/obj/ffmpeg && export PATH=$(FFMPEG_TCDIR):$(PATH) && $(REALTOP)/external/ffmpeg/configure --arch=arm --target-os=linux --prefix=/system --bindir=/system/bin --libdir=/system/lib --enable-shared --enable-gpl --disable-avdevice --enable-runtime-cpudetect --disable-libvpx --enable-libx264 --enable-cross-compile --cross-prefix=$(FFMPEG_TCPREFIX) --extra-ldflags="-nostdlib -Wl,-dynamic-linker, /system/bin/linker,-z,muldefs$(shell if test $(PRODUCT_SDK_VERSION) -lt 16;
  • 55. Basic AOSP Hacks You most likely bought this book with one thing in mind: to hack the AOSP to fit your needs. Over the next few pages, we’ll start looking into some of the most ob- vious hacks you’ll likely want to try. Of course we’re only setting the stage here with the parts that pertain to the build system, which is where you’ll likely want to start anyway. NOTE While the following explanations are based on 2.3/Gingerbread, they’ll work just the same on 4.2/Jelly Bean, and likely many versions after that one, too. The fact is, these mechanisms have been constant for quite some time. Still, where relevant, changes in 4.2/Jelly Bean are highlighted. then echo -n ',-T$(REALTOP)/$(BUILD_SYSTEM)/armelf.x'; fi),-z,nocopyreloc, --no-undefined -L$(REALTOP)/$(TARGET_OUT_STATIC_LIBRARIES) -L$(REALTOP)/$(PRODUCT_OUT)/system/lib -L$(REALTOP)/$(PRODUCT_OUT)/obj/STATIC_LIBRARIES/libvpx_intermediates -ldl -lc" --extra-cflags="$(FFMPEG_COMPILER_FLAGS) -I$(REALTOP)/bionic/libc/include -I$(REALTOP)/bionic/libc/kernel/common -I$(REALTOP)/bionic/libc/kernel/arch-arm -I$(REALTOP)/bionic/libc/arch-arm/include -I$(REALTOP)/bionic/libm/include -I$(REALTOP)/external/libvpx -I$(REALTOP)/external/x264" --extra-libs="-lgcc" && $(MAKE) TARGET_CRTBEGIN_DYNAMIC_O=$(REALTOP)/$(TARGET_CRTBEGIN_DYNAMIC_O) TARGET_CRTEND_O=$(REALTOP)/$(TARGET_CRTEND_O) $(FF_VERBOSE) && $(MAKE) install DESTDIR=$(REALTOP)/$(PRODUCT_OUT)
  • 56. Adding a Device Adding a custom device is most likely one of the topmost items (if not the top- most) on your list of reasons for reading this book. I’m about to show you how to do just that, so you’ll likely want to bookmark this section. Of course I’m actually only showing you the build aspects of the work. There are a lot more steps in- volved in porting Android to new hardware. Still, adding the new device to the build system will definitely be one of the first things you do. Fortunately, doing that is relatively straightforward. For the purposes of the current exercise, assume you work for a company called ACME and that you’re tasked with delivering its latest gizmo: the CoyotePad, in- tended to be the best platform for playing all bird games. Let’s get started by cre- ating an entry for our new device in device/: The first thing we’ll need in here is an AndroidProducts.mk file to describe the various AOSP products that could be built for the CoyotePad: While we could describe several products (see build/target/product/AndroidProducts.mk for an example), the typical case is to specify just one, as in this case, and it’s described in full_coyotepad.mk: $ cd ~/android/aosp-2.3.x $ . build/envsetup.sh $ mkdir -p device/acme/coyotepad $ cd device/acme/coyotepad PRODUCT_MAKEFILES := $(LOCAL_DIR)/full_coyotepad.mk
  • 57. It’s worth taking a closer look at this makefile. First, we’re using the inherit- product function to tell the build system to pull in other product descriptions as the basis of ours. This allows us to build on other people’s work and not have to specify from scratch every bit and piece of the AOSP that we’d like to include. languages_full.mk will pull in a vast number of locales, and full.mk will make sure we get the same set of modules as if we had built using the full-eng combo. With regard to the other variables: DEVICE_PACKAGE_OVERLAYS Allows us to specify a directory that will form the basis of an overlay that will be applied onto the AOSP’s sources, thereby allowing us to substitute default package resources with device-specific resources. You’ll find this useful if you’d like to set custom layouts or colors for Launcher2 or other apps, for instance. We’ll look at how to use this in the next section. PRODUCT_PACKAGES $(call inherit-product, $(SRC_TARGET_DIR)/product/languages_full.mk) # If you're using 4.2/Jelly Bean, use full_base.mk instead of full.mk $(call inherit-product, $(SRC_TARGET_DIR)/product/full.mk) DEVICE_PACKAGE_OVERLAYS := PRODUCT_PACKAGES += PRODUCT_COPY_FILES += PRODUCT_NAME := full_coyotepad PRODUCT_DEVICE := coyotepad PRODUCT_MODEL := Full Android on CoyotePad, meep-meep
  • 58. Allows us to specify packages we’d like to have this product include in addi- tion to those specified in the products we’re already inheriting from. If you have custom apps, binaries, or libraries located within device/acme/coyotepad/, for instance, you’ll want to add them here so that they are included in the final images generated. Notice the use of the += sign. It allows us to append to the existing values in the variable instead of substituting its content. PRODUCT_COPY_FILES Allows us to list specific files we’d like to see copied to the target’s filesys- tem and the location where they need to be copied. Each source/destination pair is colon-separated, and pairs are space-separated among themselves. This is useful for configuration files and prebuilt binaries such as firmware images or kernel modules. PRODUCT_NAME The TARGET_PRODUCT , which you can set either by selecting a lunch combo or passing it as part of the combo parameter to lunch, as in: PRODUCT_DEVICE The name of the actual finished product shipped to the customer. TARGET_DEVICE derives from this variable. PRODUCT_DEVICE has to match an entry in device/acme/, since that’s where the build looks for the corre- sponding BoardConfig.mk. In this case, the variable is the same as the name of the directory we’re already in. PRODUCT_MODEL $ lunch full_coyotepad-eng
  • 59. The name of this product as provided in the “Model number” in the “About the phone” section of the settings. This variable actually gets stored as the ro.product.model global property accessible on the device. Version 4.2/Jelly Bean also includes a PRODUCT_BRAND that is typically set to Android. The value of this variable is then available as the ro.product.brand global property. The latter is used by some parts of the stack that take action based on the device’s vendor. Now that we’ve described the product, we must also provide some information regarding the board the device is using through a BoardConfig.mk file: This is a very skinny BoardConfig.mk and ensures that we actually build success- fully. For a real-life version of that file, have a look at device/samsung/crespo/BoardConfigCommon.mk in 2.3/Gingerbread, and also at device/asus/grouper/BoardConfigCommon.mk in 4.2/Jelly Bean. You’ll also need to provide a conventional Android.mk in order to build all the modules that you might have included in this device’s directory: TARGET_NO_KERNEL := true TARGET_NO_BOOTLOADER := true TARGET_CPU_ABI := armeabi BOARD_USES_GENERIC_AUDIO := true USE_CAMERA_STUB := true LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) ifneq ($(filter coyotepad,$(TARGET_DEVICE)),)
  • 60. It’s in fact the preferred modus operandi to put all device-specific apps, binaries, and libraries within the device’s directory instead of globally within the rest of the AOSP. If you do add modules here, don’t forget to also add them to PRODUCT_PACKAGES as I explained earlier. If you just put them here and provide them valid Android.mk files, they’ll build, but they won’t be in the final images. If you have several products sharing the same set of packages, you may want to create a device/acme/common/ directory containing the shared packages. You can see an example of this in 4.2/Jelly Bean’s device/generic/ directory. In that same version, you can also check how device/samsung/maguro/device.mk inherits from device/samsung/tuna/device.mk for an example of how one device can be based on another device. Lastly, let’s close the loop by making the device we just added visible to envsetup.sh and lunch. To do so, you’ll need to add a vendorsetup.sh in your device’s directory: You also need to make sure that it’s executable if it’s to be operational: We can now go back to the AOSP’s root and take our brand-new ACME CoyotePad for a runchase: include $(call all-makefiles-under,$(LOCAL_PATH)) endif add_lunch_combo full_coyotepad-eng $ chmod 755 vendorsetup.sh
  • 61. $ croot $ . build/envsetup.sh $ lunch You're building on Linux Lunch menu... pick a combo: 1. generic-eng 2. simulator 3. full_coyotepad-eng 4. full_passion-userdebug 5. full_crespo4g-userdebug 6. full_crespo-userdebug Which would you like? [generic-eng] 3 ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=2.3.4 TARGET_PRODUCT=full_coyotepad TARGET_BUILD_VARIANT=eng TARGET_SIMULATOR=false TARGET_BUILD_TYPE=release TARGET_BUILD_APPS= TARGET_ARCH=arm HOST_ARCH=x86 HOST_OS=linux HOST_BUILD_TYPE=release BUILD_ID=GINGERBREAD ============================================ $ make -j16
  • 62. As you can see, the AOSP now recognizes our new device and prints the informa- tion correspondingly. When the build is done, we’ll also have the same type of output provided in any other AOSP build, except that it will be a product-specific directory: Also, have a look at the build.prop file in system/. It contains various global prop- erties that will be available at runtime on the target and that relate to our configu- ration and build: $ ls -al out/target/product/coyotepad/ total 89356 drwxr-xr-x 7 karim karim 4096 2011-09-21 19:20 . drwxr-xr-x 4 karim karim 4096 2011-09-21 19:08 .. -rw-r--r-- 1 karim karim 7 2011-09-21 19:10 android-info.txt -rw-r--r-- 1 karim karim 4021 2011-09-21 19:41 clean_steps.mk drwxr-xr-x 3 karim karim 4096 2011-09-21 19:11 data -rw-r--r-- 1 karim karim 20366 2011-09-21 19:20 installed-files.txt drwxr-xr-x 14 karim karim 4096 2011-09-21 19:20 obj -rw-r--r-- 1 karim karim 327 2011-09-21 19:41 previous_build_config.mk -rw-r--r-- 1 karim karim 2649750 2011-09-21 19:43 ramdisk.img drwxr-xr-x 11 karim karim 4096 2011-09-21 19:43 root drwxr-xr-x 5 karim karim 4096 2011-09-21 19:19 symbols drwxr-xr-x 12 karim karim 4096 2011-09-21 19:19 system -rw------- 1 karim karim 87280512 2011-09-21 19:20 system.img -rw------- 1 karim karim 1505856 2011-09-21 19:14 userdata.img # begin build properties # autogenerated by buildinfo.sh ro.build.id=GINGERBREAD ro.build.display.id=full_coyotepad-eng 2.3.4 GINGERBREAD eng.karim.20110921.1908 49 test-keys ro.build.version.incremental=eng.karim.20110921.190849
  • 63. ro.build.version.sdk=10 ro.build.version.codename=REL ro.build.version.release=2.3.4 ro.build.date=Wed Sep 21 19:10:04 EDT 2011 ro.build.date.utc=1316646604 ro.build.type=eng ro.build.user=karim ro.build.host=w520 ro.build.tags=test-keys ro.product.model=Full Android on CoyotePad, meep-meep ro.product.brand=generic ro.product.name=full_coyotepad ro.product.device=coyotepad ro.product.board= ro.product.cpu.abi=armeabi ro.product.manufacturer=unknown ro.product.locale.language=en ro.product.locale.region=US ro.wifi.channels= ro.board.platform= # ro.build.product is obsolete; use ro.product.device ro.build.product=coyotepad # Do not try to parse ro.build.description or .fingerprint ro.build.description=full_coyotepad-eng 2.3.4 GINGERBREAD eng.karim.20110921.190 849 test-keys ro.build.fingerprint=generic/full_coyotepad/coyotepad:2.3.4/GINGERBREAD/eng.kari m.20110921.190849:eng/test-keys # end build properties ...
  • 64. WARNING You may want to carefully vet the default properties before using the build on a real de- vice. Some developers have encountered some severe issues due to default values. In both 2.3/Gingerbread and 4.2/Jelly Bean, for instance, ro.com.android.dataroaming is set to true in some builds. Hence, if you’re doing development on a device connected to a live cell network, changing the value to false might save you some money. As you can imagine, there’s a lot more to be done here to make sure the AOSP runs on our hardware. But the preceding steps give us the starting point. However, by isolating the board-specific changes in a single directory, this config- uration will simplify adding support for the CoyotePad to the next version of the AOSP that gets released. Indeed, it’ll just be a matter of copying the corresponding directory to the new AOSP’s device/ directory and adjusting the code therein to use the new APIs. Adding an App Adding an app to your board is relatively straightforward. As a starter, try creat- ing a HelloWorld! app with Eclipse and the default SDK; all new Android projects in Eclipse are a HelloWorld! by default. Then copy that app from the Eclipse workspace to its destination: You’ll then have to create an Android.mk file in aosp- root /device/acme/coyotepad/HelloWorld/ to build that app: $ cp -a ~/workspace/HelloWorld ~/android/aosp-2.3.x/device/acme/coyotepad/ LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS)
  • 65. Given that we’re tagging this module as optional , it won’t be included by default in the AOSP build. To include it, you’ll need to add it to the PRODUCT_PACKAGES listed in the CoyotePad’s full_coyotepad.mk. If, instead of adding your app for your board only, you would like to add a default app globally to all products generated by the AOSP alongside the existing stock apps, you’ll need to put it in packages/apps/ instead of your board’s directory. You’ll also need to modify one of the built-in .mk files, such as aosp- root/build/target/product/core.mk, to have your app built by default. This is not recommended, though, as it’s not very portable since it will require you to make this modification to every new AOSP release. As I stated earlier, it’s best to keep your custom modifications in device/acme/coyotepad/ in as much as possible. Adding an App Overlay Sometimes you don’t actually want to add an app but would rather modify exist- ing ones included by default in the AOSP. That’s what app overlays are for. Overlays are a mechanism included in the AOSP to allow device manufacturers to change the resources provided (such as for apps), without actually modifying the original resources included in the AOSP. To use this capability, you must create an overlay tree and tell the build system about it. The easiest location for an overlay is within a device-specific directory such as the one we created in the previous section: LOCAL_MODULE_TAGS := optional LOCAL_SRC_FILES := $(call all-java-files-under, src) LOCAL_PACKAGE_NAME := HelloWorld include $(BUILD_PACKAGE)
  • 66. To tell the build system to take this overlay into account, we need to modify our full_coyotepad.mk such that: At this point, though, our overlay isn’t doing much. Let’s say we want to modify some of Launcher2’s default strings. We could then do something like this: You can then trim your local strings.xml to override only those strings that you need. Most importantly, your device will have a Launcher2 that has your custom strings, but the default Launcher2 will still have its original strings. So if someone relies on the same AOSP sources you’re using to build for another product, they’ll still get the original strings. You can, of course, replace most resources like this, in- cluding images and XML files. So long as you put the files in the same hierarchy as they are found in the AOSP but within device/acme/coyotepad/overlay/, they’ll be taken into account by the build system. $ cd device/acme/coyotepad/ $ mkdir overlay DEVICE_PACKAGE_OVERLAYS := device/acme/coyotepad/overlay $ mkdir -p overlay/packages/apps/Launcher2/res/values $ cp aosp-root /packages/apps/Launcher2/res/values/strings.xml > overlay/packages/apps/Launcher2/res/values/
  • 67. WARNING Overlays can be used only for resources. You can’t overlay source code. If you want to cus- tomize parts of Android’s internals, for instance, you’ll still have to make those modifica- tions in situ. There’s no way, currently at least, to isolate those changes to your board. Adding a Native Tool or Daemon Like the example above of adding an app for your board, you can add your cus- tom native tools and daemons as subdirectories of device/acme/coyotepad/. Obviously, you’ll need to provide an Android.mk in the directory containing the code to build that module: As in the app’s case, you’ll also need to make sure hello-world is part of the CoyotePad’s PRODUCT_PACKAGES . If you intend to add your binary globally for all product builds instead of just lo- cally to your board, you need to know that there are a number of locations in the tree where native tools and daemons are located. Here are the most important ones: system/core/ and system/ LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := hello-world LOCAL_MODULE_TAGS := optional LOCAL_SRC_FILES := hello-world.cpp LOCAL_SHARED_LIBRARIES := liblog include $(BUILD_EXECUTABLE)
  • 68. Custom Android binaries that are meant to be used outside the Android Framework or are standalone pieces. frameworks/base/cmds/ Binaries that are tightly coupled to the Android Framework. This is where the Service Manager and installd are found, for example. external/ Binaries that are generated by an external project that is imported into the AOSP. strace, for instance, is here. Having identified from the list above where the code generating your binary should go, you’ll also need to add it as part of one of the global .mk files such as aosp-root/build/target/product/core.mk. As I said above, however, such global addi- tions are not recommended since they can’t be transferred as easily to newer AOSP versions. Adding a Native Library Like apps and binaries, you can also add native libraries for your board. Assuming, as above, that the sources to build the library are in a subdirectory of device/acme/coyotepad/, you’ll need an Android.mk to build your library: LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := libmylib LOCAL_MODULE_TAGS := optional LOCAL_PRELINK_MODULE := false LOCAL_SRC_FILES := $(call all-c-files-under,.) include $(BUILD_SHARED_LIBRARY)
  • 69. NOTE Note that LOCAL_PRELINK_MODULE has been removed and is no longer neces- sary as of 4.0/Ice-Cream Sandwich. To use this library, you must add it to the libraries listed by the Android.mk file of whichever binaries depend on it: You’ll also likely need to add relevant headers to an include/ directory located in about the same location as you put your library, so that the code that needs to link against your library can find those headers, such as device/acme/coyotepad/include/. Should you want to make your library apply globally to all AOSP builds, not just your device, you’ll need a little bit more information regarding the various loca- tions where libraries are typically found in the tree. First, you should know that, unlike binaries, a lot of libraries are used within a single module but nowhere else. Hence, these libraries will typically be placed within that module’s code and not in the usual locations where libraries used systemwide are found. The latter are typically in the following locations: system/core/ Libraries used by many parts of the system, including some outside the Android Framework. This is where liblog is, for instance. frameworks/base/libs/ Libraries intimately tied to the framework. This is where libbinder is. LOCAL_SHARED_LIBRARIES := libmylib
  • 70. frameworks/native/libs/ In 4.2/Jelly Bean, many libraries that were in frameworks/base/libs/ in 2.3/Gingerbread have been moved out and into frameworks/native/libs/. external/ Libraries generated by external projects imported into the AOSP. OpenSSL’s libssl is here. Similarly, instead of using a CoyotePad-specific include directory, you’d use a global directory such as system/core/include/ or frameworks/base/include/ or, in 4.2/Jelly Bean, frameworks/base/include/. Again, as stated earlier, you should care- fully review whether such global additions are truly required, as they’ll represent additional work when you try to port for your device to the next version of Android.
  • 71. LIBRARY PRELINKING If you look closely at the example Android.mk we provide for the library, you’ll notice a LOCAL_PRELINK_MODULE variable. To reduce the time it takes to load libraries, Android ver- sions up to 2.3/Gingerbread used to prelink most of their libraries. Prelinking is done by specifying ahead of time the address location where the library will be loaded instead of letting it be figured out at runtime. The file where the addresses are specified in 2.3/Gingerbread is build/core/prelink-linux-arm.map, and the tool that does the mapping is called apriori. It contains entries such as these: If you want to add a custom native library to 2.3/Gingerbread, you need to either add it to the list of libraries in prelink-linux-arm.map or set the LOCAL_PRELINK_MODULE to false . The build will fail if you forget to do one of these. # core system libraries libdl.so 0xAFF00000 # [<64K] libc.so 0xAFD00000 # [~2M] libstdc++.so 0xAFC00000 # [<64K] libm.so 0xAFB00000 # [~1M] liblog.so 0xAFA00000 # [<64K] libcutils.so 0xAF900000 # [~1M] libthread_db.so 0xAF800000 # [<64K] libz.so 0xAF700000 # [~1M] libevent.so 0xAF600000 # [???] libssl.so 0xAF400000 # [~2M] ... # assorted system libraries libsqlite.so 0xA8B00000 # [~2M] libexpat.so 0xA8A00000 # [~1M] libwebcore.so 0xA8300000 # [~7M] libbinder.so 0xA8200000 # [~1M] libutils.so 0xA8100000 # [~1M] libcameraservice.so 0xA8000000 # [~1M] libhardware.so 0xA7F00000 # [<64K] libhardware_legacy.so 0xA7E00000 # [~1M] ...
  • 72. Library prelinking was dropped starting in 4.0/Ice-Cream Sandwich. If you do not provide a value, defaults will be used. For instance, all apps are set to op‐ tional by default. Also, some modules are part of GRANDFATHERED_USER_MODULES in user_tags.mk. No LOCAL_MODULE_TAGS need be specified for those; they’re always included. This file contains a set list of variables starting with the string LOCAL_ . If a variable isn’t specifically listed in this file, it won’t be taken into account by CLEAR_VARS . This version is cleaned up a little (removed commented code, for instance) and slightly reformatted. Also slightly modified to remove white space and comments. This assumes you had already run envsetup.sh and lunch. This makefile is inspired by a blog post by Row Boat developer Amit Pundir and is based on the example makefile provided in Chapter 4 of Building Embedded Linux Systems, 2nd ed. (O’Reilly). [18] [19] [20] [21] [22] [23]