Building For Mer

Editor's Notes

• #2: I'm going to talk about the Mer build system Next: OBS
• #3: OBS .... Next : Whats OBS
• #4: Summary It's two things : * a build service * a build environment manager It's not a packaging system – it works with normal debian/maemo packages Thanks to Suse Just to get out of the way... this is not a Novell proprietary service - Full GPL What does the service do? you upload (check-in) a tarball it spits out binary packages. They do provide a huge number of machines in a public build farm but you are free to setup an independent OBS instance Both openSUSE and SLES (enterprise) build on it Linux foundation will be launching a large OBS instance (~100 cores) real soon – Mer is one of the few test projects ARM joined LF last month Next: Compare to sbox
• #5: Build environment is very similar to sbox – *much* cleaner Allows you to run arm binaries on intel desktops Compile/install supported by improving qemu so it supports all required calls Local chroot into a qemu environment – uses the same binaries you (could) use on the device Multiple (clean) development environments – 1 per project. All integrated into your desktop Sbox benefits – used to be speed – but that's fixed now (talk more about how we did this cleanly later) SB reaches deeply into the buildenv and replaces sh, autoconf, perl, python, etc... means you may have different versions of apps to those on the device. Next steps involve kvm and sdk integration to improve the sdk Next: What it gives you
• #6: Why? Developers: Simple to use; reliable Truly clean/pure build environment Easy upload, At-a-glance immediate status if something is wrong web interface to track build logs in real time no home made solutions flexibility and easy building for multiple targets (armv5, v7fp, i386, Stable, Testing) Maintainers: Status overview is crucial Easy promotion Tools to support processes (like promotion) 'Make like' so entire distribution is consistent Community: Very very open Start your own project, build against Mer Build against Stable/Testing/Devel Drawbacks: Require access to the service to setup a chroot Too many :s Used to be 'emulated' compile but not now. Next Screenshots
• #7: Fully CLI osc <cmd> such as : build buildlog checkout copypac getbinaries list (ls) meta repos All written in Python / shell Scriptable eg. script that takes a local file or http location for a .dsc file, imports it, and submits it to :Extras:Testing
• #8: Simple HTTP/XML API Next web
• #9: Essentially a good gui for rarer overview operations rather than package management ones Cli used for development Web for overview Next monitor
• #10: At a glance view of package status Live logs Easy enable/disable of build/publish Mouseover says “nothing provides gnome-vfs-filechooser needed by libhildon2” (circular dependency... cheers!) Next: using OBS remote service
• #11: Version control with make-like dependencies using build-dependencies As soon as you commit the .dsc and tar.gz it is scheduled to build There is a status of where the job is. As the build progresses you can watch a live feed of the log(s)... You may have setup sbox recently; lets take a look at local builds... Next: Local Build
• #12: Really ... it's that simple The initial setup to get to that point is pretty much a 2-liner Add a repo to your apt sources Install osc and qemu Run the co command to get the source Run the build command Grab a coffee Install the .deb Next: Packaging
• #13: I want to stress : OBS has nothing to do with maemo packaging If anything it is better than autobuilder since the packages have no scratchbox-isms If a correct debian package fails then it is a bug (there are some complex conditional build-depends that fail at the moment) Next: Mer layout
• #14: Complex to explain but really quite simple. Project – main grouping mechanism. Like a directory Contains Packages you want to build Build against Repositories – bad name. Where build dependencies are satisfied; a distro like Ubuntu is an obvious repo ; 2 Types: Other projects Download on demand (cached) However packages first look locally for build dependencies, then in the repository projects and their parent repositories The repository also specifies the architecture (Ubuntu 9.04 armv5el / i586) So “Maemo:Mer:Devel:UI/MerDevel_Ubuntu9.04 armv5el” Maemo:Mer:Devel:UI project Builds against Maemo:Mer:Devel project In turn against Ubuntu_9.04 (from ports) armv5 arch Next: Mer Projects
• #15: These are OBS 'projects' which contain all the individual packages In Devel we have: Apps, Base, MaemoCommon, Qt, UI and more Essentially move from Devel to Test to a numbered release Stable then points to the latest release using kinda symlinks HW specific areas Next: Extras and user builds
• #16: One approach is to submit to our 'Extras' Very similar to Maemo Extras process Initial work is done in :Devel (or locally) move to :Testing when ready for power users move to :Stable when ready for all users Clearly you don't need all 12 builds all the time; individually enabled Most will only need to build against Stable Package releases are not tied to Core releases Rebuilds triggered whenever build dependencies in the core are changed Next : Home areas
• #17: Alternatively you can just setup your own home area and build 'against' the Mer repos. One benefit is email/rss notifications of a rebuild Next: osc
• #18: Listed the main commands to give a flavour VCS like – lots more to do with managing the remote server; getting binaries etc Main thing is to checkout packages and build them Usually you checkout, then use git to manage the code, build a package and commit the tarball Futures: We anticipate that co will do a git checkout and an osc commit <tag> will cause OBS to use that to pull a tarball and build it... Next: Move on to git branches and upstream etc
• #19: Issue is we need to manage patches against our upstream And we want to make life easy for our downstream. That means keeping the packaging and the code separate. Start with an upstream package : unpack and remove the debian/ directory. commit onto the upstream branch. pristine-tar is used to ensure that the upstream tarball can be recreated perfectly. (not used much) All features or bug fixes are broken out to additional branches; 1 branch per feature or fix. The main difference is that features are not likely to go upstream whilst fixes are. This is very similar to quilt. Very hard to manage patches in the debian/patch dir master is the main code branch and there should only be merge commits on this branch; the only non-merge commits should be in ./ChangeLog Mer should be packaging-based and the only non-merge commits should be in debian/ Next: Stable point release
• #20: Shows how we package a Stable point release Next: Move on to see how we handle an upstream release
• #21: We make changes to our fixes and features They need re-integrating when upstream releases A monolithic patch means you don't always know what a particular change is related to. Having logical changes in multiple branches makes this sane. Next: Acceleration
• #22: Discussed how qemu emulates; does this on a binary-by-binary level using binfmt-misc so ideally use 586 binaries. Just those that impact compilation Could do static builds but fiddly Use dynamic - but .so library paths collide. So we create another 'root' /lib-x86 and install all modified packages in there. Modified OBS to do this using mkbaselibs – an rpm technique. Fixup the binaries rpath on install with a static patchelf / chrpath; then symlink the binaries into the real chroot and move the real binaries to .old Toolchain replaced with a slightly modified gcc-cross (that was hard – weeks of work!) If we install these binaries in a chroot we get 5x to 10x performance increase Same source as if using arm binaries – unlike scratchbox. Disable accel and you use a pure arm chroot again Next : Issues
• #23: Mer devices are clearly multimedia oriented so music and video codecs are important Novell OBS only supports supported Ubuntu... ie not Universe. Ubuntu have some strange omissions As Mer grows we'll want to open up submit rights but not want users to write to other users packages. Need a login to Novell but soon using openID Linux Foundation OBS is big news!
• #24: Chatter on #mer is very interactive Lots of support Mer Chatter is interesting ... if you're a hardcore Mer geek. Worked hard on the docs – let me know of omissions etc Help make the SDK