Linux Management Tools
- 1. Wind River Linux and Source Control Management Tools David Reyna, Senior Member Technical Services Wind River There are several problems with this traditional methodology Table of Contents for Linux platform projects. First, these platform projects are generally far bigger than traditional projects and can include Executive Summary............................................................. 1 an entire file system of several hundred packages in addition Issues with Traditional Management Strategies. ................ 1 . to the Linux kernel. A fully expanded project can easily have Wind River Linux Management Strategies.......................... 1 more than 40,000 files and occupy several gigabytes of Preserving the Installation. ............................................. 2 . space. Using Layers to Manage Custom Project Content.......... 2 Second, the very nature of Linux packages makes projects Using Layers Now or 10 Years from Now. ...................... 2 . difficult to manage. The packages start in a tarball or SRPM form, generally together with a set of patches. To capture the Simple Single-User Strategy................................................ 2 source, the packages must be unpacked and patched. Wind River Linux Tools........................................................ 3 However, the build directories that hold the flattened source export-layer..................................................................... 3 are generally transient and may be destroyed and remade export-patch. .................................................................. 4 . several times in the course of development. This is hostile to Import Application.......................................................... 4 management systems. Import Workbench Project.............................................. 4 Third, most packages use automake to configure themselves. Eclipse Source Control Plug-Ins...................................... 4 The very nature of automake is to configure the files specifically for the particular host and the absolute location Conclusion........................................................................... 4 of the package source, so the configuration management capture and registration must occur after patching but before Executive Summary configuration. This can be done, but it is difficult to do correctly and consistently and in a way that scales. Also, the Development groups depend on source control management project itself is managed by automake, so already there are systems to maintain and scale their software projects. But site-specific files in the project tree, in other words, hard- traditional methods and workflows may not work as coded paths in the core project file that may not work for any developers shift to the complex world of Linux platform other location. development, forcing a move to new protocols. This paper highlights the difficulties of applying traditional Wind River Linux Management Strategies methods to Linux platform development, provides the Wind River Linux installations can be directly captured into a strategies to resolve these issues, and describes the tools source control system for archiving and sharing. The custom that are available with Wind River Linux to make these content of Linux projects can be exported into standalone strategies work for your environment. and portable layer directories, which in turn can be source controlled and shared. User applications that may already be Issues with Traditional Management Strategies in a source control system can be imported into projects as Projects are traditionally managed entirely within the packages and included into the exported layer, where a configuration management system. The source and second user can either utilize that captured application development files are registered with the system, the package content or reconnect it to the source control system generated object and binary files are ignored as private data, from the new project location. and the user can generally check out the same project in a Here is the recommended strategy to overcome the issues different location or at a later time and recreate the project with managing a Linux platform project. as needed.
- 2. Source Control Repository Wind River Wind River Linux Linux Installation Wind River Installation Linux Installation Project 1 Project 2 Layers Build Build Local Local Packages User Packages Applications Figure 1: Integration of user projects and a source control repository Preserving the Installation Using Layers Now or 10 Years from Now You can check the entire Wind River Linux installation into If you wish to share the project changes immediately with your source control system, to preserve the original source other developers, all they need to do is the following: content. The installation directory is designed to be portable, 1. Check out the captured layer directory. so it can be extracted to new locations now or in the future. 2. Create a local version of the project using the settings Additionally, you will have all of the user space and kernel in captured in the layer, using the local installation path and their original compressed form. the layer path. You can optionally skip the directory of prebuilt package 3. Build the project. content (in the form of RPM binary files) that are included as Once the second project is set up, a simple refresh of the part of the installation, if space is a concern. The prebuilt source control managed layer directory and a reconfig RPM files can take up to 30% to 80% of the installation’s command will assert any new changes to the second project. space, depending on how many unique board-specific If you wish to recreate the project several years later, there packages you have installed; plus this content can always be are some additional steps you should consider. Specifically, rebuilt as needed from the included original package you must preserve a copy of the development host computer sources. (or any of the Wind River Linux release’s supported hosts). Using Layers to Manage Custom Project Content This is because there are many package dependencies on the host’s tools, and while many host tools are included with the Since directly managing the actual project directory is Wind River Linux installation, not all are. These dependencies problematic, the strategy is to use Wind River Linux layer can range from obvious requirements such as the host technology to separate out and manage the changed and compiler and core utilities as well as non-obvious local project content in a safe, manageable, and scalable requirements such as asciidoc and htmlview. You will, of manner. course, also need to preserve a computer that can run this Wind River Linux layer technology is the crucial feature for host operating system. Given the rapid and tightly coupled extracting and sharing project changes between projects and Linux community development, supported hosts have a developer groups. Layers can capture any and all changes to lifetime of only one to two years, so it is important to be a project and can be included in any new project to reassert prepared. the captured changes. They also capture the state of the original project so it is easy to re-instantiate any project from Simple Single-User Strategy scratch. Not every project needs to be shared or preserved over time. Specific to source control management, a layer directory is Let us explore the simple scenario where a single user wishes easy to both check in to a source control system and to to source manage a project but does not need the full ability manage. Most importantly, a layer directory is completely to share that project with other users and other portable and re-locatable so it can be checked out and environments. reused in any environment. 2 | Wind River Linux and Source Control Management Tools
- 3. In this case, the embedded paths for the project and to read-only. While most packages can tolerate this, there are installation are not an issue because the project will be some that require that certain files remain read-write checked in and out of the same location and in the same enabled. An example is the file build/linux/wrs/meta-series, environment. This developer can check in and check out the which helps control the project’s local git tree for the kernel. project tree. You will need to either perform a non-reserved check-out or manually reset such files back to read-write. Here are a few suggestions for optimizing this arrangement. For CVS users, there will be CVS directories introduced Before doing the initial check-in, perform the clean rule for all throughout the managed tree. Unfortunately, there may be packages that you have locally instantiated, those packages packages that will fail to build with these extra directories, for that you have built locally and that are not using the prebuilt example, packages that perform recursive makefile rules, where intermediate RPM files from the installation. This will help the make will fail because the CVS directory has no makefile. reduce the burden on the source management tool to For these packages, you will need to either patch the makefiles identify which files are “source” and which are “derived,” to fix these build issues or manage only the package’s extracted which is particularly difficult to accomplish with the patch files; in other words, explicitly use the export-patch tool intermediate generated tools and help content. to capture patches to the project’s dist directory, and distclean You should also perform the distclean rule for all packages the package’s build directory prior to checking in the project that have been locally built but are not actually different from directory so that it is not added to source control. This is not an the original installation versions. For example, if you have issue for SVN, git, or Clear Case because the metadata is either performed a full build-all (or a build for debug or profiling in hidden files or is managed outside of the project directory. support), you will have locally instantiated every single package but will not have added any new information beyond Wind River Linux Tools what is in the installation and will bloat the repository image by an order of magnitude. Here are some of the workflow tools provided with Wind River Linux for helping manage platform projects for source Alternately, if your goal is to exactly capture all of the control systems. expanded source to all of the packages, perform the patch rule (after the distclean) for all packages, to flatten their export-layer source and patches into pure source trees. The export-layer build command will create a self-contained Now check the project directory into the source control layer directory either under the project’s export directory system. Optionally, check in the installation if you have not (which you can move later) or at a user-specified location. already done so because the project will have many symlinks This command will capture the project’s configuration data in that installation tree. plus the high-level changes (such as the package list and the file system layout changes). This command can be used to There are some caveats to the single-user strategy. For Clear create a second user’s project to receive the changes. Case users, the check-in will normally convert all of the files Import Project Source Control Repository Package 1 Application 1 Optional Package Rules Check-in/out SCM Rules Source Application 1 Import Package 2 Application 2 Source Package Rules External SCM Rules Source External Figure 2: Applications under source control can be imported as packages in projects 3 | Wind River Linux and Source Control Management Tools
- 4. This command will also explicitly include any kernel configure Import Workbench Project changes and any locally added or imported packages or You can use the Import Workbench Project tool (available applications. Finally, this command will also include any with Wind River Linux 4) to import a Wind River Linux extracted package or kernel patches. application or user-defined projects as a package into your project. This allows you to use the full application export-patch development project facilities in Wind River Workbench to The export-patch tool can be used to identify and extract create and develop your application and provide a way to changes to the user space packages, both SRPMs or classic capture and share that work as a formal Wind River Linux package types. These patches are placed in the local package for your platform maintainers, at any frequency that project’s dist directory, in a form that the export-layer is appropriate. command can access. Eclipse Source Control Plug-Ins Import Application If you are using source control plug-ins with Wind River If you have an application source tree, the Import Application Workbench/Eclipse, you can also use them with Wind River tool (available with Wind River Linux 4) will allow you to Linux. For example, if you are directly managing a Wind River include this source tree in your project as a formal Wind River Linux application or user-defined project, you can use the Linux package. You can leave your source in its external Import Workbench Application tool to bring that application location and have the tool provide a wrapper package. In this into a project and later have any captured changes pushed manner, there is a single source, and changes are back to the original source. Also, you can use the export-layer immediately visible to and from that location (though if you tool to lace the extracted layer directly into a generic Eclipse export this wrapper package, you will need to manually project and then use Team > Share to add this “layer” into reconnect to its external source). your source control system. Alternately, you can have the tool copy the source tree into a local package so your application package is completely Conclusion portable with the layer export tool (the import tool will Wind River recognizes the importance of configuration generate optional push and pull rules so that you can re-sync management as part of a company’s strategy to control the on demand with that external source location). Additionally, if life cycle and scalability of its software investment. With the this application tree is under your source control management tools and methodologies outlined in this paper, you can for CVS, SVN, git, or Clear Case, the generated package can extend and match those systems to the complexity, ability, optionally generate flexible rules to check out and check in and life cycle of Wind River Linux projects. These tools, changes from your development cycle to your repository. provided as part of Wind River Linux, are comprehensive If the export-patch tool is used with these special packages, enough to get the job done yet customizable enough to it can perform an intelligent check-in with the specific files meet the particular needs and requirements of developers. that were changed, using the patch description for the You can find more details on these tools, together with use check-in comment. cases, in several papers posted at the Wind River Online In all cases, these imported applications will be included by Support site at www.windriver.com/support for the Wind the export-layer rule. River Linux 4.0 product. Wind River is a leader in embedded and mobile software. We enable companies to develop, run, and manage device software faster, better, at lower cost, and more reliably. www.windriver.com © 2010 Wind River Systems, Inc. The Wind River logo is a trademark of Wind River Systems, Inc., and Wind River and VxWorks are registered trademarks of Wind River Systems, Inc. Other marks used herein are the property of their respective owners. For more information, see www.windriver.com/company/terms/trademark.html. Rev. 11/2010