P2 Introduction

From build to assembly to deployment:
Using p2 to facilitate agile software
development

Kim Moir, IBM
Ian Bull, EclipseSource

Copyright © IBM Corp., EclipseSource 2010. All
rights reserved. Licensed under EPL, v1.0.

About us

• Kim Moir is the Release Engineering lead for the Eclipse and
RT Equinox projects, and works in the IBM Ottawa Lab. She
uses p2 to provision builds for 13 platforms.

• Ian Bull works on a number of Eclipse projects, including p2,
Zest and the PDE Incubator. Ian works at EclipseSource as a
software engineer.

Copyright © IBM Corp., EclipseSource 2010. All rights reserved. Licensed under EPL, v1.0.

What about you?

• How do you use p2?
• What are you looking to get out of this tutorial?
• Are your building products based on p2?


Agile Software Development

• Key characteristic of agile software development includes Fast
Cycle Time:
• Include your customers in the development process
• Ensure your customers are continually using the latest software
• Ensure your developers are building against the latest software


Overview

• How can p2 be used to manage your update strategy?
• Update your RCP and OSGi applications
• Provide a mechanism to install and manage new features
• Provide a mechanism to manage different configurations
• Manage your developers target platforms


Outline

• Overview of p2
• p2 architecture and terminology
• Anatomy of a repository
• Using the director and publisher
• Product based builds with p2
• Repository management best practices
• Assembling products out of pre-existing components (no
compiling)


Hands-on exercises

• Building and assembling different configurations of your
application
• Building and publishing add-on bundles
• Export or build your product to a repository, and then install the
product from the repository you just created.
• Using repository tools such as the slicer, repo2runnable or mirror
to build smaller components from existing repositories


A brief history of Eclipse

• When Eclipse starts, the plug-ins are scanned and
functionality is discovered

• If a dependency is missing, all the dependent plug-ins simply
fail to start

• Old plug-ins pollute the application

• You cannot update resources, licenses, executables or the
framework itself


Unzipping is not Installing


What is p2 – p2 is Installing!

• A replacement for the old update manager
• New UI, simplified workflow
• Manage Eclipse, RCP and more (exe, ini, bundles, registry keys, …)
• Shared bundles across Eclipse-based products
• An installer
TM
• A provisioning platform for OSGi systems (in progress)
• Managing non-running instance
• Start level, framework extension
• Fine-grained dependency management


What does p2 enable?

• An extensible provisioning platform for OSGiTM systems which
provides
• Installation
• Searching for updates
• Rollback
• Dependency management
• Policy control
• UI or headless


What p2 isn’t

• An end to end build system
• PDE build provides tooling to generate p2 repositories when
building bundles
• A product – the eclipse community builds the products!


What can you build with p2?

• Add-on manager for RCP applications
• Installers
• Configuration Management Systems
• Self updating applications
• Install -> Update -> Roll back scenarios
• etc…


Who needs to provision?

Engineers Users

source code applications
open source tools
in-house built
runtimes
licensed software

Automated Builds Test
Test


p2 is for more than just bundles

• p2 can provision
• OSGiTM bundles
• Executables
• Configuration files – for instance eclipse.ini and config.ini
• Registry keys
• Native code
• Also supports bundle pooling – sharing bundles across
multiple installs


Where’s p2 ?


p2 terminology

• Artifacts is actual content being installed (bytes)
• Metadata consist of Installable units (IUs) that describes the
content that can be installed

Person Artifact IU
Name: org.eclipse.gumby
Version:3.6.0.v20100321
Requirements: org.eclipse.pokey
Provided capabilities: org.eclipse.bendy


Decouple decision making from the actual content

IU
(id, ver)

Everything is an IU
Everything is installable
It is also referred to as “metadata”


Anatomy of an IU, more

• Update description
• This allows to control of
• IU is being updated org.eclipse.osgi_3.5.1 _v20090827
• IU to update it to org.eclipse.osgi_3.5.2_v20100126
• Applicability filter
• This allows for an IU to express when it is only applicable to a
particular environment.
• Example platform specific filters for swt fragments (x86,linux,gtk)


Anatomy of an IU, requirements / capabilities

• Capabilities and requirements are the mechanism by which an IU
express what it provides and what needs.

• A capability is composed of a:
• Namespace (string), name (string) and version

• A requirement is composed of a query by default:
• Namespace, name and version range

• Namespace, name and version are open ended.

• The requirements and capabilities expressed by IUs can be arbitrary
and don’t have to all be in the same namespace.
• Resources – RPMs, .exes, docs, …
• Virtual – if you can define a capability, I can depend on it


Separation of concerns

• Installable Unit
• Exists independently of the repository

• Metadata Repository
• Store only Installable Units

• Artifact Repository
• Store only Artifacts

Metadata Artifacts


Artifacts

• Bytes/content to be installed
• Any form
• JARs (e.g., bundles, features, …)‫‏‬
• Binary executables
• RPM, MSI, …
• Defined, maintained, loaded and used separately from the
metadata


Repositories

• Defined as an interface, IRepository
• No serialization format assumed
• All repositories have
• Identity
• Version
• Description

• Repos can be queried
• Allows for delayed loading
• Captures standard questions


Metadata Repository

• Defined as an interface, IMetadataRepository

• Add/remove installable units
• Add references to other repos

• Created by factories
• Looked up in an IMetadataRepositoryManager service


Artifact Repository

• Defined as an interface, IArtifactRepository

• Add/remove descriptors
• Access artifacts by descriptor

• Created by factories
• Looked up in an IArtifactRepositoryManager service


p2 architecture

Metadata Artifacts

Director Repositories

Runtimes
Engine
Touchpoints Profiles

Eclipse
Eclipse
Classic

Other Eclipse
OS
for C++

p2 architecture

Metadata Data transfer
fetched and
constraints
Director analyzed Transports
Http/Https
File system
Volume
Mirroring
Provisioning
operation Repositories
requested p2
IU install, Update Site
uninstall, update
operations
Engine Artifact availability
and mirroring
Eclipse/OSGi
Native/OS
IUs configured Profile updated
into runtimes

Runtimes Profiles

Disk layout

Configuration area

p2 data area

Artifact repository index. This allows the
plug-ins to be served to other instances.


Disk layout

• Simple configurator provides the OSGi framework the list
of bundles to run.
• The information stored in bundles.info is
• Location, id, version, start level, started state

Configuration area

Simple configurator data


Disk layout

Download cache, where non-eclipse
pieces are mirrored before being installed.

Profile registry

Storage for the profile

MailApp.profile Lock file created when the
profile is being modified

Current and previous state of the system


A p2 profile

• A profile contains
• Properties defining the “environment” such as os, ws, arch,
install location, bundle pool location
• The list of IUs
• Properties associated with each IUs

• Profiles are held by a profile registry
• A profile registry can hold on to several profiles


Changing a profile

• A profile is usually created during the initial installation
• It is changed by provisioning operations performed using:
• The director application
• The UI
• The API
• …


Our example: Hyperbola

• RCP chat client application from the RCP book
http://eclipsercp.org
• We’ll be building this application
• Creating a repository
• Publishing add-on bundles
• Installing new bundles
• Adding self update to the product
• Using repository tools to create smaller components from the
repository


Tutorial requirements

• Eclipse 3.6M6
http://download.eclipse.org/eclipse/downloads/drops/S-
3.6M6-201003121448
• >= 1.5 VM
• Files from USB key that’s travelling around the classroom


Installing the tutorial code

• Start Eclipse 3.6M6
with a clean workspace
• The samples.zip
contains the repository
that we’ll be using
during the tutorial
• Help->Add New
Software->Add and
select the location the
file on disk
• Accept the license and
don’t worry about
unsigned content.


Introduction to the sample manager


Exercise 1 – Target Platforms

• Two large bodies of artifacts make-up your software system
1. The code you write
2. The libraries you use

• We know lots about the code you write (SCM Systems,
software processes, etc…)
• How do you manage your libraries?
• These are considered your “target”

• Eclipse developers often use the Eclipse SDK as their target
• There are better options


Exercise 1 – Target Platforms

• PDE provides facilities to help you manage your target
• We don’t need JDT or PDE in our chat client
• Create a target based on the Eclipse Platform + Delta Pack


Create a target platform

• Window -> preferences -> Plugin development -> Target
platform
• Add Empty Target
• Select directory
• Add new target platform from the directory when you have
extracted the SDK and delta pack zips


Exercise 2 – Adding a little p2

• Reusing the Eclipse SDK UI
• Reusing the UI in its entirety is easy
• A variety of techniques can be used to customize the UI


Versioning

• Every IU has the following identifiers
• Major.minor.service.qualifier
• 3.6.0.v20100129
• Major – breaking API
• Minor – new API
• Service – bug fix
• Qualifier – SCM tag or build id
• IU version must increase lexicographically for a successful
install
• NEVER rebuild and keep the same version number.
• Ensure that your plugin has a .qualifier at the end!


Configure the UI


Include the p2 bundles

• We have provided a simple p2 feature that includes everything you need
• Simply add the feature to your product

Add the feature and launch


A Variety of Options

• Use the p2 API to create your
own update UI
• Reuse the existing p2 UI in its
entirety
• Customize the existing p2 UI


Customizing the UI


Eclipse 3.6 – New UI Options
Simplifying update and extension install for RCP applications

Steffen Pingel
Susan McCourt

Wednesday, 15:45, 25 minutes
Grand Ballroom


Exercise 3 – Product Builds

• Using PDE/Build to create p2 enabled products


pde build

• PDE build generates Ant scripts based on the inputs that your
provide to the build.
• These Ant scripts are run to create the build artifacts
• Example scripts exist in the templates directory


Enabling p2 in your build

• Options to enable in your feature or product build’s
build.properties
• p2.gathering=true
• Publishes p2 metadata for all bundles, features, product builds
and pre-existing binary bundles
• p2.compress=true
• Creates artifacts.jar and content.jar files (XML by default)


Enabling p2 in your build

• p2.build.repo
• Default value is file:${buildDirectory}/buildRepo
• Build time repository
• The artifacts and metadata will be mirrored from
${p2.build.repo} and into the values of ${p2.artifacts.repo}
and ${p2.metadata.repo} are defined.


Product builds

• Eclipse bundles + custom branding
• Platform specific options for VM args, program arguments
• Specify startup levels for bundles
• Custom splash


Creating a builder
Create a builder with a top level build.properties file:

# Product and packaging control # Cross platform building
product= configs=win32, win32, x86 & linux, gtk, x86
/org.eclipsercp.hyperbola.product/hyperbola.product
runPackager=true # CVS Access control
archivePrefix=hyperbola skipMaps=true
mapsRepo=
# Build naming and locatingion :pserver:anonymous@example.com</path/to/repo>
buildDirectory=${user.home}/eclipse.build mapsRoot=<path/to/maps>
buildType=I mapsCheckoutTag=HEAD
buildId=TestBuild skipFetch=true
buildLabel=${buildType}.${buildId}
# Publish the build to a p2 repository
# Base identification and location p2.gathering = true
skipBase=true p2.metadata.repo =
pluginPath= file://${buildDirectory}/repository/hyperbola
p2.artifact.repo =
file://${buildDirectory}/repository/hyperbola
p2.compress = true

# Java class libraries and compile controls
#bootclasspath=${java.home}/lib/rt.jar
compilerArg=


Running the Builder

• Create ${buildDirectory}.
• Create ${buildDirectory}/plugins.
• Create ${buildDirectory}/features.
• Copy the required feature projects to the features directory
and plug-in projects to the plugins directory.

cd <workspace location>hyperbola.builder
c:ideeclipseeclipse.exe
-application org.eclipse.ant.core.antRunner
-buildfile productBuild.xml
-consolelog


Running the Builder


Building SDKs

• Software developer kits provide
• Binaries
• Source
• Documentation
• SDK features
• Include binary feature
• Generate source feature

generate.feature@org.yourfeature.source=org.yourfeature
SDK build.properties

individualSourceBundles=true
Top level build.properties

http://wiki.eclipse.org/PDEBuild/Individual_Source_Bundles


Exercise 4

• Creating a Voice Over IP feature
• Export from the workspace
• Install the add-on using the UI
• Install the add-on using the director


Feature builds

• Features are collections of bundles with related functionality.
Example org.eclipse.pde feature


Update your target

• Add on providers don’t have access to your SCM
• Add on providers don’t want access to your SCM

• Add on provider need access to your builds

• You can include the results of your product in your target


Update your target


Categories

• If features are not categorized they will not show up
• Most developers worry about the producers version of the
world
• Categories help define a consumers version of the world
• Use the category editor define categories


Voice Chat

• Exercise 4 contains a simple
VoIP feature
• Integrates with Hyperbola to
add voice chat
• Load the code and export the
feature
• Generate a p2 metadata
repository


Installing the feature

• We all know how to install the feature using the UI

DO NOT USE
E DROPINS!
OT US
DO N I NS ! DO
DROP DRO
NOT
U SE
PINS
!
U SE D
O
T S! D NO
NO PIN R
O T
DO RO PI U
N SE
D S!

•
We can also install it from the command line
• We use the director application to affect RCP installs


director

• Invoke provisioning
operations in a
headless manner
• Install
• Uninstall
• Update?


Director app from within IDE

Install pde
feature into new
directory


director example

d:buildereclipsec.exe -application
org.eclipse.equinox.p2.director
–metadataRepository http://download.eclipse.org/releases/helios/
–artifactRepository http://download.eclipse.org/releases/helios/
-installIU org.eclipsercp.voice.feature.feature.group
-destination d:/eclipse/
-profile SDKProfile


Voice Over IP


The p2 UI


Exercise 5 – Feature builds


Adding the voice feature

• Like product builds, PDE supports building features
• We just want a repository to represent our feature

• To build our Voice feature we need two different
requirements
• The Eclipse Platform
• The Hyperbola Platform

• For the hyperbola platform, we can use the repositories we
generated earlier


Feature build.properties

# Feature identification # SCM Access control
topLevelElementType = feature skipMaps=true
topLevelElementId = skipFetch=true
org.eclipsercp.voice.feature
archivePrefix=voip_feature # Publish the build to a p2 repository
p2.gathering = true
# Build naming and location
buildDirectory=${user.home}/feature.build # Cross platform building
configs = *, *, *
# Base identification and location
skipBase=true
baseLocation=C:/hyperbola_prereqs
repoBaseLocation=
<hyperbolaBuildLocation>/repository
transformedRepoLocation=
${buildDirectory}/transformedRepo


Running the build

• Create ${buildDirectory}
• Create ${buildDirectory}/plugins.
• Create ${buildDirectory}/features.
• Copy the required plug-ins and feature to plugins/ and
features/ directory.
• Invoke org.eclipse.ant.core.antRunner as shown below:

cd <workspace location>/feature.builder
c:ideeclipseeclipse.exe
-application org.eclipse.ant.core.antRunner
–buildfile build.xml -consolelog


Running the build


Exercise 6

• Automatically update Hyperbola each time it starts


How to update

• Two ways to update
• Look at all the IUs in your Profile
• See if there are any “updates” available
• Plan the update
• Execute the update on the engine
• Handle errors
• Restart
• Or
• Create an UpdateOperation
• Execute the Operation


Update on startup


Operation API

• We have a much simpler API for the basic use case
• Pre-canned Operations for Update, Install, Uninstall
• Available in the “operations” bundle


Setting your repositories

• You likely want a pre-canned set of repositories to check
• You can use a p2.inf file to supply this information


Exercise

• Setup hyperbola to auto update itself on startup.
• Make some changes, rebuild, and continually update


Metadata is immutable

Reuse metadata

to create subsets of functionality

p2.repository.repo2runnable

Specify source and destination


repo2runnable

• Transforms features and bundles into their installed form
• For example, bundles that are in folder format will be
extracted

<p2.repo2runnable>
<source dir="${sourcedir}“/>
<destination="${destinationdir}/eclipse“/>
</p2.repo2runnable>

• By default, all IUs will be transformed but you can also specify
nested <iu> elements


Mirroring is slicing


Mirroring repositories with p2

• Mirror a repository or a subset of the IUs to a new location
• Separate applications to mirror metadata and artifacts


p2.metadata.repository.mirrorApplication



p2.artifact.repository.mirrorApplication


Option to use a comparator
against a baseline


Exercise: Mirroring a Hyperbola subset

• Use the metadata and artifact repository mirroring tools to
mirror a subset of the Hyperbola product – perhaps one
platform.
• Not sure if we will have time for this


One of these bundles is not like the other

Bundle id + version = one set of bytes

Consistent bundles with a comparator

• If you use a different compiler to compile the same source,
it’s not necessarily the same byte code
• Use a comparator with a baseline to compare the bundles in
the repository in the current ones being built.
• Discard new bundles with the same bundle id + version
• Bundles in product zip must be identical repository bundles

Product
build
== p2 repo


Composite repositories

• Delineate artifacts and metadata into child repos
compositeArtifacts.jar
compositeContent.jar
I20100210-0950/
I20100211-0900/
I20100212-1053/
• Child repository I20100211-0900
artifacts.jar
content.jar
features/
plugins/


get involved

• p2 in the wiki http://wiki.eclipse.org/P2
• p2 mailing list p2-dev@eclipse.org
• IRC #equinox-dev
• p2 committers on twitter http://wiki.eclipse.org/Twitter
• Project sets


References

• Version numbering http://wiki.eclipse.org/Version_Numbering
• 2009 EclipseCon p2 tutorial
http://www.slideshare.net/PascalRapicault/understanding-
and-extending-p2-for-fun-and-profit
• PDE Build Integration with p2
http://help.eclipse.org/galileo/index.jsp?nav=/4_2_2
• Adding Self Update to an RCP Application
http://wiki.eclipse.org/Equinox/p2/Adding_Self-
Update_to_an_RCP_Application
• p2 director application
http://wiki.eclipse.org/Equinox_p2_director_application


Photo credits

• Orange and knife by Safari11
http://www.sxc.hu/browse.phtml?f=view&id=983682
• Oranges by Stephanie Berghaeuser
http://www.sxc.hu/photo/783917
• Sliced oranges shin0 http://www.sxc.hu/photo/278
• Recycle tag by jaylopez http://www.sxc.hu/photo/917290
• Cutlery by Oliver Delgado
• Director’s chair by Alek von Felkerzam


Legal notice

• Copyright © IBM Corp., 2007-2010. All rights reserved. This presentation
and the source code in it are made available under the EPL, v1.0.
• Java and all Java-based trademarks are trademarks of Sun Microsystems,
Inc. in the United States, other countries, or both.
• Eclipse and the Eclipse logo are trademarks of Eclipse Foundation, Inc.
• IBM and the IBM logo are trademarks or registered trademarks of IBM
Corporation, in the United States, other countries or both.
• Other company, product, or service names may be trademarks or service
marks of others.
• THE INFORMATION DISCUSSED IN THIS PRESENTATION IS PROVIDED
FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO
VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION, IT IS
PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, AND IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES
ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, SUCH
INFORMATION. ANY INFORMATION CONCERNING IBM'S PRODUCT PLANS
OR STRATEGY IS SUBJECT TO CHANGE BY IBM WITHOUT NOTICE


Extra slides


p2.mirror Ant task

• Allows you to mirror artifacts and metadata simultaneously
• Filter by platform and specify IUs

<p2.mirror source="file://${myRepo}">
<destination kind="metadata" location="file://${myNewRepo}"
format="file://${myRepo}" />
<destination kind="artifact" location="file://${myNewRepo}"
format="file://${myRepo}" />
<iu id=“org.eclipse.pde" version="" />
<iu id=“org.eclipse.pde.sdk" version="" />
<slicingOptions platformFilter="true" includeOptional="false"
includeNonGreedy="false" followStrict="true" />
</p2.mirror>


remove.iu Ant task

• Remove IUs from the repository

• Remove all packed jars from a repo

<p2.remove.iu>
<repository location="file://${myRepo}" />
<iu query="" artifacts="(format=packed)" />
</p2.remove.iu>

• Remove specific IUs

<p2.remove.iu>
<repository location="file://${myRepo}" />
<iu id="com.ibm.icu" artifacts="(format=packed)" />
</p2.remove.iu>


p2.composite.{artifact|metadata}.repository

• Create composite artifact and metadata repositories
<p2.composite.artifact.repository.create
location="file://${myRepo}" name="${p2.repo.name}"
compressed="true" failOnExists="false" />
<p2.composite.metadata.repository.create
location="file://${myRepo}" name="${p2.repo.name}"
compressed="true" failOnExists="false" />

• Add children
<p2.composite.artifact.repository.add
location="file://${myRepo}" child="${buildId}" />
<p2.composite.metadata.repository.add
location="file://${myRepo}" child="${buildId}" />
</target>


Repository management best practices

• Implement composite repositories
• Use repository tools to modify the contents of a repository
• Life cycle of a repository
• How to specify mirrors in your repository? (Not sure if this is
too much detail)


publisher

• The p2 publisher was integrated into pde build in 3.5M5.
• Features and bundles are gathered from source and published
to a repository
• Can publish
• Feature and bundles
• Products
• Categories
• Update site
• From an existing install
• Enabling the publisher will create a repository for all platforms
you are building.


p2 and agile software development

Develop

p2 repository
Test Build

Assemble

P2 Introduction

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to P2 Introduction (20)

Recently uploaded (20)

P2 Introduction