OSGi In Anger - Tara Simpson
- 1. OSGi In Anger A Case Study
- 2. Aepona? • Instil customer for 3 years • Provider of products into Telco space • Focus on ‘Network as a Service’ • Think abstractions over operator capabilities
- 3. Telecom Web Services (In a Nutshell) • SOAP/REST web services offering simplified APIs over SMS, MMS, Location, Conference Calling, etc • Endpoints provisioned on a per-partner basis • Endpoints backed by per-partner runtime policy enforcement
- 4. And Much More • Billing • Reporting • Alarms • Fault Reporting • System deployed with null implementations of optional features
- 5. Architecture container SMS (i.e. JVM) Portal OAM TLX (JBoss) MMS nodeA container group DEVKIT (Eclipse) MMS OAM TLX SMS nodeB
- 6. Container Groups Load Balancer MMS MMS Container Group
- 7. Runtime • Equinox-based stack • Initial bundles defined by config.ini • Bundles can be remotely installed, uninstalled, activated, etc • Managed by Eclipse RCP plug-in or command line (telnet & Command Provider)
- 8. A Potted History • Project started in late 2006 • Began life as Artix-based stack • Fairly large, ~10000 unit tests • Evolved to OSGi 2007
- 9. The OAM • Operations, Administration and Maintenance component spec’ed in 2007 • Used to create, delete, monitor & manage a multi-container (JVM) environment • Two key requirements: (a) 99.999% uptime (b) ability to DYNAMICALLY install, uninstall and patch components
- 10. A Leap of Faith • But for the wrong(ish) reasons • OAM provided compelling reason for OSGi • OSGi chosen mainly because it enabled hot-deploys • Not because of enforced modularity or service-oriented design
- 11. Perceived Downside • OSGi insists on everything being made ‘modular’ • But aren’t we already modular? • Don’t we partition our software effectively? 1000s of unit tests say we do!
- 12. Umm..NO! • In retrospect, system far from modular • Layers not partitioned cleanly • Code not open for extension • Free-for-all on code base
- 13. Life Before OSGi • Single source tree • Spring Configuration Hell • Extensibility issues - PS work polluting core product • Build issues
- 14. Evolving to OSGi • How do we develop? Tools? • How do we build? Ant, Maven? • What to do with Artix, Spring & Hibernate? • How to modularise existing code stream? • How much time ($) is this going to take? • No prior OSGi experience
- 15. Eclipse PDE • OSGi is first class citizen • Single project defines ‘target platform’ • Per-customer PSF (Team Project Set) files • Export bundles directly to TWS • We like PDE - but we don’t know any better!
- 16. Ant Build • build.xml copies projects into launcher- prescribed directory structure and then kicks off launcher • Heavy use of Eclipse features • Considerable effort migrating projects over, but minimal effort in maintaining • 400+ projects, features, fragments
- 17. Baby Steps • No big bang • Artix converted to single bundle • Common code refactored into core bundle • Services deployed as mega-bundles, built from single source tree • Gradual migration to OSGi services
- 18. Hibernate • Used bnd to create hibernate bundle • Used DynamicImport-Package: * to find all exported mapping files & pojos • Added Antlr & all hibernate jar files to internal classpath • Works, but less than ideal
- 19. Spring DM • Consistent programming model with existing Spring infrastructure/knowledge • No issues with proxy-based approach • Outside of core, only use Spring DM • Inside core, mixture of Spring DM and direct OSGi, but only when necessary
- 20. Early Mistakes • Failed to appreciate beauty of whiteboard pattern: publish/subscribe really should be replaced with export/track • Modules not delineated using OSGi services - services enable dynamism & enforce better design • Forgetting to unget services
- 21. Gradual Modularisation • Services migrated one at a time • Initially, layers broken into modules with public and private parts • Eventually, modules refactored to be service oriented • Code base significantly cleaner
- 22. Life Before OSGi Service ParlayX (e.g. SMS) ICallback ISmsDao SmsMessage Adapter ISmsActivityFactory SmsActivity * ALL PUBLIC Gateway/Network
- 23. Life With OSGi Service ParlayX (e.g. SMS) Adapter - public ISmsSender OSGi service SmsMessage POJO SmsSender Adapter - private ISmsDao ISmsDao ISmsDao SmsActivity Gateway
- 24. Note to Self • OSGi did not make me this way! • But it encouraged me to step back... • And be even more clinical in my separations
- 25. Reality Check • Developers rarely have the good grace to hide implementation details! • Developers who struggle with partitioning and layering will struggle even more with OSGi • OSGi requires that you think more & be more disciplined
- 26. Today • Extensive use of services & properties • Heavy use of Spring DM • Moderate use of the OSGi API • Little use of compendium services
- 27. Design • Layered WebService (e.g. SMS) • Service-Oriented OSGi services • Event-Driven Service (e.g. SMS) Shared OSGi services core • Loosely-Coupled OSGi services • Dynamic/Modular! Adapter
- 28. Overall Philosophy • Avoid the API at all costs • Necessarily complex and wonderfully subtle - e.g. returning null from a tracker • Well documented but good practice hard to come by • Hard to unit test - especially trackers • Instead use Spring DM wherever possible
- 29. Using the OSGi API • Only 1 activator across 150 production bundles • BundleContext injected via Spring DM BundleContextAware • Primarily used to track & register services • Also used to track bundles e.g. update property accessor when bundle installed, signal OAM when bundle fails to install
- 30. Idiomatic OSGi • What is good practice? A catalog of idioms/ patterns would be useful, very useful /** * Property accessor tracker that creates describer's associated * policy description based on properties extracted from exported * policy accessor service. Once created the service is immediately * 'untracked'. */ private class PropertyAccessorTracker implements ServiceTrackerCustomizer { public Object addingService(final ServiceReference serviceReference) { final IPropertyAccessor propertyAccessor = (IPropertyAccessor) bundleContext.getService(serviceReference); policyDescription = createPolicyDescription(propertyAccessor); bundleContext.ungetService(serviceReference); debug("Created policy description ", policyDescription); return null; } public void modifiedService(final ServiceReference serviceReference, final Object service) { // Do nothing } public void removedService(final ServiceReference serviceReference, final Object service) { // Do nothing - no need for unget } }
- 31. Fragments • Used extensively to configure bundles in customer specific ways • Used to house unit tests (10000+) • One test fragment for every production bundle • Fragments containing code cannot be unit tested. Therefore avoid.
- 32. Testing • Focus mostly on behavioural & interaction- based unit testing • Unit tests live in fragments • Dedicated team for end-to-end testing • Many parts of API mockable, except for trackers. Grumble.
- 33. Features • Eclipse features describe pre-canned applications/features • Parsed into config.ini files at build time • Deployed under <install-dir>/conf/ apps/<feature>/config.ini • Each feature in conf/apps presented as a deployable application on front-end
- 34. OSGi Services • Modules (mostly) communicate through services • Services enable ALL extension points • Lower layers define and export interfaces; consumed by higher layers • Lower layers define and track interfaces; implemented & exported by higher layers
- 35. Dynamic Method Interception Exported OSGi service With service properties: order=1 targetMethods=sendSms IMethodInterceptor IMethodInterceptor IMethodInterceptor Method web service Interceptor service proxy Invoker
- 36. IMethodInterceptor import org.aopalliance.intercept.MethodInterceptor; /** * Method interceptor that enables any bundle to intercept * a service invocation. Implementations of this interface * are expected to be exported as an OSGi, along with 2 * properties: * <ul> * <li>an 'order' property used to determine the order in which * exported interceptors are invoked compared to each other</li> * <li>an optional set of comma separated 'targetMethods' method * names defining the method(s) that the interceptor should be * applied to</li> * </ul> * * Typically implementations will extend {@link AbstractMethodInterceptor} * rather than implement this interface directly. * * @see AbstractMethodInterceptor */ public interface IMethodInterceptor extends MethodInterceptor { }
- 37. Canned Method Interceptors • Address Normalisation: Enables white & black listing • Group Address Resolution: Send to ‘group:abc’ => send to many • Reporting: On every system transaction
- 38. Policy Enforcement Created using IPolicyDescriber (an OSGi service) EnforceablePolicy EnforceablePolicy EnforceablePolicy web service Policy Enforcer service proxy
- 39. Tracking Lists /** * Provider of policy descriptions ({@link PolicyDescription}s), derived * from their service policy description ({@link IPolicyDescriber}) * counterpart. */ public class PolicyProvider implements IPolicyProvider { private List<IPolicyDescriber> describers; /** * Sets the list of policy describers being tracked. * The supplied list is expected to be a Spring-backed * list that automatically proxies and tracks the actual * policy describers. * @param describers a set of policy describers to be used */ @Required public void setPolicyDescribers(final List<IPolicyDescriber> describers) { this.describers = describers; } ... <bean id="policyProvider" class="com.aepona.tws.core.policy.support.PolicyProvider"> <property name="policyDescribers" ref="policyDescribers" /> </bean> <osgi:list id="policyDescribers" interface="com.aepona.tws.core.policy.IPolicyDescriber" cardinality="0..N" />
- 40. Mobile Originated Events • How to publish asynchronous SMS notifications and delivery receipts to third party applications? • More generally - how do we publish events from lower layers to higher layers?
- 41. Mobile Originated Events Third Party Service Network Application ?
- 42. Back to the Whiteboard • Lower layer defines and tracks ‘publisher’ interface • Higher layer implements and exports publisher
- 43. Whiteboard Exported as OSGi service SmsNotificationPublisher to TPA <<implements>> from SmsNotificationHandler ISmsNotificationPublisher network Tracks ISmsNotificationPublisher
- 44. Lifecycle Events • Web service endpoints come and go • How do we inform interested parties of such events? • They may need to stop ongoing transactions e.g. in the network layer
- 45. OSGi as an Event Bus • Whiteboard pattern pervades system • Core defines and tracks ‘listener’ services • Bundles implement and export listener implementations • Layers are massively decoupled from one another
- 46. Service Listener /** * Interface to be implemented classes interested in service events. * Any instance of a type that implements this interface and that is * exported as an OSGi service will be informed of service lifecycle * events. Events are fired <i>asynchronously</i> but sequentially. * <p> * Implementations should not tie up the invoking thread. In other * words the {@link #serviceChanged(ServiceEvent)} method should * return as quickly as possible. */ public interface IServiceListener { /** * Receives notification that a service lifecycle event * has occurred. * * @param serviceEvent the service event */ void serviceChanged(ServiceEvent serviceEvent); }
- 47. Service Event /** * A service event describing a service lifecycle change. * Service events are categorized as: * <pre> * create - signals that a service has been provisioned * update - signals that a service has been updated * dispose - signals that a service has been disposed * destroy - signals that a service has been destroyed * </pre> * An 'update' event signals that a service has been updated * without having to recreate the endpoint from scratch. * If an update requires that a service is re-provisioned * (because its service name has changed), then a 'destroy' * followed by a 'create' event will be emitted. */ public final class ServiceEvent { /** * Returns a service create event. * @param serviceName the newly created service name * @return a create service event */ public static ServiceEvent createEvent(final ServiceName serviceName) { return new ServiceEvent(CREATE, serviceName); } ...
- 48. Handling Service Events /** * Purges expired location notification requests from the system. * An expired notification is one that has sent the maximum number * of notifications or has reached its duration limit - both limits * are specified when the notification is created. */ public class LocationNotificationTerminator implements ILocationNotificationTerminator, IServiceListener, InitializingBean, DisposableBean { private ILocationNotificationCriteriaDao criteriaDao; /** * Deletes data associated with a destroyed service */ public void serviceChanged(final ServiceEvent serviceEvent) { if (serviceEvent.isDestroyEvent()) { logger.debug("Stopping all notifications for " + serviceEvent.getServiceName()); stopAll(criteriaDao.findByServiceName(serviceEvent.getServiceName())); } } ...
- 49. Service Adapters • When deployed inside operator TWS talks to single CORBA-based adapter • When deployed outside operator TWS proxies onto multiple operator APIs • Goal is unified API - OneAPI • Each operator has own service adapter OSGi service
- 50. Service Adapters • Need to support new operator, simply drop new adapter bundle Exported as OSGi Service With ‘name’ property name=orange OneAPI Service IServiceAdapter WebService orange IServiceAdapter vodafone IServiceAdapter sprint
- 51. Modularity Matters • TWS is large - without OSGi it would, right now, be a mess • Modularity enforces better discipline • Modular designs are more extensible • Modular designs are easier to test • Modular designs are cheaper
- 52. Personal Opinion • First I couldn’t live without Design By Contract • Then I couldn’t live without BDD & DI • Now I dread life without OSGi • Enforced modularity and OSGi services are indispensable design tools