JUC NYC 2012: Yale Build and Deployment with Jenkins

Jenkins User Conference New York, May 17 2012 #jenkinsconf

Yale Build and Deploy

E Camden Fisher
Yale University ITS


Who am I?

E Camden Fisher
Technical Lead
Unix Infrastructure and Virtualization
Information Technology Services
Yale University
camden.fisher@yale.edu
@fishnix


Who Uses Jenkins @Yale?


Let’s Not Forget Continuous Integration!

!   Maintain a Single Source Repository
!   Automate the Build
!   Make Your Build Self-Testing
!   Everyone Commits to the Mainline Every Day
!   Every Commit Should Build the Mainline on an
Integration Machine
!   Keep the Build Fast
!   Test in a Clone of the Production Environment
!   Make it Easy for Anyone to Get the Latest Executable
!   Everyone can see what's happening
!   Automate Deployment

Avoid "Integration Hell!!”


Why we chose Jenkins?

!   Easy!

!   Extensible

!   Scalable

!   Flexible

!   Open Source

!   Supported!


Yale History Lesson

Application lifecycle is a progression

!   Source Code Management

!   Maven and Artifactory

!   Building and Testing with Jenkins

!   Container Configurable artifacts

!   Runtime Configurable Containers

!   Managed deployments with Jenkins


SCM

SCCS
cp –pr
RCS

Subversion

CVS VSS


MVN

Maven

!   Project Object Model (POM)

!   Simplifies dependency resolution ("oops I forgot that .jar!")

!   Makes the build process easy and uniform

Artifactory (Maven Repository)

!   Where do I put my built artifacts?

!   Makes it easy for everyone to get the latest build!


Builds: Before Jenkins

!   We have a single source repo + a place to
store built artifacts, but…..
–  Builds still take a long time

–  Testing takes longer!

–  Code coverage take even longer!

–  Build environments are not standardized

–  Mistakes are caught later, hard to debug


Builds: Enter Jenkins

!   SCM commits automatically kick off a build
!   Testing and code coverage is automated and is run on
every commit.
!   Broken builds immediately notify the team and the
committer
!   Tests run in a Clone of the Production Environment!
!   Everyone can easily see what's happening!
!   Developers can get back to coding instead of building
and testing.
!   Releases are quick and easy


Deployment: The problem.

!   Configuration must be different per environment
!   Choice between loss of control or loss of agility
–  Often devs configure the container
–  Either devs can edit deployables or they can't
!   If they can... things are bad
–  Code is deployed, edited and removed w/out Change
Control
–  Moving quickly… "I'll clean it up later!"
–  Dev environments quickly diverge from Production
–  Security is compromised
!   If they can't... things are worse
–  Change requests are "slow”
–  Systems groups must do everything


Deployments: How do we fix them?

!   Standardize the process!
!   Promote SANE change control
!   Normalize deployables/artifacts
!   Single location for app configuration data
!   Standardize the container
!   Externalize what makes an environment
unique and special


Yale Application Installer Plugin

!   Standardize the process!
!   Why?
–  SSH creds + Delegation
–  Manage keys outside
–  Node name substitution
!   How?
–  Installer is co’d, scp’d
–  install.properties on dest
–  Pre-Install
–  SSH exec installer
–  Post-Install


Container Configurable Artifacts

!   Artifacts were built with embedded configuration.
–  ie. datasources, SSO, service endpoints, etc
–  Artifacts are different per environment! WAT!?
–  "Oops, I forgot to update that parameter!"
!   Externalize configuration parameters!
–  deployable XML using JNDIBindingServiceMgr
–  Apps self configure with JNDI
!   With Container Configurable artifacts…
–  The SAME artifact migrates between environments
–  XML configuration data (stored as build parameters in
Jenkins) is all that differentiates environments

<server>
<mbean code="org.jboss.naming.JNDIBindingServiceMgr"
name="org.jasig:service=CasConfig">

<attribute name="BindingsConfig" serialDataType="jbxb">

<jndi:bindings
xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jndi="urn:jboss:jndi-binding-service:1.0"
xs:schemaLocation="urn:jboss:jndi-binding-service:1.0 resource:jndi-binding-
service_1_0.xsd">

<jndi:binding name="jasig/cas/config">
<java:properties xmlns:java="urn:jboss:java-properties"
xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
xs:schemaLocation="urn:jboss:java-properties resource:java-properties_1_0.xsd">


<java:property>
<java:key>loginUrl</java:key>
<java:value>https://secure.its.yale.edu/cas/login</java:value>
</java:property>
<java:property>
<java:key>casServerUrl</java:key>
<java:value>https://secure.its.yale.edu/cas</java:value>
</java:property>

</java:properties>
</jndi:binding>

</jndi:bindings>
</attribute>
<depends>jboss:service=Naming</depends>
</mbean>
</server>



String jndiBinding = “JNDI_PATH_HERE”;
Context ctx = null;
try {
logger.info("Loading config from JNDI path: " + jndiBinding);
   ctx = new InitialContext();
   Map<String, String> config = (Map<String, String>) ctx.lookup(jndiBinding);
   doSomethingWithConfig(config);
} catch (Exception e) {
   throw new RuntimeException("Error getting configuration from JNDI. ", e);
} finally {
try { ctx.close(); } catch (Exception e) { }
}


Yale Standard Java Container

! JBoss EAP (5.1 now)
!   Minimal customization to externalize
configuration into runtime
!   JAAS provider for CAS SSO
!   Packaged in RPM
!   Configuration Management to install +
manage what’s “special”
!   Meets 100% of use cases (so far)
!   Runtime configurable container is key!

Yale Standard Java Container

$JBOSS_HOME/deploy/jbossweb.sar/server.xml


<Connector protocol="HTTP/1.1" SSLEnabled="true"
port="${jboss.bind.httpsport}" address="${jboss.bind.address}”
secure="true" clientAuth="false" scheme="https”
proxyName="${jboss.proxyname}" proxyPort="${jboss.proxyport}”
SSLCertificateFile="${jboss.server.home.dir}/conf/server.crt"
SSLCertificateKeyFile="${jboss.server.home.dir}/conf/server.pem”
URIEncoding="UTF-8” SSLProtocol="TLS”>

/etc/init.d/jboss_nodexx

# JBoss variables
export JBOSS_OPTS=“-Djboss.bind.httpport=${HTTPPORT}
-Djboss.bind.httpsport=${HTTPSPORT} ${ADDLJBOPTS}”

${JBOSS_HOME}/bin/run.sh ${JBOSS_OPTS} -c ${JBOSS_NODE} -g $
{JBOSS_PARTITION} -b ${JBOSS_BIND} -u ${CLSTR_ADDR}


Deployments: The result.

!   Consistency!
!   Jenkins writes configuration XML: $human_error--
!   Eliminate shells, and elevated privileges on servers
!   Container is managed by infrastructure with the O/S
!   Developers can deploy to DEV at will
!   Empowers developers to GTD
!   Puts gates at appropriate places
!   Changes to the Jenkins jobs, containers + deploys to Test/Prod
require change control
!   Frees Systems folks to work on more interesting things
!   Standard Container + Cloudy IaaS + Container Configurable
artifacts = Vendor-lock-in-less PaaS!


Drupal Continuous Deployment

!   Yale has a large shared Drupal
infrastructure
!   Shared = needs change control
!   Migration process looks similar to Java
!   Some end users want to edit themes!
!   Jenkins to the rescue!


What’s New?

!   Workflow integration through web services
!   Build + Release of Apache Servicemix
bundles and “features”
!   Spawning and Destroying Servicemix child
instances
!   Deployment of Features and OSGI
bundles to Apache Servicemix
!   Deploying Drupal 7
–  git + pull


Other Languages

! Php
–  Code coverage and unit testing available
!   Python
–  CherryPy + Filelocker deployments
!   Ruby
–  Automated unit testing
–  Automated code coverage
–  Automated deployment coming soon
! .Net
–  Build, unit test, archive creation


The Future

!   RBAC + folders
–  delegate responsibility to other systems groups
!   Ruby deployments
!   Enterprise Service Bus
!   Centralized SSH mangagement
–  the tools are better now
!   Managing/Provisioning with cfg mgmt
!   Testing and release of cfg mgmt “code”
!   Cloud deployments


We’re hiring!

http://www.yale.edu/jobs


Questions?

E Camden Fisher
camden.fisher@yale.edu
@fishnix


Thank You To Our Sponsors
Platinum
Sponsor

Gold
Sponsors

Silver
Sponsors

Bronze
Sponsors

JUC NYC 2012: Yale Build and Deployment with Jenkins

More Related Content

Similar to JUC NYC 2012: Yale Build and Deployment with Jenkins (20)

Recently uploaded (20)

JUC NYC 2012: Yale Build and Deployment with Jenkins