Automated Configuration of Firmware

APOLLO GROUP

Automated Configuration of Firmware

Michael Arnold
Principal Systems Engineer
27 September 2012

Agenda

Who
Tools
Example Workflow
Example Code
Links
Questions

APOLLO GROUP © 2012 Apollo Group 2

APOLLO GROUP

Who

APOLLO GROUP Apollo Group
© 2012 3

Who is Apollo?

Apollo Group is a leading provider of higher
education programs for working adults.


Who is Michael Arnold?

Systems Administrator
Automation geek
13 years in IT
Hadoop Cluster Admin
I deal with:
–Server hardware specification/configuration
–Server firmware
–Server operating system
–Hadoop Cluster performance and availability


How does firmware tie in with Puppet?

It has nothing to do with Puppet.
–Unless someone writes a module to program
firmware from the running OS.
• Which could work for RAID controllers.


APOLLO GROUP

Tools

© 2012 7

Hardware Toolkits

HP SmartStart Scripting Toolkit
Dell OpenManage Deployment Toolkit
IBM ServerGuide Scripting Toolkit
Fujitsu ServerView Scripting Toolkit


What do the tools provide?

Configuration of
–BIOS
–BMC (DRAC/iLO/RSA/etc.)
–Hardware RAID
–FC HBA


How does the toolkit work?

Manually configure reference hardware
Capture the firmware configuration
Clone the configuration to multiple machines


Toolkit Components

Kernel & Firmware
Toolkit
Initrd Tools


Toolkit Requirements

TFTP
DHCP NFS

Toolkit


APOLLO GROUP

Example Workflow

© 2012 13

Assumptions

Hardware is racked and cabled
Switch ports are up
Server is manually powered on
Firmware defaults are to boot from network if no
other boot device is found.
–No other boot devices are found


Constraints

All server hardware is from the same vendor (HP)


The Process

Firmware Hardware Firmware Buildsystem
Update Discovery Configuration Registration


Screencast


Workflow

Unknown system netboots


Workflow

Boot into HP SmartStart Scripting Toolkit


Workflow

Gather hardware information (CPU/RAM/disk/NIC)
via hpdiscovery


Workflow

via hpdiscovery
If this is HP hardware, then
–Update firmware
–Configure BIOS
–Configure iLO
–Configure RAID


Workflow

via hpdiscovery
–Update firmware
–Configure BIOS
–Configure iLO
–Configure RAID
cobbler-register to the hardware profile


Workflow

Gather hardware information (CPU/RAM/disk/NIC) via
hpdiscovery
–Update firmware
–Configure BIOS
–Configure iLO
–Configure RAID
cobbler-register to the hardware profile
System reboots


Workflow

Modify Cobbler system profile, giving it an identity
(hostname, IP, OS metadata)


Workflow

Known system netboots


Workflow

Boot into OS installer


Workflow

Install OS


Workflow

Install OS
Install Puppet


Workflow

Install OS
Install Puppet
…


Workflow

Install OS
Install Puppet
…
Profit!


APOLLO GROUP

Example Code

© 2012 31

Dell TFTP Config

LABEL dtk-install_hadoop
kernel /images/dtk-3.5.1/SA.1
append initrd=/images/dtk-3.5.1/SA.2
ramdisk_size=156482 Stage3_type=cdrom quiet DEBUG=0
share_location=192.168.1.1:/srv/nfs/ro/dtk-3.5.1
share_type=nfs selinux=0 share_opts=ro,nolock
share_script=install_hadoop.sh cprofile=Hadoop
ipappend 2


HP TFTP Config

LABEL sstk-install_hadoop
kernel /images/sstk-8.50/vmlinuz
append initrd=/images/sstk-8.50/initrd.img
root=/dev/ram0 rw ramdisk_size=184248 ide=nodma
ide=noraid pnpbios=off network=1
sstk_mount=192.168.1.1:/srv/nfs/ro/sstk-8.50
sstk_mount_type=nfs sstk_mount_options=ro,nolock
sstk_script=install_hadoop.sh cprofile=Hadoop
ipappend 2


Dell Config Capture

cd /opt/dell/toolkit/template/scripts
/opt/dell/toolkit/template/scripts/syscap.sh /tmp/syscap
/opt/dell/toolkit/template/scripts/raidcap.sh /tmp/raidcap
/opt/dell/toolkit/template/scripts/raccap.sh /tmp/raccap

# MegaCLI does not come with the DTK so "install" it.
cp -a /opt/dell/toolkit/systems/MegaCLI/opt/MegaRAID
/opt/MegaRAID
/opt/MegaRAID/MegaCli/MegaCli -CfgSave -a0 -f /tmp/mega.0


HP Config Capture

cd /TOOLKIT
./conrep -s -f /tmp/conrep.dat # BIOS
./hpacuscripting -c /tmp/cpqacuxe.dat # RAID
./hplpcfg /s /tmp/hplpcfg.dat # Emulex HBA
./hpqlarep /S /tmp/hpqlarep.dat # Qlogic HBA

if ./ifhw /tmp/hw_discovery.dat allboards.xml
"PCI:Integrated Lights-Out" 2> /dev/null ; then
./hponcfg -w /tmp/hponcfg.dat # iLO 2/3
else
./lo100cfg -o /tmp/lo100cfg.dat # LO100i
fi


Platform Detection

*** Detecting system type ***
System Information:
Name : ProLiant DL360 G5
Serial : USE123D456
Arch : x86_64
LNAME : ProLiantDL360G5
SNAME : DL360 G5
Asset Tag : 01234567


Some Available Variables
 HP SSSTK has a function library that sets many variables.
– $PRODUCT_NAME =
• "ProLiant SL170z G6"
• "ProLiant DL360 G6"
– $SERIAL_ID = hardware serial number
– $PRODUCT_LNAME =
• "ProLiantSL170zG6"
• "ProLiantDL360G6"
– $PRODUCT_SNAME =
• "SL170z G6"
• "DL360 G6"
– $TOOLKIT_MNTPNT = /mnt/main
– $TOOLKIT_SERVER_IP = NFS server IP
– $HWDISC_FILE = /TOOLKIT/hpdiscovery.xml


Inside install_hadoop.sh

 Set Variables Based on Hardware Type

export "`./hwquery ${HWDISC_FILE} allboards.xml SERVERNAME=SystemName`"

case "${SERVERNAME}" in
"ProLiant DL160 G6"* )
CONREPDAT=${TOOLKIT_MNTPNT}/data_files/dl160_conrep.dat
LO100CFGDAT=${TOOLKIT_MNTPNT}/data_files/dl160_lo100cfg.dat
;;
"ProLiant SL170z G6"* )
CONREPDAT="${TOOLKIT_MNTPNT}/data_files/sl170z_conrep.dat -x
${TOOLKIT_MNTPNT}/conrep_xml/conrep_SL170zg6_20100401.xml"
LO100CFGDAT=${TOOLKIT_MNTPNT}/data_files/sl170z_lo100cfg.dat
;;
#...


#...
"ProLiant DL360 G5"* )
CONREPDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_conrep.dat
if ./ifhw ${HWDISC_FILE} allboards.xml "PCI:Smart Array E200i Controller"
2> /dev/null ; then
HPACUDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_saE200i_cpqacuxe.dat
fi
if ./ifhw ${HWDISC_FILE} allboards.xml "PCI:Smart Array P400i Controller"
2> /dev/null ; then
HPACUDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_saP400i_cpqacuxe.dat
fi
HPONCFGDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_hponcfg.dat
;;
esac


if [ -n "$CONREPDAT" ] ; then
echo "** Applying BIOS Configuration"
./conrep -l -f ${CONREPDAT}
fi
if [ -n "$HPACUDAT" ] ; then
echo "** Clearing RAID Configuration"
./hpacucli/hpacuscripting -i ${HPACUDAT}_clear; sleep 5
echo "** Applying RAID Configuration"
./hpacucli/hpacuscripting -i ${HPACUDAT}
fi
if [ -n "$LO100CFGDAT" ] ; then
echo "** Applying BMC Configuration"; ./lo100cfg -i ${LO100CFGDAT}
elif [ -n "$HPONCFGDAT" ] ; then
echo "** Applying iLO Configuration"; ./hponcfg -f ${HPONCFGDAT}
fi


Register with Buildsystem
echo "** Registering with cobbler"
# Koan does not come with SSSTK, so "install" it.
${TOOLKIT_MNTPNT}/koan/install

# cprofile is set via kernel boot parameter and points at part of a
# Cobbler profile name.
# SERIAL_ID and PRODUCT_LNAME are set via SSSTK.
# PRODUCT_LNAME = ProLiantSL170zG6, ProLiantDL360G6, etc.
# SERIAL_ID = hardware serial number
echo "** Registering to profile ${cprofile}-${PRODUCT_LNAME}"
cobbler-register --server=${TOOLKIT_SERVER_IP} --fqdn=${SERIAL_ID}
--profile=${cprofile}-${PRODUCT_LNAME} --batch


Clear the HP RAID Controller
$ cat dl360g5_saE200i_cpqacuxe.dat_clear
; Version: 8.50.6.0
; http://h30499.www3.hp.com/t5/ProLiant-Servers-ML-DL-
SL/CPQACUXE-reset-utility-within-WinPE/td-p/3939475

Action= Reconfigure
Method= Custom

Controller= SLOT 0
ClearConfigurationWithDataLoss= Yes


Configure the HP RAID Controller
$ cat dl360g5_saE200i_cpqacuxe.dat ; Array Specifications
; Version: 8.50.6.0 Array= A
Action= Configure ; Array Drive Type is SAS
Method= Custom ; 1I:1:1 (72 GB), 1I:1:2 (72 GB)
; Controller Specifications Drive= 1I:1:1, 1I:1:2
; Controller HP Smart Array E200i OnlineSpare= No
; Firmware Version 1.82 ; Logical Drive Specifications
Controller= SLOT 0 LogicalDrive= 1
ReadCache= 100 RAID= 1
WriteCache= 0 Size= 69973
RebuildPriority= Medium Sectors= 32
ExpandPriority= Medium StripeSize= 128
SurfaceScanDelay= 15 ArrayAccelerator= Enabled
DriveWriteCache= Disabled


Error Handling

You may have to test for any failures when
applying configurations.
When reusing HP hardware, adding a user to the
BMC that already exists will cause a failure to
apply all of the configuration.
Newer firmware can have different config options
that will cause an older capture to fail.


Summary

Firmware Hardware Firmware Buildsystem
Update Discovery Configuration Registration


Links
 HP SmartStart Scripting Toolkit
http://h18013.www1.hp.com/products/servers/management/toolkit/index.html
 Dell OpenManage Deployment Toolkit
http://en.community.dell.com/techcenter/systems-management/w/wiki/1772.dell-
openmanage-deployment-toolkit.aspx
 IBM ServerGuide Scripting Toolkit
http://www-03.ibm.com/systems/be/management/sgstk/
 Fujitsu ServerView Scripting Toolkit
http://www.fujitsu.com/fts/products/computing/servers/primergy/management/de
ploy/
 Cobbler - Linux Installation Server
http://cobbler.github.com/
 LSI MegaRAID MegaCLI
http://www.lsi.com/downloads/Public/MegaRAID%20Common%20Files/8.02.24_
MegaCLI.zip


Contact

 https://intelligentsysadmin.wordpress.com/
 https://github.com/razorsedge
 http://forge.puppetlabs.com/razorsedge


Automated Configuration of Firmware

More Related Content

What's hot (10)

Similar to Automated Configuration of Firmware (20)

Recently uploaded (20)

Automated Configuration of Firmware