Eclipse Iot Day 2018 Presentation

Nicola La Gloria, www.kynetics.com2018 Eclipse IoT Day, Santa Clara
nicola.lagloria@kynetics.com
Enhance your embedded software
delivery pipeline with Eclipse hawkBit™

Agenda
❯ About us
❯ Motivations behind our work on the delivery pipeline for
Android and Embedded Linux devices
❯ Manage remote software updates: Eclipse hawkBit
❯ Update Factory, Kynetics’ delivery platform powered by Eclipse
hawkBit
❯ Update Factory DDI Clients for Embedded OS
》 The Android way for managing updates
》 Update Factory Android Client: Service Client and UI Client
》 Embedded Linux Client: SWUpdate
❯ Conclusions

About us
❯ Kynetics provides full software stacks for the most popular
embedded application processors focussing in particular on NXP
SoCs.
❯ Kynetics has been working on embedded products, tailoring
Android and Embedded Linux operating systems and applications
for different industries.
❯ We embrace embedded development following the best practices
to create a repeatable, reliable process for releasing
software.

Taking OS delivery to the next level
❯ Support customers during stage and production for their
medium scale general purpose SOC/SBC based products
❯ Enhance our development lifecycle by building and delivering
OS images upon specific code commits.
❯ Track updates and divide them per device type and use cases
❯ Device Metadata for
》 General device information (OS version, IP on local LAN, etc.)
》 Closing the loop: device “pull back” (sanity checks after deployment,
local target tests)

Building and Delivery pipeline

Eclipse hawkBit
The Eclipse Foundation has been very active in promoting numerous projects
for the IoT, in particular under the umbrella of the Eclipse IoT
community.
Eclipse IoT is an ecosystem of companies and individuals that are working
together to establish an Internet of Things based on open technologies.
https://iot.eclipse.org, https://eclipse.org/hawkbit/
One of the (many) projects is hawkBit “to create a domain independent back
end solution for rolling out software updates to constrained edge devices
connected to IP based networking infrastructure”

hawkBit in a nutshell
❯ Prepare the update file and upload it
❯ Create a Software Module and add an artifact to it
❯ Create a Distribution
❯ Rollout a distribution to targets
》 Manage rollout by groups
》 Group threshold for partial rollouts
❯ Set Target device metadata:
》 Attributes (i.e HW revision, custom)
》 Tags (for grouping purposes)
》 Others like: device description, what installed, logs, etc..

hawkBit overview
❯ User/Applications
》 UI
》 MGMT (API)
❯ Devices
》 DDI
(HTTP/REST/JSON)
》 DMF (AMQP)

hawkBit Architecture

High Availability
<<VM>>
RabbitMQ
<<Exchange>>
<<VM>> <<VM>> <<VM>>
hawkBit
Node 3
hawkBit
Node 2
hawkBit
Node 1
Caches CachesCaches
request
User action
Storage

❯ Update Factory is our OS image and application delivery
platform powered by Eclipse hawkBit™.
❯ Update Factory serves our software delivery pipeline during
development stages.
❯ Update Factory manages software rollout campaigns in
production.

Update Factory Architecture
❯ Client Service on the embedded device
❯ Update Server featuring hawkBit™
❯ IAM Server (Custom)
❯ Artifact Repository (AWS S3)
❯ Metadata Repository (AWS RDS)
❯ High Availability
》 Clusters IAM and Update Server
》 RDS
》 S3

Update Factory Target Device Clients
❯ Hawkbit DDI APIs allow to develop clients on the target device
❯ Client implements the server state machine and update
workflow.
❯ A first implementation on Linux Embedded is provided by
SWUpdate Suricata Daemon (GPL2)
❯ An Android complete implementation is provided by Kynetics
(EPL 1.0)

Device Update: Approaches
❯ Double copy:
》 The devices features two copies
of the Application/OS/RootFS
》 Each copy must contain the
kernel, the root file system,
and each further component that
can be updated
》 Cooperation with the boot
loader is necessary to decide
which copy should be booted
❯ Single copy:
》 A single copy of the system is
present
》 An independent bootable system is
required to manage the update
》 Possible to update the Kernel if
the update environment is
segregated from the regular OS
》 Cooperation with the bootloader is
necessary to boot in update mode

Double copy
Dual Boot Partition
Bootable system 1
ramdisk
rootfs
kernel
device tree
bootscript
Boot partition 1
rootfs partition 1
Bootable system 2
ramdisk
rootfs
kernel
device tree
bootscript
Boot partition 2
rootfs partition 2
bootloader
bootenv boot selection
Unpartitioned Space

Double Copy: Pros and Cons
❯ Pros:
》 Fallback in case of failure
》 Pretty easy to implement
❯ Cons:
》 Expensive in terms of storage resources, double the space
》 Requires quite a mechanism to switch between the running and the other copy
if multiple partitions are doubled (e.g. boot, root, cache, etc.)
》 Identify which copy is running

Single copy
bootloader
bootenv
Unpartitioned Space
ramdisk
kernel
device tree
bootscript
Boot partition
Independent
System
rootfs
rootfs partition

Single Copy: Pros and Cons
❯ Pros:
》 Requires smaller amount of space
》 “Update mode” lives in RAM
》 Can freely access whole storage (rewrite from scratch, including partition
table)
》 Can be used for factory reset (i.e by accessing external storage)
❯ Cons:
》 No fallback if write fails (e.g. power interruption). Restart recovery mode
to try again

Android approach to OTA updates
❯ Android uses the single copy approach
❯ Android approach splits the upgrade process in two phases:
》 preparation for the upgrade → performed in the full fledged Regular OS
》 execution of the upgrade → performed in a purpose built Recovery OS
❯ Execution performed by the recovery binary
❯ System is rebooted with the new OS
❯ Update system itself is eventually updated

Android update: preparation
❯ Device update preparation flow:
》 Usually your phone registers to the manufacturer cloud
》 polls for available updates
》 notifies update is available (Download? Y/n)
》 notifies update is ready to install (Proceed? Y/n)
》 reboot to Recovery OS and apply the update...

Android update: update execution
❯ Bootloader boots in ramdisk-based Recovery Mode
❯ recovery starts
❯ recovery unpacks the update file provided (signed zip)
❯ update-binary executes actions in the updater-script (edify)
❯ log and result files are written in the partition
❯ reboot to Regular OS

Recovery Partition
Android update Workflow
Recovery OS
Regular OS
Recovery bin
Bootloader
Update bin
Update script
Reboot

Android Update: advantages
❯ Single copy update featuring a recovery OS
❯ OTA agent runs in regular OS
》 No need to interrupt normal operation (yet)
》 Network access (e.g. Wifi setup by the user)
》 Interaction with the user (notifications / acknowledgment)
》 Full API access (Wifi or 3G/4G? Low battery?)
❯ Recovery has no need of network access, all artifacts are
pre-fetched
❯ Update script support binary writing (no mount is required)
❯ Recovery environment is RO, minimal, isolated

Update Factory Android Android Client
uramdisk
kernel
device tree
bootscript
Boot partition
>> Android UF Service
>> Android UF Client App
kernel
device tree

Recovery Partition
Tenant = foo
Id = bar
Url = https://updatefactory.io"
uramdisk
cache partition
.zip

Other default partitions
SoC
Space non partitioned
bootloader

Update Factory Android Client Resources:
❯ https://github.com/Kynetics/UfAndroidClient
❯ https://github.com/Kynetics/uf-ddiclient
❯ https://docs.updatefactory.io/devices/android/android-client-p
ackages/
❯ https://www.kynetics.com/docs/2018/uf_android_client/

Update Factory OE Linux Client (OELC)
Update Factory Client for Open Embedded implements all of the
missing bits to have an Android-like OTA mechanism on an Embedded
Linux OS
❯ Device to cloud communication
❯ Bootloader coordination (boot OS selection)
❯ Recovery partition
❯ Recovery bootscript
❯ Recovery ramdisk
❯ Update installation feedback to the cloud

Update Factory OELC featuring SWUpdate
❯ A good option for building in Linux, a recovery system
“Android Like” is by using SWUpdate as an agent:
》 Written in C by Stefano Babic (Denx) and contributors
》 Runs as Daemon or direct invocation
》 Update files (.swu) based on CPIO format
》 Several handlers (e.g. write raw data, write single file)
》 Signature and encryption

SWUpdate: Architecture
Notifier
Installer
Default
Parser
(.swu)
LUA Parser
Handler Manager
UBI MTD RAW ENV LUA
Local
Storage
Remote file
server
Web Server
Custom
protocol
MCU
hawkBit
START, RUN, SUCCESS, FAILURE, DOWLOAD, DONE

SWUpdate: update .swu file format
CPIO Header
sw-descriptor
Artifact (1)
Artifact (2)
Artifact (n)
software =
{
version = "0.1.0";
target = {
hardware-compatibility: [ "1.0"];
Images: (
{
filename = “rootfs.ext4.gz”;
device = /dev/mmcblk0p2”;
type = “raw”;
compressed = true;
}
);
scripts:(
{
filename = “installscript.sh”;
type = “schellscript”;
sha256 = “faaaa30c….”;
}
);
}
}

Update Factory OELC: device to cloud
SWUpdate implements the suricatta daemon mode which polls the
remote update server, hawkBit.
Configuration file:
suricatta :
{
tenant = "AcmeCorp";
id = "device-beep";
url = "https://stage.updatefactory.io";
artifactsstorage = "/recovery/updates";
};
isolated set of data
and configuration
unique device identifier
baseurl for request
URL generation
new option to download
update files (no install)

Update Factory: Linux Update Anatomy
Bootloader
bootenv boot_mode
ramdisk
kernel
device tree
bootscript
Boot partition
rootfs partition
ramdisk
kernel
device tree
bootscript

Recovery Partition
.swu
cache partition
surricata =
{
Tenant = “foo”;
Id = “bar”;
Url = "
https://stage.updatefactory.io";
};
Regular OS Recovery OS

Update Factory OE Linux Client Resources:
❯ https://github.com/Kynetics/meta-updatefactory
❯ https://docs.updatefactory.io/devices/linux/update-files/
❯ https://sbabic.github.io/swupdate

Conclusions
❯ We needed a building block for our delivery pipeline.
❯ hawkBit is architected and designed with proven technologies
and frameworks.
❯ hawkBit’s deployment model uses docker containers for its
evaluation. Easy to evaluate.
❯ Kynetics built Update Factory as its delivery platform on top
of hawkBit update server.
❯ We provide an Android client implementation upon DDI APIs.
❯ We provide a Linux client based on SWUpdate which embraces the
Android update philosophy.

Other Links
❯ https://www.kynetics.com/update-factory
❯ https://docs.updatefactory.io/
❯ http://warpx.io/blog/tutorial/easy-os-upgrades-swupdate
❯ https://eclipse.org/hawkbit/
This presentation: https://www.slideshare.net/kynetics/

Thanks:
Diego Rondini, Andrea Zoleo, Will Martindale,
Daniele Sergio, Roberto Sartori, Eric Nelson,
Gary Bisson (Boundary Devices), Gabriel Huau
(witekio), Benjamin Cabe’ and Roxanne Joncas
from the Eclipse Foundation.

Contacts:
USA
Kynetics LLC
2040 Martin Ave, Santa Clara CA 95050
Ph: +1 (408) 475 7760
Italy
Kynetics Srl
Via G. Longhin 23, Padova (PD) 35129
Ph: +39 (049) 781 1091
info@kynetics.com | www.kynetics.com

Thanks!

Security
❯ SWUpdate combines signed sw-description with the verification
of hashes for each single image.
》 RSA PKCS#1 (public/private)
》 CMS PKCS#7 (certificates)
❯ This means that only signed sw-description, generated by a
verified source, can be trusted by the installer.
》 sw-description.sig
》 Public.pem or public.key can be passed to SWUpdate daemon (on the device)
❯ sw-description contains hashes for each sub-image to verify
that each delivered subimage really belongs to the release.
》 Each image inside sw-description must have the attribute “sha256”

Android Device Deployment
uramdisk
kernel
device tree
bootscript
Boot partition
kernel
device tree

Recovery Partition
uramdisk
cache partition
.zip

Other default partitions
SoC
bootloader

Eclipse Iot Day 2018 Presentation

More Related Content

What's hot (20)

Similar to Eclipse Iot Day 2018 Presentation (20)

More from Kynetics (8)

Recently uploaded (20)

Eclipse Iot Day 2018 Presentation