Newlug presentation- OpenSolaris

04/15/2010
1

OpenSolaris
Overview
Ivan Richwalski
ivan@seppuku.net
http://seppuku.net/

04/15/2010
2

A brief history
● SunOS 1 - 4
 Based on original BSD
 Sun 680x0 workstations
 Later added Sun SPARC
● AT&T & Sun collaboration in 1987
 Merging the best parts to create SVR4
● New operating system named Solaris 2
 ( SunOS 4 retroacively named Solaris 1 )
 Solaris brand to identify whole system
 OS, Graphics system, Desktop software

04/15/2010
3

A brief history
● 1989 – AT&T forms Unix System Labs
● 1991 – First Solaris 2 release
● 1993 – AT&T sells USL to Novell
● 1994 – Sun buys out license from Novell
 No more license/royalties, Sun owns the code
● Early 2004 – OpenSolaris planning begins
● Late 2004 – OpenSolaris pilot project starts
● 2005 – Open to the public
 Dtrace source, website & lists, more code mid-year
● 2007 – Ian Murdock joins Sun, project Indiana starts

04/15/2010
4

Solaris/OpenSolaris Today
● Supports SPARC, x86 CPUs
● Both 32 and 64 bit kernels
 SPARC systems 64 bit kernel only
 Mixed 32 & 64 bit programs on 64 bit kernel
● Highly scaleable
 32 CPUs on 32 bit x86, 256 on 64 bit, 512 on SPARC

04/15/2010
5

Licensing
● CDDL - Common Development & Distribution License
 Kernel, drivers, and most of userland programs
● OpenSolaris Binary License
 Not open source, but is freely redistributable
 http://hub.opensolaris.org/bin/view/Main/no_source
● Lots of other code with their own licences
 GPL, BSD, MIT, Apache, Perl Artistic, etc.

04/15/2010
6

About the CDDL
● Comon Development & Distribution License
● Based on Mozilla Public License v1.1
● File Based
● Allows combination with code that has other
compatible licenses
● Requires modifications shared back
● Explicit patent license, “patent peace” clause
● http://hub.opensolaris.org/bin/view/Main/licensing_faq

04/15/2010
7

CDDL Misconceptions
● Gives Sun ownership of your code
 You retain ownership/copyright of your code
 Seperate contributor agreement for including your source
code in Sun's code base
● Designed to be GPL incompatible
 Compatible with lots of other licences
 File based, does not require everything to be under CDDL

● http://blogs.sun.com/cmh/entry/cddl_is_it_so_bad
● http://blogs.sun.com/harpster/entry/cddl_me_elmo

04/15/2010
8

Building an operating system
● Many seperate consolidations
 Operating System/Networking ( aka ON )
 Kernel, drivers, networking, libraries, services, & commands
 X11
 Desktop
 Gnome, Firefox, Thunderbird
 SFW ( Sun Freeware, various open source software )
 Globalization ( includes internationalization & localization )
● Not all consolidations open sourced
 SPARC Graphics Drivers
 OpenWindows, CDE ( obsolete, removed )

04/15/2010
9

SMF: Service Management Facility
● Framework for defining persistent system services
● Each service identified by FMRI
( Fault Management Resource Identifier )

● A service can have multiple instances

04/15/2010
10

● Configuration stored in repository
 Persistant, transaction based, safe rollback
● XML manifests used to import/export service
● Tracks dependencies, manages parallel startup

04/15/2010
11

● Restarters responsible for managing services
● New process attribute: contract
 Restarter alerted if contract becomes empty
 Decides what action to take

04/15/2010
12

● Long running & transient ( run once ) services
● Service manifest defines how service runs
 User/Group Credentials
 Process Privileges
 Working Directory
 Environment variables
 Project ID / Resource Pool
● Delegated administration of services
● Users assigned profiles for managing services
 [ demo 1 ]

04/15/2010
13

Fault Management Daemon
● Manages diagnosis engines, response agents, logging
● Recieves telemetry from other parts of system
 Hardware error reports ( CPU, memory, IO bus bit errors )
 Driver faults ( NIC reporting corrupted data )
 Disk errors ( bad blocks, disk not responding )
● Diagnoses problem, takes action when needed
 Offline/retire faulty CPUs & memory pages
 Reroute IO ( Multipath disks, aggregated NICs )
 Activate disk hot spares
 Restart affected services via SMF
 Send alerts via syslog, SNMP

04/15/2010
14

ZFS
● Pooled storage
 Does away with dealing with partitions/slices
 Eliminates extra volume manager layer
 Simplifies administration, improves performance

Volume Filesystem FS

Storage Pool

04/15/2010
15

ZFS
● Pool built on one or more vdevs
 Stripe, Mirror, Raidz1, Raidz2, Raidz3
 Able to add additional vdevs to a pool
● NFSv4/NTFS style ACLs
● Instant read-only snapshots
 Can be cloned to create new writeable filesystem
● Delegated administration
 Allow users & groups ability to manage own filesystems
 Fine grained control of allowed tasks
● Compression, deduplication
● In place upgrades for new capabilities

04/15/2010
16

ZFS
● Guaranteed data protection
 Checksums used end to end, catches & repairs corruption
 Transactional writes, always consistent state on disk
 Copy on write, never alters live data, no “write hole”
 Checksum Trees, checksum stored in parent block pointer
 Avoids corruption hitting both data & checksum together
 Validate entire pool by walking whole tree

 Background scrubbing & resilvering
● Maximum performance
 Writes spread across all devices in pool
 Most disk writes sequential due to copy on write/transactions
 Smart prefetch, detects length & striding reads

04/15/2010
17

ZFS
● Self healing ( as long as there's redundancy )
● Redundant copies of all filesystem metadata
● Copies of user data optional, setable on the fly

App App App

Mirror Mirror Mirror

04/15/2010
18

ZFS & SSDs
● Seperate Intent Logs ( aka “slogs” )
 Intent log used for synchronous writes
 Normally uses space from disks in pool
 Move to SSD for fast, short lived writes ( NFS, iSCSI )
● Level 2 ARC ( aka “L2ARC” )
 ARC == Adaptive Replacement Cache in memory
 L2ARC best for read optimized SSDs
 Faster than disk, slower & cheaper than RAM

04/15/2010
19

ZFS vs btrfs
● 2001 start, 2005 release ● 2007 development start
● CDDL License ● GPL License
● Checksum: fletcher2, ● Checksum: crc32, off
fletcher4, sha256, off
● Always copy-on-write ● Selective copy-on-write
● Device removal, defrag
● No fsck needed ● Online/offline fsck
● Rollback from snapshot

04/15/2010
20

DTrace
● Dynamic Tracing of kernel & userland code
● Simplified C-like “D” language
 Added functions and variables for tracing
 Thread-local variables
● Completely safe to run on production systems
● Zero probe effect when dtrace not running

04/15/2010
21

DTrace
● Provider: syscall, pid, proc, io, fbt ( functional block trace )
● Module: Process executable, library, driver
● Function
● Name: entry, return, connection-start, connection-done
● Applications can also have their own providers

04/15/2010
22

IPS: Image Packaging System
● Replaces old SVR4 packaging system
● Network based software repository & delivery
● GUI and CLI package tools
● Multi-variant packages

04/15/2010
23


04/15/2010
24

● Uses ZFS to create & manage boot environments

04/15/2010
25

Desktop
● Gnome 2.28
● Xorg 1.7
● Compiz ( on live CD )
● Firefox 3.5.8
● Thunderbird 3.0.1
● Device Driver Utility

04/15/2010
26

Desktop
● Time Slider
 Automated ZFS Snapshots
 Addon to Nautilus File Manager
 Enable/disable per filesystem

● [ demo ]

● Future improvements
 Backup to external media
 Better retention configuration

04/15/2010
27

Crossbow: Network Virtualization
● Great for server & network consolidation
● Add virtual NICs, switches, & vlans
● Assign cpu resources, priorities, bandwidth limits
 Local IP/network & remote IP/network
 Local & remote network port
 Transport type ( tcp, udp, sctp, icmp, icmpv6 )
 Limit max bandwidth, set relative priority
 Extended accounting for flows for tracking usage

04/15/2010
28

Zones
● OS level virtualization, “chroot on steroids”
 One kernel, isolated userland instances
 Seperate naming services inside each zone ( DNS, NIS, LDAP )
 Processes isolated within zones
 Broad resource controls
 Processor pools, process, thread, memory, swap limits
 Limited privileges & device access
 Must be granted by configuration
 Shared ip stack networking or exclusive access to NIC

04/15/2010
29

Branded Zones
● Support for non-native environments
 Interpretation layer to translate from brand to native
 System calls, signals, process & thread handling
 Different brand types
 native: on Solaris 10 / Solaris Express releases
 ipkg: for OpenSolaris releases
 lx: Linux 2.4 kernel & 32 bit apps
 lx26: Experimental Linux 2.6 support
 solaris8 & solaris9: on SPARC ( commercial support )
 s10brand: to support transition of Solaris 10 to future versions

● [ zones demo ]

04/15/2010
30

xVM
● OpenSolaris as dom0 on top of Xen
 Xen 3.4 hypervisor & tools
 Live migration support
 Full & Paravirtualized domU on top of hypervisor
 Intel VT or AMD-V instruction set needed for Full HVM
 OpenSolaris as full PV guest
● Use Crossbow for managing networking
● Can use ZFS volumes for storage

04/15/2010
31

COMSTAR
● Common Multiprotocol SCSI Target
● Seperates storage backend from access methods
● LUN masking & mapping
● Multipathing over different transports

COMSTAR

04/15/2010
32

COMSTAR
● Logical Unit Providers
 Disk, Tape, Virtual Tape Library logical units
 Actual storage via ZFS volume, raw disk, file, tape
 Supports thin provisioning

COMSTAR

04/15/2010
33

COMSTAR
● Port Providers
 Exports LUNs to other hosts
 Looks like normal block device to the host
 iSCSI, FibreChannel, FCoE, SAS, iSER ( iSCSI over RDMA )

Server 1

Server 2
COMSTAR

Server 3

04/15/2010
34

CIFS Server
● Native, in kernel CIFS server
● Can join Windows domain
● LDAP/Windows AD user maps
● Map Windows Security ID to
Solaris user/group
● NT ACL stored natively in ZFS
● ZFS snapshots work with
Windows Shadow Copy
● Virus scan engine support
 Integrated into ZFS, uses ICAP
 Protects NFS & local access too

04/15/2010
35

Trusted Extensions
● Replaces previous seperate “Trusted Solaris” OS
● System enforced mandatory access control policies
● Explicit authorization needed to cross security labels
● Device & network access also labeled

● [ demo ]

04/15/2010
36

Drawbacks
● Uses more memory
● No KDE ( Other distros include KDE )
● ZFS block pointer rewrite a ways off yet
 For online pool resizing, re-compress/dedup/defrag
● ZFS encryption still being developed
● IPS going through heavy development
● Power Management, Suspend/Resume
● Limited Wi-Fi chipsets supported
● Proprietary media codecs
 Fluendo commercial gstreamer plugins

04/15/2010
37

Other Distributions
● OpenSolaris: http://opensolaris.com/
 Official Sun/Oracle sponsored distribution

● BeleniX: http://belenix.org/
 First non-Sun Live CD & GUI distro
 KDE, GNOME, & XFCE packages

● Nexenta: http://nexenta.org/
 OpenSolaris kernel, Ubuntu userland & packaging
 Easy NAS/SAN Seutp
 Commercially supported NexentaStor storage platform

04/15/2010
38

Other Distributions
● EON: http://sites.google.com/site/eonstorage/
 Embedded Operating System/Networking
 Small NAS focused distribution

● MilaX: http://www.milax.org/
 Started as mini/rescue CD experiment
 Full CD/USB based distro, enough for older hardware

● Jaris: http://jaris.jp/
 Japanese based distribution

04/15/2010
39

Other Distributions
● Schillix: http://schillix.berlios.de/
 First non-sun distribution
 Lead by Joerg Schilling

● MarTux
 First SPARC distribution

● Korona:
http://techbase.kde.org/Projects/KDE_on_Solaris/OpenSolaris/Korona
 KDE4 based test drive Live DVD
 Experimental, currenly on KDE 4.3.98

Newlug presentation- OpenSolaris

More Related Content

What's hot (20)

Viewers also liked (7)

Similar to Newlug presentation- OpenSolaris (20)

More from NEWLUG (10)

Recently uploaded (20)

Newlug presentation- OpenSolaris