Dag Sonstebo - CloudStack usage service

The Cloud Specialists
CloudStackUsage Service Deep Dive
Dag Sonstebo
Cloud Architect
dag.sonstebo@shapeblue.com
Twitter: @dagsonstebo

C l i c k t o e d i t
ShapeBlue.com @ShapeBlue
Ab ou t m e
• Cloud Architect @ ShapeBlue
• Background:
• Cloud and virtualization architect with 20 years experience
from the service provider, financial and manufacturing
industries.
• Specialize in:
• Cloud infrastructure architecture and engineering.
• Virtualization - VMware vSphere, Citrix XenServer, KVM.
• Automationand configurationmanagement, love Ansible.
• Involvedwith CloudStacksince version 2.1.
• Downtime: traveland motorbikes.

The Cloud Specialists ShapeBlue.com @ShapeBlue
“ShapeBlue are expert builders of public & private
clouds. They are the leading global CloudStack
services company.”
Ab o u t S h ap eB l u e

S h a p e B l u e c u s t o m e r s

CloudStack usage service

O ver vi ew
CloudStack usage is a complimentary service
which:
• tracks end user consumption of CloudStack
resources
• summarizes all data in a separate database for
reporting or billing
The usage database can be:
• queried directly
• queried through the CloudStack API
• or it can be integrated into external billing or reporting systems.

H ow d o I i n s t al l i t ?
CentOS:
# yum install cloudstack-usage
# chkconfig cloudstack-usage on
# service cloudstack-usage on
Ubuntu:
# apt-get install cloudstack-usage
# update-rc.d cloudstack-usage defaults
# service cloudstack-usage on
Configuration files:
# ls -l /etc/cloudstack/usage/
total 4
lrwxrwxrwx. 1 root root 40 Sep 8 08:18 db.properties -> /etc/cloudstack/management/db.properties
lrwxrwxrwx. 1 root root 30 Sep 8 08:18 key -> /etc/cloudstack/management/key
-rw-r--r--. 1 root root 2968 Jul 12 10:36 log4j-cloud.xml

G l ob al s et t i n g s
Setting Comment
usage.stats.job.aggregation.range The range of time for aggregating the user statistics specified in
minutes (e.g. 1440 for daily (default), 60 for hourly)
usage.stats.job.exec.time The time at which the usage statistics aggregation job will run
(default 00:15). This is utilized together with the
usage.stats.job.aggregation.range to determine service scheduling.
enable.usage.server Switches usage service on/off
usage.aggregation.timezone Timezone used for usage aggregation (default “GMT”)
usage.execution.timezone Timezone for usage job execution
usage.sanity.check.interval Interval (in days) to check sanity of usage data
usage.snapshot.virtualsize.select Set the value to true if snapshot usage need to consider virtual size,
else physical size is considered (default “false”)

U s ag e even t s an d t y p es
• All lifecycle events for resources in
CloudStack are tracked in the
cloud.usage_event table.
• This table lists all timestamps for:
• Create / destroy
• Start / stop
• Request / release
• These events are processed by the
usage service and categorized as usage
types.
Create
Start
Request
Release
Stop
Destroy

U s ag e t y p es
# cloudmonkey list usagetypes
count = 21
usagetype:
+-------------+--------------------------------------+
| usagetypeid | description |
+-------------+--------------------------------------+
| 1 | Running Vm Usage |
| 2 | Allocated Vm Usage |
| 3 | IP Address Usage |
| 4 | Network Usage (Bytes Sent) |
| 5 | Network Usage (Bytes Received) |
| 6 | Volume Usage |
| 7 | Template Usage |
| 8 | ISO Usage |
| 9 | Snapshot Usage |
| 10 | Security Group Usage |
| 11 | Load Balancer Usage |
| 12 | Port Forwarding Usage |
| 13 | Network Offering Usage |
| 14 | VPN users usage |
| 21 | VM Disk usage(I/O Read) |
| 22 | VM Disk usage(I/O Write) |
| 23 | VM Disk usage(Bytes Read) |
| 24 | VM Disk usage(Bytes Write) |
| 25 | VM Snapshot storage usage |
| 27 | VM Snapshot on primary storage usage |
| 28 | VM Backup storage usage |
+-------------+--------------------------------------+

P r o c e s s f l o w – AP I c a l l a n d u s a g e j o b s
API
• Call received
API
• Call processed
DB
• Lifecycle action
written to
cloud.usage_event
Start
• Execute usage job at:
• usage.stats.job.exec.time
• + usage.stats.job.aggregation.range
Copy
• Copy all new events:
• FROM: cloud.usage_event
• TO: cloud_usage.usage_event
• Copy account table for reference
Process
• Summarise each usage type per account:
• Populate helper tables, e.g. usage_storage,
usage_vm_instance, etc
• Aggregate usage types for all accounts
• in cloud_usage.cloud_usage
• based on usage.status.job.aggregation.range

U s ag e even t s

H el p er t ab l es
• Helper tables summarise each resource with:
• start and end datestamp
• usage type (if required)
• Note though – every table has a different schema depending on
the resource usage type
• Example (all in the cloud_usage) DB:
• usage_vm_instance
• usage_storage
• usage_vm_disk
• Etc.

H e l p e r t a b l e s - e xa m p l e s

U s ag e ag g r eg at i on
• Once helper tables are processed the
cloud_usage.cloud_usage table is populated.
• The usage service analyses the helper table
entries and divides the time usage for each by
the time period defined in
“usage.stats.job.aggregation.range”.
• E.g.:
• how many hours has a VM been running in each 1440
minute (24 hour) period
• how many hours has each GB of storage been used in
1440 minute period

C l o u d _ u s a g e . c l o u d _ u s a g e
Usage
type
Raw
usage
Resource
ID
Vol/snap
size
Start /
end date

Q u er y t h r ou g h AP I w i t h C l o u d Mo n k e y

R e p o r t i n g e xa m p l e – r u n n i n g VMs

R ep or t i n g exam p l e – n et w or k u t i l i z at i on

R ep or t i n g exam p l e – vol u m es

R e p o r t i n g e x a mp l e – I P a d d r e s s e s , p o r t f o r w a r d i n g a n d VP N

Tr o u b l e s h o o t i n g
• Check usage job status in the database:
• Usage jobs:
• On service startup the usage service will check the DB whether last job was ran – if not this job is
ran once.
• Note if execution time takes longer than “usage.stats.job.aggregation.range” – default 1 day. If so
jobs may overlap and cause problems, hence avoid this situation.
• Logs:
• /var/log/cloudstack/usage/usage.log
• Relativelyverbose – will show processing of usage types, accounts, etc.

Tr o u b l e s h o o t i n g a n d h o u s e k e e p i n g
• DB inconsistencies:
• Back up databases before editing…….. ☺
• If usage job fails it will report in logs what the problem is.
• Watch out for inconsistencies like STOP entry without START, DESTROY without CREATE or double
entries – e.g. 2 x START entries for a VM.
• If required you can add/delete entries in the cloud.usage_event table.
• Housekeeping of cloud_usage table:
• Use the removeRawUsageRecords API call to delete entries older than 5 days.
# cloudmonkey removeRawUsageRecords interval=5
success = true
• Regenerating usage data:
• Use the generateUsageRecords API call to rerun usage in case of failure:
# cloudmonkey generateUsageRecords startdate=2018-09-01 enddate=2018-09-30
success = true

A b r i ef w or d on t h e q u ot a s er vi c e
• In the cloud_usage database you will see a number of “quota_*” tables.
• These are not directly used by the usage service, rather the separate
Quota service.
• Check the following for more information:
• https://cwiki.apache.org/confluence/display/CLOUDSTACK/Quota+Service+-
+FS
• http://docs.cloudstack.apache.org/projects/cloudstack-
administration/en/4.11/plugins/quota.html

Any questions?

Some references
• Blog post:
http://www.shapeblue.com/cloudstack-usage-service-deep-dive
This has more technicaldetail.
• CloudStack usage references:
installation/en/4.11/optional_installation.html
administration/en/4.11/usage.html

Mo r e i n f o r m a t i o n
• Slide decks: http://www.slideshare.net/shapeblue
• Blog: http://shapeblue.com/blog
http://dsonstebo.wordpress.com
• Email: dag.sonstebo@shapeblue.com
• Twitter: @dagsonstebo
• Web: http://shapeblue.com

Dag Sonstebo - CloudStack usage service

More Related Content

What's hot (20)

Similar to Dag Sonstebo - CloudStack usage service (20)

More from ShapeBlue (20)

Recently uploaded (20)

Dag Sonstebo - CloudStack usage service