SlideShare a Scribd company logo
Troubleshooting the Virtual Router - Run and Get
Diagnostics
CCC / ApacheCon NA Las Vegas 19
Boris Stoyanov
Software Engineer in Test
boris.stoyanov@shapeblue.com
twitter: @shapeblue
about me
•Break Stuff @ ShapeBlue
•Background:
•More than 10 years in Software Development
and Testing
•Specialize in:
•Test Management
•Automated Testing
•Testing Frameworks
•Joined ShapeBlue and CloudStack 2016
•Recently invited to join PMC
Troubleshooting the Virtual Router - Run and Get
Diagnostics
The Virtual Router is essential to
CloudStack Networking model
Get Diagnostics
Allows you to execute commands and gather log and config files from
the VR and System VMs
Get Diagnostics - Defaults
VR – ‘diagnostics.data.router.defaults’ global setting
“[IPTABLES], [IFCONFIG], [ROUTE], /etc/dnsmasq.conf, /etc/resolv.conf, /etc/haproxy.conf,
/etc/hosts.conf, /etc/dnsmaq-resolv.conf, /var/log/cloud.log, /var/log/routerServiceMonitor.log,
/var/log/dnsmasq.log”
System Vms – ‘diagnostics.data.systemvm.defaults’ global setting
“[IPTABLES], [IFCONFIG], [ROUTE], /usr/local/cloud/systemvm/conf/agent.properties,
/usr/local/cloud/systemvm/conf/consoleproxy.properties, /var/log/cloud.log”
Get Diagnostics - Customs
One can get custom files or execute scripts on the VR
• Custom scripts needs to be within brackets ‘ [] ‘. Any custom script
needs to be in ‘/opt/cloud/bin/‘
• Files needs to be pointed with the absolute path of the file
Get Diagnostics - Settings
diagnostics.data.gc.enable
Enable the garbage collector background task to delete old files from
secondary storage. Requires management server restart
true/false
diagnostics.data.gc.interval
The interval at which the garbage collector background tasks in
seconds. Requires management server restart
3600
diagnostics.data.retrieval.timeout Overall data retrieval timeout in seconds 86400 (1 day)
diagnostics.data.max.file.age
Sets the maximum time in seconds a file can stay in secondary
storage before it is deleted.
86400
diagnostics.data.disable.threshold
Sets the secondary storage disk utilisation percentage for file retrieval.
Used to look for suitable secondary storage with enough space,
otherwise an exception is thrown when no secondary store is found.
0.9
Get Diagnostics - more info
• PR has been submitted: 3350
• https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+G
Get Diagnostics - limitations
• For successful file retrieval, operator needs to specify the correct
absolute path of file to be retrieved.
• A working/running SSVM is required in order to create/generate
the public file download URL.
• The Zone where target VM is running should have at least 1
secondary storage that has a disk quota utilization of less than 90
%.
• Only supports system VMs
• Any script that must be executed as part of this API is expected to
be present in the system VM under the directory ‘/opt/cloud/bin/’
• The response from the API is only a URL for download.
Demo
Run Diagnostics
Allows the admin to execute network utility commands remotely on
any System VM (VR, SSVM, CPVM)
Run Diagnostics - supported
commands
• ping - test if any destination is reachable by tge System VM
• traceroute - check path and trasits to destination
• arping - test if destination is reachable through a specific NIC
Run Diagnostics - more info
• PR 2833 - Merged in 4.12
• Feature Spec:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Clo
udStack+Remote+Diagnostics+API
• Blog: https://www.shapeblue.com/troubleshooting-
cloudstack-virtual-routers/
Demo
Credits
• Dingane Hlaluku, Rohit Yadav and Shapelue Dev Team
• ACS Community – Code reviews and further testing
We’re hiring!
https://www.shapeblue.com/careers/
Q&A

More Related Content

PPTX
Nagios Conference 2014 - Jeff Mendoza - Monitoring Microsoft Azure with Nagios
PDF
SaltConf14 - Ryan Lane, Wikimedia - Immediate consistency with Trebuchet Depl...
PPT
SaltConf14 - Ben Cane - Using SaltStack in High Availability Environments
PPTX
Deploy RvSIEM (eng)
PPTX
Apache Performance Tuning: Scaling Out
PDF
25 Apache Performance Tips
PPTX
Ansible Automation - Enterprise Use Cases | Juncheng Anthony Lin
PPTX
WordPress + NGINX Best Practices with EasyEngine
Nagios Conference 2014 - Jeff Mendoza - Monitoring Microsoft Azure with Nagios
SaltConf14 - Ryan Lane, Wikimedia - Immediate consistency with Trebuchet Depl...
SaltConf14 - Ben Cane - Using SaltStack in High Availability Environments
Deploy RvSIEM (eng)
Apache Performance Tuning: Scaling Out
25 Apache Performance Tips
Ansible Automation - Enterprise Use Cases | Juncheng Anthony Lin
WordPress + NGINX Best Practices with EasyEngine

What's hot (20)

PDF
Salt conf 2014-installing-openstack-using-saltstack-v02
PDF
SaltConf14 - Forrest Alvarez, Choice Hotels - Salt Formulas and States
PDF
Puppet Camp Charlotte 2015: Use Puppet to Manage your NetApp Storage Infrastr...
PDF
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...
PPTX
Uponor Exadata e-Business Suite Migration Case Study
PDF
Apache Traffic Server & Lua
PPTX
Automatic Backup in Ceph
PPTX
Neutron upgrades
PPTX
Global State Management of Micro Services
PPTX
wordpress-performance-presentation
PPT
Roy foubister (hosting high traffic sites on a tight budget)
PPT
Managing Oracle Enterprise Manager Cloud Control 12c with Oracle Clusterware
PPTX
Deep dive networking
PDF
Web crawlers part-2-20161104
PDF
Automation using power cli
PDF
OpenStack Tempest and REST API testing
PPTX
Aegir Introduction
PDF
How to monitor and manage Apache Tomcat
DOC
EMC Networker installation Document
PPTX
2016 oSC MySQL Firewall
Salt conf 2014-installing-openstack-using-saltstack-v02
SaltConf14 - Forrest Alvarez, Choice Hotels - Salt Formulas and States
Puppet Camp Charlotte 2015: Use Puppet to Manage your NetApp Storage Infrastr...
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...
Uponor Exadata e-Business Suite Migration Case Study
Apache Traffic Server & Lua
Automatic Backup in Ceph
Neutron upgrades
Global State Management of Micro Services
wordpress-performance-presentation
Roy foubister (hosting high traffic sites on a tight budget)
Managing Oracle Enterprise Manager Cloud Control 12c with Oracle Clusterware
Deep dive networking
Web crawlers part-2-20161104
Automation using power cli
OpenStack Tempest and REST API testing
Aegir Introduction
How to monitor and manage Apache Tomcat
EMC Networker installation Document
2016 oSC MySQL Firewall
Ad

Similar to Boris Stoyanov - Troubleshooting the Virtual Router - Run and Get Diagnostics (20)

PDF
Boris Stoyanov - some new features in Apache cloudStack
PDF
New stuff in CloudStack!
PDF
Paul Angus - what's new in ACS 4.11
PDF
Whats new in Cloudstack 4.11 - behind the headlines
PDF
CloudStack - Top 5 Technical Issues and Troubleshooting
PDF
Giles Sirett: Introduction and CloudStack news
PPTX
Using the KVMhypervisor in CloudStack
PDF
CCNA17 KVM and CloudStack
PDF
Cloud stack troubleshooting
PDF
CloudStack and NFV
PPTX
CCNA17 CloudStack and NFV
PPTX
Designing CloudStack Clouds
PDF
ApacheCon Miami / CCCNA17 Using KVM in CloudStack
PDF
Rohit Yadav - The future of the CloudStack Virtual Router
PPTX
CloudStack EU user group - Trillian
PPTX
Introduction to Apache cloudstack - Linuxcon
PPTX
Nested CloudStack with VMware
PPTX
Silicon Valley CloudStack User Group - Designing CloudStack Clouds
PPTX
CloudStack networking
PPTX
Building virtualised CloudStack test environments
Boris Stoyanov - some new features in Apache cloudStack
New stuff in CloudStack!
Paul Angus - what's new in ACS 4.11
Whats new in Cloudstack 4.11 - behind the headlines
CloudStack - Top 5 Technical Issues and Troubleshooting
Giles Sirett: Introduction and CloudStack news
Using the KVMhypervisor in CloudStack
CCNA17 KVM and CloudStack
Cloud stack troubleshooting
CloudStack and NFV
CCNA17 CloudStack and NFV
Designing CloudStack Clouds
ApacheCon Miami / CCCNA17 Using KVM in CloudStack
Rohit Yadav - The future of the CloudStack Virtual Router
CloudStack EU user group - Trillian
Introduction to Apache cloudstack - Linuxcon
Nested CloudStack with VMware
Silicon Valley CloudStack User Group - Designing CloudStack Clouds
CloudStack networking
Building virtualised CloudStack test environments
Ad

More from ShapeBlue (20)

PPTX
The Yotta x CloudStack Advantage: Scalable, India-First Cloud
PPTX
Simplifying End-to-End Apache CloudStack Deployment with a Web-Based Automati...
PPTX
Extensions Framework (XaaS) - Enabling Orchestrate Anything
PDF
CloudStack GPU Integration - Rohit Yadav
PPTX
Building and Operating a Private Cloud with CloudStack and LINBIT CloudStack ...
PDF
Ampere Offers Energy-Efficient Future For AI And Cloud
PDF
Empowering Cloud Providers with Apache CloudStack and Stackbill
PDF
Apache CloudStack 201: Let's Design & Build an IaaS Cloud
PDF
Meetup Kickoff & Welcome - Rohit Yadav, CSIUG Chairman
PDF
Fully Open-Source Private Clouds: Freedom, Security, and Control
PPTX
Pushing the Limits: CloudStack at 25K Hosts
PPTX
Stretching CloudStack over multiple datacenters
PPTX
Proposed Feature: Monitoring and Managing Cloud Usage Costs in Apache CloudStack
PPSX
CloudStack + KVM: Your Local Cloud Lab
PDF
I’d like to resell your CloudStack services, but...
PDF
Storage Setup for LINSTOR/DRBD/CloudStack
PDF
Apache CloudStack 101 - Introduction, What’s New and What’s Coming
PDF
Development of an Оbject Storage Plugin for CloudStack, Christian Reichert, s...
PDF
VM-HA with CloudStack and Linstor, Rene Peinthor
PDF
How We Use CloudStack to Provide Managed Hosting, Swen Brüseke, proIO
The Yotta x CloudStack Advantage: Scalable, India-First Cloud
Simplifying End-to-End Apache CloudStack Deployment with a Web-Based Automati...
Extensions Framework (XaaS) - Enabling Orchestrate Anything
CloudStack GPU Integration - Rohit Yadav
Building and Operating a Private Cloud with CloudStack and LINBIT CloudStack ...
Ampere Offers Energy-Efficient Future For AI And Cloud
Empowering Cloud Providers with Apache CloudStack and Stackbill
Apache CloudStack 201: Let's Design & Build an IaaS Cloud
Meetup Kickoff & Welcome - Rohit Yadav, CSIUG Chairman
Fully Open-Source Private Clouds: Freedom, Security, and Control
Pushing the Limits: CloudStack at 25K Hosts
Stretching CloudStack over multiple datacenters
Proposed Feature: Monitoring and Managing Cloud Usage Costs in Apache CloudStack
CloudStack + KVM: Your Local Cloud Lab
I’d like to resell your CloudStack services, but...
Storage Setup for LINSTOR/DRBD/CloudStack
Apache CloudStack 101 - Introduction, What’s New and What’s Coming
Development of an Оbject Storage Plugin for CloudStack, Christian Reichert, s...
VM-HA with CloudStack and Linstor, Rene Peinthor
How We Use CloudStack to Provide Managed Hosting, Swen Brüseke, proIO

Recently uploaded (20)

PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PDF
Encapsulation_ Review paper, used for researhc scholars
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Encapsulation theory and applications.pdf
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
PDF
Empathic Computing: Creating Shared Understanding
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
KodekX | Application Modernization Development
PDF
Approach and Philosophy of On baking technology
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
“AI and Expert System Decision Support & Business Intelligence Systems”
CIFDAQ's Market Insight: SEC Turns Pro Crypto
Encapsulation_ Review paper, used for researhc scholars
The AUB Centre for AI in Media Proposal.docx
Building Integrated photovoltaic BIPV_UPV.pdf
Diabetes mellitus diagnosis method based random forest with bat algorithm
Reach Out and Touch Someone: Haptics and Empathic Computing
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Encapsulation theory and applications.pdf
The Rise and Fall of 3GPP – Time for a Sabbatical?
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Advanced methodologies resolving dimensionality complications for autism neur...
Empathic Computing: Creating Shared Understanding
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
KodekX | Application Modernization Development
Approach and Philosophy of On baking technology

Boris Stoyanov - Troubleshooting the Virtual Router - Run and Get Diagnostics

  • 1. Troubleshooting the Virtual Router - Run and Get Diagnostics CCC / ApacheCon NA Las Vegas 19 Boris Stoyanov Software Engineer in Test boris.stoyanov@shapeblue.com twitter: @shapeblue
  • 2. about me •Break Stuff @ ShapeBlue •Background: •More than 10 years in Software Development and Testing •Specialize in: •Test Management •Automated Testing •Testing Frameworks •Joined ShapeBlue and CloudStack 2016 •Recently invited to join PMC
  • 3. Troubleshooting the Virtual Router - Run and Get Diagnostics
  • 4. The Virtual Router is essential to CloudStack Networking model
  • 5. Get Diagnostics Allows you to execute commands and gather log and config files from the VR and System VMs
  • 6. Get Diagnostics - Defaults VR – ‘diagnostics.data.router.defaults’ global setting “[IPTABLES], [IFCONFIG], [ROUTE], /etc/dnsmasq.conf, /etc/resolv.conf, /etc/haproxy.conf, /etc/hosts.conf, /etc/dnsmaq-resolv.conf, /var/log/cloud.log, /var/log/routerServiceMonitor.log, /var/log/dnsmasq.log” System Vms – ‘diagnostics.data.systemvm.defaults’ global setting “[IPTABLES], [IFCONFIG], [ROUTE], /usr/local/cloud/systemvm/conf/agent.properties, /usr/local/cloud/systemvm/conf/consoleproxy.properties, /var/log/cloud.log”
  • 7. Get Diagnostics - Customs One can get custom files or execute scripts on the VR • Custom scripts needs to be within brackets ‘ [] ‘. Any custom script needs to be in ‘/opt/cloud/bin/‘ • Files needs to be pointed with the absolute path of the file
  • 8. Get Diagnostics - Settings diagnostics.data.gc.enable Enable the garbage collector background task to delete old files from secondary storage. Requires management server restart true/false diagnostics.data.gc.interval The interval at which the garbage collector background tasks in seconds. Requires management server restart 3600 diagnostics.data.retrieval.timeout Overall data retrieval timeout in seconds 86400 (1 day) diagnostics.data.max.file.age Sets the maximum time in seconds a file can stay in secondary storage before it is deleted. 86400 diagnostics.data.disable.threshold Sets the secondary storage disk utilisation percentage for file retrieval. Used to look for suitable secondary storage with enough space, otherwise an exception is thrown when no secondary store is found. 0.9
  • 9. Get Diagnostics - more info • PR has been submitted: 3350 • https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+G
  • 10. Get Diagnostics - limitations • For successful file retrieval, operator needs to specify the correct absolute path of file to be retrieved. • A working/running SSVM is required in order to create/generate the public file download URL. • The Zone where target VM is running should have at least 1 secondary storage that has a disk quota utilization of less than 90 %. • Only supports system VMs • Any script that must be executed as part of this API is expected to be present in the system VM under the directory ‘/opt/cloud/bin/’ • The response from the API is only a URL for download.
  • 11. Demo
  • 12. Run Diagnostics Allows the admin to execute network utility commands remotely on any System VM (VR, SSVM, CPVM)
  • 13. Run Diagnostics - supported commands • ping - test if any destination is reachable by tge System VM • traceroute - check path and trasits to destination • arping - test if destination is reachable through a specific NIC
  • 14. Run Diagnostics - more info • PR 2833 - Merged in 4.12 • Feature Spec: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Clo udStack+Remote+Diagnostics+API • Blog: https://www.shapeblue.com/troubleshooting- cloudstack-virtual-routers/
  • 15. Demo
  • 16. Credits • Dingane Hlaluku, Rohit Yadav and Shapelue Dev Team • ACS Community – Code reviews and further testing