Sprint 71

ManageIQ
Sprint 71 Review - Sprint End October 16, 2017
October 18, 2017
Meeting Start: 10:30am EST

Overview
● Sprint Statistics (Oleg Barenboim)
● Community Update (Carol Chen)
● Classic UI (Dan Clarizio)
● Providers (Bronagh Sorota, Adam Grare, Rich Oliveri)
● Automate (Tina Fitzgerald)
● Platform (Gregg Tanzillo)
● API (Alberto Bellotti)
● Documentation (Chris Budzilowicz)
● Discussion

All Repo Stats - Top 10
Top Repositories # Closed
ManageIQ/manageiq-ui-classic 109
ManageIQ/integration_tests 105
ManageIQ/manageiq 87
ManageIQ/manageiq-ui-service 35
ManageIQ/manageiq-api 29
ManageIQ/manageiq-schema 12
ManageIQ/manageiq-automation-engine 11
ManageIQ/manageiq-providers-amazon 11
ManageIQ/manageiq_docs 11
ManageIQ/manageiq-gems-pending 9
Total of 537 merged PRs across 87 ManageIQ Repos
Pull Requests from: 2017-10-03 00:00:00 UTC to: 2017-10-16 00:00:00 UTC

Distribution and Types of Merged PRs across ManageIQ
Repositories

PRs Merged per Sprint across ManageIQ Repositories

Repository Health Report
October 16, 2017

Community Update
(Carol Chen)
● Last Week in ManageIQ
○ http://manageiq.org/blog/2017/10/yep/ by Drew Bomhof
○ http://manageiq.org/blog/2017/10/how-was-your-last-week/ by Libor Pichler
● Blog Posts
○ "MiqLdap To SSSD Conversion Tool" by Joe VLcek
http://manageiq.org/blog/2017/09/miqldap-to-sssd/
○ "H-release Rearchitecture" by Jason Frey
http://manageiq.org/blog/2017/10/h-release-rearchitecture/
● Events
○ Open Source Summit EU, Oct 23 - 26 in Prague
http://events.linuxfoundation.org/events/open-source-summit-europe
○ USENIX LISA, Oct 29 - Nov 3 in San Francisco
https://www.usenix.org/conference/lisa17

Classic UI
(Dan Clarizio)
PRs merged - 109
● UI Repo: Enhancements - 26, Bugs - 37, Technical Debt/Refactoring - 28/9
● UI Components Repo - 8
Refactoring/Technical Debt:
● Toolbars refactored to pass in send_checked params explicitly when sending transactions from the
list views (on-going)
● Moved SubstMixin used in the expression editor to the core repo
● Merged similar Ansible Tower and Foreman provider form controllers
● Replaced ui_lookup calls with gettext calls in many controllers
● Background PNG images in quadicons replaced with SVGs
● Deduplicated/unified tree building code in controllers

Classic UI
(Dan Clarizio)
Enhancements:
● Generic Objects
○ Custom image support for Generic Object classes
○ Tagging support for Generic Object instances
○ Support to delete Generic Object Classes (last CRUD piece)
● New delay based retirement support
● Alerts - Added required field ‘severity’ to editor and list/summary screens
● Tag expression support was added to the Group editor
● Automate
○ Support for embedded methods
○ Support for Playbook type methods
● Physical Servers support added (physical servers team)
○ Refresh & delete
○ Manage Policies, Policy simulation, Tagging
● Infrastructure Provider dashboard converted to Angular & Patternfly components.

Classic UI
Custom Image support in Generic Object Class editor

Classic UI
Delay based Retirement

Classic UI
Tag expressions in Group editor

Classic UI
Embedded methods in Automate

Classic UI
Playbook type Automate Methods

Providers
(Bronagh Sorota, Adam Grare, Rich Oliveri)

Merged Provider PRs
Provider PRs Provider PRs
amazon 11 scvmm 4
openstack 9 vmware 4
ovirt 8 kubernetes 3
hawkular 7 lenovo 3
vmware 6 openshift 3
azure 6 ansible_tower 2
nuage 5 google 1

Cloud Providers
Amazon
● CloudVolumeSnapshot delete operation
● OrchestrationStack event support
● Network router inventory

Azure
● Drastic performance improvements to private
image gathering
● Azure-armrest 0.9.1
● Combined with improvements from Sprint 70:
○ Memory consumption reduced by 50%
○ Real time reduced by 25%
Cloud Providers cont.

Providers
RHV/oVirt
● Async requests in the provider inventory refresh
● Integrating OVN management (to manage RHV network provider)
Middleware
● Enhancement - when server or domain operation are requested, an event in
the timeline is logged (success/failure)
● Fixed handling of downloading attempt of an already removed JDR report
● Fixed - Events don't show on Middleware Provider Timeline

Container Providers
● Image Inspector OpenSCAP CVE URL is now globally configurable in the
Advanced Settings
○ This allows disconnected cluster to define a locally-hosted copy of the CVE definition to be
used for the Container Images SSA
● Following an update from OpenShift the Component Status inventory
collection has been removed
ems:
ems_kubernetes:
image_inspector_cve_url: https://my-specific/cve-base-url/

Container Providers
● Following up the Projects Quotas addition to reports; a new Out-Of-The-Box
example was added:

Container Providers
● Group by docker label in chargeback for container images

Container Providers
● Prometheus Alerts Collection Manager were added to the Container Providers

OpenStack
● Orchestration Stack related updates
○ Updated Infra provider refresher for OSP12
○ Enhanced orchestration template parameter type
support
● Finished model actions for OpenStack Image (Template in
MIQ)

OpenStack
● Fixed empty timelines on Hosts
● Fixed attach/detach disks automate methods
● Fixed error messages for Cloud Tenant operations
● Fixed Use Floating IP address creation messages

Lenovo
● Added support to collect Config Patterns
● Will be used in the future to support provisioning to physical
servers

SmartState Analysis
(Rich Oliveri)
Accommodate new Azure features.
(Total of 22 PRs across 5 repositories)
○ Added support for Managed Images.
○ Added support for Managed Instances.
○ Verified support for encrypted managed disks.
○ Added support for locked disks - through snapshots.

SmartState Analysis
(Rich Oliveri)
SmartState Analysis for Amazon - Work in progress
○ Added amazon_ssa_support GEM
■ Agent SSA code and communication.
○ PR pending for amazon_agent_coordinator worker.
○ Asynchronous response handler in development.
○ New Agent deployment scheme:
■ Instances based on Atomic OS
■ Code deployed and executed through Docker
containers.

Automate
(Tina Fitzgerald)
PRs Merged: 37
15 - Enhancements, 11 - Bugs, 6 - Refactoring, 5 - Technical Debt
● Ansible Playbook Method
○ Can be used as a State Machine or regular method
○ Input parameters as extra vars, substitution enabled
● Service Resource Linking
○ Links VMs to Service via API
● Generic Object
○ Added support for calling Generic Object methods via API

Automate
(Tina Fitzgerald)
● Automate Workspace
○ Method lifetime
○ Persists through retries
○ Only accessible to Ansible Methods today
● Orchestration Templates
○ OpenStack - Enhanced non-string parameter type support
○ Amazon - Added necessary constraint for text area
● Service Models
○ Exposed hardware disks/volumes

Automate
(Greg McCullough)
Embedded Methods - UI Changes Merged

Automate
(Greg McCullough)
Embedded Methods - UI Changes

Automate
(Greg McCullough)
Embedded Methods
dump_root_objects
logging
InspectMe

Automate
(Greg McCullough)
Embedded Methods - All together

Automate
(Greg McCullough)
Embedded Methods - Logging
[09:16] INFO -- : Invoking [inline] method [/Demo/System/Request/InspectMe] with inputs [{}]
[09:16] INFO -- : Updated namespace [Shared/Methods/dump_root_objects Demo/Shared]
[09:16] INFO -- : Loading embedded method Demo/Shared/Methods/dump_root_objects
[09:16] INFO -- : Loading embedded method Demo/Shared/Methods/logging
[09:16] INFO -- : <AEMethod [/Demo/System/Request/InspectMe]> Starting
[09:17] INFO -- : <AEMethod inspectme> <main> - Inspect is running
[09:17] INFO -- : <AEMethod inspectme> Root:<$evm.root> Attributes - Begin
[09:17] INFO -- : <AEMethod inspectme> Attribute - message: create
[09:17] INFO -- : <AEMethod inspectme> Attribute - miq_server: #<MiqAeMethodService::MiqAeServ...
[09:17] INFO -- : <AEMethod inspectme> Attribute - miq_server_id: 14
[09:17] INFO -- : <AEMethod inspectme> Attribute - object_name: Request
[09:17] INFO -- : <AEMethod inspectme> Attribute - request: inspectme
[09:17] INFO -- : <AEMethod inspectme> Root:<$evm.root> Attributes - End
[09:17] INFO -- : <AEMethod [/Demo/System/Request/InspectMe]> Ending
[09:17] INFO -- : Method exited with rc=MIQ_OK
[09:17] INFO -- : Followed Relationship [miqaedb:/System/Request/inspectme#create]

Automate
(Greg McCullough)
Embedded Methods - Errors
Add error into
Embedded method:
INFO -- : <AEMethod [/Demo/System/Request/InspectMe]> Starting
INFO -- : <AEMethod inspectme> <main> - Inspect is running
INFO -- : <AEMethod inspectme> Root:<$evm.root> Attributes - Begin
ERROR -- : <AEMethod inspectme> The following error occurred during method evaluation:
ERROR -- : <AEMethod inspectme> RuntimeError: fake error on line 100]
ERROR -- : <AEMethod inspectme> Demo/Shared/Methods/dump_root_objects:5:in `dump_root'
ERROR -- : Method STDERR: Demo/Shared/Methods/dump_root_objects:5:in `dump_root': fake error on line 100]
(RuntimeError)

Automate
(Greg McCullough)
Embedded Methods - Notes
● Embedded methods support domains lookup at runtime
○ Specifying the full path, including the domain, will prevent it from getting
overridden at runtime from a different domain
● No support for nested embedded methods
● We recommend that embedded methods contain classes or a collection of
methods.
○ Code will execute in the order it was listed in the Embedded Methods list.

Platform
(Gregg Tanzillo)
PRs Merged: 52
11 - Enh, 13 - Bugs, 14 - Tech Debt, 6 - Refactoring, 8 - Other
Enhancements
● Replication - Updated pglogical to version 2.1
● Restrict PostgreSQL to currently supported versions 9.4, 9.5
● Added severity to alert definitions (info, warning, error)
● Added Metering Used Hours to chargeback
● New Report: Projects by Quota Items (zakiva)

Platform
(Gregg Tanzillo)
PRs Merged: 73
29 - Enh, 16 - Bugs, 12 - Tech Debt, 4 - Refactoring, 8 - Other
Bug Fixes
● RBAC -
○ Add network models to Hosts & Clusters (belongsto) filter
■ CloudNetwork, CloudSubnet, FloatingIp, LoadBalancer, NetworkPort, NetworkRouter,
SecurityGroup
○ Fixed missing providers in ‘Optimize > Utilization’ tree
■ Properly scope provider classes (STI)
● Fixed report import - symbolize keys only in 'db_options:' section
Technical Debt
● Dropped Ruby 2.2 support

Platform
Metering Used Hours to chargeback report

Platform
Projects by Quota Items

API
(Alberto Bellotti)
● Added support for Generic Object subcollection (Jillian T.)
● Creating generic object for a specific generic object definition:
POST /api/generic_object_definitions/:id/generic_objects
● Getting the generic objects for a generic object definition:
GET /api/generic_object_definitions/:id/generic_objects
GET /api/generic_object_definitions/:id/generic_objects/:id
GET /api/generic_object_definitions/:id/generic_objects/:id?associations=vms,services
● Getting the generic objects associated with a service:
GET /api/services/:id/generic_objects

API
● Added security group subcollection (Tzu-Mainn C.)
● For Cloud Tenants:
GET /api/cloud_tenants/:id/security_groups
GET /api/cloud_tenants/:id/security_groups/:id
● For Providers:
GET /api/providers/:id/security_groups
GET /api/providers/:id/security_groups/:id
● For VMs:
GET /api/vms/:id/security_groups
GET /api/vms/:id/security_groups/:id

API
● Added support for Custom Attributes on Services (Bo Y.)
● POST /api/services/:id/custom_attributes
● Adding custom attributes
{
“action” : “add”,
“resources” : [ { “name” : “attr1”, “value” : “value1” }, { “name” : “attr2”, “value” : “value2” } ]
}
● Editing custom attributes
{
“action” : “edit”,
“resources” : [ { “name” : “attr1”, “value” : “updated value1” } ]
}
● Deleting custom attributes
{
“action” : “delete”,
“resources” : [ { “name” : “attr1” }, { “name” : “attr2” } ]
}
● GET /api/services/:id/custom_attributes
● DELETE /api/services/:id/custom_attributes/:id

API
● Added support for Generic Object methods (Jillian T.)
Methods defined in Automate for Generic Object definitions are exposed as custom actions on the
related generic objects.
GET /api/generic_objects/5
{
“href” : “http://localhost:3000/api/generic_objects/5”,
“name” : “Test Generic Object”,
“property_attributes” : { “state” : “enabled”, “num_connections” : 3 },
“actions” : [
{ “name” : “edit”, “method” : “post”, … },
{ “name” : “test_method1”, “method” : “post”, …}
]
}
POST /api/generic_objects/5
{
“action” : “test_method1”,
“parameters” : [ … ]
}
{
“success” : true,
“message” : “Invoked method Test Method1 for Generic Object id: 5”,
“task_id” : “1001”,
“task_href” : “http://localhost:3000/api/tasks/1001”,
“href” : “http://localhost:3000/api/generic_objects/5”
}

API
● Added new add_provider_vms action for Services (Jillian T.)
When vms are created externally, i.e. through Ansible Playbooks, this enhancement
allows us to link a service to the provider resources that were generated for it.
POST /api/services/7
{
“action” : “add_provider_vms”,
“resource” : {
“provider” : { “href” : “http://localhost:3000/api/providers/:id” },
“uid_ems” : [
"420c0d63-5270-443c-61b9-8270a876db16",
"420c7853-af6f-50ce-35ba-4935cfcba4e8"
]
}
}
{
“success” : true,
“message” : “Adding provider vms for Service id: 7”,
“task_id” : “22”,
“task_href” : “http://localhost:3000/api/tasks/22”,
“href” : “http://localhost:3000/api/services/7”
}

API
● Added support for Custom Button Sets (Aparna K.)
○ New primary collection: /api/custom_button_sets
○ Full CRUD capability
POST /api/custom_button_sets
{
"name" : "CustomButtonSet",
"description" : "CustomButtonSet description",
"set_data": {
"button_order" : [ 10000000000013, 10000000000016 ],
"button_icon" : "ff ff-view-expanded",
"button_color" : "#4727ff",
"display" : true,
"applies_to_class" : "GenericObjectDefinition",
"applies_to_id " : "10000000000050"
}
}
GET /api/custom_button_sets
GET /api/custom_button_sets/:id
POST /api/custom_button_sets/:id - action “edit”
PUT /api/custom_button_sets/:id
PATCH /api/custom_button_sets/:id
POST /api/custom_button_sets/:id - action “delete”
DELETE /api/custom_button_sets/:id
POST /api/custom_button_sets - Bulk action “delete”

API
● Removed required filters for event streams (Tim W.)
● In the initial GET implementation for /api/event_streams target_type and timestamp filters were required
● With the default paging of 1000 items we apply on that collection, those filters were not needed but still
generally honored by the API.
○ GET /api/event_streams?filter[]=timestamp>2017-01-03&filter[]=timestamp<2017-01-05
○ GET /api/event_streams?filter[]=target_type=Host
● This gives the API callers more flexibility on the types of queries they could perform.

API
● Added picture support for Generic Object Definitions (Jillian T.)
Pictures depicting Generic Objects can now be added when Generic Object Definitions are created:
POST /api/generic_object_definitions
{
“name” : “LoadBalancer”,
“description” : “LoadBalancer description”,
“properties” : { … },
“picture” : {
“extension” : “png”,
“content” : “base64 content of image …”
}
}
Pictures already added can be referenced above by id or href instead.

API
● Image_href now returned with Pictures (Jillian T.)
- Previously, the href of a picture had to be asked for when querying the picture.
- With this enhancement, the image_href is returned upon getting the picture resources.
GET /api/pictures?expand=resources
GET /api/pictures
GET /api/services/:id?expand=generic_objects&attributes=generic_objects.picture
GET /api/pictures/5
{
"href" : "http://localhost:3000/api/pictures/5",
"id" : "5",
"resource_id" : "8",
"resource_type" : "GenericObjectDefinition",
==> "image_href" : "http://localhost:3000/pictures/5.png",
"extension" : "png"
}

● PRs merged (10)
○ Enhancements 6
○ Bugs 4
● Highlights
○ #523 Cloud volume restore and delete operations.
○ #528 Managing Security Groups for cloud instances.
○ #531 Updated markup around procedure to upload appliance to RHV .
Documentation
(C. Budzilowicz)

Discussion
Sprint 72 Review - November 01
Meeting Start: 10:30am EST

Sprint 71

More Related Content

What's hot (20)

Similar to Sprint 71 (16)

More from ManageIQ (20)

Recently uploaded (20)

Sprint 71