Smart city IT operations- manage solutions complexity
- 1. Operating IOT and Smart cities - a real world use case IOT changes the game: The internet of things changes the prevailing model where humans interact with machines via a web browser. As devices start peering with devices, and processes get interconnected, devices can interoperate both locally and globally. Decisions are made as per predetermined rules, and responses to stimulus occur locally. The changes that have created the Internet of Things aren’t changes to the internet, but rather changes to the things connected to the internet—the devices and gateways on the edge of the network that are now able to request a service or start an action without human intervention. Today the momentous development in Social Media, Mobility, Analytics and Cloud has created the perfect storm. These are further snowballing thanks to –lower manufacturing cost of Smart devices, availability of High Bandwidth connectivity via 4G mobile networks, Proliferation of Flexible cloud computing models, emergence of Application Aware Software Defined Network architectures and Government policies and incentives to promote investment in these area Complexity of IOT –Platform view: This proliferation of interconnected multi-vendor devices creates enormous management complexity. The devices must communicate and coordinate seamlessly, and exchange data collected via disparate protocols, creating a need for Protocol/API integration layer that scales across thousands of devices. Next, the ability to Onboard (Provision) thousands of these edge devices almost instantaneously and automatically. When was the last time anyone called the technician? With everyone on-board and transacting using mobile wallets, and sharing all manner of personal information, secure multi-tenancy is a table stake. The Cloud Services Platform plays a critical role in Orchestrating these complex tasks. Complexity of IOT –Infrastructure view: Under the hood a secure, scalable and flexible infrastructure provides the foundation to run complex applications. Inside the data center, an intelligent stack of server, storage, load balancers, security firewalls, virtual machines collaborate securely with intelligent gateways and wireless LANs to provide secure peering and aggregation of IOT devices. Today these devices have intelligence that dynamically alters the profile of the infrastructure stack to match the application running on top. Quite like the edge devices, these devices need to be provisioned with minimal human intervention, and periodic OS and firmware updates need to be pushed. When things break down, or performance of the application degrades, the problem could be on any device or a combination of devices that are not configured properly or have inter working issues Context for Smart city/ IOT use case: A 1500 acre township developer decided to deploy IOT in every home, and building common area to automate lighting and cooling, to secure homes through IOT based surveillance, and to run tutoring and concierge services using high definition video conference. The developer needed a city operations center to run the township. A service delivery platform (SDP) was deployed to provide monitoring dashboard and controls, and an application layer to mediate between edge devices. An Integrated Operation center (IOC) was created to provide IT Services to manage these IOT devices sourced from multiple vendors, an application platform, and IT infrastructure for an entire city block, providing a consistent customer experience. Managing Complexity- IOT Managed Services: So what are these challenges? these include technical issue management and resolution, troubleshooting, tracking, managing
- 2. configurations, changes, software upgrades, knowledge management, hardware procurement / replacement, and reporting. Processes and procedures were developed to solve these problems. A key feature of this IOC architecture was that, in addition to providing on-site resources to manage IT operations and a help desk, a significant chunk of proactive and reactive support was provide remotely through a global shared services model that leveraged expertise without locational constraints, and access to entire network and application stack remotely. What was Built- the IOC architecture: The Integrated Service desk (ISD): is the entry point for all customer service calls. It presents an integrated service view for facilities and IT service requests coming from residents such as “video surveillance is down” or “send me a plumber”. The ISD team uses the management portal and trouble ticketing system provided in the service delivery platform (SDP) to log and manage service requests and escalations. Integrated into this service desk is also Level 1 reactive IT support; this team leverages the same management portal to manage IT incidents. They observe the incidents tickets proactively to provide knowledgeable response to service requests. The team is dedicated to the local facility, and their skill set is broad but not deep. Solutions specialists &maintenance team: This is a core technical level solution support team that provides Fulfillment (provisioning devices & managing change) and Assurance (end to end reactive and proactive technical support) for the entire solution. Through well managed processes, leveraging skilled resources available 24X7 (across global delivery centers) and tooling (on the service delivery platform) this activity is performed remotely with the support of a handful of on-site technical resources to provide hands and feet local support. Their responsibilities include: Intermediate incident and problem management Troubleshooting & Parts replacement using Remote Technical Support Proactively manage solutions leveraging monitoring tools embedded in the SDP Recover and replace faulty devices by peering with onsite field support provision new services or make changes to existing services Maintain a known error database to continually improve service response Day 2 Operations for IT & network Infrastructure services Technology to enable Domain services -LAN, WAN, Data centers & all applications Day 2 Operations for S+CC Domain Services Facility Management, Physical Safety & Security management, Video Concierge Integrated Service desks to provide all Customer interface Call center and Technical service desk
- 3. Command center: The IOC command center was designed to analyze incidents and perform cross-domain correlation, improving P1 incident response and resource coordination, reporting, managing critical escalations between service desk, on-site resources, remote TAC and development teams. They handle all special requests and provide the “One Stop Shop” for critical problems until resolution. With the availability of superior analytics capability, machine learning and automation, flat management style, the IOC Command center in hind-sight seems a redundant appendage. The solution support model? how was it developed into a profitable service: How did we assemble and sustainably build a group of support specialists with knowledge and expertise to support 5-10-20 solutions involving data center components, unified communication architectures, video surveillance equipment, HD video conference, and building HVAC and lighting controls? How did we bring multiple experts under one roof, and pay for their expertise while ensuring profitable service delivery? How to work with multiple vendors and their support and development organizations? How to anticipate interoperability and configuration issues that contribute more than 70% of incidents? How to give customers the confidence of single window support, and price them attractively? Solution Support is an innovative service built on a remote, shared support model developed in response to the above challenges. A team of experienced engineers were trained on multi- vendor technologies that constitute various solutions. This global, virtual team was housed inside existing global technical delivery centers, leveraging and contributing to existing pools of TAC expertise. Global processes were developed and linked through information exchanges (e- ticketing) interconnecting vendor TACs to manage support cases. Our proactive response and accountability for supporting our solutions was welcomed by the customer(s) who paid premium support price. These premium multi-vendor support services were offered as sku’d offers- easier to sell and reorder. The shared support model built service profitability. Proactive and timely resolution meant lower demands on backbone TAC. Multi-vendor support expertise was built over time, and several TAC engineers got new career paths, improving overall employee engagement. How was this SOW based offer priced: These solution sku’d support services were combined with existing support contracts, and dedicated on-site resources through a be-spoke statement of work driven (SOW) offer. Having the expertise to defend and support the entire solution meant we could offer SLAs. In response to customer’s demand, we prepared a blended delivery team comprising customer’s on-site support engineers o worked in concert with the remote solution support team. Through remote operations, we were able to retain critical solution IP – ensuring longer term service renewals; a Build operate and transfer (BOT) plan ensured gradual transition to customer led on-site support. How was the service built: An internal service development process was leveraged to review and validate offer design and development. Clear milestones ensured timely delivery against pre-defined goals
- 4. How was the project managed, including presales and deal support: The project was defined and managed using a PMO approach. The PMO was tasked with assessment of IOC architecture, schedule, budget and risks. The PMO drove the development and assessment of the statement of work (SOW), definition of measurement systems and operate SLAs, and a cross organization deployment plan. An IOC Managed Services business development team dovetailed into front-line sales to capture customer support requirements, and provide customer education and deal support. On the backend it peered with the professional services sales, solution and service delivery platform (SDP) development teams, Technical services product development team and solution TAC delivery teams, ensuring we built an inventory of re-usable service assets.