Voxxed Days Thesaloniki 2016 - A journey to Open Source Technologies on Azure
0 likes316 views
1) Microsoft Azure provides a platform for building and deploying Java applications on virtual machines, containers, and platform as a service (PaaS) offerings. 2) Azure supports the full Java ecosystem including frameworks, tools, and databases and has strong partnerships with the Eclipse Foundation and Linux Foundation. 3) Many large Java projects like Jenkins use Azure to host their infrastructure due to Azure's support for open source technologies and large Java developer community.
Voxxed Days Thesaloniki 2016 - A journey to Open Source Technologies on Azure
- 1. Java on Azure Developer's Guide Alex Yochev DX Technical Evangelist @_xayo_
- 2. Traditional Microsoft … Databases Applications Operations Application Frameworks Infrastructure SQL Server .NET SharePoint Dynamics System Center Exchange MS Traditional Monetization
- 3. Microsoft + Open Source Azure Marketplace 60% of all images in Azure Marketplace are based on Linux/OSS 1 out of 3 1 out of 3 VMs on Azure run Linux, and more than half of all new VMs run Linux Marketplace Microsoft joins Eclipse Foundation Partnership with the Linux Foundation for Linux on Azure certification Jenkins project on Azure Partnership Partnership SQL Server on Linux HD Insight PaaS on Linux Acquisition Product Run Linux on Windows natively C:Usersmarkhill> bash root@localhost: # Windows Subsystemfor Linux Operations Management Suite = Windows + Linux 500 Million+ Lines of open source code submitted to GitHub by Microsoft engineersMicrosoftOpenSourceHub Wim Coekaerts Oracle’s Mr. Linux joins Microsoft Ross Gardler President Apache SW Foundation Ingesting Open Source DNA @MSFT Michael Kelley Open Source Technology Center
- 4. A New Era for Microsoft • Microsoft Cloud Platform can now address ALL of our customers’ cloud needs, including enterprise grade support for their open source solutions as well as Linux management Databases Applications Operations Application Frameworks Infrastructure SQL Server .NET SharePoint Dynamics System Center Exchange Microsoft Azure
- 5. 5 Microsoft love Linux and OSS
- 6. Build on the platformBuild on the infrastructure Preview GA Stakeholders Developers Microsoft Azure Virtual Machines Stateful and stateless one- off solutions VM Scale Sets Scalable solutions Container Service Scalable, orchestrated Docker images deployed into containers Batch Custom High Performance Computing solutions Open Source PaaS Cloud Foundry, OpenShift Service Fabric Custom Microservice- based stateless and stateful solutions App Service Template based rapid development web, mobile and API solutions Orchestrated workflow based integration solutions LogicWeb, Mobile & API All the goodness of App Service in a dedicated environment Environments … Functions Operations DBAs and data scientists Business Office PowerBI Cloud-born LOBLift-and-shift + net new on Linux Docker Azure Open Source App Platform
- 7. Developer & Data Agility with Containers in the Cloud Microsoft Azure Azure Open Source Container Portfolio Developers Cross-cloud orchestration Tools integration Operations OMS Data scientists Workload portability Open source container-based PaaS platforms in Azure Container-ready application platforms that benefit from Azure’s native partitioning, capacity management and high availability Azure Container Service Optimized container hosting in the cloud with familiar tooling and your choice of orchestrator Docker VM Extension for Azure Easy and programmatic way to add Docker capabilities to your VMs Azure Marketplace container partners Partner solutions that address management challenges of containers
- 8. Java deployment in Azure Ultimate Control Cost & Agility Virtual Machines VM Extensions + Containers VM ScaleSets Cloud Foundry, Container Services Service Fabric IaaS PaaS Bring your Java Automate everything Elastic Declarative Stateless & Stateful
- 9. Java ?
- 10. DevOps Infrastructure Java apps & tools Databases & middleware Azure Use your OS, frameworks, and data sources
- 11. Java on Azure Customers
- 12. Java Dev Center – azure.com/java
- 14. azure.github.com
- 15. Javadoc views on Azure are 2nd (after .NET) Azure is the only Oracle-certified cloud platform HDInsight (Hadoop), Azure Search (Elastic Search) Your OS, frameworks, and data sources MICROSOFT CONFIDENTIAL Popular on Azure1 in 3 Linux VMsWe Use Java
- 16. SDKs for Java Storage Service Bus Management Key Vault Resource Manager Data Lake SQL Database Active Directory ApplicationInsights DocumentDB Web Apps Media Services Scheduler Notification Hubs Full developer ecosystems in Azure for your deployment needs and beyond SQL Azure
- 19. Azure Toolkit for Eclipse/for IntelliJ Deploy to Azure Web Apps or Azure PaaS
- 20. • Performance enhancements so far • Patches have been accepted and back ported to Java 7 & 8 • Increase throughput and reduce latency for local tcp connections: • 8060170: Support SIO_LOOPBACK_FAST_PATH option on Windows • http://hg.openjdk.java.net/jdk9/dev/jdk/rev/26e6402772c8 • Increase throughput for transfering files over tcp: 8064407: • FileChannel transferTo should use TransmitFile on Windows • http://hg.openjdk.java.net/jdk9/dev/jdk/rev/ba6d8f56003b Microsoft is CONTRIBUTING to Java!
- 21. Jenkins Running their infrastructure on Azure! Situation Infrastructure in four different locations, with four different infrastructure providers, each with their own APIs and tools for managing resources, each with varying capabilities and capacities. Project infrastructure is managed by a team of volunteers, operating more than 15 different services and managing a number of additional external services. Solution A cloud platform consisting of: compute, CDN, storage and data- store services. Azure also enables End-to-end TLS encrypted distribution of Jenkins packages, plugins and metadata via the Azure CDN. Benefits More complete build/test/release support and capacity on ci.jenkins.io for plugin developers using Azure Container Service and generic VMs. Jenkins provides continuous integration services for software development. It supports popular SCM tools and can execute Apache Ant and Apache Maven based projects as well as shell scripts and Windows batch commands. “Migrating to Azure simplifies and improves our infrastructure in a dramatic way that would not be possible without a comprehensive platform”
- 22. Jenkins Plugins Azure Storage plugin for Jenkins • Easily deploy artifacts to Azure Storage Jenkins Slave Plugin for Azure • Azure Cloud Profile Configuration • Azure subscription info • Deploy and manage Azure Slaves • Azure Slave Template Configuration • Azure Slave Templates tell Azure what type of Azure Slave to deploy • Windows or Linux
- 23. Microsoft joins the Eclipse Foundation Microsoft Eclipse
- 24. What Microsoft has to Offer Broad Linux Support We know that most Java shops run Linux. Azure supports all popular Linux and UNIX distributions SDK • Rapidly develop Azure-enabled applications using Azure Java SDK and access all Azure services ranging from Event Hub to App Insights • Use Azure Android SDK for mobile applications that needs to access storage Azure Jenkins support • Jenkins CI supports Azure slave-plugin and Azure storage- plugin. • Jenkins partnership with Microsoft will result in them (Jenkins) running their infrastructure on Azure. • Jenkins is the most popular CI tool amongst Java developers Azure support for Java Enterprise Platforms Intrinsic Java support in Azure PaaS offerings and services • Azure Web Apps supports Tomcat • Azure Web Apps and Service Fabric support Java constructs intrinsically Development • Azure Toolkits for Eclipse and IntelliJ, including Team Foundation Services (TFS) Plugin for Eclipse enable developers to seamlessly write, test and deploy code on Azure from their favorite IDEs • Team Explorer Everywhere Plugin for Eclipse enables developers using Eclipse to collaborate using Visual Studio Team Services. They are not required to choose Visual Studio as the IDE anymore • Automated builds and tests using open source tools such as Jenkins, Ant, Maven, Gradle, etc. • Huge repository of Java code samples on GitHub, Java Development Center • Azure support for Java-enabled platforms like Cloud Foundry, OpenShift, JBoss, WebLogic, etc.
- 26. IoT is made up of four core elements Connectivity Data AnalyticsThings
- 27. IoT Device & Cloud Patterns Field Gateway Device Connectivity & Management Devices RTOS,Linux,Windows,Android,iOS Field Gateway Protocol Adaptation Protocol Adaptation Analytics & Operationalized Insights Batch Analytics & Visualizations Azure HDInsight, AzureML, Power BI, Azure Data Lake, Azure Data Factory Hot Path Analytics Azure Stream Analytics, Azure HDInsight Storm Hot Path Business Logic Service Fabric & Actor Framework Presentation & Business Connectivity Presentation & Business Connectivity App Service, Websites Dynamics, BizTalk Services, Notification Hubs IoT Hub Cloud Gateway
- 28. Microsoft Azure IoT Services Devices Device Connectivity Storage Analytics Presentation & Action Event Hub SQL Database Machine Learning App Service IoT Hub Table/Blob Storage Stream Analytics Power BI Service Bus DocumentDB HDInsight Notification Hubs External Data Sources 3rd party Databases Data Factory Mobile Services Data Lake* BizTalk Services { } (*) service in Preview
- 29. Microsoft Cloud offer for IoT Remote monitoring Predictive maintenance Azure IoT SuiteAzure services (IoT and others)
- 30. Accelerate time to value with preconfigured solutions Finish with your Internet of Things applicationStart quickly with preconfigured solutions Modify existing rules and alerts Fine-tuned to specific assets and processes Integrate with back-end systems Highly visual for your real-time operational data Get started in minutes Add your devices and begin tailor to your needs
- 31. Security & Management Platform Services Infrastructure Services Virtual Machines Windows Linux Containers Network Attached Storage BLOB Storage Azure Files Premium Storage Datacenter Infrastructure (24 Regions, 19 Online) Web and Mobile Web Apps Mobile Apps API Management API Apps Logic Apps Notification Hubs Media & CDN Content Delivery Network (CDN) Media Services Analytics & IoT HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile Engagement Active Directory Multi-Factor Authentication Automation Portal Key Vault Integration Biztalk Services Hybrid Connections Service Bus Storage Queues Store / Marketplace Hybrid Operations Backup StorSimple Site Recovery Import/Export Software Defined Networking Virtual Network Express Route Traffic Manager Load Balancer DNS VPN Gateway Application Gateway Data SQL Database DocumentDB Redis Cache Search Tables SQL Data Warehouse Azure AD Connect Health AD Privileged Identity Mngt Operational Insights Compute Cloud Services Batch Remote App Service Fabric Developer Services Visual Studio Application Insights Azure SDK Team Project VM Image Gallery & VM Depot IoT Hubs
- 32. IoT Hub endpoints device Event processing (hot and cold path) Device provisioning and management Your IoT Hub Device id C2D queue endpoint D2C send endpoint Device … Device … Device … D2C receive endpoint C2D send endpoint Msg feedback and monitoring endpoint Device identity managementIoT Hub management Device business logic, Connectivity monitoring Field GW / Cloud GW 10 101 0 1 0 0 1 1 1 0 1 1 0 10 1 0 0 1 1 1 0 1 0 0 0 1 1 0 1 0
- 33. • Azure SDKs • Open source SDKs for Azure IoT Hub • github.com/azure/azure-iot-sdks • Service Client SDK – C#, Java, JavaScript • Device Client SDK – C, JavaScript, Java, C# (including PCL), Python • Packages • NuGet • Npm • Maven • Apt-get • Open source preconfigured solutions for IoT Suite • Tools • X-Plat CLI tool: iothub-explorer • Windows tool: DeviceExplorer Developer SDKs & Tools
- 34. Azure IoT Suite
- 35. Microsoft Azure IoT Services and Java Devices Device Connectivity Storage Analytics Presentation & Action Event Hub SQL Database Machine Learning App Service IoT Hub Table/Blob Storage Stream Analytics Power BI Service Bus DocumentDB HDInsight Notification Hubs External Data Sources 3rd party Databases Data Factory Mobile Services Data Lake* BizTalk Services { } (*) service in Preview
- 36. Microsoft Azure IoT Services and HD Insight Devices Device Connectivity Storage Analytics Presentation & Action Event Hub SQL Database Machine Learning App Service IoT Hub Table/Blob Storage Stream Analytics Power BI Service Bus DocumentDB HDInsight Notification Hubs External Data Sources 3rd party Databases Data Factory Mobile Services Data Lake* BizTalk Services { } (*) service in Preview
- 37. We do Hadoop together Engineering alignment Corporate alignment Field alignment
- 38. Who is Hortonworks? Open leadership Drive innovation in the open exclusively via the Apache community-driven open source process Enterprise rigor Engineer, test and certify apache Hadoop with the enterprise in mind Ecosystem endorsement Focus on deep integration with existing data center technologies and skills Our mission: Enable your modern data architecture by delivering one enterprise Hadoop Headquarters: Palo Alto, CA
- 40. Azure HDInsight Hadoop Meets the Cloud Microsoft’s managed Hadoop as a Service 100% open source Apache Hadoop Compatible for .Net and Java tools Built on the latest releases across Hadoop (2.6) Up and running in minutes with no hardware to deploy Run on Windows or Linux Spin up, do something, tear down – without losing any data Supported by Microsoft
- 41. HDInsight Supports Hive SQL-like queries on Hadoop data in HDInsight HDInsight provides easy-to-use graphical query interface for Hive HiveQL is a SQL-like language (subset of SQL) Hive structures include well-understood database concepts such as tables, rows, columns, partitions Compiled into MapReduce jobs that are executed on Hadoop Microsoft contribution to Apache code Hadoop 2.0 1400s 44.3s 35.1s Sample Query Hive 10 HDP 1.3 / Hive 11 HDP 2.0 32x Speedup 40X Speedup HDP 2.1 15s 100x Speedup
- 42. HDInsight Supports HBase Data NodeData Node Data NodeData Node Data NodeData Node Data NodeData Node Task TrackerTask Tracker Task TrackerTask Tracker Task TrackerTask Tracker Task TrackerTask Tracker Name NodeName Node Job TrackerJob Tracker HMaster Coordination Region Server Region Server Region Server Region Server NoSQL database on data in HDInsight Columnar, NoSQL database Runs on top of the Hadoop Distributed File System (HDFS) Provides flexibility in that new columns can be added to column families at any time
- 43. HDInsight Supports Mahout Machine learning library A library of machine learning algorithms to execute on data in HDFS Algorithms are not dependent on size of data and can scale with large datasets Library includes: Collaborative Filtering, Classification, Clustering, Dimensionality Reduction, Topic Models
- 44. HDInsight supports Storm Stream analytics for near-real-time processing Consumes millions of real-time events from a scalable event broker (Apache Kafka, Azure Event Hub) Performs time-sensitive computation Output to persistent stores, dashboards or devices Bolt Bolt Bolt Spout Spout Bolt
- 45. HDInsight Supports Spark In Memory Processing on Multiple Workloads Single execution model for multiple tasks (SQL queries, Streaming, Machine Learning, and Graph) Processing up to 100x faster performance Developer friendly (Java, Python, Scala) BI tool of choice (Power BI)
- 46. Additional Components and Utilities • Ambari: Cluster provisioning, management, and monitoring. • Avro (Microsoft .NET Library for Avro): Data serialization for the Microsoft .NET environment • Hive & HCatalog: Structured Query Language (SQL)-like querying, and a table and storage management layer • Mahout: Machine learning • MapReduce and YARN: Distributed processing and resource management • Oozie: Workflow management • Phoenix: Relational database layer over HBase • Pig: Simpler scripting for MapReduce transformations • Sqoop: Data import and export • Tez: Allows data-intensive processes to run efficiently at scale • ZooKeeper: Coordination of processes in distributed systems
- 47. … Add Hadoop Projects to HDInsight Modify HDInsight clusters with custom script Add Apache Hadoop projects to HDInsight Documented for Spark, R, Giraph, Solr HDInsight Allows You To Add Hadoop Projects
- 48. Advantages of Hadoop in the Azure • Automatic provisioning of Hadoop clusters • State-of-the-art Hadoop components • High availability and reliability of clusters • Efficient and economical data storage with Azure Blob storage, a Hadoop-compatible option • Integration with other Azure services, including Web apps and SQL Database • Low entry cost
- 49. IoT – Reference Architecture Protocol Adaptation Device Connectivity & Management Environmental sensors Air conditioning and temperature control Health monitoring Cloud Gateway Cold Path Analysis Data Factory HDInsight Spark Machine Learning Storage Document DB Storage SQL DB SQL Server Analytics & Operationalized Insights HDInsight Spark/Storm Stream Analytics Notification Hub Presentation & Business Connectivity Web App Mobile App Logic App Hot Path Analysis Field Gateway Devices (Linux, Windows, RTOS) IoT Hub Event Hub Apache Kafka Apache Kafka Apache Spark Apache Storm
- 50. Presentation and action Streaming Analytics TransformationCollection Event Queuing System Long-term storage Search and query Data analytics (Excel) Web/thick client dashboards Devices to take action Event hub Event producers Applications Web and social Devices Sensors Live Dashboards Apache HBase on HDInsight DocumentDB Solr Azure Search MongoDB Cloud gateways (web APIs) Field gateways Kafka/RabbitMQ/ ActiveMQ Event hubs Azure ML Storage adapters Stream processing Storm / Spark HDInsight
- 51. IoT – Reference Architecture Protocol Adaptation Device Connectivity & Management Environmental sensors Air conditioning and temperature control Health monitoring Cloud Gateway Cold Path Analysis Data Factory HDInsight Spark Machine Learning Storage Document DB Storage SQL DB SQL Server Analytics & Operationalized Insights HDInsight Spark/Storm Stream Analytics Notification Hub Presentation & Business Connectivity Web App Mobile App Logic App Hot Path Analysis Field Gateway Devices (Linux, Windows, RTOS) IoT Hub Event Hub Apache Kafka Apache Kafka Apache Spark Apache Storm
- 52. Apache Spark Apache Storm Big Players in Hot Path Analysis
- 55. Apache Storm Distributed, Scalable, Real-Time Computation System Fast – benchmarked as processing one million 100 byte messages per second per node Fault-tolerant – when workers die, Storm will automatically restart them. If a node dies, the worker will be restarted on another node. Reliable – Storm guarantees that each unit of data (tuple) will be processed at least once or exactly once. Tuple Streams Spouts Bolt
- 56. Provision HDI Storm cluster
- 58. HDI Storm Dashboard in Azure
- 59. Adding services to HDI Storm
- 60. Apache Storm for Microsoft Azure Self Hosted Managed Service Resource Manager Template Script Actions Enterprise Data Hub Data Platform
- 61. Real Time Streaming - Apache Storm with IoT Hub IoT Device SDK for Node.js IoT Hub Apache Storm Node Server with Socket.IO Data Driven Documents (D3.js) Ingestion and Management Analysis Presentation & VisualizationEvent Generation IoT Hub ParserBolt Dashboard Bolt
- 62. Apache Spark Initially started at UC Berkeley in 2009 10x (on disk) – 100x (in-memory) faster Provides high-level APIs in Java, Scala and Python Integration with Hadoop eco-system
- 65. Spark Dashboards
- 66. Spark Dashboards
- 67. Spark ML with Jupyter
- 68. Spark ML with Jupyter - 2
- 69. Spark ML with Jupyter - 3
- 70. Spark ML with Jupyter - 4
- 71. Spark for Microsoft Azure - Tools
- 72. Containers
- 73. The fuss about containers and why? Virtual Machines Each virtual machine includes the application, the necessary binaries and libraries and an entire guest operating system - all of which may be tens of GBs in size. Containers Containers include the application and all of its dependencies, but share the kernel with other containers. They run as an isolated process in user-space on the host operating system. They’re also not tied to any specific infrastructure –So containers can run on any computer, on any infrastructure and in any cloud.
- 74. Standard Docker tooling and API support Streamlined provisioning of DCOS and Docker Swarm Linux and Windows Server containers Azure and Azure Stack Azure Container Service SwarmDC/OS
- 76. Docker Datacenter: Integrated, end-to-end platform for agile application development and management Enables hybrid use cases Azure / On-premise
- 77. Not everything is a Docker Container
- 78. Not Everything is a Docker Container
- 79. DCOS Empowers your developers and operations teams with simplified app deployment. Enables hybrid use cases Azure / On- premise Use Docker Images or other containerized workloads
- 81. DC/OS - Mesosphere Universe of services
- 82. Marathon
- 83. VM Scale Sets
- 84. Compute requirements of a modern cloud app Virtual Network Availability Set Availability Set Load Balancing Compute nodes Control nodes • Transparent, automatic, rapid scale • Simple manageability • High availability • Security - Patching / upgrades
- 85. • Scale - Deploy identical resources • Easily • Rapidly • At scale.. 10s – 10,000s of cores • Customization – PaaS-like ease of use with custom infrastructure requirements • Availability - Roll out updates without service interruption • Low cost - Dynamically increase/decrease compute power to optimize costs • Elasticity - Automatically scale to changing demand VM Scale Sets provide https://azure.microsoft.com/en-us/documentation/services/virtual-machines-scale-sets/
- 86. VM Scale Sets in Azure Resource Manager Resource Group Subnet Scalable Storage V M Scalable NIC … V M V M V M Scale Set Extensions https://github.com/Azure/azure-quickstart-templates - *vmss* VNet
- 87. Autoscale
- 88. • Portal: get started in a few clicks • Resource Manager Template: have full control • CLI, PowerShell, Resource Explorer, SDKs, etc. • In-VM Agent • Windows Azure Diagnostics agent • Linux Azure Diagnostics agent Autoscale: How do I get it?
- 89. • Simultaneously: • Scale on a specific day • Scale on a recurring schedule • Scale on multiple resource metrics • Defined in Microsoft.Insights Autoscale rules resource Autoscale: What about complex scenarios?
- 90. • Auto-email your ops team! • Integrate with external APIs via Event Hooks • Comes with basic payload • Add your own payload Autoscale: Can Azure notify me?
- 91. Connecting to Scale Set VMs – Jump box Subnet V M VNET V M V M Scale Set Mesos master (VM) Public IP Browser Port 5050 Mesos slave Mesos slave Mesos slave https://github.com/gbowerman/azure-myriad/blob/master/mesos-vmss-simple-cluster.json Create a VNET, public IP, single VM, scaleset Scaleset nodes communicate with VM
- 92. Try Azure! Learn more about Java and OSS on Azure Check out Java samples on GitHub Contribute https://azure.microsoft.com/develop/java/ What’s next?
- 93. Java Reference • All about Java on Azure – Azure Blog • VM Depot • Azure Java Developer Center (SDK and More) • Azure Toolkit for Eclipse • MS Open Tech Tools for IntelliJ • Java Microsoft Virtual Academy Course • Azure Chat for Java
- 94. Next steps Try Hadoop on-premises Free download Personal, portable Hadoop environment Free Trial: http://www.hortonworks.com/sandbox/ Hortonworks partner: http://hortonworks.com/partners/become-a-partne r/ Try Hadoop in the Cloud Up and running in minutes Spin up without hardware Free Trial: http://azure.microsoft.com/en-us/services/hdinsigh t/ Free Trial: www.powerbi.com
- 95. © 2016 Microsoft Corporation. All rights reserved. Try Azure Container Service http://aka.ms/try-acs
- 96. © 2015 Microsoft Corporation. All rights reserved.
- 97. Layer Supported Technologies Configuration as Code ARM, Dockerfile, Docker Compose, Marathon.json Host cluster management VM Scale Sets Container orchestration Docker Swarm, Chronos, Marathon, Apache Mesos Monitoring OMS, Statsd Layered for flexibility and agility Containers VMs and VM Scale Sets ARM Template Container Services (1st party, 3rd party) Windows Server Linux Azure Stack Azure Container ToolingService Tooling Layer Supported Technologies Configuration as Code ARM, Dockerfile, Docker Compose, Marathon.json Host cluster management VM Scale Sets, Availability Sets Container orchestration Docker Swarm, Marathon, Apache Mesos Monitoring OMS, Statsd, Fluentd, cAdvisor, ELK Networking Docker Network Plugin Storage Docker Storage Plugin ???
- 98. Building for the Ecosystem Azure Container Service is the Infrastructure Production grade Enterprise scale Hybrid Empowering the Ecosystem Open source Choice: Docker Native or Apache Mesos Community defined APIs
- 99. Marathon
- 100. VM Scale Sets
- 101. Compute requirements of a modern cloud app Virtual Network Availability Set Availability Set Load Balancing Compute nodes Control nodes • Transparent, automatic, rapid scale • Simple manageability • High availability • Security - Patching / upgrades
- 102. • Scale - Deploy identical resources • Easily • Rapidly • At scale.. 10s – 10,000s of cores • Customization – PaaS-like ease of use with custom infrastructure requirements • Availability - Roll out updates without service interruption • Low cost - Dynamically increase/decrease compute power to optimize costs • Elasticity - Automatically scale to changing demand VM Scale Sets provide https://azure.microsoft.com/en-us/documentation/services/virtual-machines-scale-sets/
- 103. VM Scale Sets in Azure Resource Manager Resource Group Subnet Scalable Storage V M Scalable NIC … V M V M V M Scale Set Extensions https://github.com/Azure/azure-quickstart-templates - *vmss* VNet
- 104. Autoscale
- 105. • Portal: get started in a few clicks • Resource Manager Template: have full control • CLI, PowerShell, Resource Explorer, SDKs, etc. • In-VM Agent • Windows Azure Diagnostics agent • Linux Azure Diagnostics agent Autoscale: How do I get it?
- 106. • Simultaneously: • Scale on a specific day • Scale on a recurring schedule • Scale on multiple resource metrics • Defined in Microsoft.Insights Autoscale rules resource Autoscale: What about complex scenarios?
- 107. • Auto-email your ops team! • Integrate with external APIs via Event Hooks • Comes with basic payload • Add your own payload Autoscale: Can Azure notify me?
- 108. Connecting to Scale Set VMs – Jump box Subnet V M VNET V M V M Scale Set Mesos master (VM) Public IP Browser Port 5050 Mesos slave Mesos slave Mesos slave https://github.com/gbowerman/azure-myriad/blob/master/mesos-vmss-simple-cluster.json Create a VNET, public IP, single VM, scaleset Scaleset nodes communicate with VM
- 109. Try Azure IoT Hubs for free! Learn more about Azure IoT Suite Check out Azure certified devices Contribute http://aka.ms/azureiotdev What’s next?
- 110. Java Reference • All about Java on Azure – Azure Blog • VM Depot • Azure Java Developer Center (SDK and More) • Azure Toolkit for Eclipse • MS Open Tech Tools for IntelliJ • Java Microsoft Virtual Academy Course • Azure Chat for Java
- 111. Next steps Try Hadoop on-premises Free download Personal, portable Hadoop environment Free Trial: http://www.hortonworks.com/sandbox/ Hortonworks partner: http://hortonworks.com/partners/become-a-partne r/ Try Hadoop in the Cloud Up and running in minutes Spin up without hardware Free Trial: http://azure.microsoft.com/en-us/services/hdinsigh t/ Free Trial: www.powerbi.com
- 112. © 2016 Microsoft Corporation. All rights reserved. Try Azure Container Service http://aka.ms/try-acs
- 113. © 2015 Microsoft Corporation. All rights reserved.
- 114. Java on Azure Customers
- 115. The Market • Run scalable Java Enterprise web servers, application servers and mobile services on the cloud and in a hybrid environment – to leverage cloud economics at scale • Leverage Azure PaaS for rapid development and reduced time-to-market Java Enterprise Platforms BUSINESS NEEDS • Tap into the large and growing pool of expert Java developers • Modernize their investments in Java-based application stacks through adoption of new development patterns and practices
- 116. What Microsoft has to Offer • Traditional Data Stores • Run Linux-based Oracle, MySQL, MongoDB, PostgreSQL and other Data Management Solutions • Fully Managed Database-as-a-service • for both NoSQL and RDBMS options • Big Data Analytics • HDInsight – Fully Managed Hadoop, Spark or Storm Clusters, also supports Hive, R, Hbase • Partnership with Hortonworks (the #1 Apache Hadoop committer) allows us to influence roadmap on behalf of our customers • Data Ingestion and Storage tools • like Azure Data Lake, Microsoft Stream Analytics, Event Hub Managed Service with full Java SDK and REST API integration • Azure Storage • Java, Python, Ruby, PHP, Node, SDKs, CLI and REST support • Azure support for Open Source Data Infrastructure Azure Search Fully indexed and searchable websites Data Marketplace with Tiered Monetization Capabilities End to end Managed Analytics solutions Cortana Intelligence Suite including Azure ML High Performance Computing A8, A9, A10, A11 “compute-intensive” instances supporting InfiniBand Network Standard OTHER SQL Server on Linux Application for private preview now open, incl. for RHEL customers R support “Microsoft R Open” (open source) and “Microsoft R Server 2016” (commercial offering for enterprises)
- 117. The Market Application Lifecycle Management addresses DevTest, DevOps, monitoring and issue-tracking across the development lifecycle. Customers want to • Increase developer productivity by eliminating resource contention • Improve provisioning agility by eliminating procurement delays • Manage cost by reducing budgets for under- utilized bare metal • Run realistic load tests and benchmarking • Reduce downtime, perform rolling upgrades, achieve stability in production • Respond quickly to market needs by adopting more frequent iterations Developers say it is too complicated and time-consuming to get development and test resources DevTest for Linux-based Workloads 65% Business Case for Test Environment Management Whitepaper, Cognizant
- 118. What Microsoft has to Offer • Script Based Automation • Azure CLI (Azure Command Line Interface) • Custom Script Extension • Cloud-Init for Ubuntu • DevTest Labs • Provides ease of administration for dev and test teams by enforcing quotas and policies on Azure Resources. Has notifications and reports built in • Reduces entry barrier for moving to cloud or introducing cloud culture • Comprehensive support for Container Technologies • Docker, Docker Swarm, Docker Compose • Docker Datacenter • Mesosphere DC/OS, both natively as well as via Azure Container Service • Azure support for DevTest for Linux-based Workloads Large and growing Marketplace • With 3rd party solutions for DevTest and DevOps like Chef, Puppet, Jenkins, etc. Azure Functions (in Preview) • Event-driven, compute-on-demand experience • Completely open source. Supports custom CI/CD implementations and connections to repositories like GitHub or VSTS, allowing for a slimmer and more efficient CI/CD pipeline Team Services Release Management Pipeline • For fully automated CI/CD capabilities