TDC2017 | Florianopolis - Trilha DevOps How we figured out we had a SRE team at .NET

Editor's Notes

• #2: Welcome: Hello guys! How are you doing? Welcome to this presentation. It’s a great pleasure to be here today in order to learn togtheter a little bit more about Azure and microservices Present myself: I’m Fabricio, Technical Evangelist in Brazil Talking point 1: Talk a little bit about the day-by-day in touch with customers Talking point 2: Talk about conversations with customers about the possibility to break their applications in microservices in order to get a more lightwight structure Talking point 3: Talk about the decision to create Arda to test microservices concept and Azure in this aspect
• #3: Topic 1: When we think about an modern application... What architectural and pratical aspects we need to considering to create them? Topic 2: Analysing a monolithic architecture, what insights can we extract in order to compare it with modern applications requirements? Topic 3: Analysing a microservices architecture, what insights can we extract in order to compare it with modern applications requirements? Topic 4: Here we will take a look in more technical aspects about the application: architecture, technologies, implementation, security, etc. Topic 5: Here we will take a look in Visual Studio Team Services (VSTS) in order to understand how we manage this application in a DevOps context Topic 6: We will extract some insights and recomendations to our customers about this process Topic 7: Q&A
• #4: Requirement 1: Yes, modern application are highly concurrent. Is super common any web application, for example, have thousands connect people at the same time through diferent devices. Requirement 2: To cover all the different scenarios, applications tend to become complex. Some important aspects that collaborate to this: 1: Writting and consumig data from different data sources 2: Spread informations along diferent devices and environments (mobile, desktop, Windows, Android, iOS) 3: Tallking with different APIs models 4: Providing best UI experience for final users Requirement 3: In general modern applications “talk” with another applications (3rd-party) in order to get informations unknown of application. This is usual. Requirement 4: Mobile communication is mandatory. Modern applications must provide information to mobile apps. This is a huge requirement for modern applications. Requirement 5: Companies have to be free to choose the development language and environment to delivery their application. Why not have in the same application C#, Python and Node.js? Why not deliver part of this application in Linux and another part in Windows? Requirement 6: Technology market is very fast and the delivery of features in modern application must be as well. How can we delivery software with quality and high speed? Requirement 7: Modern application must be secure. In some cases people input very sensitive information in our applications so this way, need to guarantee that application is completely safe.
• #5: Talking point 1: For years companies has been developing their applications using monolithic approach. Somes whys: 1: Applications were restricted to company domain without need expose APIs outside 2: In general applications used a desktop approach to build their solutions 3: To communicate with another applications, another standards were used, like EDB, EDI, EDX and after, WCF Talking point 2: Monolithic architecture incentive the tightly coupling between layers of application. This implies mainly in one environment hard to test and maintain Talking point 3: In general this applications depends of environment (OS and Database) Talking point 4: Deploy this applications in general is a little bit more difficult to PaaS environments Talking point 5: To scale out this applications on the cloud we must scale entire application. This aspect can be expensive for majority scenarios
• #6: Talking point 1: Divide the application in small pieces called “services”. Each service can be developed, tested and deployed individually Talking point 2: This services exchange info between theirself using JSON (RESTFul APIs) Talking point 3: Each service has their own database repository Talking point 4: Each service is 100% isonomic each other Talking point 5: Each service run in a shared cluster environment, in one single runner instance (Web App, for example) our in a single container (Docker, for example). Each service has their own process in memory Talking point 6: Each service can be develop in a different technology each other and can run with different OS and databases at same time Talking point 7: Each service can be scaled out (or up) individually Talking point 8: This services naturally works as APIs this way, share info with another applications or mobile apps through the internet is very simple and built-in Talking point 9: To implement security is considering more simple. We need just protect APIs individually (using Active Directory, for example)
• #7: Arda is a free management tool designed to help managers have some insights about workloads and time offenders for its teams. We developed the first version specifically to Microsoft but with some small adjustments can be used for any time of companies and teams. Soon as possible we intend share the solution through Microsoft’s GitHub. As we will see, Arda is fully developed with Microsoft’s technologies stack, is 100% microservices-based and fully hosted on Azure. So... Let’s get started with Arda. Link to production environment: https://ardaapp.azurewebsites.net.
• #8: Run Arda (https://ardaapp.azurewebsites.net) using Chrome anonymous mode Access the tool by clicking in “Sign in” icon Provide requested informations on the login page Provide requested informations by “Two factor authentication process” Navigate through somes application features: Kanban, Workloads and Reports
• #9: This is the general view about Arda architecture: Layer 1: Here we have the front-end layer. This layer is responsable for support user navigation or another applications nature (for example: A mobile application, etc). This layer only execute assyncronous (using AJAX) calls in order to consume microservices informations. Layer 2: We have internal microservices environment. Currently we have 3 microservices living: Kanban, Permissions and Reports. It responsabilities are described below: Kanban: microservice responsable for manage all activities regarding to kanban: workloads, appointments, etc. Permissions: microservice responsable for control authentication, users it permissions to resources Reports: microservice responsable for consolidate informations provided by another services and send it to ArdaApp microservice ArdaApp (Main): microservice that run front-end application and show the informations to final users All microservices exchange informations using JSON format through REST APIs Permissions and Kanban has it own SQL databases Every microservice implements a replication mechanism to aspects that must be replicated Authentication will be implemented in the future to companies that would like to use Arda but don’t have Azure AD
• #10: This is the current architecture in which system run today: Using browser the user access Arda. Every user final request is happen in “ardaapp” Web App When the sign in process is requested, “ardaapp” goes to Azure AD and request the authorization for this user. If ok (user exists) Azure AD return the requested informations, Arda store this informations in a cookie and create an entry in Redis Cache Each microservice can store data in it own Azure SQL Database in a 100% isonomic process Each microservice can consume and share informations with any other All this structure is managed for the the same Resource Group (Arda)
• #11: Open Azure Portal Select “Arda” dashboard WEB APP Navigate to “ardaapp” Talk about the monitoring panel and alerts configuration Talk about Kudu in order to get a more advanced management to the environment Show the IIS configuration featuring “Always On” and “Affinity by Cookies”. Always On is important for us because the services must be immediately accessable in any time Show the backup configuration Talking a little bit about the possibilites: Web Jobs Automate deploy Create load test Automatically enable Application Insights SQL AZURE DATABASES Navigate to “Arda_Permissions Talk about the database Properties Talk about the Georeplication Talk about the TDE (Transfer Encription Data) Talk about the performance and querie monitoring AZURE REDIS CACHE Navigate to “Arda” Talk about monitoring panel and alerts configuration Talk about possibilities (network, persistence, etc) using Premium Tier AZURE STORAGE Navigate to “Arda” Talk about encryption possibility Talk about Georedundance Talk about Premium tier (SSD) possibility SEND GRID Talk about “Configuration”
• #12: This is another option to deliver Arda microservices on Azure: using Docker containers We have almost the same environment. The difference here is: we have a Virtual Machine running Ubuntu that acts how Docker Host. The Docker containers that run Arda microservices are hosted in this VM. The request continues get in “ardaapp”. This microservice “ardaapp” direct their calls to public ports in Docker Host VM. Docker Engine convert this public ports to internal ports (NAT) and then the microservice can answer We are trying this approach, making the first tests. Next tests will be: ACS and Service Fabric
• #13: Navigate to “ardadockerhost” Acquire public IP from VM Open Putty and sign in with public ip + username + password Type “docker ps –a” to see all containers running Type “docker logs {container name} to see activity in this container
• #14: We choose .NET Core and ASP.NET Core with aim of testing the maturity of the platform. We started working with RC1 and delivered first release of Arda with 1.0 During the development process we found some troubles related to missing resources. For example: we couldn’t send e-mail messages in a built-in way. We need to develop our own email sender. In the 1.0 release, this and another gaps was solved In Web Apps we are using native IIS integration. In containers we are using selfhost model
• #15: Open Visual Studio Open Arda solution Talk quickly about the new project structure Project.json Startup.cs Repository pattern MVC to web api Manage secrets
• #17: The Azure AD protection happens in two different moments: 1. When user request “sign in” Arda application send a request for one “authorization code” to Azure AD. Azure AD verify if caller user exists. If “yes” the authorization code is returned to Arda. Arda store this authentication code in one cookie and creates a “session” with this information on Azure Redis Cache. 2. When an API needs to consume data from another API Arda APIs are protected by Azure AD. This means that when a microservice need consume some information provided by other, the caller service must validate with AD if the caller have permissions to execute that access and if access is happening in a write way (the audience information is correct). What happens follow: Caller service read cookie in order to get “authentication code” Caller service request (doing a POST to AD Token Endpoint) for an “access token”. AD Token Endpoint consult related AD application to know what can be accessed by another service Caller service get the returned “access token” Caller service call target service sending “access token” on header. Target service validate information provided by “access token” and, if is everything ok, returns content requested. If not, return a 401 error (unauthorized) This flow guarantee security on APIs access.
• #18: Open Azure Portal (old): https://manage.windowsazure.com Navigate to Active Directory option Navigate to “Microsoft” Navigate to “Applications” tab Search for “Arda” Navigate inside “Arda” application
• #19: Topic 1: When we think about an modern application... What architectural and pratical aspects we need to considering to create them? Topic 2: Analysing a monolithic architecture, what insights can we extract in order to compare it with modern applications requirements? Topic 3: Analysing a microservices architecture, what insights can we extract in order to compare it with modern applications requirements? Topic 4: Here we will take a look in more technical aspects about the application: architecture, technologies, implementation, security, etc. Topic 5: Here we will take a look in Visual Studio Team Services (VSTS) in order to understand how we manage this application in a DevOps context Topic 6: We will extract some insights and recomendations to our customers about this process Topic 7: Q&A