![name: Java CI with Gradle
on:
push:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK 11
uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'adopt'
- name: Build with Gradle
run: ./gradlew
GITHUB ACTIONS YML FILE](https://image.slidesharecdn.com/devopsbliss-240209150804-30258a95/85/Simplified-DevOps-Bliss-with-OpenAI-API-8-320.jpg)
![<springProperty scope="context" name=“APP_NAME"
source="vcap.application.name" defaultValue="local_app"/>
<springProperty scope="context" name="APP_SPACE"
source="vcap.application.space_name" defaultValue="${HOSTNAME}"/>
<appender name="SLACK" class="com.github.maricn.logback.SlackAppender">
<webhookUri>${SLACK_INCOMING_WEB_HOOK}</webhookUri>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%-4relative [%thread] %-5level %class - %msg%n</pattern>
</layout>
<username>${APP_NAME}@${APP_SPACE}</username>
<iconEmoji>:rage:</iconEmoji>
</appender>
<appender name="ASYNC_SLACK" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="SLACK" />
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>ERROR</level>
</filter>
</appender>
LOGBACK.XML EXAMPLE
(ERROR-LEVEL LOG MESSAGES DIRECTLY TO SLACK)](https://image.slidesharecdn.com/devopsbliss-240209150804-30258a95/85/Simplified-DevOps-Bliss-with-OpenAI-API-20-320.jpg)
Simplified DevOps Bliss -with OpenAI API
- 1. Victor Szoltysek - Feb 8th / 2024 AWS, OpenAI API, and GitHub Actions Unleashed! Simplified DevOps Bliss
- 2. DevOps was meant to streamline our workflow, yet here we are, navigating a labyrinth of complexity. Tools and processes, meant to liberate, have become chains. Devops is a mess It's time to break free and find the simplicity DevOps was supposed to offer.
- 3. Embrace a new paradigm for operational excellence • Speed: Prioritize actions that yield quick wins and fast feedback loops • Simplicity: Choose the simplest effective solution to avoid unnecessary complexity • Self-Serve: Empower teams to manage their own processes and tools Redefining DevOps: Speed, Simplicity, Self-Serve
- 4. #1 - Version your artifacts • No More “is Feature X in this deploy?” , or “What version is deployed?” • Major / Minor (hot fix) / Build Number (i.e GITHUB_BUILD_NUMBER) • Include in file name, REST endpoint, directly in Index.html • Include - version, GitHash tag, and branch What the Hell is in This Binary?
- 6. plugins { id 'org.springframework.boot' version '2.3.1.RELEASE' id 'io.spring.dependency-management' version '1.0.9.RELEASE' id "com.gorylenko.gradle-git-properties" version "2.2.2" id 'java' } version = "1.0.${System.env.BUILD_NUMBER ?: System.env.GITHUB_RUN_NUMBER ?: '0- SNAPSHOT'}" springBoot { buildInfo() } GRADLE CODE
- 7. #2 - Use GitHub Actions (or similar) • CI script becomes part of application SCM • CI shouldn’t be this hard - you don’t need a DevOps Team • No more Waiting on Build agents • No agent management required (including need to. keep build version tools up-to-date) Stop Using Jenkins: Embrace Modern CI/CD
- 8. name: Java CI with Gradle on: push: branches: [ master ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up JDK 11 uses: actions/setup-java@v3 with: java-version: '11' distribution: 'adopt' - name: Build with Gradle run: ./gradlew GITHUB ACTIONS YML FILE
- 9. #3 - Keep CI Logic in Application Repos Instead • Eases testing and maintenance of CI scripts • Ensure CI and local environment parity • Simplifies the process of updating and debugging CI/CD pipelines You Don't Need Complicated Pipelines!
- 10. #4 - Build Once, Deploy anywhere • Don’t rebuild the app on deploy (i.e. deploy the source code) — note the LeftPad NPM Package Issue • Use a single artifact for all environments with dependencies • Use Maven or Gradle Wrappers (no more build script version issues) • Build the artifact once - Every build is a release candidate Every Build a Release Candidate
- 11. #5 - Keep builds green with proactive notifications • Implement automated blame and notification systems (mS teams messages) • Foster a culture of immediate response to broken builds • Allow rollback if build isn’t getting fixed (~ 15 minutes) Who Broke the Build?
- 12. #6 - Use modern communication tools for alerts • Use Incoming webhooks for alerts with Slack and MS Teams (super easy to setup) • Connect JIRA, GitHub, Service Now, and PagerDuty for issue tracking • Use SMS (via AWS SNS) for urgent issue notifications Emails Are the Worst Notifications
- 13. SUPER EASY WEBHOOK CREATION
- 14. curl -s -d "payload={"text":"Test Message"}" INSERT_YOUR_WEB_HOOK_URL_HERE WEBHOOK CALL EXAMPLE
- 16. #7 - Automate your Deploy in Your Build Script • Keep deployment logic in version control • Embed deployment steps into build scripts ( Use plugins, don’t reinvent the wheel) • Automate deployment On Commits (No more tedious deployment plans) • Auto Notify on Successful Deploys - No more “Is it deployed yet?” • Integrate Automatic Database Scheme versioning (i.e. liquid base) • Correlate Stories to Deploys - no more “Is feature X in Prod Yet?” Where’s the Deploy Button?
- 17. #8 - Use PaaS instead • Skip Docker, Kubernetes and Terraform unless absolutely necessary • Deploy into Amazon Beanstalk (or similar PaaS) for simplicity • Rely on platform services for operational tasks • Minimize the “undifferentiated Heavy Lifting” and “Resume Driven Development” • Infrastructure should be fully self-healing (App crashes , JVM Failures, etC) — add health check endpoints and config if needed Containers are overkill majority of time
- 18. plugins { id "fi.evident.beanstalk" version "0.3.2" } beanstalk { profile = ‘my-profile' s3Endpoint = "s3-eu-west-1.amazonaws.com" beanstalkEndpoint = "elasticbeanstalk.eu-west-1.amazonaws.com" deployments { dev { file = app.jar application = 'my-app' environment = 'my-app-staging' } } } GRADLE BEANSTALK DEPLOY PLUGIN EXAMPLE Deploy by running gr a dle deploy<deployment-n a me>, i.e. gr a dle deployDev
- 19. #9 - Proactively monitor and respond to application health • Alert on Error Level Exceptions and action on them immediately • Alert via webhooks for quick issue notification (can be done via Logback.xml) • Notify directly to Developers as errors Happen • Also Alert on known critical Failures (Communication failures, SQL Failures, ETC) — you can alert directly to responsible teams (including via SMS!!) • Implement a 'fix it or forget it' policy to ensure errors are either resolved or reclassified Catch Failures Before They Catch You
- 20. <springProperty scope="context" name=“APP_NAME" source="vcap.application.name" defaultValue="local_app"/> <springProperty scope="context" name="APP_SPACE" source="vcap.application.space_name" defaultValue="${HOSTNAME}"/> <appender name="SLACK" class="com.github.maricn.logback.SlackAppender"> <webhookUri>${SLACK_INCOMING_WEB_HOOK}</webhookUri> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%-4relative [%thread] %-5level %class - %msg%n</pattern> </layout> <username>${APP_NAME}@${APP_SPACE}</username> <iconEmoji>:rage:</iconEmoji> </appender> <appender name="ASYNC_SLACK" class="ch.qos.logback.classic.AsyncAppender"> <appender-ref ref="SLACK" /> <filter class="ch.qos.logback.classic.filter.ThresholdFilter"> <level>ERROR</level> </filter> </appender> LOGBACK.XML EXAMPLE (ERROR-LEVEL LOG MESSAGES DIRECTLY TO SLACK)
- 21. • Implement Unknown exception Handling on the browsers Side • Makes sure to Track User-Agent • Log Back issues to Backend • You’d be surprised how many users use out-of-date browsers that snap your latest SPA’s #10 - Proactively Monitor and Fix Front-End Issues Ensuring Your Web Pages Always Load
- 22. <script> window.onerror = function(message, url, lineno, colno, error) { $.post({ url: 'logging/client-error', contentType: 'text/plain', data: 'Source:' + url + ':' + lineno + ' Error:' + message + ' UserAgent:’ + navigator.userAgent }); } </script> JAVASCRIPT CODE (QUICK AND DIRTY WEB LOGGING)
- 23. #11 - Automate code quality with static analysis • Agreed on and enforce coding standards (Doesn’t matter which ones) • Allow developers to focus on functionality, not formatting • Eliminates petty disputes over coding conventions Stop Nitpicking About Coding Styles
- 24. #12 - Adopt trunk-based development and WIP commits • Encourage Daily commits (to mainline) to minimize conflicts • Use feature toggles to manage incomplete features in production (implicit initially) • Reduces the fingers-crossed fear of pressing the ‘use-mine’ merge option No More Merge Conflicts and Regressions
- 25. #13 - Allow direct commits for efficiency and agility • Trust developers to commit directly to trunk • Foster a culture of responsibility and peer review through pairing or targeted PRs • Speeds up minor updates, like fixing a typo in the README.md Stop Waiting for PRs
- 26. #14 - Implement blue-green deployments for zero downtime • Ensure seamless user experience during updates • Reduce risk by having a readily available (and fast) rollback option • no more outage windows - deploys can be done during business hours • Note the need for Stateless Applications (which also make scaling easier) No More Outage Windows
- 27. Live Traffic Old Deploy Live Users
- 28. Live Traffic Old Deploy Live Users New Deploy New version Pushed and tested
- 29. Live Traffic Old Deploy Live Users New Deploy
- 30. #15 - Optimize Logging for Actionable Insights • Stop logging to local files – Use centralized logging • Use structured (key-value pair) logging instead of unstructured • Add trace-ids for logging between processes (spring Cloud Sleuth) • Focus on logging actionable information (not exiting / entering methods) • Aim to log strategically rather than voluminously (Focus on exceptions) • Consider Metrics instead of Logging (especially for high Volume - metrics scale linearly) I Have No Idea Where to Look for Errors
- 31. #16 - Leverage AI for faster problem resolution - or Just for having fun! • Integrate OpenAI with webhooks and AWS Lambda for rapid problem analysis • Automate preliminary PR reviews with AI to catch issues before human review • Send error messages to AI for first opinions — like a virtual rubber duck for debugging • Leverages AI to provide quick solutions or code suggestions, streamlining the troubleshooting process Use AI to Speed Up Troubleshooting
- 32. curl -X POST https://api.openai.com/v1/engines/gpt-4/ completions -H "Content-Type: application/json" -H "Authorization: Bearer YOUR_API_KEY" -d '{"prompt": "Tell me an Agile software development joke.", "max_tokens": 50}' OPENAI API CALL EXAMPLE
- 33. GitHub MS Teams OpenAI API Webhook Event - PR - Build Failure - Deploy - Etc .. Webhook Notification - enriched data AI API Call AI API response AWS Lambda
- 34. ANALYZE THE FOLLOWING CODE DIFF SUBMITTED AS A PR REQUEST FOR A JAVA PROJECT AND PROVIDE FEEDBACK. 1) DETERMINE IF THE PR IS WARRANTED BASED ON THE CHANGES ( FOR SMALL AND LOW RISK CHANGES LIKE README UPDATES, TEXT CHANGES, COLOUR CHANGES, ETC .. I DON’T CARE ABOUT PRS). 2) IDENTIFY ANY ERRORS 3) OFFER RECOMMENDATIONS FOR IMPROVEMENT. <APPENDED PR CODE DIFF> OpenAI PR Aid - Prompt
- 35. GIVEN THE FOLLOWING USER STORIES, GENERATE A SUMMARY FOR RELEASE NOTES, IDENTIFY THE SYSTEMS AFFECTED BY THESE CHANGES, AND OUTLINE THE KEY AREAS OF FOCUS FOR QA TESTING. 1. GENERATE CONCISE RELEASE NOTES SUMMARIZING THE NEW FEATURES, IMPROVEMENTS, AND BUG FIXES INCLUDED IN THIS RELEASE BASED ON THE PROVIDED USER STORIES. 2. IDENTIFY WHICH SYSTEMS OR COMPONENTS ARE AFFECTED BY THE CHANGES IN THESE USER STORIES. 3. OUTLINE THE KEY AREAS OF FOCUS FOR QA TESTING, INCLUDING SPECIFIC FUNCTIONALITIES THAT NEED THOROUGH TESTING, POTENTIAL REGRESSION AREAS, AND NEW FEATURES THAT REQUIRE VALIDATION. <APPENDED PR CODE DIFF> Release Note Generation - Prompt
- 36. GIVEN AN SQL ERROR CAUSING APPLICATION OUTAGES, CREATE A PASSIVE-AGGRESSIVE MESSAGE DIRECTED AT THE DBA TEAM, REQUESTING THEIR ATTENTION TO RESOLVE THE ISSUE. THE ACTUAL SQL ERROR WILL BE APPENDED TO THIS MESSAGE. CRAFT THE MESSAGE TO IMPLY THE NECESSITY OF DBA INTERVENTION, ASSUMING THE ISSUE IS IDENTIFIED AS STEMMING FROM THE DATABASE SIDE RATHER THAN DEVELOPMENT. HOWEVER, PROCEED WITH CREATING THE MESSAGE ONLY IF IT'S CLEAR THAT THE ERROR ORIGINATES FROM A DATABASE ADMINISTRATION OVERSIGHT OR MISTAKE. Passive aggressive Fix you stuff Message - Prompt