Automating Cloud Operations: Everything You Wanted to Know about cURL and REST
0 likes31 views
The document presents the 2022 Michigan Oracle Users Summit focused on automating cloud operations using REST APIs and cURL, led by presenter Ahmed Aboulnaga. It covers key concepts including REST architecture, JSON, cURL commands, and creating an Oracle Autonomous Database, along with challenges faced during API interactions and solutions. The presentation aims to enhance understanding of REST APIs for automating cloud operations, targeting attendees interested in DevOps and cloud technologies.
![WHATWASTHE SUCCESSFUL OUTCOME?
Final successful command:
curl -H 'Content-Type: application/x-www-form-urlencoded' -H 'Authorization:Basic
d1VibG9naWM6d2VsY29tZTE='
'http://soadev.revtech.com:7701/oam/services/rest/ssa/api/v1/oauthpolicyadmin/oauthidentitydomai
n' -d '{"name":"AhmedWebGateDomain", "identityProvider":"AhmedOUDStore", "description":"Ahmed
OIDC WebGate Domain",
"tokenSettings":[{"tokenType":"ACCESS_TOKEN","tokenExpiry":3600,"lifeCycleEnabled":false,"refres
hTokenEnabled":false,"refreshTokenExpiry":86400,"refreshTokenLifeCycleEnabled":false},
{"tokenType":"AUTHZ_CODE","tokenExpiry":3600,"lifeCycleEnabled":false,"refreshTokenEnabled":fals
e,"refreshTokenExpiry":86400,"refreshTokenLifeCycleEnabled":false},
{"tokenType":"SSO_LINK_TOKEN","tokenExpiry":3600,"lifeCycleEnabled":false,"refreshTokenEnabled":
false,"refreshTokenExpiry":86400,"refreshTokenLifeCycleEnabled":false}],
"errorPageURL":"/oam/pages/servererror.jsp", "consentPageURL":"/oam/pages/consent.jsp"}'
Impossible to determine accurate command without support,examples,or thorough documentation
Required Oracle Support assistance to get these details](https://image.slidesharecdn.com/2022revtechmousautomatingcloudoperations-221028001100-1e8f45ef/85/Automating-Cloud-Operations-Everything-You-Wanted-to-Know-about-cURL-and-REST-14-320.jpg)
![JSON
JavaScript Object Notation
Pronounced“Jason”
An object surrounded by { }
An array or ordered list
REST can support both JSON and XML
Less verbose than XML,but lacks metadata support
//JSON Object
{
"employee": {
"id": 12,
"name": "Kobe",
"location": "USA"
}
}
//JSON Array
{
"employees": [
{
"id": 12,
"name": "Kobe",
"location": "USA"
},
{
"id": 13,
"name": "Jordan",
"location": "Canada"
},
{
"id": 14,
"name": "Barkley",
"location": "USA"
}
]
}](https://image.slidesharecdn.com/2022revtechmousautomatingcloudoperations-221028001100-1e8f45ef/85/Automating-Cloud-Operations-Everything-You-Wanted-to-Know-about-cURL-and-REST-21-320.jpg)
Automating Cloud Operations: Everything You Wanted to Know about cURL and REST
- 1. MICHIGAN ORACLE USERS SUMMIT 2022 WEDNESDAY OCTOBER 26,2022 1:15PM @W210C AUTOMATING CLOUD OPERATIONS EverythingYou Needed to Know about REST and cURL PRESENTER NAME: AHMED ABOULNAGA PRESENTERTITLE: TECHNICAL DIRECTOR
- 2. TABLE OF CONTENTS Introduction 3 REST and Cloud 6 Introduction to REST 9 Introduction to cURL 23 Creating an Oracle Autonomous Database 30 Figuring Out Authentication and Headers 41 Demo 56
- 3. INTRODUCTION
- 4. ABOUT ME Ahmed Aboulnaga Master’s degree in Computer Science from George Mason University Recent emphasis on cloud,DevOps,middleware,security in current projects OracleACE Pro, OCE, OCA Author, Blogger,Presenter @Ahmed_Aboulnaga
- 6. CLOUD APIS All cloud vendors provide some type of API to their services This allows for programmatic access to cloud services A basic understanding of cURL, REST, and JSON is helpful Most cloud providers use the REST architectural style for their APIs Client REST API Cloud Service JSON / XML GET / POST / PUT / DELETE
- 7. GOAL OFTHIS PRESENTATION Better understanding of REST and JSON Understand and interpret REST API documentation Familiarization with authorization when calling REST APIs Become capable of automating your cloud operations through REST APIs Target Audience https://globalacademy-eg.com/training-courses/oracle-dba.htm
- 8. MY STORY
- 9. MY USE CASE What I was trying to do? Create an Identity Domain in Oracle Access Manager OAM 12.2.1.3 had a web-based console for all OAuth configuration
- 10. MY USE CASE What was my challenge? OAM 12.2.1.4 provided no web- based interface and only supported a RESTAPI https://docs.oracle.com/en/middleware/idm/access-manager/12.2.1.3/oroau/op-oam-services-rest-ssa-api-v1-oauthpolicyadmin-oauthidentitydomain-post.html
- 11. STARTWITHTHE DOCUMENTATION Was the documentation helpful? Not all RESTAPI documentation is created equally Fortunately,the OAM REST API documentation provided an example request and example response
- 12. PREPARINGTO LOGIN What did I do first? Passwords can be passed as a “username:password” combination or encoded Encoding converts literal text to a humanly unreadable format Used online to encode“weblogic:welcome1” Authentication Options: curl -u 'weblogic':'welcome1' curl -H 'Authorization:Basic d2VibG9naWM6d2VsY29tZTE=' https://www.base64encode.org
- 13. DID ITWORK? Initial invocation failed command: curl -H 'Content-Type: application/x-www-form-urlencoded' -H 'Authorization:Basic d1VibG9naWM6d2VsY29tZTE=' --request POST http://soadev.revtech.com:7701/oam/services/rest/ssa/api/v1/oauth policyadmin/oauthidentitydomain -d ' { "name" : "AhmedWebGateDomain", "identityProvider" : "AhmedOUDStore", "description" : "Ahmed OIDC Domain" } ' Output: Mandatory param not found. Entity - IdentityDomain, paramName - name The documentation states that only “name” is mandatory So what’s the problem?
- 14. WHATWASTHE SUCCESSFUL OUTCOME? Final successful command: curl -H 'Content-Type: application/x-www-form-urlencoded' -H 'Authorization:Basic d1VibG9naWM6d2VsY29tZTE=' 'http://soadev.revtech.com:7701/oam/services/rest/ssa/api/v1/oauthpolicyadmin/oauthidentitydomai n' -d '{"name":"AhmedWebGateDomain", "identityProvider":"AhmedOUDStore", "description":"Ahmed OIDC WebGate Domain", "tokenSettings":[{"tokenType":"ACCESS_TOKEN","tokenExpiry":3600,"lifeCycleEnabled":false,"refres hTokenEnabled":false,"refreshTokenExpiry":86400,"refreshTokenLifeCycleEnabled":false}, {"tokenType":"AUTHZ_CODE","tokenExpiry":3600,"lifeCycleEnabled":false,"refreshTokenEnabled":fals e,"refreshTokenExpiry":86400,"refreshTokenLifeCycleEnabled":false}, {"tokenType":"SSO_LINK_TOKEN","tokenExpiry":3600,"lifeCycleEnabled":false,"refreshTokenEnabled": false,"refreshTokenExpiry":86400,"refreshTokenLifeCycleEnabled":false}], "errorPageURL":"/oam/pages/servererror.jsp", "consentPageURL":"/oam/pages/consent.jsp"}' Impossible to determine accurate command without support,examples,or thorough documentation Required Oracle Support assistance to get these details
- 16. WHAT IS REST? REpresentational StateTransfer Architectural style for distributed hypermedia system Proposed in 2000 by Roy Fielding in his dissertation Web Service implemented with REST is called RESTful web service REST is not a protocol like SOAP.It is rather an architectural style REST services typically use HTTP/HTTPS,but can be implemented with other protocols like FTP
- 17. REST ARCHITECTURAL CONSIDERATIONS Uniform interface: Easy to understand and readable results and can be consumed by any client or programming language over basic protocols. URI-based access: Using the same approach to a human browsing a website where all resource are linked together. Stateless communication: Extremely scalable since no client context is stored on the server between requests.
- 18. REST METHODS The HTTP protocol provides multiple methods which you can utilize for RESTful web services The table maps the HTTP method to the typical REST operation Some firewalls may limit some HTTP methods for security reasons HTTP Method REST Operation GET Read POST Create PUT Update DELETE Delete OPTIONS List of available methods HEAD Get version PATCH Update property/attribute Most common in web applications Most common in REST to provide CRUD functionality
- 19. RESOURCES Requests are sent to resources (i.e., URLs) Each resource represents an object which identified by a noun (e.g., employee,etc.) Each resource has a unique URL When performing a POST (create) or PUT (update),you must pass additional values Resource HTTP Method REST Output https://hostname/hr/employee GET Retrieve a list of all employees https://hostname/hr/employee/12 GET Retrieve details for employee #12 https://hostname/hr/employee POST Create a new employee https://hostname/hr/employee/12 PUT Update employee #12 https://hostname/hr/employee/12 DELETE Delete employee #12 https://hostname/hr/employee/12/address GET Retrieve address for employee #12
- 20. HTTP RESPONSE CODES HTTP response codes determine the overall response of the REST invocation HTTP Code Status Description 2XX (200,201,204) OK Data was received and operation was performed 3XX (301,302) Redirect Request redirected to another URL 4XX (403,404) Client Error Resource not available to client 5XX (500) Server Error Server error
- 21. JSON JavaScript Object Notation Pronounced“Jason” An object surrounded by { } An array or ordered list REST can support both JSON and XML Less verbose than XML,but lacks metadata support //JSON Object { "employee": { "id": 12, "name": "Kobe", "location": "USA" } } //JSON Array { "employees": [ { "id": 12, "name": "Kobe", "location": "USA" }, { "id": 13, "name": "Jordan", "location": "Canada" }, { "id": 14, "name": "Barkley", "location": "USA" } ] }
- 23. WHAT IS CURL? Open-source command-line tool Supports more than 22 different protocols (e.g., HTTP,HTTPS,FTP,etc.) For HTTP,supports all methods (e.g., GET, POST, PUT,DELETE, etc.) Very useful for testing RESTful web services Other advanced tools available include Postman, SoapUI,Oracle SQL Developer,etc. Example service: https://api.weather.gov/alerts/active?area=MI
- 24. VIEWING HEADER ATTRIBUTES Fetch the HTTP header attributes only using the -i or -I option:
- 25. AUTHENTICATION OPTIONS Many authentication options are supported;access token, bearer token, OAuth, basic auth, username/password,client certificate,etc. Authentication Examples: curl -H 'token:T3jaD5aDHJLr3idjCma894DAWzk49opp' curl -u 'weblogic':'welcome1' curl -H 'Authorization:Basic d2VibG9naWM6d2VsY29tZTE=' Token-basedAuthentication:
- 26. POSTMAN PopularAPI client Free version available www.postman.com Numerous features that include: ‒ Create API documentation ‒ Automated testing ‒ Design and mock APIs ‒ MonitorAPIs ‒ Etc.
- 27. BENEFITS OF CURL Free Command-line based tool Useful for non-interactive scripts Can pass HTTP headers,cookies,and authentication information Support for SSL, proxy,numerous protocols (e.g., LDAP, SMB,SCP,IMAP,FILE,TELNET,etc.), etc.
- 28. RECAP Recap of previous cURL request: curl -H 'Content-Type: application/x-www-form-urlencoded' -H 'Authorization:Basic d1VibG9naWM6d2VsY29tZTE=' -X POST http://dev.revtech.com:7701/oam/services/rest/ssa/api/v1/oauthpolicyadmin/oauthidentitydomain -d ' { "name" : "AhmedWebGateDomain", "identityProvider" : "AhmedOUDStore", "description" : "Ahmed OIDC Domain" } '
- 29. CREATING AN AUTONOMOUS DATABASE
- 30. MANUALLY CREATE AN AUTONOMOUS DATABASE (1 OF 4)
- 31. MANUALLY CREATE AN AUTONOMOUS DATABASE (2 OF 4)
- 32. MANUALLY CREATE AN AUTONOMOUS DATABASE (3 OF 4)
- 33. MANUALLY CREATE AN AUTONOMOUS DATABASE (4 OF 4)
- 34. NAVIGATETO DOCUMENTATION CreateAutonomousDatabase Reference https://docs.cloud.oracle.com/en-us/iaas/api/#/en/database/20160918/AutonomousDatabase/CreateAutonomousDatabase Note: ‒ API reference ‒ CreateAutonomousD atabase operation ‒ REST API endpoint ‒ API version
- 35. VIEW RESOURCE DETAILS AND EXAMPLE The resource details provides a list of all required parameters, often beyond what is demonstrated in the example Use the example as a starting point { "compartmentId" : "ocid1.tenancy.oc1..d6cpxn…dbx", "displayName" : "MOUS DB 2020 Auto", "dbName" : "MOUSDBAUTO", "adminPassword" : "Kobe_24_24_24", "cpuCoreCount" : 1, "dataStorageSizeInTBs" : 1 }
- 36. FIRST ATTEMPT… FAILED! Unable to authenticate upon first try despite all parameters/settings correct per the documentation…
- 37. EXECUTE COMMAND Eventually got it working…
- 38. PROVISIONING…
- 39. AVAILABLE!
- 40. FIGURING OUT AUTHENTICATION & HEADERS
- 41. DISCOVERY IS PAINFUL API References and Endpoints https://docs.cloud.oracle.com/en- us/iaas/api/#/en/database/20160918/ Oracle Cloud Infrastructure Documentation https://docs.cloud.oracle.com/en- us/iaas/Content/API/Concepts/apisigningkey.htm#How Managing Autonomous Data Warehouse Using oci-curl https://blogs.oracle.com/datawarehousing/managing- autonomous-data-warehouse-using-oci-curl Oracle Cloud Infrastructure (OCI) REST call walkthrough with curl https://www.ateam-oracle.com/oracle-cloud- infrastructure-oci-rest-call-walkthrough-with-curl But why didn’t the docs point me to this? Found this on my own, has some helpful info… I don’t want to use “oci- curl”! This is complicated! Thank God they have a script! Blog? Another blog?
- 42. PIECING IT TOGETHER If you want to use cURL to invoke OCI REST APIs… 1. Get information from the OCI Console a. Get theTenancy ID b. Get the User ID 2. Generate and configure an API Signing Key a. Create public/private key pair b. Get the fingerprint of the key c. Get the public key from the private key in PEM format d. Add the API Key to the OCI user (by uploading the public key) 3. Prepare and execute script a. Ensure private key is available b. Create the JSON request c. Update the custom script d. Execute!
- 43. 1A. GET THETENANCY ID
- 44. 1B. GET THE USER ID
- 45. 2A. CREATE PUBLIC/PRIVATE KEY PAIR Use ssh-keygen to create a public/private key pair The public key will be added as an “API Key” to your OCI account The private key will be used by your client (i.e., cURL)
- 46. 2B. GET THE FINGERPRINT OFTHE KEY Use openssl to view the X.509 MD5 PEM certificate fingerprint
- 47. 2C. GET PUBLIC KEY FROMTHE PRIVATE KEY IN PEM FORMAT OCI requires that the public key is imported in PEM format Use openssl to get the public key in PEM format
- 48. 2D.ADDTHE API KEY TOTHE OCI USER The public key is added to the OCI user’sAPI Key Must be in PEM format Can be uploaded or pasted
- 49. 3A. ENSURE PRIVATE KEY IS AVAILABLE The private key created earlier (and in PEM format) is used when invoking the REST service
- 50. 3B. CREATE THE JSON REQUEST The payload is created in JSON format
- 51. 3C. UPDATE CUSTOM SCRIPT The various elements (tenancy id, user id, private key, key fingerprint,etc.) are parameterized OCI’s REST API requires additional calculated elements,all taken care of here (oci-curl takes care of all of this for you) The cURL command is eventually called using a combination of static and dynamic values
- 52. 3D. EXECUTE! The cURL command is expanded The cURL command is executed The HTTP status code is observed to obtain the result of the invocation
- 53. PROVISIONING…
- 54. AVAILABLE!
- 55. www.mous.us THANKYOU SAVE THE DATE • ASCEND CONFERENCE 2023 June 11-14,2023 Caribe Royale Resort Orlando,Florida https://ascendusersconference.com • MOUS 2023 October 25, 2023 Schoolcraft College -VisTaTech Center, 18600 Haggerty Rd, Livonia,MI https://www.mous.us
- 56. www.mous.us THANKYOU SURVEYS • Session Surveys Please complete the session survey for this session. The survey will be provided to each attendee.