A Cost-Effective Approach to Integrate Edgio RTLD WAF Logs into Microsoft Sentinel

Summary:

In this article, we will present our custom-developed, cost-effective methodology for on-boarding Edgio RTLD WAF logs into Microsoft Sentinel. When it comes to SOC monitoring, Edgio provides a rich set of RTLD WAF logs that offer significant security value for web application protection. As shown in the diagram below, Edgio RTLD logs can be delivered to various destinations using multiple methods. If you're using Microsoft Sentinel as your SIEM and want to ingest Edgio RTLD WAF logs, the only direct method available is through Azure Blob Storage. This method of using Azure Blob Storage has its own cost and can be quite expensive. During a recent client engagement, we were tasked with developing a more budget-friendly alternative to this solution.

As always, we accepted the challenge and began exploring a solution by utilizing the web server log delivery methodology mentioned above.

Web Server Log Delivery Method - How It works

Edgio uses the web server log delivery method to send RTLD WAF logs in compressed gzip format via HTTPS POST requests to a specified web server

There are two authentication methods supported by this method:

• HTTP Basic Authentication: For web server that does not require authorization

• HTTP Custom Authentication: For web server that authorizes requests through the Authorization header

In this article, we focus on the Custom Authentication method. With this approach, the web server authorizes requests through the Authorization header. Edgio transmits the bearer token, as part of the HTTP request header in the Authorization field as shown. Once the web server receives the correct bearer token, it authorizes the request and accepts the logs.

Based on the workflow outlined above, we chose to use the Logs Ingestion API in place of the web server for our destination endpoint. Azure Monitor's Logs Ingestion API facilitates data transmission to a Log Analytics workspace through REST API calls. This communication process aligns with the custom authentication flow mentioned earlier, requiring a bearer token for authorization to push the logs. Once the logs are received via this API, they can be directed to both predefined Azure tables and custom tables, such as Edgio_CL. Before pushing logs to the respective tables, we can also parse them using the TransformKQL feature from the DCR as shown below.

To send logs to Microsoft Sentinel using Azure Monitor's REST API, Edgio should be able to provide the following values. As noted earlier(Edgio HTTPS request parameters), Edgio supports sending all these values through its interface.

HTTP Headers:

• Endpoint URI(Request URL): Destination url that has DCE and DCR Immutable ID.

• Authorization: Bearer <tokenvalue> (which is obtained via submitting appid & secret values to registered entra ID app)

• Content-Type: application/json

• Content-Encoding: gzip

• Body: JSON Array formatted log fields

How can we obtain a Bearer Token that Edgio can submit and that will be accepted by the Azure Monitor REST API?r

The Log Ingestion API utilizes a registered Entra ID application for authentication and authorization. This registered application consists of an App ID (similar to a username) and a secret value (akin to a password), which are used for authentication with the Entra ID app. Once authenticated, you receive a bearer token. Unfortunately, this process cannot be performed directly from Edgio, as there is no mechanism to pass the App ID and secret value. Therefore, after obtaining the bearer token from the Entra ID app, you provide it to Edgio. This token is then included in the Authorization header of the HTTP request to the Log Ingestion REST API. If the token is valid, the API authorizes the log-pushing action, ensuring secure and efficient access.

The Limitation - Token Updation Challenge:

Using the RTLD profile, you can leverage the HTTP custom authentication method provided by Edgio to push logs to the Logs Ingestion REST API. The Edgio RTLD profile offers a user-friendly GUI for entering all the necessary information to pass to the Azure Monitor REST API, including the bearer token in the RTLD profile section "token" as shown.

However, since this token is dynamically generated, managing it can be cumbersome. Each time the bearer token expires—typically within 50 minutes to 1 hour—you must manually access the Edgio profile GUI to input a new token. If not, Edgio cannot authenticate with the Azure Log Ingestion REST API using an expired token. Even if you extend the token's expiry to 1 day, regular updates remain necessary, which is not a feasible or practical solution.

Token Maximum Size Limit Issue

Another issue with Edgio is rejecting the bearer token fetched from the Entra ID application due to its maximum character length restriction, resulting in the following error.

Error:Could not save log profile: 'token' must be between 1 and 1024 characters. You entered 1301 characters.

Usually, the best practice is to limit the maximum size of bearer tokens in kilobytes (KB). However, Edgio enforces a maximum character length restriction. This limitation hinders the successful update of the bearer token, complicating the integration process. We informed the Edgio team about the issue, and fortunately, they resolved it in time for our customer. However, it remains unclear whether this fix has been applied for all their customers across the world. 

Overcoming Token Management Challenges in Edgio RTLD Profile:

To address the challenge of manually updating the bearer token each time it expires, we aim to implement an automated solution by creating a python automation script that interfaces with registered Entra ID application to fetch a new bearer token whenever it expires by passing the following values.

• Entra ID Application Client ID

• Secret Value

Once the bearer token is fetched, the script will automatically update the Edgio RTLD profile via the Edgio API endpoint. By automating this workflow, we can eliminate manual interventions and ensure seamless log delivery to Azure Sentinel without interruptions.

Implementing the solution

Azure Side

Steps:

• Create a Microsoft Entra application to authenticate against the API.

• Create a custom table(Edgio_CL) in a Log Analytics workspace. This is the table you'll be sending log data to.

• Create a data collection rule (DCR) to direct the data to the target table.

• Give the registered Entra ID application to access the DCR.

Step1: Create Microsoft Entra application

Follow the section "Create Microsoft Entra application" from the article for creating Entra ID application. Be sure to note down the application ID and its secret in a separate, secure location to use later.

Step2: Create a custom table(Edgio_CL) in Microsoft Sentinel.

You can create the table by running following command.

After creating the table, you can verify it by navigating to Microsoft Sentinel Workspace > Settings > Workspace Settings > Settings > Tables -> Search for Edgio_CL

Also from Microsoft Sentinel side you can verify as shown:

Step3: Create DCE and DCR

First, create the DCE using the template provided below. Navigate to Azure Portal > Template Specs and create the template using the following data. Once the template is created, proceed to deploy it.

Once the DCE is created open the DCE and copy the Log Ingestion URL as shown.

Once the DCE is created, copy the Log Ingestion URL from the DCE and paste it into the DCR template under the "logsIngestion" field as shown below:

Next Create the DCR by using the template provided below. Navigate to Azure Portal > Template Specs and create the template using the following data. Once the template is created, proceed to deploy it.

While deploying DCR it asks for the

• Log Analytics Workspace Resource ID: Fetch this value by navigating to Microsoft Sentinel->Settings->Workspace Settings -> Properties-> Resource ID

• DCE Resource Id: Open the DCE in JSON View then copy the Resource ID and fill.

Step4: Provide Microsoft Entra ID App with Monitoring Metrics Publisher role

Assigning the Monitoring Metrics Publisher role to an Azure Entra ID application that was created in step1 on created DCR resource is necessary when pushing logs to azure monitor from custom log sources such as edgio. This role grants the application permission to publish custom metrics to Azure Monitor.

Open DCR-> Access Control -> Add -> Add Role Assignment -> Monitoring Metrics Publisher -> Select the Entra Id App -> assign the role.

Edgio Side:

So far, we have registered the Entra ID application and created a Data Collection Rule (DCR), a Data Collection Endpoint (DCE), and a custom table in Log Analytics for receiving logs from Edgio. This setup allows us to process and push these logs to a custom table called Edgio_CL in Microsoft Sentinel. WNow, let's create the RTLD WAF profile and API clients in Edgio for pushing logs to the Azure Monitor Log Ingestion REST API.

Here are the steps you need to follow:

• Create an Edgio RTLD WAF Profile: Fill in all the required fields, including a random token value (any placeholder string).

• Create an Edgio API Client: Set up an API client for accessing Edgio through API calls. This will allow you to automate interactions with the Edgio service.

Step1: Create Edgio RTLD WAF profile.

To enable the web server log delivery method, create an Edgio RTLD profile that includes all the necessary HTTP request parameters. Follow the steps below to create the profile:

• From the left pane, click Realtime Log Delivery.

• Click + New Log Delivery Profile and then select WAF option

• From the Profile Name option, assign a name to this log delivery profile.

• From the Log Delivery Method option, select HTTP POST.

• Define how RTLD will communicate with your web server(s).

• Set the Request URL option to (DCE LogIngestion URL)

• From the Authentication Type option, select Custom Authentication option.

• Set the Token option to a random value. Later we can update this with the real bearer token that we obtain via automation script.

• Select Log Format option to JSON Array

• within the Fields section Select the log fields that you want to send.

• Click on Apply Changes.

Step2: Create Edgio API Client:

Previously, we created the RTLD WAF profile that uses the web server custom authentication method for log delivery. Now, we need to update the existing WAF profile with a dynamic bearer token value each time. To achieve this, we will use an automation script that interacts with Edgio via its API interface to perform the update.

To enable the automation script to connect to Edgio via API, we first need to create an API client in Edgio, which will provide us with the API Client ID and Secret values.

You must assign one or more scope(s) when creating an API client. Each scope identifies the set of actions that the API client is authorized to perform.

To create an API client

1. Navigate to the API Clients page.

2. Click Create API Client

• Select either your private space or the desired organization. Your new API client will only be authorized for the properties associated with either your private space or the selected organization. Click API Clients.

• In the Name option, assign a name to this API client.

• In the Description option, describe this API client.

• In the Access Token Lifetime (Seconds) option, determine the number of seconds that an access token will remain valid after being issued.

• In the Allowed API Scopes section, mark each scope that will be assigned to the API client.

A security best practice is to only grant the set of scope(s) required for the automation task(s) that the client will perform.

Upon successfully registering your client application, the following information will be available for your client application:

• Client ID: Identifies an API client by its system-defined ID.

• Secret: A client must pass this private key for identity verification when requesting an access token.

• Scopes: Defines the set of operations that this API client is authorized to perform.

Use this information, along with scopes, to generate an access token from automation script.

Automation Script

As discussed above, to address the dynamic generation of the bearer token from Entra ID and to update it in Edgio, we will utilize a Python script. This script will be placed on an on-premises syslog server(/etc/edgioprofileupdate.py that has connectivity with our on-prem Edgio setup.

Use the following python script for automating whole process. Please update the below values from the python script with your custom values to your environment.

• tenantID: Your Azure tenant ID.

• app_id: The client ID of the registered Entra ID application.

• app_secret: The secret value of the Entra ID application.

• dcr_immutable_id: Collect this value by accessing the JSON view of your Data Collection Rule (DCR).

• dce_endpoint: {Data Collection Endpoint URI}/dataCollectionRules/{DCR Immutable ID}/streams/{Stream Name}?api-version=2023-01-01

• client_id: Replace this with your actual Edgio API client ID.

• client_secret: Replace this with your actual Edgio API secret value.

• environment_id: Indicates the system-defined ID for the environment through which this request was served. Fetch this value from your network team.

• <profileId>: /rtld/v1/waf/profiles/<profileId> RTLD Waf profile Id

Note: To enhance the security of your API keys in the script, consider using Azure Key Vault, which securely stores sensitive information and allows for programmatic access without hard coding. Alternatively, you can use the python-dotenv library to manage your API keys in a .env file, keeping them separate from your code.

Also ensure the file can only be accessed by root user.

In our setup the bearer token expires every 40 minutes for our Entra App. So we need to run the python script every 40minutes to update the WAF profile. So we are scheduling a root-level linux cron job( a linux job scheduling service) to run it every 35 minutes(its better to run a little early before expiry) as shown.

Once the entire setup is complete, you can view the logs in Microsoft Sentinel as illustrated below:

If you’re having trouble copying the scripts and JSON templates from the article, you can also find them in my GitHub repository link.

By Ashokkrishna Vemuri