Salesforce AppExchange App Analytics

How do you know if anyone is actively using your AppExchange app? How can you tell which features are being utilized and which are left untouched? Is there a way to identify top customers and determine which ones need attention to prevent churn? Can you find out where your users are located to focus your marketing efforts there?

Great questions, my fellow ISV! The answer to these and many more lies in AppExchange App Analytics!

The original and more interactive version of this article is published on our blog - you can access it here - original article version - AppExchange App Analytics.

What is AppExchange App Analytics and what can I gain from it?

Before learning how to configure all the fancy stuff, let’s first understand the topic and set clear expectations.

Salesforce provides a native mechanism to collect usage logs generated by your app – AppExchange App Analytics. Various types of logs are collected, but their main purpose is to gather information on how your clients are using your app.

There is tons of information you can obtain from these logs. Some examples include:

• How many times a particular component/feature was used and by which customer?
• Are there features that are rarely used or not used at all?
• What unhandled exceptions were raised and which clients encountered them?
• How many users are actively using the app?
• What is the trend of app usage – is it increasing over time?
• In which countries are most of your users located? Where are your top customers?
• Are there users testing your app in trial orgs? Which features are they testing?
• Were any specific events triggered? You can define your own custom event types and log them in your app.

You probably already have some ideas on how this data can help your business and product development. However, let me throw a few of mine:

• More informed product roadmap decisions.
• Identification of top customers.
• Churn prevention (if you see that a customer is not actively using the app or is encountering exceptions, maybe you should react quickly to prevent uninstallation?).
• Bug identification.
• More informed licensing strategy decisions – perhaps you should switch to a usage-based model if there are significant differences in how your customers use the app?
• Identification of high-potential marketing regions.
• … and a lot more.

Is this as perfect as it sounds? I will cover this in the next sections, but for now, I hope you have a good understanding of the data that can be available to you.

And what data is not?

I’d like to note that AppExchange App Analytics data is strictly related to your app’s usage. Statistics about SEO, most-relevant search terms, and events that lead to your app’s installation are tracked separately as part of Salesforce Marketplace Analytics.

Types of Collected Logs

Salesforce collects three types of logs for your app:

• Package Usage Summary Logs – A monthly summary of your app’s usage. It contains information about how many times specific components/features were accessed in certain months. Salesforce keeps usage summary logs for 10 years (you can download summary logs from up to 10 years ago).
• Subscriber Snapshots – Snapshots of interactions with your app on specific days. They are similar to summary logs but are collected daily and contain slightly less information (e.g., they do not include details about the users who triggered the interaction – you wouldn’t be able to count unique users interacting with your app based solely on these logs). The retention period for subscriber snapshots is significantly shorter than for usage summaries – 45 days.
• Package Usage Logs – The most detailed of the three; each log represents a single interaction with your app, such as opening a Lightning page, reading or creating a record, making a REST call, and so on. These logs are very granular and can provide great insights into your customers’ behavior, but the data volumes can get wild ;). Like snapshots, package usage logs are retained for 45 days from the interaction occurrence.

Due to the data granularity and retention period considerations, different logs are collected from different customer orgs. Here’s a comparison from Salesforce documentation:

Prerequisites for Log Collection

To use AppExchange App Analytics, your package must pass the security review and be registered in your License Management Org.

The Package Usage Summary logs are enabled by default, but to start collecting Subscriber Snapshots and Package Usage Logs, you need to take additional action - different depending on whether you have a 1GP (first-generation packaging) or 2GP (second-generation packaging) managed package:

• 1GP – Toggle the Enable AppExchange App Analytics option in Package Manager of your packaging org (see this article for reference).
• 2GP – Execute a CLI command on your Devhub org (see this article for reference).

sf package update --package "Your Package Alias" --enable-app-analytics         

How to Download App Analytics Logs

Ok, you’ve met all the requirements. What now? The only way to download the App Analytics logs is by creating AppAnalyticsQueryRequest records. These represent requests sent to Salesforce to receive logs of a specific type for a specified timeframe. You can create such a record via an API call or by executing Apex code.

Probably the easiest way to do this yourself is by using Workbench – a simple tool that allows you to send API requests to your Salesforce org using a visual interface.

Here is a step-by-step example where I request and download Package Usage Logs for my app for the period August 10-16, 2024:

1. Go to https://workbench.developerforce.com and log in using your License Management Org (LMO) credentials (unless you specifically requested otherwise, these are just the credentials for your Partner Business Org – the org created for you when you became a Salesforce ISV/OEM Partner).

2. In the Select an action to perform section, select the Insert action, choose the AppAnalyticsQueryRequest object, and click Select.

3. Choose the Single Record option on the next screen and click Next.

4. Now, fill in the fields for your request and click Confirm Insert.

I populated only the required fields (type of logs and start/end time), but you can fill in the others to customize your request. Here’s a quick summary of the AppAnalyticsQueryRequest object fields:

• AvailableSince – It takes time for Salesforce to collect all the logs as they are collected from instances worldwide. If you request logs today, tomorrow Salesforce might have additional logs collected for the same period. You can use Available Since to request only the logs that Salesforce collected after a specific date.
• DataType – The type of logs you request. Valid values are PackageUsageSummary, PackageUsageLog, and SubscriberSnapshot.
• EndTime – The end time of the logging period. E.g., if you want to download logs collected between August 10-16, the EndTime should be August 16.
• FileCompression – Useful if you build custom logic to retrieve the logs automatically; allows you to request a compressed version of the logs. Valid values are gzip, snappy, and none.
• FileType – The format of the resulting logs. Valid values are csv (default) and parquet.
• OrganizationIds – If you want to receive logs only for the app usage by specific customers, you can specify their organization IDs (separated by commas) here.
• PackageIds – If you have published multiple packages and would like to retrieve logs only for specific ones, you can enter the package IDs (separated by commas) here.
• StartTime – The start of the logging period. E.g., if you want to download logs collected between August 10-16, the StartTime should be August 10.

To learn more about the AppAnalyticsQueryRequest structure, refer to the official documentation.

5. Click on the Salesforce ID value displayed on the confirmation screen.

6. You will see the AppAnalyticsQueryRequest record you’ve just created. Salesforce has 15 minutes to update its state and populate the DownloadUrl field. Unless your app is used very heavily, this usually takes up to a minute. Refresh the page until you see the RequestState change to Complete and the DownloadUrl field populated with a URL.

7. The final step is to copy the DownloadUrl value to a new browser tab and download the logs (note: the download link expires in 60 minutes). Here’s a screenshot of what these logs look like:

If you want to learn about the columns displayed here, scroll down to the App Analytics Logs Structure section.

How to Visualize App Analytics Logs

You need to handle the logs visualization yourself. Salesforce recommends uploading them to CRM Analytics (Salesforce’s native tool for data visualization). Here’s an end-to-end guide on how to utilize CRM Analytics to visualize your app usage logs. If you have plenty of time to dedicate or plan to build your own automation for the logs retrieval process, or you’re already familiar with CRM Analytics, this is a great option. The product is very powerful and allows you to create very great visualizations. But if you’re not a CRM Analytics expert (or don’t have a developer who is) and want to see your data quickly, you should probably stick to Excel or your own BI tool.

Here’s a sample Excel visualization of the number of interactions with your app’s components by specific customers ("Foo, Inc." and "Bar, Inc."):

If you’re thinking now "Uhh, not too sexy", let me assure you that this is just a simple example. If you really spend some time on the visualization preparation or invest in a ready-to-go solution, the results can be truly impressive.

Here is a screenshot of app usage visualization in our own data analytics solution – ISV Analytics:

App Analytics Logs Structure

The three types of logs contain different information with varying levels of granularity. I won’t describe all the log columns here, but you can reference them in the official Salesforce Documentation:

• Package Usage Summary logs
• Subscriber Snapshot logs
• Package Usage Logs

However, there are some common concepts and columns across all these log types. Here are the most important columns that provide an overall understanding of what is being collected:

• custom_entity – Probably the most important column. It contains the name of the component accessed by the user (e.g., the name of a Lightning page or Apex class). Groups of custom entities can be treated as specific features of the app.
• custom_entity_type – The type of entity accessed by the user (e.g., LightningPage, ApexClass, CustomObject, etc.). Entities of different types can have the same names (e.g., an Apex class and a Lightning page both named OrderManagement), so it is important to consider the custom_entity_type.
• organization_name, organization_id, organization_status, and other organization-related columns – These provide information about your customer’s Salesforce org. This allows you to group your usage data by specific orgs or org types (e.g., you may want to see data only for production instances or only for trial instances).
• Timestamp columns (timestamp_derived, date, or month depending on the log type) – You might need to filter the usage data by the time of the interaction. These columns help with that.

Your Custom Logs

Salesforce collects a lot of entity types out of the box, which already provide great insights into your app’s usage. However, there are certain use cases where you may need to collect custom logs to track app-specific events. Here are a few examples:

• A user completes a business process with success or failure.
• A user reaches a milestone, indicating they have begun to derive real value from your app (have you heard of the aha moment concept?).
• An automated process is launched for the n-th time and exceeds a specific threshold.
• A user completes the end-to-end path of a process – you can log a custom event at each stage to understand the detailed customer journey.

This is where Custom Interactions come into play. I will leave you with the pleasure of reading the details in the official documentation, but to summarize, you can create custom interaction logs directly from your Apex code - giving you the ability to decide on the logging granularity and frequency.

How to Make It All Easier

Let’s recap all the steps needed to view your app usage data.

1. Create an AppAnalyticsQueryRequest record via API or code for the specific logging period.
2. Wait for Salesforce to update the record with a download URL.
3. Download the logs using the download URL from the previous step.
4. Insert the logs into your data visualization tool of choice.
5. Visualize.

Doing this manually every time you want to see up-to-date usage data for your app seems like a hopeless quest. And what if you want to compare usage trends over time? Would you append the new data to the same Excel sheet manually all over again?

Fortunately, there are ways to improve this process drastically.

1. Build your own automations and visualizations.
2. Use a free semi-manual app.
3. Use a powerful paid solution that does everything for you.

Build Your Own Automations and Visualizations

This one is pretty easy to describe. You can automate the entire process yourself – request the logs every X days/hours, download them automatically, inject them into the visualization tool, and visualize.

The unquestionable advantage of this approach is that you control everything and build a solution that 100% matches your needs.

The downside is that you will likely spend much more money (and time) than buying a ready-to-go solution – apart from the usual development challenges, there are many restrictions on App Analytics requests (e.g., the max number of parallel requests, the max amount of data you can download in 24 hours, testing restrictions, etc.).

Use a Free Semi-Manual App

You can use one of the free AppExchange App Analytics apps published by Salesforce Labs – App Analytics and Package Visualizer. I will cover the latter one as it has more additonal features. It allows you to see some basic information about your app (such as the package details, package versions list, list of subscriber licenses) and gives you the ability to request App Analytics logs from the UI interface.

When Salesforce is ready with the download URL, you can download the logs directly from the app:

The rest of the process remains the same – you need to upload the logs to your visualization tool of choice (Package Visualizer prompts you to use CRM Analytics, but unfortunately, it doesn’t upload and configure it automatically) and visualize.

How is this better than using the fully manual approach with Workbench I showed you earlier? You can install Package Visualizer on your Partner Business Org and request logs directly from there. Moreover, the app shows you some basic information about your package and includes a great additional feature (not strictly related to App Analytics, but still very useful for ISVs) – it allows you to manage push upgrades of your apps.

In conclusion, this approach is perfect if you just want to check whether anybody is using your app from time to time. However, it is still a manual process that doesn’t offer any built-in visualization options, so it’s only applicable for very simple use cases.

Use a Powerful Paid (or Freemium) Solution That Does Everything for You

The last option I’ll discuss here is using a paid solution that handles the entire process and visualizes the App Analytics data for you – you’re responsible only for opening the app and drawing strategic conclusions for your product.

Spoiler alert: if you’ve published your package for free and don’t want to spend money on such an app, don’t worry! I will suggest a free version of this kind of app below.

We have built such an app – ISV Analytics. It is designed for both Salesforce OEM and ISV Partners and includes a free plan for partners distributing only free apps. Similar to Package Visualizer, it can be installed directly on your Partner Business Org. You can find it here.

It manages the log collection process behind the scenes and provides access to multiple App Analytics dashboards. Here are some examples of the data we visualize out of the box:

• Feature usage per customer and unique users per client company
• App usage trends over time
• App usage in different regions
• Package errors
• App installation trends, licenses breakdown
• Top customers based on different metrics
• Customers that need your attention
• Detailed customer summary
• …and more.

A few screenshots that might encourage you to try it out:

OK, enough of this self-promotion, let’s get back to the objective pros and cons of this approach.

The undeniable advantage of this option is that you get a plug-and-play solution with no development effort on your side.

However, if you have very unique needs and the app doesn’t satisfy them, building your own automations might be a better choice.

App Analytics In Practice – Approach Comparison

Conclusion

Salesforce AppExchange App Analytics can provide invaluable insights into your AppExchange app’s usage and performance. Combine it with Marketplace Analytics, and you’ll be well-equipped to make strategic product decisions, drive growth, prevent churn, and enhance marketing activities.

Using it in practice might be challenging due to the number of manual actions required, but there are ready-to-go solutions that can help, as well as the possibility to automate the process yourself.

If you need help configuring App Analytics for your app or have follow-up questions about this post, don’t hesitate to contact me (https://www.linkedin.com/in/bartosz-suchocki) or Beyond The Cloud via email (contact@beyondthecloud.dev) or LinkedIn (https://linkedin.com/company/beyondtheclouddev).

Resources

• AppExchange App Analytics Intro
• Package Usage Summaries
• Subscriber Snapshots
• Package Usage Logs
• AppAnalyticsQueryRequest Object
• Custom Interactions
• App Analytics Simulation Mode
• Marketplace Analytics
• Package Visualizer
• ISV Analytics