What Does API Monitoring Mean for Product Managers?

Conﬁdential and Proprietary. © 2020 Moesif, Inc. All Rights Reserved
moesif
What Does API Monitoring Mean for
Product Managers?

Who am I?
● Co-founder and CEO of Moesif,
an API analytics platform
● Focus on API strategy, platform
growth, and analytics
● I love IPAs!
derric@moesif.com

Lots of infrastructure level metrics for
engineering goals
Uptime
and SLA
Resource
Usage
Average
Latency
Request
Per Minute
Errors
Per Minute
1 2 3 4 5

However, product metrics should be aligned
to one of three product/business goals:
Adoption
(Evangelism)
Engagement Retention

Similarly, many top of funnel metrics, but are
these the best for product success
Page views Sign ups/
Sign ins
Requests Per
Minute
Average
Latency
Visits to a landing page is
not engagement
Can be faked by unqualified
leads signing up
Could be artificially high
from simple health probes
A consistently slow
API can be fine

Adoption: Map out your funnel to ﬁnd
barriers to adoption
The developer funnel tracks a user’s journey from initial sign up
to shipping a working app using your API
Pre-Integration
Test/Sandbox
Production

What to measure?
Unlike the mobile app acquisition funnel, an API user could stay
in a single stage for days. You should track both:
Conversion rate
for each step
Time to reach
next step

Track both conversion rate and
time to next step
Sign Up First API Call First Working App
30%
2 days
30%
6 hours

Pre-integration stage: Get them to sign up
Content, conferences,
and 1:1 meetings
Stellar and up to
date docs
Paid ads on developer
focused networks

Sandbox stage: Get them to their first hello
world
TTFHW (Time to First Hello World)
is the time a new developer takes to:
Sign up
Create a simple test app
Make first transaction and
verify API fits their needs
1
2
3

Lack of docs/simple onboarding slows
TTFHW
Ambiguous or lengthy
onboarding
No framework for their
environment
Lack of documentation

Production stage: Get them to ship a
working app
TTFWA (Time to First Working App)
or TTFPA (Time to First Paid App)
is the time a new developer takes
to ship a fully working solution
to their users.

Internal stakeholders slow TTFPA
Legal and
compliance risk
Project
priorities
Functional &
performance
testing

Let’s create example funnel goals for Algolia
Pre-integration stage:
Developers who signed-up and viewed onboarding docs or their API key
Sandbox stage:
Developers who completed at least:
One POST /v1/indexes operation AND
One POST /v1/indexes/{indexName}/query operation
WHERE response.headers.Content-Length > 0 .
Production stage:
Developers who completed at least:
1,000 POST /v1/indexes/{indexName}/query operations
WHERE response.headers.Content-Length > 0 .

API Engagement: Weekly active tokens can
be a good north star metric
The more accurate version is Weekly Active API Users.
This is because each user can create multiple keys, such as for dev and prod.

Once you deﬁne a north star metric,
understand what’s driving it
Which SDKs have
higher or lower
integration rates?
Which marketing
channels drive the most
integrations?
2%30%20%2%

With proper attribution, you can measure the
effectiveness of acquisition channels

It’s also important to look at API volume such
as which partners are most active

For most APIs, inconsistent performance is
worse for users than a high average latency

API Retention: What causes customers to
become loyal?

Programing languages
or SDK
API versions Features used
Digging into your API Retention
Find areas of improvement by grouping retention curves by:
V 3.281.32
V 2.812.93
V 1.472.48

Finding SDK issues via retention curves

What Does API Monitoring Mean for Product Managers?

More Related Content

What's hot (15)

Similar to What Does API Monitoring Mean for Product Managers? (20)

Recently uploaded (20)

What Does API Monitoring Mean for Product Managers?