SlideShare a Scribd company logo
©2008–22 New Relic, Inc. All rights reserved
@LeonAdato
bit.ly/miami-signup
IT’S FREE!!
FOREVER
You also need it
for this workshop
© 2022 New Relic, Inc. All rights reserved.
Closing gaps in visibility
With New Relic Flex
Leon Adato
$date();
© 2022 New Relic, Inc. All rights reserved.
Leon Adato
• Principal DevRel Advocate @ New Relic
• Started in tech ~33 yrs ago
• Focus on monitoring for ~20 yrs
• Tivoli, BMC, OpenView, Nagios, SolarWinds
Perl scripts, batch files,
and New Relic (duh)
• Built monitoring for small, medium,
and ludicrous size orgs.
• @LeonAdato almost everywhere
© 2022 New Relic, Inc. All rights reserved.
This is an Oyster Talk™
● Installing the Infrastructure agent
● YAML and the Flex directory
● Flex and command line output
● Flex and script output
● Flex and JSON output
● Troubleshooting
©2008–22 New Relic, Inc. All rights reserved
@LeonAdato
© 2022 New Relic, Inc. All rights reserved.
What isn’t (always) covered
● Standard commercial off-the-shelf (COTS) software
○ Exchange, Sharepoint, Blackberry, SAP, Storage
management, etc.
● Customized standard software
○ SAP, Wordpress, Drupal, etc)
● Custom data found in custom-built applications
© 2022 New Relic, Inc. All rights reserved.
Why you need* custom monitoring values
If what you need isn’t built into your monitoring / observability tool, you can:
● Place a feature request with the vendor, asking to add the metric.
○ And then wait. And pray. And maybe scream if you think it will help**
● Ask the community if/how they've solved the issue
○ IF there's an active community.
● Start shopping for yet-another-monitoring-tool
* probably
** spoiler: It won’t.
© 2022 New Relic, Inc. All rights reserved.
That sounds dumb
Or - and just hear me out - you could use your
monitoring / observability tool’s BUILT-IN ability to
collect custom metrics.
DUH!
© 2022 New Relic, Inc. All rights reserved.
What is Flex?
● Application-agnostic integration into New Relic
● Samples & more
https://github.com/newrelic/nri-flex
© 2022 New Relic, Inc. All rights reserved.
Agent
YAML
How does Flex ACTUALLY work?
Data!
© 2022 New Relic, Inc. All rights reserved.
YAML starter kit for the uninitiated
integrations:
- name: nri-flex
config:
name: NameyMcNameyFace
apis:
- event_type: <something here>
<lots more stuff here>
© 2022 New Relic, Inc. All rights reserved.
What can Flex do?
Collect metrics from a variety of outputs, and seamlessly send them
to New Relic for display on dashboards, use in alerts, etc. It can grab
data from
● A utility’s <stdout>.
● The output or return values of a script or program.
● JSON data generated by a local application.
● JSON data returned from a web page or external API call.
● Data scraped from a web page.
© 2022 New Relic, Inc. All rights reserved.
OK, but what is that in real life?
● LinuxAteMyRam.com - why sysadmins drink
● The IOT coffee pot - why security admins drink
● Everyone blaming the network - why network engineers drink
● A bunch of Oracle servers, and no way to get specific metrics.
● A different customer had >4k batch jobs per day, and needed
specific information on completion, size, return codes, etc.
● Or even finding the IIS space station and checking if your
current weather conditions will let you see it.
© 2022 New Relic, Inc. All rights reserved.
LESSON 1: Install the New Relic infra agent
© 2022 New Relic, Inc. All rights reserved.
LESSON 2: Your First YAML
integrations:
- name: nri-flex
config:
name: linuxFileSystemIntegration
apis:
- name: TestFlex
commands:
- run: 'df -PT -B1 -x tmpfs -x xfs -x vxfs -x btrfs -x ext -x ext2 -x ext3'
split: horizontal
split_by: s+
row_start: 1
set_header: [fs,fsType,capacityBytes,usedBytes,availableBytes,usedPerc,mountedOn]
perc_to_decimal: true
© 2022 New Relic, Inc. All rights reserved.
LESSON 3: The Necessary NRQL
FROM TestFlexSample SELECT average(availableBytes) TIMESERIES
© 2022 New Relic, Inc. All rights reserved.
LESSON 4: Flex with script output
integrations:
- name: nri-flex
timeout: 5m
interval: 10m
config:
name: linuxspeedtest
apis:
- name: pyspeedtest
commands:
- run: sudo python3 /flexlab/NR_checkspeed.py
split: horizontal
split_by: s+
set_header: [latency,download,upload]
timeout: 300000
© 2022 New Relic, Inc. All rights reserved.
LESSON 5: More NRQL
FROM pyspeedtestSample
SELECT average(download), average(upload)
TIMESERIES
© 2022 New Relic, Inc. All rights reserved.
LESSON 6: Flex with JSON output
integrations:
- name: nri-flex
timeout: 5m
interval: 10m
config:
name: linuxspeedtest
apis:
- name: speedtest
commands:
- run: speedtest --accept-license --accept-gcp -f json
timeout: 300000
remove_keys:
- timestamp
© 2022 New Relic, Inc. All rights reserved.
LESSON 7: Still more NRQL
FROM speedtestSample
select average(download.bandwidth)/125000,
average(upload.bandwidth)/125000
TIMESERIES
© 2022 New Relic, Inc. All rights reserved.
LESSON 8: Troubleshooting*
sudo /var/db/newrelic-infra/newrelic-integrations/bin/nri-flex
--verbose --pretty --config_file <path>/<your yaml>.yml
Edit /etc/newrelic-infra.yml
log:
file: '/var/log/newrelic-infra/newrelic-infra.log'
level: debug
forward: true
stdout: false
* and other swear words

More Related Content

PPTX
Using Queryable State for Fun and Profit
PDF
Combining Logs, Metrics, and Traces for Unified Observability
PPTX
Time to say goodbye to your Nagios based setup
PDF
OSMC 2014: Time to say goodbye to your Nagios setup | Oliver Jan
PPTX
Evolution of Monitoring and Prometheus (Dublin 2018)
PDF
Making operations visible - Nick Gallbreath
PDF
Making operations visible - devopsdays tokyo 2013
PDF
New Relic
Using Queryable State for Fun and Profit
Combining Logs, Metrics, and Traces for Unified Observability
Time to say goodbye to your Nagios based setup
OSMC 2014: Time to say goodbye to your Nagios setup | Oliver Jan
Evolution of Monitoring and Prometheus (Dublin 2018)
Making operations visible - Nick Gallbreath
Making operations visible - devopsdays tokyo 2013
New Relic

Similar to Flex Workshop, New Relic O11y Days (20)

PPTX
Real-Time Metrics and Distributed Monitoring - Jeff Pierce, Change.org - Dev...
PDF
Microservices and Prometheus (Microservices NYC 2016)
PDF
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)
PPTX
Prometheus - Open Source Forum Japan
PDF
OSDC 2018 | Distributed Monitoring by Gianluca Arbezzano
PDF
OSDC 2018 - Distributed monitoring
PPTX
An Introduction to Prometheus (GrafanaCon 2016)
PPTX
New Relic Basics
PPTX
Observability for Application Developers (1)-1.pptx
PPTX
NewRelic.pptx
PDF
Monitoring Big Data Systems - "The Simple Way"
PPTX
The Art of Container Monitoring
PDF
The present and future of serverless observability (QCon London)
PDF
The present and future of Serverless observability
PDF
The present and future of Serverless observability
PDF
Meetup OpenTelemetry Intro
PDF
New Relic Zapier Case Study Platform
PDF
Prometheus (Microsoft, 2016)
PDF
OSDC 2014: Devdas Bhagat - Graphite: Graphs for the modern age
PPTX
Rootconf 2017 - State of the Open Source monitoring landscape
Real-Time Metrics and Distributed Monitoring - Jeff Pierce, Change.org - Dev...
Microservices and Prometheus (Microservices NYC 2016)
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)
Prometheus - Open Source Forum Japan
OSDC 2018 | Distributed Monitoring by Gianluca Arbezzano
OSDC 2018 - Distributed monitoring
An Introduction to Prometheus (GrafanaCon 2016)
New Relic Basics
Observability for Application Developers (1)-1.pptx
NewRelic.pptx
Monitoring Big Data Systems - "The Simple Way"
The Art of Container Monitoring
The present and future of serverless observability (QCon London)
The present and future of Serverless observability
The present and future of Serverless observability
Meetup OpenTelemetry Intro
New Relic Zapier Case Study Platform
Prometheus (Microsoft, 2016)
OSDC 2014: Devdas Bhagat - Graphite: Graphs for the modern age
Rootconf 2017 - State of the Open Source monitoring landscape
Ad

Recently uploaded (20)

PPT
Teaching material agriculture food technology
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
NewMind AI Monthly Chronicles - July 2025
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Empathic Computing: Creating Shared Understanding
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
KodekX | Application Modernization Development
DOCX
The AUB Centre for AI in Media Proposal.docx
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
cuic standard and advanced reporting.pdf
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PPTX
A Presentation on Artificial Intelligence
Teaching material agriculture food technology
Understanding_Digital_Forensics_Presentation.pptx
Mobile App Security Testing_ A Comprehensive Guide.pdf
Unlocking AI with Model Context Protocol (MCP)
Encapsulation_ Review paper, used for researhc scholars
NewMind AI Monthly Chronicles - July 2025
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Empathic Computing: Creating Shared Understanding
Diabetes mellitus diagnosis method based random forest with bat algorithm
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
20250228 LYD VKU AI Blended-Learning.pptx
KodekX | Application Modernization Development
The AUB Centre for AI in Media Proposal.docx
Digital-Transformation-Roadmap-for-Companies.pptx
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
cuic standard and advanced reporting.pdf
Chapter 3 Spatial Domain Image Processing.pdf
A Presentation on Artificial Intelligence
Ad

Flex Workshop, New Relic O11y Days

  • 1. ©2008–22 New Relic, Inc. All rights reserved @LeonAdato bit.ly/miami-signup IT’S FREE!! FOREVER You also need it for this workshop
  • 2. © 2022 New Relic, Inc. All rights reserved. Closing gaps in visibility With New Relic Flex Leon Adato $date();
  • 3. © 2022 New Relic, Inc. All rights reserved. Leon Adato • Principal DevRel Advocate @ New Relic • Started in tech ~33 yrs ago • Focus on monitoring for ~20 yrs • Tivoli, BMC, OpenView, Nagios, SolarWinds Perl scripts, batch files, and New Relic (duh) • Built monitoring for small, medium, and ludicrous size orgs. • @LeonAdato almost everywhere
  • 4. © 2022 New Relic, Inc. All rights reserved. This is an Oyster Talk™ ● Installing the Infrastructure agent ● YAML and the Flex directory ● Flex and command line output ● Flex and script output ● Flex and JSON output ● Troubleshooting
  • 5. ©2008–22 New Relic, Inc. All rights reserved @LeonAdato
  • 6. © 2022 New Relic, Inc. All rights reserved. What isn’t (always) covered ● Standard commercial off-the-shelf (COTS) software ○ Exchange, Sharepoint, Blackberry, SAP, Storage management, etc. ● Customized standard software ○ SAP, Wordpress, Drupal, etc) ● Custom data found in custom-built applications
  • 7. © 2022 New Relic, Inc. All rights reserved. Why you need* custom monitoring values If what you need isn’t built into your monitoring / observability tool, you can: ● Place a feature request with the vendor, asking to add the metric. ○ And then wait. And pray. And maybe scream if you think it will help** ● Ask the community if/how they've solved the issue ○ IF there's an active community. ● Start shopping for yet-another-monitoring-tool * probably ** spoiler: It won’t.
  • 8. © 2022 New Relic, Inc. All rights reserved. That sounds dumb Or - and just hear me out - you could use your monitoring / observability tool’s BUILT-IN ability to collect custom metrics. DUH!
  • 9. © 2022 New Relic, Inc. All rights reserved. What is Flex? ● Application-agnostic integration into New Relic ● Samples & more https://github.com/newrelic/nri-flex
  • 10. © 2022 New Relic, Inc. All rights reserved. Agent YAML How does Flex ACTUALLY work? Data!
  • 11. © 2022 New Relic, Inc. All rights reserved. YAML starter kit for the uninitiated integrations: - name: nri-flex config: name: NameyMcNameyFace apis: - event_type: <something here> <lots more stuff here>
  • 12. © 2022 New Relic, Inc. All rights reserved. What can Flex do? Collect metrics from a variety of outputs, and seamlessly send them to New Relic for display on dashboards, use in alerts, etc. It can grab data from ● A utility’s <stdout>. ● The output or return values of a script or program. ● JSON data generated by a local application. ● JSON data returned from a web page or external API call. ● Data scraped from a web page.
  • 13. © 2022 New Relic, Inc. All rights reserved. OK, but what is that in real life? ● LinuxAteMyRam.com - why sysadmins drink ● The IOT coffee pot - why security admins drink ● Everyone blaming the network - why network engineers drink ● A bunch of Oracle servers, and no way to get specific metrics. ● A different customer had >4k batch jobs per day, and needed specific information on completion, size, return codes, etc. ● Or even finding the IIS space station and checking if your current weather conditions will let you see it.
  • 14. © 2022 New Relic, Inc. All rights reserved. LESSON 1: Install the New Relic infra agent
  • 15. © 2022 New Relic, Inc. All rights reserved. LESSON 2: Your First YAML integrations: - name: nri-flex config: name: linuxFileSystemIntegration apis: - name: TestFlex commands: - run: 'df -PT -B1 -x tmpfs -x xfs -x vxfs -x btrfs -x ext -x ext2 -x ext3' split: horizontal split_by: s+ row_start: 1 set_header: [fs,fsType,capacityBytes,usedBytes,availableBytes,usedPerc,mountedOn] perc_to_decimal: true
  • 16. © 2022 New Relic, Inc. All rights reserved. LESSON 3: The Necessary NRQL FROM TestFlexSample SELECT average(availableBytes) TIMESERIES
  • 17. © 2022 New Relic, Inc. All rights reserved. LESSON 4: Flex with script output integrations: - name: nri-flex timeout: 5m interval: 10m config: name: linuxspeedtest apis: - name: pyspeedtest commands: - run: sudo python3 /flexlab/NR_checkspeed.py split: horizontal split_by: s+ set_header: [latency,download,upload] timeout: 300000
  • 18. © 2022 New Relic, Inc. All rights reserved. LESSON 5: More NRQL FROM pyspeedtestSample SELECT average(download), average(upload) TIMESERIES
  • 19. © 2022 New Relic, Inc. All rights reserved. LESSON 6: Flex with JSON output integrations: - name: nri-flex timeout: 5m interval: 10m config: name: linuxspeedtest apis: - name: speedtest commands: - run: speedtest --accept-license --accept-gcp -f json timeout: 300000 remove_keys: - timestamp
  • 20. © 2022 New Relic, Inc. All rights reserved. LESSON 7: Still more NRQL FROM speedtestSample select average(download.bandwidth)/125000, average(upload.bandwidth)/125000 TIMESERIES
  • 21. © 2022 New Relic, Inc. All rights reserved. LESSON 8: Troubleshooting* sudo /var/db/newrelic-infra/newrelic-integrations/bin/nri-flex --verbose --pretty --config_file <path>/<your yaml>.yml Edit /etc/newrelic-infra.yml log: file: '/var/log/newrelic-infra/newrelic-infra.log' level: debug forward: true stdout: false * and other swear words