[APM] Storage explorer improvements #143179

gbamparop · 2022-10-12T11:39:41Z

Summary

Summary stats

Add total APM size
Add percentage of disk space used by the APM indices compared to the overall storage configured for ES
Improve loading state

Callouts

Add callout when using cross-cluster search
Add callout for enabling progressive loading of data or optimised sorting for the services table

Performance

Add support for optimised sorting to the services table
Rows can now be expanded while loading when using the random_sampler aggregation or optimised sorting

Services table and details

Improve loading states
Display index breakdown table for each service

Documentation
Add tips and trick section

Closes #142333

… overall storage configured for ES

…r-improvements

gbamparop · 2022-10-12T12:48:32Z

/oblt-deploy

…s' into apm-storage-explorer-improvements

…r-improvements

gbamparop · 2022-10-18T21:15:21Z

x-pack/plugins/observability/server/ui_settings.ts

@@ -197,13 +197,13 @@ export const uiSettings: Record<string, UiSettings> = {
  [apmServiceInventoryOptimizedSorting]: {
    category: [observabilityFeatureId],
    name: i18n.translate('xpack.observability.apmServiceInventoryOptimizedSorting', {
-      defaultMessage: 'Optimize APM Service Inventory page load performance',
+      defaultMessage: 'Optimize services list load performance in APM',
    }),
    description: i18n.translate(
      'xpack.observability.apmServiceInventoryOptimizedSortingDescription',


We can check if renaming it will break telemetry

x-pack/plugins/apm/public/components/app/storage_explorer/resources/tips_and_resources.tsx

gbamparop · 2022-10-18T21:49:35Z

/oblt-deploy

elasticmachine · 2022-10-18T21:56:20Z

Pinging @elastic/apm-ui (Team:APM)

gbamparop · 2022-10-19T08:07:54Z

/oblt-deploy

cauemarcondes

Added some comments

x-pack/plugins/apm/server/routes/storage_explorer/route.ts

x-pack/plugins/apm/server/routes/storage_explorer/is_cross_cluster_search.ts

cauemarcondes · 2022-10-19T18:01:10Z

x-pack/plugins/apm/server/routes/storage_explorer/get_storage_details_per_service.ts

+              indices: {
+                terms: {
+                  field: INDEX,
+                  size: 500,


We've had multiple SDHs around this kind of limitation, should we already anticipate it and create a setting for this?

I don't think it's worth adding another setting for the number of indices returned for each service as we wouldn't want for the settings to keep growing for each type of terms aggregation. I see the point though that if there are more than 500 indices the sum of their size / docs wouldn't match the one for the service. We could increase the number in case they have a very aggressive ILM policy, display a message if there are more or just change the table to top 500? @sqren WDYT?

Imo configuration options are escape hatches when we cannot come up with better solutions. In this case, increasing the size will cause a perf penalty which is also not great for the user.
I'm leaning towards just showing a warning if we hit the limit.

cauemarcondes · 2022-10-19T18:04:33Z

x-pack/plugins/apm/server/routes/storage_explorer/get_storage_details_per_service.ts

+      const size =
+        allIndicesStats &&
+        getEstimatedSizeForDocumentsInIndex({
+          allIndicesStats,
+          indexName,
+          numberOfDocs,
+        });


This looks odd, so if allIndicesStats is 0 you don't want to use getEstimatedSizeForDocumentsInIndex? and if allIndicesStats is >= 1 you'll use the value of getEstimatedSizeForDocumentsInIndex?

Do you mean the check that allIndicesStats is defined?

is size a boolean?

No it's the estimated index size

x-pack/plugins/apm/server/routes/storage_explorer/get_storage_details_per_service.ts

cauemarcondes · 2022-10-19T18:10:01Z

x-pack/plugins/apm/public/components/app/storage_explorer/summary_stats.tsx

+      {hasData && (
+        <EuiText
+          css={css`
+            ${xlFontSize}


Why is this xlFontSize necessary? can't you use the size prop instead?

It seems to only accept s, m, xs, relative.

Maybe create an issue on EUI to add it?

I think this is meant to be used for font sizes https://elastic.github.io/eui/#/theming/typography/values

x-pack/plugins/apm/public/components/app/storage_explorer/index.tsx

…r-improvements

gchaps

doclinks look good

cauemarcondes

LGTM.

cauemarcondes · 2022-10-25T17:36:19Z

x-pack/plugins/apm/server/routes/storage_explorer/is_cross_cluster_search.test.ts

+      } as ApmIndicesConfig,
+    } as unknown as Setup;
+
+    expect(isCrossClusterSearch(mockedSetup)).toBe(false);


Suggested change

expect(isCrossClusterSearch(mockedSetup)).toBe(false);

expect(isCrossClusterSearch(mockedSetup)).toBeFalsy()

This will still pass with all falsy values (false, 0, '', null, undefined, and NaN)

cauemarcondes · 2022-10-25T17:54:08Z

x-pack/plugins/apm/public/components/app/storage_explorer/summary_stats.tsx

+      {hasData && (
+        <EuiText
+          css={css`
+            ${xlFontSize}


Maybe create an issue on EUI to add it?

x-pack/plugins/apm/server/routes/storage_explorer/get_storage_details_per_service.ts

cauemarcondes · 2022-10-25T17:56:40Z

x-pack/plugins/apm/server/routes/storage_explorer/get_storage_details_per_service.ts

+      const size =
+        allIndicesStats &&
+        getEstimatedSizeForDocumentsInIndex({
+          allIndicesStats,
+          indexName,
+          numberOfDocs,
+        });


is size a boolean?

…r-improvements

gbamparop · 2022-10-26T10:53:48Z

/oblt-deploy

kibana-ci · 2022-10-26T11:00:11Z

💚 Build Succeeded

Buildkite Build
Commit: 1c4b800

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`apm`	1331	1334	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`apm`	3.1MB	3.1MB	+9.3KB
`lists`	192.2KB	192.5KB	+271.0B
total			+9.6KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`core`	369.1KB	369.4KB	+271.0B

History

💚 Build #82415 succeeded b50e5a9
💚 Build #82256 succeeded 3eb643d
💚 Build #81442 succeeded ca5cd36
💚 Build #81441 succeeded 169a742
💛 Build #81203 was flaky 5fb2c33
💔 Build #80992 failed 43b88c8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

gbamparop added 6 commits October 6, 2022 14:48

Add total APM size to summary stats

b89c6cd

Show percentage of disk space used by the APM indices compared to the…

11e59a8

… overall storage configured for ES

Add tips and references section

fb1790c

Add tooltips to summary metrics

7ec2791

Add indices breakdown table

2fdd001

Merge remote-tracking branch 'upstream/main' into apm-storage-explore…

dce68e4

…r-improvements

gbamparop and others added 8 commits October 12, 2022 17:02

Add callout for cross-cluster search

0271eef

Update storage explorer link

d122eec

removed the coloring of the summary card titles and updated the icons

43b88c8

Storage explorer improvements

280dc79

Merge remote-tracking branch 'origin/apm-storage-explorer-improvement…

185addf

…s' into apm-storage-explorer-improvements

Merge remote-tracking branch 'upstream/main' into apm-storage-explore…

d495acf

…r-improvements

Add check for storageExplorerIndicesStatsTable to e2e tests

f783b1b

Fix loading indicator

58a1cad

gbamparop commented Oct 18, 2022

View reviewed changes

x-pack/plugins/apm/public/components/app/storage_explorer/resources/tips_and_resources.tsx Show resolved Hide resolved

Remove unnecessary fragment

5fb2c33

gbamparop marked this pull request as ready for review October 18, 2022 21:56

gbamparop requested review from a team as code owners October 18, 2022 21:56

gbamparop added the Team:APM - DEPRECATED Use Team:obs-ux-infra_services. label Oct 18, 2022

gbamparop added the release_note:skip Skip the PR/issue when compiling release notes label Oct 18, 2022

gbamparop mentioned this pull request Oct 19, 2022

[APM] Implement Storage Explorer improvements #142333

Closed

10 tasks

gbamparop added 2 commits October 19, 2022 15:42

Add formatting to the number of docs per index

169a742

Remove beta badge

ca5cd36

cauemarcondes reviewed Oct 19, 2022

View reviewed changes

gbamparop added 6 commits October 24, 2022 12:01

Merge remote-tracking branch 'upstream/main' into apm-storage-explore…

a807976

…r-improvements

Add unit tests for isCrossClusterSearch

b49f669

Update isCrossClusterSearch

3e5a8cf

Remove cast to QueryDslQueryContainer

4e38047

Remove optional chaining

3eb643d

Update tips and tricks section

b50e5a9

gchaps approved these changes Oct 25, 2022

View reviewed changes

cauemarcondes approved these changes Oct 25, 2022

View reviewed changes

Merge remote-tracking branch 'upstream/main' into apm-storage-explore…

1c4b800

…r-improvements

gbamparop merged commit e390743 into elastic:main Oct 26, 2022

kibanamachine added v8.6.0 backport:skip This commit does not require backporting labels Oct 26, 2022

cauemarcondes mentioned this pull request Nov 28, 2022

[DOCS] Adds the 8.6.0 release notes #145768

Merged

gbamparop mentioned this pull request Feb 15, 2023

[APM] Storage explorer docs are missing cluster monitoring privileges #151302

Closed

gbamparop deleted the apm-storage-explorer-improvements branch May 10, 2023 17:01

	expect(isCrossClusterSearch(mockedSetup)).toBe(false);
	expect(isCrossClusterSearch(mockedSetup)).toBeFalsy()

[APM] Storage explorer improvements #143179

[APM] Storage explorer improvements #143179

Uh oh!

Conversation

gbamparop commented Oct 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

gbamparop commented Oct 12, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gbamparop commented Oct 18, 2022

Uh oh!

elasticmachine commented Oct 18, 2022

Uh oh!

gbamparop commented Oct 19, 2022

Uh oh!

cauemarcondes left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gbamparop Oct 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sorenlouv Oct 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gchaps left a comment

Choose a reason for hiding this comment

Uh oh!

cauemarcondes left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gbamparop commented Oct 26, 2022

Uh oh!

kibana-ci commented Oct 26, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

History

Uh oh!

gbamparop commented Oct 12, 2022 •

edited

Loading

gbamparop Oct 24, 2022 •

edited

Loading

sorenlouv Oct 24, 2022 •

edited

Loading