Skip to content

test: update test filter in perf test yml file to select cases by gpu name and add cases for RTX 6000 pro #4282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 20, 2025

Conversation

ruodil
Copy link
Collaborator

@ruodil ruodil commented May 14, 2025

PR title

Please write the PR title by following template:

[JIRA ticket link/nvbug link/github issue link][fix/feat/doc/infra/...] <summary of this PR>

For example, assume I have a PR hope to support a new feature about cache manager of Jira TRTLLM-1000 ticket, it would be like

[TRTLLM-1000][feat] Support a new feature about cache manager

Description

  1. update test filter by adding gpu configuration
  2. add mixtral_8x7b_v0.18_fp8 model path, lama_v3.1_70b_instruct_fp8 model path and llama_v3.1_8b_instruct_fp4 model path, correct model path for llama_v3.1_405b_instruct_fp4
  3. add more cases for RTX Pro 6000 SE according to this doc and add some case into waive list.

Test Coverage

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--disable-fail-fast --skip-test --stage-list "A10-1, xxx" --gpu-type "A30, H100_PCIe" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-[Post-Merge]-1, xxx"]

Launch build/test pipelines. All previously running jobs will be killed.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests. Will also run L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-[Post-Merge]-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-[Post-Merge]-1, xxx".

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

@ruodil ruodil requested review from LarryXFly and venkywonka May 14, 2025 02:25
@ruodil ruodil force-pushed the user/ruodil/test_perf branch from 1baaf77 to 2976fdf Compare May 14, 2025 08:02
@ruodil
Copy link
Collaborator Author

ruodil commented May 14, 2025

/bot run

@ruodil ruodil force-pushed the user/ruodil/test_perf branch from 2976fdf to a932934 Compare May 14, 2025 08:03
@tensorrt-cicd
Copy link
Collaborator

PR_Github #5146 [ run ] triggered by Bot

@venkywonka
Copy link
Collaborator

venkywonka commented May 14, 2025

This PR seems to do multiple things but the PR subject says only a portion of it. The PR description says nothing - would help if you could add that thanks!

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5146 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #3753 completed with status: 'SUCCESS'

@ruodil ruodil force-pushed the user/ruodil/test_perf branch from a932934 to bda0f02 Compare May 15, 2025 03:12
@ruodil
Copy link
Collaborator Author

ruodil commented May 15, 2025

This PR seems to do multiple things but the PR subject says only a portion of it. The PR description says nothing - would help if you could add that thanks!

add more detailed info in Desciprtion, thanks!

@ruodil ruodil force-pushed the user/ruodil/test_perf branch 2 times, most recently from 6551936 to 0a31e5a Compare May 19, 2025 01:10
Signed-off-by: Ruodi <200874449+ruodil@users.noreply.github.com>
@ruodil ruodil force-pushed the user/ruodil/test_perf branch 3 times, most recently from e0a68d0 to a11c0a4 Compare May 19, 2025 07:24
@ruodil ruodil changed the title test: update test filter in perf test yml file to select cases by gpu name test: update test filter in perf test yml file to select cases by gpu name and add cases for RTX 6000 pro May 19, 2025
@ruodil ruodil force-pushed the user/ruodil/test_perf branch from a11c0a4 to b786af1 Compare May 20, 2025 02:53
@LarryXFly LarryXFly merged commit b5edf13 into NVIDIA:main May 20, 2025
2 checks passed
ruodil and others added 2 commits May 20, 2025 03:00
… for rtx pro 6000 and waive_list

Signed-off-by: Ruodi <200874449+ruodil@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants