[refactor] Simplification of Speculative decoding configs

[wili-65535]

Description

An update of PR5296.

• We met a weird error in thread-leak of pytest in the old PR (https://nvidia.slack.com/archives/C059LSY62BT/p1751905741648539).

  • We worked around the error in this PR with a unusual way (https://github.com/NVIDIA/TensorRT-LLM/pull/5639/files#diff-8dae39367d8c7b1d446c04cb21055db2b911c88bf1661a8f1e9a9e696b3cf4a6R160).
  • If we removed class SpecConfig directly, the error would be raised stablely.
  • We will fix it in a later PR.

• Replace class SpecConfig of tensorrt_llm/_torch/speculatice/interface.py into class DecodingBaseConfig of tensorrt_llm/llmapi/llm_args.py.

  • Remove derived classes of class SpecConfig such as DraftTargetConfig, NGramConfig, etc..
  • Some small adjust of class DecodingBaseConfig for related usage.

• Adopt Fanrong's suggestions for the old PR.

• Unify pytorch_weights_path or speculative_model into speculative_model_dir in many places (to align with model_dir).

• Unify max_draft_tokens into max_draft_len in many places.

• Replace prompt_lookup_num_tokens in NGram pytorch workflow into max_draft_len.

  • The remained prompt_lookup_num_tokens is for C++ code, examples/draft_target_model and related tests, we will remove it in a later PR.

• Rewrite speculative decoding tests in tests/unittest/_torch/speculative/ to use a unified code style.

• A small difference than the old PR:

  • We keep the methods update_from_model_config, get_draft_model_prompt, and get_num_extra_kv_tokens as class methods rather than stand-alone tool functions, since I find the errors raise again when I split them out of the class, we may fix later.

Test Coverage

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--disable-fail-fast --skip-test --stage-list "A10-1, xxx" --gpu-type "A30, H100_PCIe" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-[Post-Merge]-1, xxx"]

Launch build/test pipelines. All previously running jobs will be killed.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests. Will also run L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-[Post-Merge]-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-[Post-Merge]-1, xxx".

For guidance on mapping tests to stage names, see docs/source/reference/ci-overview.md.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

[wili-65535]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #11388 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #11388 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #8421 completed with status: 'FAILURE'

[wili-65535]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #11498 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #11498 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #8507 completed with status: 'FAILURE'

[wili-65535]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #11533 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #11533 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #8536 completed with status: 'SUCCESS'

wili-65535: Strange error, all tests passed but the pipeline failed at "Kill previous jobs".

[wili-65535]

/bot run

[tensorrt-cicd]

PR_Github #11554 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #11554 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #8555 completed with status: 'SUCCESS'

[wili-65535]

Finally here we have a healthy commit with pipeline passed for this PR!!!
Ready for review now!

[mikeiovine]

Thanks, let's fix the thread leak stuff in a follow up.

[lucaslie]

Approving AutoDeploy-related changes