Search results

[Bug] Zero-std reward groups produce spurious KL gradients when beta > 0 in GRPO/RLOO

Status: Open.

#5588 In huggingface/trl;

· SwayamInSync opened on Apr 18, 2026

[Bug] DistillationTrainer fails with Qwen3.5 due to nested config.vocab_size attribute

Status: Open.

#5585 In huggingface/trl;

· ZephronCoder opened on Apr 17, 2026

Add entropy regularization (e.g., initial_entropy_coef) to GRPOTrainer for improved exploration

Status: Open.

#5584 In huggingface/trl;

· lailanelkoussy opened on Apr 17, 2026

fix: DataCollatorForRewardModelingDataset checks margin only on examples[0], causing silent data loss after shuffle

Status: Open.

#5539 In huggingface/trl;

· kuishou68 opened on Apr 14, 2026

OnlineDPOTrainer._generate_vllm_server() flattens vllm-serve completion_ids twice

Status: Open.

#5514 In huggingface/trl;

· JohnGiorgi opened on Apr 10, 2026

Expose trainer dataset type metadata

Status: Open.

#5511 In huggingface/trl;

· JohnGiorgi opened on Apr 10, 2026

[AsyncGRPO] AsyncGRPOTrainer ignores provided processing_class when initializing AsyncRolloutWorker

Status: Open.

#5498 In huggingface/trl;

· xuanduy04 opened on Apr 10, 2026

CI fails with dev dependencies for Llava models: AssertionError: Param is not updated

Status: Open.

Bug

#5497 In huggingface/trl;

· albertvillanova opened on Apr 10, 2026

Ascend NPU Error with CANN 8.5.0 and TRL 0.25.1

Status: Open.

#5495 In huggingface/trl;

· shibizhao opened on Apr 10, 2026

Tracking: Add {% generation %} chat templates for common model families

Status: Open.

#5471 In huggingface/trl;

· qgallouedec opened on Apr 7, 2026

Tracking: tool calling support across chat templates

Status: Open.

#5460 In huggingface/trl;

· qgallouedec opened on Apr 6, 2026

AsyncGRPO: support async tool calls in AsyncRolloutWorker

Status: Open.

#5444 In huggingface/trl;

· PoilZero opened on Apr 3, 2026