gliner.modeling.loss_functions module

gliner.modeling.loss_functions.span_dice_loss(inputs, targets, gamma=1.0, reduction='none', ignore_index=-100, eps=1e-06)[source]

Per-element self-adjusting Dice loss for span-level NER.

Adapted from Li et al. (ACL 2020) “Dice Loss for Data-imbalanced NLP Tasks” to operate element-wise over the (B, L*K, C) span-prediction tensor.

Unlike BCE, Dice is directly optimizing a surrogate of the F1 coefficient and is immune to O-label domination because false negatives and false positives are balanced in the denominator regardless of class prevalence.

The self-adjusting modulator (1-p)^gamma down-weights easy-to-classify negative spans (high confidence, low p), focusing gradient on hard positives.

Loss per element:

adjusted_p = (1 - sigmoid(input))^gamma * sigmoid(input) DSC_adj = (2 * adjusted_p * y + eps) / (adjusted_p + y + eps) loss = 1 - DSC_adj

Parameters:

  • inputs (Tensor) – Predicted logits of arbitrary shape.

  • targets (Tensor) – Ground truth binary labels, same shape as inputs. Values in {0, 1} or ignore_index.

  • gamma (float) – Self-adjustment exponent. Higher values focus more on hard negatives. Defaults to 1.0. Set to 0.0 to recover standard soft Dice.

  • reduction (str) – ‘none’, ‘mean’, or ‘sum’. Defaults to ‘none’.

  • ignore_index (int) – Target values to mask out from the loss. Defaults to -100.

  • eps (float) – Numerical stability epsilon. Defaults to 1e-6.

Returns:

Loss tensor. Shape depends on reduction.

Return type:

Tensor

gliner.modeling.loss_functions.focal_loss_with_logits(inputs, targets, alpha=0.25, gamma=2, prob_margin=0.0, reduction='none', label_smoothing=0.0, normalize_prob=True, ignore_index=-100, eps=1e-06)[source]

Compute focal loss for binary classification with logits.

Focal loss is designed to address class imbalance by down-weighting easy examples and focusing on hard examples. Originally proposed in RetinaNet (https://arxiv.org/abs/1708.02002) for dense object detection.

The focal loss is defined as:

FL(p_t) = -α_t * (1 - p_t)^γ * log(p_t)

where p_t is the model’s estimated probability for the correct class.

Parameters:

  • inputs (torch.Tensor) – Predicted logits of arbitrary shape. The predictions for each example (not probabilities).

  • targets (torch.Tensor) – Ground truth binary labels with the same shape as inputs. Should contain 0 for negative class and 1 for positive class.

  • alpha (float, optional) – Weighting factor in range (0, 1) to balance positive vs negative examples. Set to -1 to disable. The weight for positive examples is alpha, and for negative examples is (1 - alpha). Defaults to 0.25.

  • gamma (float, optional) – Exponent of the modulating factor (1 - p_t) to balance easy vs hard examples. Higher values give more weight to hard examples. Set to 0 to disable focal modulation. Defaults to 2.

  • prob_margin (float, optional) – Margin to subtract from predicted probabilities for negative examples. Can help with hard negative mining. Defaults to 0.

  • reduction (str, optional) – Specifies the reduction to apply to the output. Options are: - ‘none’: No reduction, return loss for each element - ‘mean’: Return the mean loss (normalized by valid elements) - ‘sum’: Return the sum of all losses Defaults to ‘none’.

  • label_smoothing (float, optional) – Amount of label smoothing to apply. Targets become: target * (1 - label_smoothing) + 0.5 * label_smoothing. Value should be in range [0, 1]. Defaults to 0.0 (no smoothing).

  • normalize_prob (bool, optional) – If True, apply sigmoid to inputs to get probabilities. If False, treat inputs as probabilities directly. Defaults to True.

  • ignore_index (int, optional) – Specifies a target value that is ignored and does not contribute to the loss gradient. Defaults to -100.

  • eps (float, optional) – Small epsilon value for numerical stability when computing logarithms. Defaults to 1e-6.

Returns:

Loss tensor. Shape depends on reduction parameter:

  • If reduction=’none’: same shape as inputs

  • If reduction=’mean’ or ‘sum’: scalar tensor

Return type:

torch.Tensor

Raises:

ValueError – If an invalid reduction mode is specified.

Example

>>> inputs = torch.randn(32, 100)  # logits
>>> targets = torch.randint(0, 2, (32, 100)).float()
>>> loss = focal_loss_with_logits(inputs, targets, reduction="mean")
gliner.modeling.loss_functions.cross_entropy_loss(inputs, targets, reduction='sum', label_smoothing=0.0, ignore_index=-100)[source]

Compute cross-entropy loss for multi-class classification.

A wrapper around PyTorch’s F.cross_entropy that reshapes inputs and targets for convenient use in GLiNER models. This function flattens batch and sequence dimensions before computing the loss.

Parameters:

  • inputs (torch.Tensor) – Predicted logits of shape […, num_classes]. Typically shape [batch_size, seq_len, num_classes] or similar. Will be reshaped to [-1, num_classes] internally.

  • targets (torch.Tensor) – Ground truth class indices with shape […]. Should have one fewer dimension than inputs (no class dimension). Values should be in range [0, num_classes - 1] or equal to ignore_index. Will be reshaped to [-1] internally.

  • reduction (str, optional) – Specifies the reduction to apply to the output. Options are: - ‘none’: No reduction, return loss for each element - ‘mean’: Return the mean loss - ‘sum’: Return the sum of all losses Defaults to ‘sum’.

  • label_smoothing (float, optional) – Amount of label smoothing to apply. Value should be in range [0, 1]. Defaults to 0.0 (no smoothing).

  • ignore_index (int, optional) – Specifies a target value that is ignored and does not contribute to the loss gradient. Defaults to -100.

Returns:

Loss tensor. Shape depends on reduction parameter:

  • If reduction=’none’: shape [batch_size * seq_len] (flattened)

  • If reduction=’mean’ or ‘sum’: scalar tensor

Return type:

torch.Tensor

Example

>>> inputs = torch.randn(8, 50, 10)  # batch=8, seq_len=50, classes=10
>>> targets = torch.randint(0, 10, (8, 50))  # class indices
>>> loss = cross_entropy_loss(inputs, targets, reduction="mean")