gliner.modeling.span_rep module

class gliner.modeling.span_rep.SpanQuery(hidden_size, max_width, trainable=True)

Bases: Module

Span representation using learned query vectors.

This layer learns a set of query vectors, one for each span width, and projects token representations onto these queries to produce span representations.

query_seg

Learnable query matrix of shape [hidden_size, max_width].

Type:

nn.Parameter

project

MLP projection layer with ReLU activation.

Type:

nn.Sequential

Initialize the SpanQuery layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• trainable (bool, optional) – Whether query parameters are trainable. Defaults to True.

__init__(hidden_size, max_width, trainable=True)

Initialize the SpanQuery layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• trainable (bool, optional) – Whether query parameters are trainable. Defaults to True.

forward(h, *args)

Compute span representations using query projection.

Parameters:

• h (torch.Tensor) – Token representations of shape [B, L, D].

• *args – Additional arguments (unused).

Returns:

Span representations of shape [B, L, max_width, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.SpanMLP(hidden_size, max_width)

Bases: Module

Span representation using a simple MLP.

This layer applies a linear transformation to produce multiple span representations per position.

mlp

Linear layer that expands hidden_size to hidden_size * max_width.

Type:

nn.Linear

Initialize the SpanMLP layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

__init__(hidden_size, max_width)

Initialize the SpanMLP layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

forward(h, *args)

Compute span representations using MLP projection.

Parameters:

• h (torch.Tensor) – Token representations of shape [B, L, D].

• *args – Additional arguments (unused).

Returns:

Span representations of shape [B, L, max_width, D]

with ReLU activation applied.

Return type:

torch.Tensor

class gliner.modeling.span_rep.SpanCAT(hidden_size, max_width)

Bases: Module

Span representation using concatenation with learned queries.

This layer concatenates token representations with learnable query vectors and projects them to produce span representations.

max_width

Maximum span width to represent.

Type:

int

query_seg

Learnable query matrix of shape [128, max_width].

Type:

nn.Parameter

project

MLP projection layer with ReLU activation.

Type:

nn.Sequential

Initialize the SpanCAT layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

__init__(hidden_size, max_width)

Initialize the SpanCAT layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

forward(h, *args)

Compute span representations by concatenating with queries.

Parameters:

• h (torch.Tensor) – Token representations of shape [B, L, D].

• *args – Additional arguments (unused).

Returns:

Span representations of shape [B, L, max_width, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.SpanConvBlock(hidden_size, kernel_size, span_mode='conv_normal')

Bases: Module

A single convolutional block for span representation.

This block applies either convolution or pooling operations with a specific kernel size to capture span information.

conv

Convolution or pooling layer.

Type:

nn.Module

span_mode

Type of operation (‘conv_conv’, ‘conv_max’, ‘conv_mean’, ‘conv_sum’).

Type:

str

pad

Padding size for the operation.

Type:

int

Initialize the SpanConvBlock.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• kernel_size (int) – Size of the convolution/pooling kernel.

• span_mode (str, optional) – Type of operation to use. Options are: ‘conv_conv’, ‘conv_max’, ‘conv_mean’, ‘conv_sum’. Defaults to ‘conv_normal’.

__init__(hidden_size, kernel_size, span_mode='conv_normal')

Initialize the SpanConvBlock.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• kernel_size (int) – Size of the convolution/pooling kernel.

• span_mode (str, optional) – Type of operation to use. Options are: ‘conv_conv’, ‘conv_max’, ‘conv_mean’, ‘conv_sum’. Defaults to ‘conv_normal’.

forward(x)

Apply the convolutional block.

Parameters:

x (torch.Tensor) – Input tensor of shape [B, L, D].

Returns:

Output tensor of shape [B, L, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.SpanConv(hidden_size, max_width, span_mode)

Bases: Module

Span representation using multiple convolutional layers.

This layer uses convolutions with different kernel sizes to capture spans of different widths.

convs

List of convolutional blocks with varying kernel sizes.

Type:

nn.ModuleList

project

MLP projection layer with ReLU activation.

Type:

nn.Sequential

Initialize the SpanConv layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• span_mode (str) – Type of convolution operation to use.

__init__(hidden_size, max_width, span_mode)

Initialize the SpanConv layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• span_mode (str) – Type of convolution operation to use.

forward(x, *args)

Compute span representations using multiple convolutions.

Parameters:

• x (torch.Tensor) – Input tensor of shape [B, L, D].

• *args – Additional arguments (unused).

Returns:

Span representations of shape [B, L, max_width, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.SpanEndpointsBlock(kernel_size)

Bases: Module

Extract start and end token representations for spans.

This block extracts the first and last token of each span.

kernel_size

The span width (kernel size).

Type:

int

Initialize the SpanEndpointsBlock.

Parameters:

kernel_size (int) – The span width to extract endpoints for.

__init__(kernel_size)

Initialize the SpanEndpointsBlock.

Parameters:

kernel_size (int) – The span width to extract endpoints for.

forward(x)

Extract start and end representations for all spans.

Parameters:

x (torch.Tensor) – Input tensor of shape [B, L, D].

Returns:

Start and end representations of shape [B, L, 2, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.ConvShare(hidden_size, max_width)

Bases: Module

Span representation using shared convolution weights.

This layer uses a single set of convolution weights shared across different span widths.

max_width

Maximum span width to represent.

Type:

int

conv_weigth

Shared convolution weights of shape [hidden_size, hidden_size, max_width].

Type:

nn.Parameter

project

MLP projection layer with ReLU activation.

Type:

nn.Sequential

Initialize the ConvShare layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

__init__(hidden_size, max_width)

Initialize the ConvShare layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

forward(x, *args)

Compute span representations using shared convolutions.

Parameters:

• x (torch.Tensor) – Input tensor of shape [B, L, D].

• *args – Additional arguments (unused).

Returns:

Span representations of shape [B, L, max_width, D].

Return type:

torch.Tensor

gliner.modeling.span_rep.extract_elements(sequence, indices)

Extract elements from a sequence using provided indices.

Parameters:

• sequence (torch.Tensor) – Input sequence of shape [B, L, D].

• indices (torch.Tensor) – Indices to extract, shape [B, K].

Returns:

Extracted elements of shape [B, K, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.SpanMarker(hidden_size, max_width, dropout=0.4)

Bases: Module

Span representation using marker-based approach.

This layer projects start and end positions separately and combines them to form span representations.

max_width

Maximum span width to represent.

Type:

int

project_start

MLP for projecting start positions.

Type:

nn.Sequential

project_end

MLP for projecting end positions.

Type:

nn.Sequential

out_project

Final projection layer.

Type:

nn.Linear

Initialize the SpanMarker layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• dropout (float, optional) – Dropout rate. Defaults to 0.4.

__init__(hidden_size, max_width, dropout=0.4)

Initialize the SpanMarker layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• dropout (float, optional) – Dropout rate. Defaults to 0.4.

forward(h, span_idx)

Compute span representations using start and end markers.

Parameters:

• h (torch.Tensor) – Token representations of shape [B, L, D].

• span_idx (torch.Tensor) – Span indices of shape [B, *, 2] where span_idx[…, 0] are start indices and span_idx[…, 1] are end indices.

Returns:

Span representations of shape [B, L, max_width, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.SpanMarkerV0(hidden_size, max_width, dropout=0.4)

Bases: Module

Marks and projects span endpoints using an MLP.

A cleaner version of SpanMarker using the create_projection_layer utility.

max_width

Maximum span width to represent.

Type:

int

project_start

MLP for projecting start positions.

Type:

nn.Module

project_end

MLP for projecting end positions.

Type:

nn.Module

out_project

Final projection layer.

Type:

nn.Module

Initialize the SpanMarkerV0 layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• dropout (float, optional) – Dropout rate. Defaults to 0.4.

__init__(hidden_size, max_width, dropout=0.4)

Initialize the SpanMarkerV0 layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• dropout (float, optional) – Dropout rate. Defaults to 0.4.

forward(h, span_idx)

Compute span representations using start and end markers.

Parameters:

• h (torch.Tensor) – Token representations of shape [B, L, D].

• span_idx (torch.Tensor) – Span indices of shape [B, *, 2].

Returns:

Span representations of shape [B, L, max_width, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.SpanMarkerV1(hidden_size, max_width, dropout=0.4)

Bases: Module

Marks span endpoints and augments them with the first-token embedding.

For each candidate span we build

[ start_proj ‖ end_proj ‖ first_token_proj ] → MLP → span_rep

and finally reshape to [B, L, max_width, D].

max_width

Maximum span width to represent.

Type:

int

project_start

MLP for projecting start positions.

Type:

nn.Module

project_end

MLP for projecting end positions.

Type:

nn.Module

project_first

MLP for projecting the average token.

Type:

nn.Module

out_project

Final projection layer.

Type:

nn.Module

Initialize the SpanMarkerV1 layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• dropout (float, optional) – Dropout rate. Defaults to 0.4.

__init__(hidden_size, max_width, dropout=0.4)

Initialize the SpanMarkerV1 layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• dropout (float, optional) – Dropout rate. Defaults to 0.4.

forward(h, span_idx)

Compute span representations with average token augmentation.

For each span, concatenates start marker, end marker, and average token embedding, then projects to produce the final representation.

Parameters:

• h (torch.Tensor) – Token representations, shape [B, L, D].

• span_idx (torch.Tensor) – Indices of candidate spans, shape [B, , 2] ( can be L x max_width or any flattened span dimension).

Returns:

Span representations, shape [B, L, max_width, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.ConvShareV2(hidden_size, max_width)

Bases: Module

Span representation using shared convolution weights (version 2).

Similar to ConvShare but uses Xavier initialization and no projection layer.

max_width

Maximum span width to represent.

Type:

int

conv_weigth

Shared convolution weights of shape [hidden_size, hidden_size, max_width].

Type:

nn.Parameter

Initialize the ConvShareV2 layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

__init__(hidden_size, max_width)

Initialize the ConvShareV2 layer.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

forward(x, *args)

Compute span representations using shared convolutions.

Parameters:

• x (torch.Tensor) – Input tensor of shape [B, L, D].

• *args – Additional arguments (unused).

Returns:

Span representations of shape [B, L, max_width, D].

Return type:

torch.Tensor

class gliner.modeling.span_rep.SpanRepLayer(hidden_size, max_width, span_mode, **kwargs)

Bases: Module

Factory class for various span representation approaches.

This class provides a unified interface to instantiate different span representation methods based on the specified mode.

span_rep_layer

The underlying span representation layer.

Type:

nn.Module

Initialize the SpanRepLayer with the specified mode.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• span_mode (str) – Type of span representation to use. Options: - ‘marker’: SpanMarker - ‘markerV0’: SpanMarkerV0 - ‘markerV1’: SpanMarkerV1 - ‘query’: SpanQuery - ‘mlp’: SpanMLP - ‘cat’: SpanCAT - ‘conv_conv’: SpanConv with convolution - ‘conv_max’: SpanConv with max pooling - ‘conv_mean’: SpanConv with mean pooling - ‘conv_sum’: SpanConv with sum pooling - ‘conv_share’: ConvShare

• **kwargs – Additional arguments passed to the span representation layer.

Raises:

ValueError – If an unknown span_mode is provided.

__init__(hidden_size, max_width, span_mode, **kwargs)

Initialize the SpanRepLayer with the specified mode.

Parameters:

• hidden_size (int) – Dimension of the hidden representations.

• max_width (int) – Maximum span width to represent.

• span_mode (str) – Type of span representation to use. Options: - ‘marker’: SpanMarker - ‘markerV0’: SpanMarkerV0 - ‘markerV1’: SpanMarkerV1 - ‘query’: SpanQuery - ‘mlp’: SpanMLP - ‘cat’: SpanCAT - ‘conv_conv’: SpanConv with convolution - ‘conv_max’: SpanConv with max pooling - ‘conv_mean’: SpanConv with mean pooling - ‘conv_sum’: SpanConv with sum pooling - ‘conv_share’: ConvShare

• **kwargs – Additional arguments passed to the span representation layer.

Raises:

ValueError – If an unknown span_mode is provided.

forward(x, *args)

Forward pass through the selected span representation layer.

Parameters:

• x (torch.Tensor) – Input tensor, typically of shape [B, L, D].

• *args – Additional arguments passed to the underlying layer.

Returns:

Span representations, typically of shape

[B, L, max_width, D].

Return type:

torch.Tensor