Attribution Text Analysis
Analyses token-level attributions for text inputs. Extracts the top-k most-attributed words, computes the stop-word ratio of the top-k set, and measures span statistics (mean gap between top-k tokens, number of contiguous runs). Useful for diagnosing whether text explanations focus on meaningful content words.
attribution_text_analysis
attribution_text_analysis(
attributions: tuple[Tensor, ...] | list[tuple[Tensor, ...]],
tokens: list[list[str]],
token_embeddings: list[Tensor] | None = None,
target_indices: list[int] | None = None,
token_labels: list[list[str]] | None = None,
feature_mask: tuple[Tensor, ...] | None = None,
k: int | float = 0.1,
use_weighted_sum: bool = False,
) -> list[list[dict[Any, Any]]] | list[dict[Any, Any]]
Analyses token-level attributions and extracts interpretability diagnostics for text inputs.
For each sample, extracts the top-k most-attributed words, computes the content-to-stopword attribution ratio, measures the mean and max distance of top-k tokens from the target token, and optionally correlates attributions with NER labels or token embeddings.
Parameters:
-
(attributionstuple[Tensor, ...] or list[tuple[Tensor, ...]]) –Attribution tensors, one per modality, each of shape
(batch_size, n_features). For multi-target mode, pass a list of such tuples. -
(tokenslist[list[str]]) –Word-level token strings per sample, length matching the number of attribution feature groups
G. -
(token_embeddingslist[Tensor] or None, default:None) –Precomputed word embeddings per sample, each of shape
(G, hidden_size). When provided, a Spearman correlation between attribution and semantic similarity to the target token is computed. Default:None. -
(target_indiceslist[int] or None, default:None) –Index of the target token per sample. Used to exclude the target from top-k and to compute distance metrics. Default:
None. -
(token_labelslist[list[str]] or None, default:None) –NER or other string label per token per sample. When provided, per-label attribution sums and a label-match correlation are computed. Default:
None. -
(feature_masktuple[Tensor, ...] or None, default:None) –Feature group masks of the same shape as
attributions, used to pool sub-word tokens into word-level groups before scoring. IfNone, each feature is its own group. -
(kint or float, default:0.1) –Number of top tokens to extract. An
intselects exactlyktokens; afloatin(0.0, 1.0]selects that fraction of all tokens. Default:0.1. -
(use_weighted_sumbool, default:False) –If
True, use weighted-sum pooling when reducing features to groups viafeature_mask. Default:False.
Returns:
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–list[dict] or list[list[dict]]: Per-sample result dicts (or a list thereof for multi-target
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–input). Each dict contains:
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–topk_words– list of top-k token strings.
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–topk_scores– normalised attribution scores for each top-k token.
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–topk_indices– token indices of the top-k tokens.
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–topk_is_stopword– boolean list indicating whether each top-k token is a stopword.
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–content_stop_ratio– ratio of mean content-word attribution to mean stopword attribution.
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–topk_mean_dist– mean absolute distance of top-k tokens from the target token.
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–topk_max_dist– maximum absolute distance of top-k tokens from the target token.
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–ner– per-label attribution sums (only whentoken_labelsis provided).
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–ner_corr– Spearman correlation between label match and attribution (only whentoken_labelsandtarget_indicesare provided).
-
list[list[dict[Any, Any]]] | list[dict[Any, Any]]–semantic_corr– Spearman correlation between cosine similarity to the target embedding and attribution (only whentoken_embeddingsandtarget_indicesare provided).
Example
import torch attr = (torch.tensor([[0.05, 0.4, 0.1, 0.3, 0.15]]),) tokens_batch = [["The", "cat", "sat", "on", "mat"]] results = attribution_text_analysis(attr, tokens=tokens_batch, k=0.4) results[0]["topk_words"] ['cat', 'on']
Source code in torchxai/metrics/diagnosis/attribution_text_analysis.py
254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 | |