GradientShapExplainer

GradientShap explainer for computing noise-based Shapley value approximations.

This explainer computes attributions using GradientShap, which combines ideas from Integrated Gradients and SHAP. It uses a distribution of baselines rather than a single baseline and adds noise to create more robust attribution estimates. The method approximates Shapley values through gradient-based computations. Supports both single-target and multi-target modes for both single-target and multi-target scenarios.

GradientShap provides more robust attributions by using baseline distributions and noise, making it less sensitive to specific baseline choices.

Parameters:

  • model

    (Module) –

    The PyTorch model whose output is to be explained.

  • multi_target

    (bool, default: False ) –

    Whether to use multi-target mode. When True, can compute attributions for multiple targets simultaneously. Defaults to False.

  • internal_batch_size

    (int, default: 1 ) –

    Batch size for internal computations. Defaults to 64.

  • grad_batch_size

    (int, default: 64 ) –

    Batch size for gradient computations. Defaults to 64.

  • n_samples

    (int, default: 25 ) –

    Number of random samples used to approximate Shapley values. Defaults to 25.

  • return_convergence_delta

    (bool, default: False ) –

    Whether to return convergence delta for completeness check. Defaults to False.

Examples:

Single-target usage:

>>> import torch
>>> from torchxai.data_types import SingleTargetAcrossBatch
>>>
>>> model = torch.nn.Linear(10, 2)
>>> explainer = GradientShapExplainer(model)
>>> inputs        = torch.randn(1, 10)
>>> baselines_dist = torch.zeros(1, 10).expand(5, -1)   # 5-sample reference distribution
>>> attributions = explainer.explain(
...     inputs=inputs,
...     baselines=baselines_dist,
...     target=SingleTargetAcrossBatch(index=0),
... )
>>> attributions.shape   # (1, 10)

Multi-target usage:

>>> explainer_mt = GradientShapExplainer(model, multi_target=True)
>>> mt_attributions = explainer_mt.explain(
...     inputs=inputs,
...     baselines=baselines_dist,
...     target=[SingleTargetAcrossBatch(index=0), SingleTargetAcrossBatch(index=1)],
... )
>>> len(mt_attributions), mt_attributions[0].shape   # 2, (1, 10)

Methods:

  • explain

    Compute GradientShap attributions for the given inputs.

Source code in torchxai/explainers/_grad/_gradient_shap.py
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
class GradientShapExplainer(FeatureAttributionExplainer):
    """GradientShap explainer for computing noise-based Shapley value approximations.

    This explainer computes attributions using GradientShap, which combines ideas from
    Integrated Gradients and SHAP. It uses a distribution of baselines rather than a
    single baseline and adds noise to create more robust attribution estimates. The method
    approximates Shapley values through gradient-based computations. Supports both
    single-target and multi-target modes for both single-target and multi-target scenarios.

    GradientShap provides more robust attributions by using baseline distributions
    and noise, making it less sensitive to specific baseline choices.

    Args:
        model: The PyTorch model whose output is to be explained.
        multi_target: Whether to use multi-target mode. When True, can compute
            attributions for multiple targets simultaneously. Defaults to False.
        internal_batch_size: Batch size for internal computations. Defaults to 64.
        grad_batch_size: Batch size for gradient computations. Defaults to 64.
        n_samples: Number of random samples used to approximate Shapley values.
            Defaults to 25.
        return_convergence_delta: Whether to return convergence delta for
            completeness check. Defaults to False.

    Examples:
        Single-target usage:
        >>> import torch
        >>> from torchxai.data_types import SingleTargetAcrossBatch
        >>>
        >>> model = torch.nn.Linear(10, 2)
        >>> explainer = GradientShapExplainer(model)
        >>> inputs        = torch.randn(1, 10)
        >>> baselines_dist = torch.zeros(1, 10).expand(5, -1)   # 5-sample reference distribution
        >>> attributions = explainer.explain(
        ...     inputs=inputs,
        ...     baselines=baselines_dist,
        ...     target=SingleTargetAcrossBatch(index=0),
        ... )
        >>> attributions.shape   # (1, 10)

        Multi-target usage:
        >>> explainer_mt = GradientShapExplainer(model, multi_target=True)
        >>> mt_attributions = explainer_mt.explain(
        ...     inputs=inputs,
        ...     baselines=baselines_dist,
        ...     target=[SingleTargetAcrossBatch(index=0), SingleTargetAcrossBatch(index=1)],
        ... )
        >>> len(mt_attributions), mt_attributions[0].shape   # 2, (1, 10)
    """

    __repr_attrs__ = [
        "_multi_target",
        "_internal_batch_size",
        "_grad_batch_size",
        "_n_samples",
        "_return_convergence_delta",
        "_stdevs",
    ]

    def __init__(
        self,
        model: Module,
        multi_target: bool = False,
        internal_batch_size: int = 1,
        grad_batch_size: int = 64,
        n_samples: int = 25,
        stddevs: float = 0.0,
        return_convergence_delta: bool = False,
    ) -> None:
        """Initialize the GradientShapExplainer.

        Args:
            model: The model whose output is to be explained.
            multi_target: Whether to use multi-target mode. Defaults to False.
            internal_batch_size: Batch size for internal computations. Defaults to 1.
            grad_batch_size: Batch size for gradient computations. Defaults to 64.
            n_samples: Number of random samples for Shapley value approximation.
                Defaults to 25.
            return_convergence_delta: Whether to return convergence delta for
                completeness check. Defaults to False.
        """
        self._n_samples = n_samples
        self._return_convergence_delta = return_convergence_delta
        self._stdevs = stddevs

        super().__init__(
            model, multi_target, internal_batch_size, grad_batch_size=grad_batch_size
        )

    def _init_single_target_explanation_fn(self) -> Callable:
        """Initialize single-target GradientShap attribution function.

        Returns:
            Custom GradientShap attribution function for single targets.
        """
        return partial(
            GradientShap_(self._model).attribute,
            n_samples=self._n_samples,
            n_samples_batch_size=self._internal_batch_size,
            return_convergence_delta=self._return_convergence_delta,
            stdevs=self._stdevs,
        )

    def _init_multi_target_explanation_fn(self) -> Callable:
        """Initialize multi-target GradientShap attribution function.

        Returns:
            MultiTargetGradientShap attribution function for multiple targets.
        """
        return partial(
            MultiTargetGradientShap(
                self._model, grad_batch_size=self._grad_batch_size
            ).attribute,
            n_samples=self._n_samples,
            n_samples_batch_size=self._internal_batch_size,
            return_convergence_delta=self._return_convergence_delta,
            stdevs=self._stdevs,
        )

    def explain(
        self,
        inputs: TensorOrTupleOfTensorsGeneric,
        target: ExplanationTargetType | list[ExplanationTargetType],
        baselines: TensorOrTupleOfTensorsGeneric | None = None,
        additional_forward_args: tuple[Any, ...] | None = None,
    ) -> TensorOrTupleOfTensorsGeneric | list[TensorOrTupleOfTensorsGeneric]:
        """Compute GradientShap attributions for the given inputs.

        Args:
            inputs: Input tensor(s) for attribution computation.
            target: An `ExplanationTargetType` (e.g. `SingleTargetAcrossBatch`) for single-target
                mode, or a list of them for multi-target mode.
            baselines: Baseline distribution for GradientShap. Must be provided as
                a tensor distribution or callable that generates baseline samples.
            additional_forward_args: Additional arguments for model forward pass.

        Returns:
            Tensor in single-target mode. List of Tensors, one per target, in multi-target mode.

        Note:
            GradientShap requires a distribution of baselines rather than a single baseline.
            The number of samples and noise parameters are controlled by initialization settings.

        Examples:
            >>> # With baseline distribution
            >>> baseline_dist = torch.randn(50, 10)  # 50 baseline samples
            >>> attributions = explainer.explain(
            ...     inputs=OrderedDict({"input": torch.randn(2, 10)}),
            ...     target=torch.tensor([0, 1]),
            ...     baselines=OrderedDict({"input": baseline_dist}),
            ... )
        """
        return self._default_explain(
            inputs=inputs,
            target=target,
            baselines=baselines,
            additional_forward_args=additional_forward_args,
        )

explain

explain(
    inputs: TensorOrTupleOfTensorsGeneric,
    target: ExplanationTargetType | list[ExplanationTargetType],
    baselines: TensorOrTupleOfTensorsGeneric | None = None,
    additional_forward_args: tuple[Any, ...] | None = None,
) -> TensorOrTupleOfTensorsGeneric | list[TensorOrTupleOfTensorsGeneric]

Compute GradientShap attributions for the given inputs.

Parameters:

  • inputs
    (TensorOrTupleOfTensorsGeneric) –

    Input tensor(s) for attribution computation.

  • target
    (ExplanationTargetType | list[ExplanationTargetType]) –

    An ExplanationTargetType (e.g. SingleTargetAcrossBatch) for single-target mode, or a list of them for multi-target mode.

  • baselines
    (TensorOrTupleOfTensorsGeneric | None, default: None ) –

    Baseline distribution for GradientShap. Must be provided as a tensor distribution or callable that generates baseline samples.

  • additional_forward_args
    (tuple[Any, ...] | None, default: None ) –

    Additional arguments for model forward pass.

Returns:

  • TensorOrTupleOfTensorsGeneric | list[TensorOrTupleOfTensorsGeneric]

    Tensor in single-target mode. List of Tensors, one per target, in multi-target mode.

Note

GradientShap requires a distribution of baselines rather than a single baseline. The number of samples and noise parameters are controlled by initialization settings.

Examples:

>>> # With baseline distribution
>>> baseline_dist = torch.randn(50, 10)  # 50 baseline samples
>>> attributions = explainer.explain(
...     inputs=OrderedDict({"input": torch.randn(2, 10)}),
...     target=torch.tensor([0, 1]),
...     baselines=OrderedDict({"input": baseline_dist}),
... )
Source code in torchxai/explainers/_grad/_gradient_shap.py
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
def explain(
    self,
    inputs: TensorOrTupleOfTensorsGeneric,
    target: ExplanationTargetType | list[ExplanationTargetType],
    baselines: TensorOrTupleOfTensorsGeneric | None = None,
    additional_forward_args: tuple[Any, ...] | None = None,
) -> TensorOrTupleOfTensorsGeneric | list[TensorOrTupleOfTensorsGeneric]:
    """Compute GradientShap attributions for the given inputs.

    Args:
        inputs: Input tensor(s) for attribution computation.
        target: An `ExplanationTargetType` (e.g. `SingleTargetAcrossBatch`) for single-target
            mode, or a list of them for multi-target mode.
        baselines: Baseline distribution for GradientShap. Must be provided as
            a tensor distribution or callable that generates baseline samples.
        additional_forward_args: Additional arguments for model forward pass.

    Returns:
        Tensor in single-target mode. List of Tensors, one per target, in multi-target mode.

    Note:
        GradientShap requires a distribution of baselines rather than a single baseline.
        The number of samples and noise parameters are controlled by initialization settings.

    Examples:
        >>> # With baseline distribution
        >>> baseline_dist = torch.randn(50, 10)  # 50 baseline samples
        >>> attributions = explainer.explain(
        ...     inputs=OrderedDict({"input": torch.randn(2, 10)}),
        ...     target=torch.tensor([0, 1]),
        ...     baselines=OrderedDict({"input": baseline_dist}),
        ... )
    """
    return self._default_explain(
        inputs=inputs,
        target=target,
        baselines=baselines,
        additional_forward_args=additional_forward_args,
    )