Tests that attributions remain unchanged when a constant shift is added to inputs and the model bias is adjusted to compensate — a formal axiom from Kindermans et al.

input_invariance

Functions:

  • input_invariance

    Implementation of Input Invariance test by Kindermans et al., 2017. This implementation

input_invariance

input_invariance(
    explainer: FeatureAttributionExplainer | Attribution,
    inputs: TensorOrTupleOfTensorsGeneric,
    constant_shifts: TensorOrTupleOfTensorsGeneric,
    input_layer_names: tuple[str],
    multi_target: bool = False,
    return_intermediate_results: bool = False,
    return_dict: bool = False,
    baselines: TensorOrTupleOfTensorsGeneric | None = None,
    shift_baselines: TensorOrTupleOfTensorsGeneric | None = None,
    feature_mask: TensorOrTupleOfTensorsGeneric | None = None,
    additional_forward_args: Any = None,
    target: ExplanationTarget | list[ExplanationTarget] = NoTarget(),
    sliding_window_shapes: Any | None = None,
    strides: Any | None = None,
    **kwargs,
) -> dict | tuple | Tensor | list[Tensor]

Implementation of Input Invariance test by Kindermans et al., 2017. This implementation reuses the batch-computation ideas from captum and therefore it is fully compatible with the Captum library. In addition, the implementation takes some ideas about the implementation of the metric from the python Quantus library.

To test for input invariance, we add a constant shift to the input data and a mean shift to the model bias, so that the output of the original model on the original data is equal to the output of the changed model on the shifted data. The metric returns True if batch attributions stayed unchanged too. Currently only supporting constant values for the shift.

References

Pieter-Jan Kindermans et al.: "The (Un)reliability of Saliency Methods." Explainable AI (2019): 267-280

Args:

explainer (Union[FusionExplainer, Attribution]): The explainer instance that is used to
        compute the explanations. The explainer must be an instance of either captum Attribution class or
        the FusionExplainer instance.

inputs (Tensor or tuple[Tensor, ...]): Input for which
        explanations are computed. If `explainer` takes a
        single tensor as input, a single input tensor should
        be provided.
        If `explainer` takes multiple tensors as input, a tuple
        of the input tensors should be provided. It is assumed
        that for all given input tensors, dimension 0 corresponds
        to the number of examples (aka batch size), and if
        multiple input tensors are provided, the examples must
        be aligned appropriately.

constant_shifts (Tensor or tuple[Tensor, ...]): Constant shifts defined for each input tensor.
        If `inputs` is single tensor, a single constant_shifts tensor should be provided.
        If `inputs` consists of multiple tensors, a tuple
        of the constant_shifts tensors should be provided. It is assumed
        that for all given input tensors, dimension 0 corresponds
        to the number of examples (aka batch size), and if
        multiple input tensors are provided, the examples must
        be aligned appropriately. This constant_shift is subtracted from the input tensor and is used
        as an input baseline to shift the bias of the model in `create_model_with_shifted_bias`.
        Note that this is opposite to the original paper where the constant_shift is added to the input)

input_layer_names (List[str]): The names of the input layers of the model that should be shifted. Each layer
        should be unique for each input constant shift tensor.

delta (float, optional): The absolute tolerance parameter for the allclose function which checks whether
    the explanations generated for original inputs and shifted model and inptus are equal.
    Default is 1e-8.

multi_target (bool, optional): A boolean flag that indicates whether the metric computation is for
        multi-target explanations. if set to true, the targets are required to be a list of integers
        each corresponding to a required target class in the output. The corresponding metric outputs
        are then returned as a list of metric outputs corresponding to each target class.
        Default is False.
return_intermediate_results (bool, optional): A boolean flag that indicates whether the intermediate results
        of the metric computation are returned.
        Default is False.
return_dict (bool, optional): A boolean flag that indicates whether the metric outputs are returned as a dictionary
        with keys as the metric names and values as the corresponding metric outputs.
        Default is False.
**kwargs (Any, optional): Contains a list of arguments that are passed
        to `explanation_func` explanation function which in some cases
        could be the `attribute` function of an attribution algorithm.
        Any additional arguments that need be passed to the explanation
        function should be included here.
        For instance, such arguments include:
        `additional_forward_args`, `baselines` and `target`.

Returns:

input_invariance (Tensor): A boolean tensor of per
    input example showing whether the explanation is invariant to the input.
    The output dimension is equal to the number of examples in the input batch.
inputs_expl (Tensor or tuple[Tensor, ...]): The explanation for the original input.
    If `inputs` is a single tensor, a single tensor is returned.
    If `inputs` consists of multiple tensors, a tuple of tensors is returned.
shifted_inputs_expl (Tensor or tuple[Tensor, ...]): The explanation generated for the shifted input and
    the shifted model.
    If `inputs` is a single tensor, a single tensor is returned.
    If `inputs` consists of multiple tensors, a tuple of tensors is returned.

Examples:: >>> # ImageClassifier takes a single input tensor of images Nx3x32x32, >>> # and returns an Nx10 tensor of class probabilities. >>> net = ImageClassifier() >>> saliency = Saliency(net) >>> input = torch.randn(2, 3, 32, 32, requires_grad=True) >>> # Computes sensitivity score for saliency maps of class 3 >>> input_invarance_score, inputs_expl, shifted_inputs_expl = input_invarance(saliency, input, target = 3)

Source code in torchxai/metrics/axiomatic/input_invariance.py
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
def input_invariance(
    explainer: FeatureAttributionExplainer | Attribution,
    inputs: TensorOrTupleOfTensorsGeneric,
    constant_shifts: TensorOrTupleOfTensorsGeneric,
    input_layer_names: tuple[str],
    multi_target: bool = False,
    return_intermediate_results: bool = False,
    return_dict: bool = False,
    # explainer kwargs
    baselines: TensorOrTupleOfTensorsGeneric | None = None,
    shift_baselines: TensorOrTupleOfTensorsGeneric | None = None,
    feature_mask: TensorOrTupleOfTensorsGeneric | None = None,
    additional_forward_args: Any = None,
    target: ExplanationTarget | list[ExplanationTarget] = NoTarget(),
    sliding_window_shapes: Any | None = None,
    strides: Any | None = None,
    **kwargs,
) -> dict | tuple | torch.Tensor | list[torch.Tensor]:
    """
    Implementation of Input Invariance test by Kindermans et al., 2017. This implementation
    reuses the batch-computation ideas from captum and therefore it is fully compatible with the Captum library.
    In addition, the implementation takes some ideas about the implementation of the metric from the python
    Quantus library.

    To test for input invariance, we add a constant shift to the input data and a mean shift to the model bias,
    so that the output of the original model on the original data is equal to the output of the changed model
    on the shifted data. The metric returns True if batch attributions stayed unchanged too. Currently only
    supporting constant values for the shift.

    References:
        Pieter-Jan Kindermans et al.: "The (Un)reliability of Saliency Methods." Explainable AI (2019): 267-280

    Args:

        explainer (Union[FusionExplainer, Attribution]): The explainer instance that is used to
                compute the explanations. The explainer must be an instance of either captum Attribution class or
                the FusionExplainer instance.

        inputs (Tensor or tuple[Tensor, ...]): Input for which
                explanations are computed. If `explainer` takes a
                single tensor as input, a single input tensor should
                be provided.
                If `explainer` takes multiple tensors as input, a tuple
                of the input tensors should be provided. It is assumed
                that for all given input tensors, dimension 0 corresponds
                to the number of examples (aka batch size), and if
                multiple input tensors are provided, the examples must
                be aligned appropriately.

        constant_shifts (Tensor or tuple[Tensor, ...]): Constant shifts defined for each input tensor.
                If `inputs` is single tensor, a single constant_shifts tensor should be provided.
                If `inputs` consists of multiple tensors, a tuple
                of the constant_shifts tensors should be provided. It is assumed
                that for all given input tensors, dimension 0 corresponds
                to the number of examples (aka batch size), and if
                multiple input tensors are provided, the examples must
                be aligned appropriately. This constant_shift is subtracted from the input tensor and is used
                as an input baseline to shift the bias of the model in `create_model_with_shifted_bias`.
                Note that this is opposite to the original paper where the constant_shift is added to the input)

        input_layer_names (List[str]): The names of the input layers of the model that should be shifted. Each layer
                should be unique for each input constant shift tensor.

        delta (float, optional): The absolute tolerance parameter for the allclose function which checks whether
            the explanations generated for original inputs and shifted model and inptus are equal.
            Default is 1e-8.

        multi_target (bool, optional): A boolean flag that indicates whether the metric computation is for
                multi-target explanations. if set to true, the targets are required to be a list of integers
                each corresponding to a required target class in the output. The corresponding metric outputs
                are then returned as a list of metric outputs corresponding to each target class.
                Default is False.
        return_intermediate_results (bool, optional): A boolean flag that indicates whether the intermediate results
                of the metric computation are returned.
                Default is False.
        return_dict (bool, optional): A boolean flag that indicates whether the metric outputs are returned as a dictionary
                with keys as the metric names and values as the corresponding metric outputs.
                Default is False.
        **kwargs (Any, optional): Contains a list of arguments that are passed
                to `explanation_func` explanation function which in some cases
                could be the `attribute` function of an attribution algorithm.
                Any additional arguments that need be passed to the explanation
                function should be included here.
                For instance, such arguments include:
                `additional_forward_args`, `baselines` and `target`.

    Returns:

        input_invariance (Tensor): A boolean tensor of per
            input example showing whether the explanation is invariant to the input.
            The output dimension is equal to the number of examples in the input batch.
        inputs_expl (Tensor or tuple[Tensor, ...]): The explanation for the original input.
            If `inputs` is a single tensor, a single tensor is returned.
            If `inputs` consists of multiple tensors, a tuple of tensors is returned.
        shifted_inputs_expl (Tensor or tuple[Tensor, ...]): The explanation generated for the shifted input and
            the shifted model.
            If `inputs` is a single tensor, a single tensor is returned.
            If `inputs` consists of multiple tensors, a tuple of tensors is returned.

    Examples::
        >>> # ImageClassifier takes a single input tensor of images Nx3x32x32,
        >>> # and returns an Nx10 tensor of class probabilities.
        >>> net = ImageClassifier()
        >>> saliency = Saliency(net)
        >>> input = torch.randn(2, 3, 32, 32, requires_grad=True)
        >>> # Computes sensitivity score for saliency maps of class 3
        >>> input_invarance_score, inputs_expl, shifted_inputs_expl = input_invarance(saliency, input, target = 3)

    """
    if multi_target:
        assert isinstance(explainer, FeatureAttributionExplainer), (
            "The explainer must be an instance of Explainer."
        )
        assert explainer.multi_target, "The explainer must be a multi-target explainer."
        assert isinstance(target, list), (
            "The target must be a list of ExplanationTarget for multi-target input invariance."
        )
        input_invarance_score, inputs_expl, shifted_inputs_expl = (
            _multi_target_input_invariance(
                explainer=explainer,
                inputs=inputs,
                constant_shifts=constant_shifts,
                input_layer_names=input_layer_names,
                baselines=baselines,
                shift_baselines=shift_baselines,
                feature_mask=feature_mask,
                additional_forward_args=additional_forward_args,
                target=target,
                sliding_window_shapes=sliding_window_shapes,
                strides=strides,
            )
        )
    else:
        assert not isinstance(target, list), (
            "The target must be a single ExplanationTarget for single-target input invariance."
        )
        input_invarance_score, inputs_expl, shifted_inputs_expl = _input_invariance(
            explainer=explainer,
            inputs=inputs,
            constant_shifts=constant_shifts,
            input_layer_names=input_layer_names,
            baselines=baselines,
            shift_baselines=shift_baselines,
            feature_mask=feature_mask,
            additional_forward_args=additional_forward_args,
            target=target,
            sliding_window_shapes=sliding_window_shapes,
            strides=strides,
        )

    if return_intermediate_results:
        if return_dict:
            return {
                "input_invarance_score": input_invarance_score,
                "inputs_expl": inputs_expl,
                "shifted_inputs_expl": shifted_inputs_expl,
            }
        else:
            return (input_invarance_score, inputs_expl, shifted_inputs_expl)
    else:
        if return_dict:
            return {"input_invarance_score": input_invarance_score}
        else:
            return input_invarance_score