Tests that attributions remain unchanged when a constant shift is added to inputs and the model bias is adjusted to compensate — a formal axiom from Kindermans et al.
input_invariance
Functions:
-
input_invariance–Implementation of Input Invariance test by Kindermans et al., 2017. This implementation
input_invariance
input_invariance(
explainer: FeatureAttributionExplainer | Attribution,
inputs: TensorOrTupleOfTensorsGeneric,
constant_shifts: TensorOrTupleOfTensorsGeneric,
input_layer_names: tuple[str],
multi_target: bool = False,
return_intermediate_results: bool = False,
return_dict: bool = False,
baselines: TensorOrTupleOfTensorsGeneric | None = None,
shift_baselines: TensorOrTupleOfTensorsGeneric | None = None,
feature_mask: TensorOrTupleOfTensorsGeneric | None = None,
additional_forward_args: Any = None,
target: ExplanationTarget | list[ExplanationTarget] = NoTarget(),
sliding_window_shapes: Any | None = None,
strides: Any | None = None,
**kwargs,
) -> dict | tuple | Tensor | list[Tensor]
Implementation of Input Invariance test by Kindermans et al., 2017. This implementation reuses the batch-computation ideas from captum and therefore it is fully compatible with the Captum library. In addition, the implementation takes some ideas about the implementation of the metric from the python Quantus library.
To test for input invariance, we add a constant shift to the input data and a mean shift to the model bias, so that the output of the original model on the original data is equal to the output of the changed model on the shifted data. The metric returns True if batch attributions stayed unchanged too. Currently only supporting constant values for the shift.
References
Pieter-Jan Kindermans et al.: "The (Un)reliability of Saliency Methods." Explainable AI (2019): 267-280
Args:
explainer (Union[FusionExplainer, Attribution]): The explainer instance that is used to
compute the explanations. The explainer must be an instance of either captum Attribution class or
the FusionExplainer instance.
inputs (Tensor or tuple[Tensor, ...]): Input for which
explanations are computed. If `explainer` takes a
single tensor as input, a single input tensor should
be provided.
If `explainer` takes multiple tensors as input, a tuple
of the input tensors should be provided. It is assumed
that for all given input tensors, dimension 0 corresponds
to the number of examples (aka batch size), and if
multiple input tensors are provided, the examples must
be aligned appropriately.
constant_shifts (Tensor or tuple[Tensor, ...]): Constant shifts defined for each input tensor.
If `inputs` is single tensor, a single constant_shifts tensor should be provided.
If `inputs` consists of multiple tensors, a tuple
of the constant_shifts tensors should be provided. It is assumed
that for all given input tensors, dimension 0 corresponds
to the number of examples (aka batch size), and if
multiple input tensors are provided, the examples must
be aligned appropriately. This constant_shift is subtracted from the input tensor and is used
as an input baseline to shift the bias of the model in `create_model_with_shifted_bias`.
Note that this is opposite to the original paper where the constant_shift is added to the input)
input_layer_names (List[str]): The names of the input layers of the model that should be shifted. Each layer
should be unique for each input constant shift tensor.
delta (float, optional): The absolute tolerance parameter for the allclose function which checks whether
the explanations generated for original inputs and shifted model and inptus are equal.
Default is 1e-8.
multi_target (bool, optional): A boolean flag that indicates whether the metric computation is for
multi-target explanations. if set to true, the targets are required to be a list of integers
each corresponding to a required target class in the output. The corresponding metric outputs
are then returned as a list of metric outputs corresponding to each target class.
Default is False.
return_intermediate_results (bool, optional): A boolean flag that indicates whether the intermediate results
of the metric computation are returned.
Default is False.
return_dict (bool, optional): A boolean flag that indicates whether the metric outputs are returned as a dictionary
with keys as the metric names and values as the corresponding metric outputs.
Default is False.
**kwargs (Any, optional): Contains a list of arguments that are passed
to `explanation_func` explanation function which in some cases
could be the `attribute` function of an attribution algorithm.
Any additional arguments that need be passed to the explanation
function should be included here.
For instance, such arguments include:
`additional_forward_args`, `baselines` and `target`.
Returns:
input_invariance (Tensor): A boolean tensor of per
input example showing whether the explanation is invariant to the input.
The output dimension is equal to the number of examples in the input batch.
inputs_expl (Tensor or tuple[Tensor, ...]): The explanation for the original input.
If `inputs` is a single tensor, a single tensor is returned.
If `inputs` consists of multiple tensors, a tuple of tensors is returned.
shifted_inputs_expl (Tensor or tuple[Tensor, ...]): The explanation generated for the shifted input and
the shifted model.
If `inputs` is a single tensor, a single tensor is returned.
If `inputs` consists of multiple tensors, a tuple of tensors is returned.
Examples:: >>> # ImageClassifier takes a single input tensor of images Nx3x32x32, >>> # and returns an Nx10 tensor of class probabilities. >>> net = ImageClassifier() >>> saliency = Saliency(net) >>> input = torch.randn(2, 3, 32, 32, requires_grad=True) >>> # Computes sensitivity score for saliency maps of class 3 >>> input_invarance_score, inputs_expl, shifted_inputs_expl = input_invarance(saliency, input, target = 3)
Source code in torchxai/metrics/axiomatic/input_invariance.py
135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 | |