Max & Avg Sensitivity
Measures explanation stability under small input perturbations. Returns two values: max sensitivity (worst-case attribution distance across perturbations, ↓ better) and avg sensitivity (average distance, ↓ better). Does not require a baseline.
sensitivity_max_and_avg
sensitivity_max_and_avg(
explainer: Explainer | Attribution,
inputs: TensorOrTupleOfTensorsGeneric,
perturb_func: Callable = default_perturb_func,
perturb_radius: float = 0.02,
n_perturb_samples: int = 10,
norm_ord: str = "fro",
max_examples_per_batch: int | None = None,
multi_target: bool = False,
return_dict: bool = False,
**kwargs: Any,
) -> (
Tensor
| tuple[Tensor, Tensor]
| list[Tensor]
| tuple[list[Tensor], list[Tensor]]
| dict[str, Tensor]
| dict[str, list[Tensor]]
)
This is a modified version of the captum captum.metric.sensitivity_max (see: from captum.metrics import sensitivity_max)
function that repeats the feature masks when performing perturbations. This function returns the sensitivity
scores with both average and max aggregation resulting in sensitivity_max and sensitivity_avg together.
The metric returns a list of senstivity scores if multi_target is True using the
torchxai.metrics.robustness.multi_target._multi_target_sensitivity_scores,
otherwise it returns a single sensitivity score same as the captum.metric.sensitivity_max.
Explanation sensitivity measures the extent of explanation change when
the input is slightly perturbed. It has been shown that the models that
have high explanation sensitivity are prone to adversarial attacks:
Interpretation of Neural Networks is Fragile
https://www.aaai.org/ojs/index.php/AAAI/article/view/4252
sensitivity_max metric measures maximum sensitivity of an explanation
using Monte Carlo sampling-based approximation. By default in order to
do so it samples multiple data points from a sub-space of an L-Infinity
ball that has a perturb_radius radius using default_perturb_func
default perturbation function. In a general case users can
use any L_p ball or any other custom sampling technique that they
prefer by providing a custom perturb_func.
Note that max sensitivity is similar to Lipschitz Continuity metric however it is more robust and easier to estimate. Since the explanation, for instance an attribution function, may not always be continuous, can lead to unbounded Lipschitz continuity. Therefore the latter isn't always appropriate.
More about the Lipschitz Continuity Metric can also be found here
On the Robustness of Interpretability Methods
https://arxiv.org/abs/1806.08049
and
Towards Robust Interpretability with Self-Explaining Neural Networks
https://papers.nips.cc/paper\
8003-towards-robust-interpretability-
with-self-explaining-neural-networks.pdf
More details about sensitivity max can be found here:
On the (In)fidelity and Sensitivity of Explanations
https://arxiv.org/abs/1901.09392
Args:
explanation_func (Callable):
This function can be the `attribute` method of an
attribution algorithm or any other explanation method
that returns the explanations.
inputs (Tensor or tuple[Tensor, ...]): Input for which
explanations are computed. If `explanation_func` takes a
single tensor as input, a single input tensor should
be provided.
If `explanation_func` takes multiple tensors as input, a tuple
of the input tensors should be provided. It is assumed
that for all given input tensors, dimension 0 corresponds
to the number of examples (aka batch size), and if
multiple input tensors are provided, the examples must
be aligned appropriately.
perturb_func (Callable):
The perturbation function of model inputs. This function takes
model inputs and optionally `perturb_radius` if
the function takes more than one argument and returns
perturbed inputs.
If there are more than one inputs passed to sensitivity function those
will be passed to `perturb_func` as tuples in the same order as they
are passed to sensitivity function.
It is important to note that for performance reasons `perturb_func`
isn't called for each example individually but on a batch of
input examples that are repeated `max_examples_per_batch / batch_size`
times within the batch.
Default: default_perturb_func
perturb_radius (float, optional): The epsilon radius used for sampling.
In the `default_perturb_func` it is used as the radius of
the L-Infinity ball. In a general case it can serve as a radius of
any L_p norm.
This argument is passed to `perturb_func` if it takes more than
one argument.
Default: 0.02
n_perturb_samples (int, optional): The number of times input tensors
are perturbed. Each input example in the inputs tensor is
expanded `n_perturb_samples` times before calling
`perturb_func` function.
Default: 10
norm_ord (int, float, or str, optional): The type of norm that is used to
compute the norm of the sensitivity matrix which is defined as the
difference between the explanation function at its input and perturbed
input. Acceptable values are either a string of 'fro' or 'nuc', or a
number in the range of [-inf, inf] (including float("-inf") &
float("inf")).
Default: 'fro'
max_examples_per_batch (int, optional): The number of maximum input
examples that are processed together. In case the number of
examples (`input batch size * n_perturb_samples`) exceeds
`max_examples_per_batch`, they will be sliced
into batches of `max_examples_per_batch` examples and processed
in a sequential order. If `max_examples_per_batch` is None, all
examples are processed together. `max_examples_per_batch` should
at least be equal `input batch size` and at most
`input batch size * n_perturb_samples`.
Default: None
multi_target (bool, optional): A boolean flag that indicates whether the metric computation is for
multi-target explanations. if set to true, the targets are required to be a list of integers
each corresponding to a required target class in the output. The corresponding metric outputs
are then returned as a list of metric outputs corresponding to each target class. For multi-target
infidelity, captum implementation is extened in _multi_target_infidelity function.
Default is False.
return_dict (bool, optional): A boolean flag that indicates whether the metric outputs are returned as a dictionary
with keys as the metric names and values as the corresponding metric outputs.
Default is False.
**kwargs (Any, optional): Contains a list of arguments that are passed
to `explanation_func` explanation function which in some cases
could be the `attribute` function of an attribution algorithm.
Any additional arguments that need be passed to the explanation
function should be included here.
For instance, such arguments include:
`additional_forward_args`, `baselines` and `target`.
Returns:
sensitivities (Tensor): A tensor of scalar sensitivity scores per
input example. The first dimension is equal to the
number of examples in the input batch and the second
dimension is one. Returned sensitivities are normalized by
the magnitudes of the input explanations.
Examples:: >>> # ImageClassifier takes a single input tensor of images Nx3x32x32, >>> # and returns an Nx10 tensor of class probabilities. >>> net = ImageClassifier() >>> saliency = Saliency(net) >>> input = torch.randn(2, 3, 32, 32, requires_grad=True) >>> # Computes sensitivity score for saliency maps of class 3 >>> sens = sensitivity_max(saliency.attribute, input, target = 3)
Source code in torchxai/metrics/robustness/sensitivity.py
196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 | |