Verifies the summation-to-delta (conservation) axiom: attributions must sum exactly to the difference between the model output on the input and on the baseline. A score of zero means perfect completeness. ↓ better.

completeness

Functions:

  • completeness

    Implementation of Completeness test by Sundararajan et al., 2017, also referred

completeness

completeness(
    forward_func: Callable,
    inputs: TensorOrTupleOfTensorsGeneric,
    attributions: list[TensorOrTupleOfTensorsGeneric] | TensorOrTupleOfTensorsGeneric,
    baselines: BaselineType,
    additional_forward_args: Any = None,
    target: ExplanationTarget | list[ExplanationTarget] = NO_TARGET,
    multi_target: bool = False,
    return_dict: bool = False,
) -> Tensor | list[Tensor] | dict[str, Tensor | list[Tensor]]

Implementation of Completeness test by Sundararajan et al., 2017, also referred to as Summation to Delta by Shrikumar et al., 2017 and Conservation by Montavon et al., 2018. This implementation reuses the batch-computation ideas from captum and therefore it is fully compatible with the Captum library. In addition, the implementation takes some ideas about the implementation of the metric from the python Quantus library.

Attribution completeness asks that the total attribution is proportional to the explainable evidence at the output/ or some function of the model output. Or, that the attributions add up to the difference between the model output F at the input x and the baseline b. This is essentially the same as convergence_delta returned from captum however this works independently of the attribution method and the availability of convergence_delta calculation in Captum.

References

1) Completeness - Mukund Sundararajan et al.: "Axiomatic attribution for deep networks." International Conference on Machine Learning. PMLR, 2017. 2) Summation to delta - Avanti Shrikumar et al.: "Learning important features through propagating activation differences." International Conference on Machine Learning. PMLR, 2017. 3) Conservation - Grégoire Montavon et al.: "Methods for interpreting and understanding deep neural networks." Digital Signal Processing 73 (2018): 1-15.

Parameters:

  • forward_func

    (Callable) –
    The forward function of the model or any modification of it.
    
  • inputs

    (Tensor or tuple[Tensor, ...]) –
    Input for which
    attributions are computed. If forward_func takes a single
    tensor as input, a single input tensor should be provided.
    If forward_func takes multiple tensors as input, a tuple
    of the input tensors should be provided. It is assumed
    that for all given input tensors, dimension 0 corresponds
    to the number of examples (aka batch size), and if
    multiple input tensors are provided, the examples must
    be aligned appropriately.
    
  • baselines

    (scalar, Tensor, tuple of scalar, or Tensor) –
    Baselines define reference values against which the completeness is measured which sometimes
    represent ablated values and are used to compare with the actual inputs to compute
    importance scores in attribution algorithms. They can be represented
    as:
    
    - a single tensor, if inputs is a single tensor, with
      exactly the same dimensions as inputs or the first
      dimension is one and the remaining dimensions match
      with inputs.
    
    - a single scalar, if inputs is a single tensor, which will
      be broadcasted for each input value in input tensor.
    
    - a tuple of tensors or scalars, the baseline corresponding
      to each tensor in the inputs' tuple can be:
    
    - either a tensor with matching dimensions to
      corresponding tensor in the inputs' tuple
      or the first dimension is one and the remaining
      dimensions match with the corresponding
      input tensor.
    
    - or a scalar, corresponding to a tensor in the
      inputs' tuple. This scalar value is broadcasted
      for corresponding input tensor.
    
    Default: None
    
  • attributions

    (Tensor or tuple[Tensor, ...]) –
    Attribution scores computed based on an attribution algorithm.
    This attribution scores can be computed using the implementations
    provided in the `captum.attr` package. Some of those attribution
    approaches are so called global methods, which means that
    they factor in model inputs' multiplier, as described in:
    https://arxiv.org/abs/1711.06104
    Many global attribution algorithms can be used in local modes,
    meaning that the inputs multiplier isn't factored in the
    attribution scores.
    This can be done duing the definition of the attribution algorithm
    by passing `multipy_by_inputs=False` flag.
    For example in case of Integrated Gradients (IG) we can obtain
    local attribution scores if we define the constructor of IG as:
    ig = IntegratedGradients(multipy_by_inputs=False)
    
    Some attribution algorithms are inherently local.
    Examples of inherently local attribution methods include:
    Saliency, Guided GradCam, Guided Backprop and Deconvolution.
    
    For local attributions we can use real-valued perturbations
    whereas for global attributions that perturbation is binary.
    https://arxiv.org/abs/1901.09392
    
    Attributions have the same shape and dimensionality as the inputs.
    If inputs is a single tensor then the attributions is a single
    tensor as well. If inputs is provided as a tuple of tensors
    then attributions will be tuples of tensors as well.
    
  • additional_forward_args

    (Any, default: None ) –
    If the forward function
    requires additional arguments other than the inputs for
    which attributions should not be computed, this argument
    can be provided. It must be either a single additional
    argument of a Tensor or arbitrary (non-tuple) type or a tuple
    containing multiple additional arguments including tensors
    or any arbitrary python types. These arguments are provided to
    forward_func in order, following the arguments in inputs.
    Note that the perturbations are not computed with respect
    to these arguments. This means that these arguments aren't
    being passed to `perturb_func` as an input argument.
    
    Default: None
    
  • target

    (int, tuple, Tensor, or list, default: NO_TARGET ) –
    Indices for selecting
    predictions from output(for classification cases,
    this is usually the target class).
    If the network returns a scalar value per example, no target
    index is necessary.
    For general 2D outputs, targets can be either:
    
    - A single integer or a tensor containing a single
      integer, which is applied to all input examples
    
    - A list of integers or a 1D tensor, with length matching
      the number of examples in inputs (dim 0). Each integer
      is applied as the target for the corresponding example.
    
      For outputs with > 2 dimensions, targets can be either:
    
    - A single tuple, which contains #output_dims - 1
      elements. This target index is applied to all examples.
    
    - A list of tuples with length equal to the number of
      examples in inputs (dim 0), and each tuple containing
      #output_dims - 1 elements. Each tuple is applied as the
      target for the corresponding example.
    
    Default: None
    
  • multi_target

    (bool, default: False ) –
    A boolean flag that indicates whether the metric computation is for
    multi-target explanations. if set to true, the targets are required to be a list of integers
    each corresponding to a required target class in the output. The corresponding metric outputs
    are then returned as a list of metric outputs corresponding to each target class.
    Default is False.
    
  • return_dict

    (bool, default: False ) –
    A boolean flag that indicates whether the metric outputs are returned as a dictionary
    with keys as the metric names and values as the corresponding metric outputs.
    Default is False.
    

Returns:

  • Tensor | list[Tensor] | dict[str, Tensor | list[Tensor]]
    Either  tensor of scalar completeness scores per
    input example. The first dimension is equal to the
    number of examples in the input batch and the second
    dimension is one. If multi_target is set to True, then a list of
    completeness scores is returned, each respective to the (target, attribution) pairs.
    If return_dict is set to True, then a dictionary is returned with key "score" and value as the
    completeness score tensor or list of tensors.
    

Examples:

>>> # ImageClassifier takes a single input tensor of images Nx3x32x32,
>>> # and returns an Nx10 tensor of class probabilities.
>>> net = ImageClassifier()
>>> saliency = Saliency(net)
>>> input = torch.randn(2, 3, 32, 32, requires_grad=True)
>>> baselines = torch.zeros(2, 3, 32, 32)
>>> # Computes saliency maps for class 3.
>>> attribution = saliency.attribute(input, target=3)
>>> # define a perturbation function for the input
>>> # Computes completeness score for saliency maps
>>> completeness = completeness(net, input, attribution, baselines)
Source code in torchxai/metrics/axiomatic/completeness.py
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
def completeness(
    forward_func: Callable,
    inputs: TensorOrTupleOfTensorsGeneric,
    attributions: list[TensorOrTupleOfTensorsGeneric] | TensorOrTupleOfTensorsGeneric,
    baselines: BaselineType,
    additional_forward_args: Any = None,
    target: ExplanationTarget | list[ExplanationTarget] = NO_TARGET,
    multi_target: bool = False,
    return_dict: bool = False,
) -> Tensor | list[Tensor] | dict[str, Tensor | list[Tensor]]:
    """
    Implementation of Completeness test by Sundararajan et al., 2017, also referred
    to as Summation to Delta by Shrikumar et al., 2017 and Conservation by
    Montavon et al., 2018. This implementation reuses the batch-computation ideas from captum and therefore it is fully
    compatible with the Captum library. In addition, the implementation takes some ideas about the implementation
    of the metric from the python Quantus library.

    Attribution completeness asks that the total attribution is proportional to the explainable
    evidence at the output/ or some function of the model output. Or, that the attributions
    add up to the difference between the model output F at the input x and the baseline b. This is essentially
    the same as `convergence_delta` returned from captum however this works independently of the attribution method
    and the availability of `convergence_delta` calculation in Captum.

    References:
        1) Completeness - Mukund Sundararajan et al.: "Axiomatic attribution for deep networks."
        International Conference on Machine Learning. PMLR, 2017.
        2) Summation to delta - Avanti Shrikumar et al.: "Learning important
        features through propagating activation differences." International Conference on Machine Learning. PMLR, 2017.
        3) Conservation - Grégoire Montavon et al.: "Methods for interpreting
        and understanding deep neural networks." Digital Signal Processing 73 (2018): 1-15.

    Args:
        forward_func (Callable):
                The forward function of the model or any modification of it.

        inputs (Tensor or tuple[Tensor, ...]):
                Input for which
                attributions are computed. If forward_func takes a single
                tensor as input, a single input tensor should be provided.
                If forward_func takes multiple tensors as input, a tuple
                of the input tensors should be provided. It is assumed
                that for all given input tensors, dimension 0 corresponds
                to the number of examples (aka batch size), and if
                multiple input tensors are provided, the examples must
                be aligned appropriately.

        baselines (scalar, Tensor, tuple of scalar, or Tensor):
                Baselines define reference values against which the completeness is measured which sometimes
                represent ablated values and are used to compare with the actual inputs to compute
                importance scores in attribution algorithms. They can be represented
                as:

                - a single tensor, if inputs is a single tensor, with
                  exactly the same dimensions as inputs or the first
                  dimension is one and the remaining dimensions match
                  with inputs.

                - a single scalar, if inputs is a single tensor, which will
                  be broadcasted for each input value in input tensor.

                - a tuple of tensors or scalars, the baseline corresponding
                  to each tensor in the inputs\' tuple can be:

                - either a tensor with matching dimensions to
                  corresponding tensor in the inputs\' tuple
                  or the first dimension is one and the remaining
                  dimensions match with the corresponding
                  input tensor.

                - or a scalar, corresponding to a tensor in the
                  inputs\' tuple. This scalar value is broadcasted
                  for corresponding input tensor.

                Default: None

        attributions (Tensor or tuple[Tensor, ...]):
                Attribution scores computed based on an attribution algorithm.
                This attribution scores can be computed using the implementations
                provided in the `captum.attr` package. Some of those attribution
                approaches are so called global methods, which means that
                they factor in model inputs' multiplier, as described in:
                https://arxiv.org/abs/1711.06104
                Many global attribution algorithms can be used in local modes,
                meaning that the inputs multiplier isn't factored in the
                attribution scores.
                This can be done duing the definition of the attribution algorithm
                by passing `multipy_by_inputs=False` flag.
                For example in case of Integrated Gradients (IG) we can obtain
                local attribution scores if we define the constructor of IG as:
                ig = IntegratedGradients(multipy_by_inputs=False)

                Some attribution algorithms are inherently local.
                Examples of inherently local attribution methods include:
                Saliency, Guided GradCam, Guided Backprop and Deconvolution.

                For local attributions we can use real-valued perturbations
                whereas for global attributions that perturbation is binary.
                https://arxiv.org/abs/1901.09392

                Attributions have the same shape and dimensionality as the inputs.
                If inputs is a single tensor then the attributions is a single
                tensor as well. If inputs is provided as a tuple of tensors
                then attributions will be tuples of tensors as well.

        additional_forward_args (Any, optional):
                If the forward function
                requires additional arguments other than the inputs for
                which attributions should not be computed, this argument
                can be provided. It must be either a single additional
                argument of a Tensor or arbitrary (non-tuple) type or a tuple
                containing multiple additional arguments including tensors
                or any arbitrary python types. These arguments are provided to
                forward_func in order, following the arguments in inputs.
                Note that the perturbations are not computed with respect
                to these arguments. This means that these arguments aren't
                being passed to `perturb_func` as an input argument.

                Default: None
        target (int, tuple, Tensor, or list, optional):
                Indices for selecting
                predictions from output(for classification cases,
                this is usually the target class).
                If the network returns a scalar value per example, no target
                index is necessary.
                For general 2D outputs, targets can be either:

                - A single integer or a tensor containing a single
                  integer, which is applied to all input examples

                - A list of integers or a 1D tensor, with length matching
                  the number of examples in inputs (dim 0). Each integer
                  is applied as the target for the corresponding example.

                  For outputs with > 2 dimensions, targets can be either:

                - A single tuple, which contains #output_dims - 1
                  elements. This target index is applied to all examples.

                - A list of tuples with length equal to the number of
                  examples in inputs (dim 0), and each tuple containing
                  #output_dims - 1 elements. Each tuple is applied as the
                  target for the corresponding example.

                Default: None
        multi_target (bool, optional):
                A boolean flag that indicates whether the metric computation is for
                multi-target explanations. if set to true, the targets are required to be a list of integers
                each corresponding to a required target class in the output. The corresponding metric outputs
                are then returned as a list of metric outputs corresponding to each target class.
                Default is False.
        return_dict (bool, optional):
                A boolean flag that indicates whether the metric outputs are returned as a dictionary
                with keys as the metric names and values as the corresponding metric outputs.
                Default is False.

    Returns:
        (Tensor | list[Tensor] | dict[str, Tensor | list[Tensor]]):
                Either  tensor of scalar completeness scores per
                input example. The first dimension is equal to the
                number of examples in the input batch and the second
                dimension is one. If multi_target is set to True, then a list of
                completeness scores is returned, each respective to the (target, attribution) pairs.
                If return_dict is set to True, then a dictionary is returned with key "score" and value as the
                completeness score tensor or list of tensors.

    Examples:
        >>> # ImageClassifier takes a single input tensor of images Nx3x32x32,
        >>> # and returns an Nx10 tensor of class probabilities.
        >>> net = ImageClassifier()
        >>> saliency = Saliency(net)
        >>> input = torch.randn(2, 3, 32, 32, requires_grad=True)
        >>> baselines = torch.zeros(2, 3, 32, 32)
        >>> # Computes saliency maps for class 3.
        >>> attribution = saliency.attribute(input, target=3)
        >>> # define a perturbation function for the input

        >>> # Computes completeness score for saliency maps
        >>> completeness = completeness(net, input, attribution, baselines)
    """
    metric_func = _multi_target_completeness if multi_target else _completeness
    kwargs = {
        "forward_func": forward_func,
        "inputs": inputs,
        "baselines": baselines,
        "additional_forward_args": additional_forward_args,
    }
    if multi_target:
        assert isinstance(target, list), (
            "For multi-target completeness, the target must be a list of target indices."
        )
        assert isinstance(attributions, list), (
            "For multi-target completeness, attributions must be a list of attributions."
        )
        kwargs["attributions_list"] = attributions  # type: ignore
        kwargs["targets_list"] = [t.value for t in target]  # type: ignore
    else:
        assert not isinstance(target, list), (
            "For single-target completeness, the target must be a scalar or a tensor."
        )
        assert not isinstance(attributions, list), (
            "For single-target completeness, the attributions must be a tensor or a tuple of tensors."
        )
        kwargs["attributions"] = attributions  # type: ignore
        kwargs["target"] = target.value

    score = metric_func(**kwargs)
    if return_dict:
        return {"score": score}
    return score