Improved signature reporting

Whether it is for building documentation or minimizing repetition for the users of your library or framework, inspecting callables might be something that could help you achieve it.

Python has provided tools to help you do this for a while. The inspect module in version 2.1 added getargspec which made use of code and function attributes dating back to Python 1.3. Python 3.3 introduced inspect.signature which polished the concept such that a inspect.Signature object describes a function’s parameters, annotations and default values.

sigtools.signature(obj, auto=True, args=(), kwargs={})

Improved version of inspect.signature. Takes into account decorators from sigtools.specifiers if present or tries to determine a full signature automatically.

See sigtools.specifiers.signature for detailed reference.

sigtools.signature is a drop-in replacement for inspect.signature, with a few key improvements:

  • It can automatically traverse through decorators, while keeping track of which functions owns each parameter

  • It helps you evaluate parameter annotations in the proper context, for instance when the parameter annotation is defined in a module that enables postponed evaluation of annotations.

  • It supports a mechanism for functions to dynamically alter their reported signature

Using sigtools.signature

Python’s inspect module can produce signature objects, which represent how a function can be called. Their textual representation roughly matches the parameter list part of a function definition:

import inspect


def func(abc, *args, **kwargs):
    ...


print(inspect.signature(func))
# (abc, *args, **kwargs)

You can do the same with sigtools.signature:

from sigtools import signature

print(signature(func))
# (abc, *args, **kwargs)

You can use the resulting object the same way as inspect.Signature, for example:

sig = signature(myfunc)
for param in sig.parameters.values():
    print(param.name, param.kind)
# param POSITIONAL_OR_KEYWORD
# decorator_param KEYWORD_ONLY

Introspection through decorators

As alluded to above, sigtools.signature will look through decorators and produce a signature that takes parameters that such decorators add or remove into account:

import inspect
import functools

from sigtools import signature


def decorator(value_for_first_param):
    def _decorate(wrapped):
        @functools.wraps(wrapped)
        def _wrapper(*args, extra_arg, **kwargs):
            wrapped(value_for_first_param, *args, **kwargs)

        return _wrapper

    return _decorate


@decorator("eggs")
def func(ham, spam):
    return ham, spam


print("inspect:", inspect.signature(func))
# inspect: (ham, spam)

print("sigtools:", specifiers.signature(func))
# sigtools: (spam, *, extra_arg)

Where inspect.signature simply sees the signature of func itself, sigtools.signature sees how things fit together:

  • It understands that _wrapper uses its *args and **kwargs by passing them to wrapped, which is func,

  • It sees that one argument to func is supplied by _wrapper

  • It sees that _wrapper has a parameter of its own.

This lets sigtools.signature determine the effective signature of func as decorated.

Note

Is functools.wraps necessary?

wraps is recommended when writing decorators, because it copies attributes from the wrapped function to the wrapper, such as the name and docstring, which is generally useful.

sigtools.signature will continue working the same, but inspect.signature will show _wrapper’s signature:

print("inspect w/o wraps:", inspect.signature(func_without_wraps))
# inspect w/o wraps: (*args, extra_arg, **kwargs


print("sigtools w/o wraps:", specifiers.signature(func_without_wraps))
# sigtools w/o wraps: (spam, *, extra_arg)

In the example above this note, inspect.signature uses inspect.unwrap to find the function to get the innermost function (func as defined in the source, before decorators), and takes the signature from that. If the decorator alters the effective signature of whatever it wraps, like above, this will probably produce an incorrect signature.

Note

While sigtools.signature should generally work with most python code (see Limitations of automatic signature discovery), sigtools recommends sigtools.wrappers.decorator as the simplest way to write custom decorators while preserving as much information as possible.

For instance, decorators defined with decorator are set up in a way that the source of each parameter points to a function on which the docstring is not overwritten, unlike in the example above.

Evaluating PEP 563 stringified annotations

sigtools.signature returns instances of UpgradedSignatures, on which parameters have a new attribute upgraded_annotation. The return signature is also available as upgraded_return_annotation. Both have a value of type UpgradedAnnotation, which allow you to get the value of an annotation as seen in the source.

from __future__ import annotations

import attrs

from sigtools import signature


def myfunc(my_param: MyInput) -> MyOutput:
    return MyOutput(ham=my_param.spam)


@attrs.define
class MyInput:
    spam: str


@attrs.define
class MyOutput:
    ham: str


sig = signature(myfunc)

print(sig)
# (my_param: 'MyInput') -> 'MyOutput'

print(repr(sig.return_annotation))
# 'MyOutput'
print(repr(sig.upgraded_return_annotation.source_value()))
# <class '__main__.MyOutput'>

print(repr(sig.parameters["my_param"].annotation))
# 'MyInput'
print(repr(sig.parameters["my_param"].upgraded_annotation.source_value()))
# <class '__main__.MyInput'>

print(sig.evaluated())
# (my_param: __main__.MyInput) -> __main__.MyOutput
class sigtools.signatures.UpgradedAnnotation[source]

Represents an annotation, whether already evaluated, or deferred by PEP 563.

abstract source_value()[source]

Value of this annotation as would be evaluated at the site of its definition.

class sigtools.signatures.UpgradedSignature(parameters=None, *args, upgraded_return_annotation=EmptyAnnotation, _stacklevel=0, **kwargs)[source]

A Signature augmented with parameter sources and upgraded annotations, as returned by sigtools.signature or sigtools.signatures.signature

parameters: sigtools.signatures.UpgradedParameter
upgraded_return_annotation

Return annotation.

Type

sigtools.signatures.UpgradedAnnotation

class sigtools.signatures.UpgradedParameter(*args, function=None, sources=[], source_depths={}, upgraded_annotation=EmptyAnnotation, **kwargs)[source]

A Parameter augmented with parameter sources and upgraded annotations, as found on signatures returned by sigtools.signature or sigtools.signatures.signature.

upgraded_annotation

Annotation of this parameter.

Type

sigtools.signatures.UpgradedAnnotation

Parameter provenance

Warning

Interface is likely to change in sigtools 5.0.

sigtools.signature adds a sources attribute to the signature object. For each parameter, it lists all functions that will receive this parameter:

for param in sig.parameters.values():
    print(param.name, sig.sources[param.name])
# param [<function myfunc at ...>]
# decorator_param [<function decorator.<locals>._wrapper at ...>]

Additionally, this attribute contains the depth of each function, if you need a reliable order for them:

print(sig.sources['+depths'])
# {<function decorator.<locals>._wrapper at 0x7f354829c6a8>: 0,
#  <function myfunc at 0x7f354829c730>: 1}

Limitations of automatic signature discovery

sigtools.signature is able to examine a function to determine how its *args, **kwargs parameters are being used, even when no information is otherwise provided.

This is very useful for documentation or introspection tools, because it means authors of documented or introspected code don’t have to worry about providing this meta-information.

It should handle almost all instances of decorator code, though more unusual code could go beyond its ability to understand it. If this happens it will fall back to a generic signature.

Here is a list of the current limitations:

  • It requires the source code to be available. This means automatic introspection of functions that were defined in missing .py files, in code passed to eval() or in an interactive session will fail.

  • It doesn’t handle transformations or resetting of args and kwargs

  • It doesn’t handle Python 3.5’s multiple *args and **kwargs support

  • It doesn’t handle calls to the superclass

In some other instances, the signature genuinely can’t be determined in advance. For instance, if you call one function or another depending on a parameter, and these functions have incompatible signatures, there wouldn’t be one common signature for the outer function.

If you still need accurate signature reporting when automatic discovery fails, you can use the decorators from the specifiers module:

See also

Picking the appropriate arguments for forwards

Getting more help

If there is anything you wish to discuss more thoroughly, feel free to come by the sigtools gitter chat.