Upgrading to Newer Releases

This section documents migration paths to new releases.

Upgrading to 8.0

In 8.0, the default values for unknown were changed. When the location is set to json, form, or json_or_form, the default for unknown is now None. Previously, the default was RAISE.

Because RAISE is the default value for unknown on marshmallow schemas, this change only affects usage in which the following conditions are met:

• A schema with unknown set to INCLUDE or EXCLUDE is passed to webargs use_args, use_kwargs, or parse

• unknown is not passed explicitly to the webargs function

• location is not set (default of json) or is set explicitly to json, form, or json_or__form

For example

import marshmallow as ma


class BodySchema(ma.Schema):
    foo = ma.fields.String()

    class Meta:
        unknown = ma.EXCLUDE


@parser.use_args(BodySchema)
def foo(data): ...

In this case, under webargs 7.0 the schema unknown setting of EXCLUDE would be ignored. Instead, unknown=RAISE would be used.

In webargs 8.0, the schema unknown is used.

To get the webargs 7.0 behavior (overriding the Schema unknown), simply pass unknown to use_args, as in

@parser.use_args(BodySchema, unknown=ma.RAISE)
def foo(data): ...

Upgrading to 7.0

unknown is Now Settable by the Parser

As of 7.0, Parsers have multiple settings for controlling the value for unknown which is passed to schema.load when parsing.

To set unknown behavior on a parser, see the advanced doc on this topic: Setting unknown.

Importantly, by default, any schema setting for unknown will be overridden by the unknown settings for the parser.

In order to use a schema’s unknown value, set unknown=None on the parser. In 6.x versions of webargs, schema values for unknown are used, so the unknown=None setting is the best way to emulate this.

To get identical behavior:

# assuming you have a schema named MySchema


# webargs 6.x
@parser.use_args(MySchema)
def foo(args): ...


# webargs 7.x
# as a parameter to use_args or parse
@parser.use_args(MySchema, unknown=None)
def foo(args): ...


# webargs 7.x
# as a parser setting
# example with flaskparser, but any parser class works
parser = FlaskParser(unknown=None)


@parser.use_args(MySchema)
def foo(args): ...

Upgrading to 6.0

Multiple Locations Are No Longer Supported In A Single Call

The default location is JSON/body.

Under webargs 5.x, code often did not have to specify a location.

Because webargs would parse data from multiple locations automatically, users did not need to specify where a parameter, call it q, was passed. q could be in a query parameter or in a JSON or form-post body.

Now, webargs requires that users specify only one location for data loading per use_args call, and "json" is the default. If q is intended to be a query parameter, the developer must be explicit and rewrite like so:

# webargs 5.x
@parser.use_args({"q": ma.fields.String()})
def foo(args):
    return some_function(user_query=args.get("q"))


# webargs 6.x
@parser.use_args({"q": ma.fields.String()}, location="query")
def foo(args):
    return some_function(user_query=args.get("q"))

This also means that another usage from 5.x is not supported. Code with multiple locations in a single use_args, use_kwargs, or parse call must be rewritten in multiple separate use_args or use_kwargs invocations, like so:

# webargs 5.x
@parser.use_kwargs(
    {
        "q1": ma.fields.Int(location="query"),
        "q2": ma.fields.Int(location="query"),
        "h1": ma.fields.Int(location="headers"),
    },
    locations=("query", "headers"),
)
def foo(q1, q2, h1): ...


# webargs 6.x
@parser.use_kwargs({"q1": ma.fields.Int(), "q2": ma.fields.Int()}, location="query")
@parser.use_kwargs({"h1": ma.fields.Int()}, location="headers")
def foo(q1, q2, h1): ...

Fields No Longer Support location=…

Because a single parser.use_args, parser.use_kwargs, or parser.parse call cannot specify multiple locations, it is not necessary for a field to be able to specify its location. Rewrite code like so:

# webargs 5.x
@parser.use_args({"q": ma.fields.String(location="query")})
def foo(args):
    return some_function(user_query=args.get("q"))


# webargs 6.x
@parser.use_args({"q": ma.fields.String()}, location="query")
def foo(args):
    return some_function(user_query=args.get("q"))

location_handler Has Been Replaced With location_loader

This is not just a name change. The expected signature of a location_loader is slightly different from the signature for a location_handler.

Where previously a location_handler code took the incoming request data and details of a single field being loaded, a location_loader takes the request and the schema as a pair. It does not return a specific field’s data, but data for the whole location.

Rewrite code like this:

# webargs 5.x
@parser.location_handler("data")
def load_data(request, name, field):
    return request.data.get(name)


# webargs 6.x
@parser.location_loader("data")
def load_data(request, schema):
    return request.data

Data Is Not Filtered Before Being Passed To Schemas, And It May Be Proxified

In webargs 5.x, the deserialization schema was used to pull data out of the request object. That data was compiled into a dictionary which was then passed to the schema.

One of the major changes in webargs 6.x allows the use of unknown parameter on schemas. This lets a schema decide what to do with fields not specified in the schema. In order to achieve this, webargs now passes the full data from the specified location to the schema.

Therefore, users should specify unknown=marshmallow.EXCLUDE on their schemas in order to filter out unknown fields. Like so:

# webargs 5.x
# this can assume that "q" is the only parameter passed, and all other
# parameters will be ignored
@parser.use_kwargs({"q": ma.fields.String()}, locations=("query",))
def foo(q): ...


# webargs 6.x, Solution 1: declare a schema with Meta.unknown set
class QuerySchema(ma.Schema):
    q = ma.fields.String()

    class Meta:
        unknown = ma.EXCLUDE


@parser.use_kwargs(QuerySchema, location="query")
def foo(q): ...


# webargs 6.x, Solution 2: instantiate a schema with unknown set
class QuerySchema(ma.Schema):
    q = ma.fields.String()


@parser.use_kwargs(QuerySchema(unknown=ma.EXCLUDE), location="query")
def foo(q): ...

This also allows usage which passes the unknown parameters through, like so:

# webargs 6.x only! cannot be done in 5.x
class QuerySchema(ma.Schema):
    q = ma.fields.String()


# will pass *all* query params through as "kwargs"
@parser.use_kwargs(QuerySchema(unknown=ma.INCLUDE), location="query")
def foo(q, **kwargs): ...

However, many types of request data are so-called “multidicts” – dictionary-like types which can return one or multiple values. To handle marshmallow.fields.List and webargs.fields.DelimitedList fields correctly, passing list data, webargs must combine schema information with the raw request data. This is done in the MultiDictProxy type, which will often be passed to schemas.

This means that if a schema has a pre_load hook which interacts with the data, it may need modifications. For example, a flask query string will be parsed into an ImmutableMultiDict type, which will break pre-load hooks which modify the data in-place. Such usages need rewrites like so:

# webargs 5.x
# flask query params is just an example -- applies to several types
from webargs.flaskparser import use_kwargs


class QuerySchema(ma.Schema):
    q = ma.fields.String()

    @ma.pre_load
    def convert_nil_to_none(self, obj, **kwargs):
        if obj.get("q") == "nil":
            obj["q"] = None
        return obj


@use_kwargs(QuerySchema, locations=("query",))
def foo(q): ...


# webargs 6.x
class QuerySchema(ma.Schema):
    q = ma.fields.String()

    # unlike under 5.x, we cannot modify 'obj' in-place because writing
    # to the MultiDictProxy will try to write to the underlying
    # ImmutableMultiDict, which is not allowed
    @ma.pre_load
    def convert_nil_to_none(self, obj, **kwargs):
        # creating a dict from a MultiDictProxy works well because it
        # "unwraps" lists and delimited lists correctly
        data = dict(obj)
        if data.get("q") == "nil":
            data["q"] = None
        return data


@parser.use_kwargs(QuerySchema, location="query")
def foo(q): ...

DelimitedList Now Only Takes A String Input

Combining List and string parsing functionality in a single type had some messy corner cases. For the most part, this should not require rewrites. But for APIs which need to allow both usages, rewrites are possible like so:

# webargs 5.x
# this allows ...?x=1&x=2&x=3
# as well as ...?x=1,2,3
@use_kwargs({"x": webargs.fields.DelimitedList(ma.fields.Int)}, locations=("query",))
def foo(x): ...


# webargs 6.x
# this accepts x=1,2,3 but NOT x=1&x=2&x=3
@use_kwargs({"x": webargs.fields.DelimitedList(ma.fields.Int)}, location="query")
def foo(x): ...


# webargs 6.x
# this accepts x=1,2,3 ; x=1&x=2&x=3 ; x=1,2&x=3
# to do this, it needs a post_load hook which will flatten out the list data
class UnpackingDelimitedListSchema(ma.Schema):
    x = ma.fields.List(webargs.fields.DelimitedList(ma.fields.Int))

    @ma.post_load
    def flatten_lists(self, data, **kwargs):
        new_x = []
        for x in data["x"]:
            new_x.extend(x)
        data["x"] = new_x
        return data


@parser.use_kwargs(UnpackingDelimitedListSchema, location="query")
def foo(x): ...

ValidationError Messages Are Namespaced Under The Location

Code parsing ValidationError messages will notice a change in the messages produced by webargs. What would previously have come back with messages like {"foo":["Not a valid integer."]} will now have messages nested one layer deeper, like {"json":{"foo":["Not a valid integer."]}}.

To rewrite code which was handling these errors, the handler will need to be prepared to traverse messages by one additional level. For example:

import logging

log = logging.getLogger(__name__)


# webargs 5.x
# logs debug messages like
#   bad value for 'foo': ["Not a valid integer."]
#   bad value for 'bar': ["Not a valid boolean."]
def log_invalid_parameters(validation_error):
    for field, messages in validation_error.messages.items():
        log.debug("bad value for '{}': {}".format(field, messages))


# webargs 6.x
# logs debug messages like
#   bad value for 'foo' [query]: ["Not a valid integer."]
#   bad value for 'bar' [json]: ["Not a valid boolean."]
def log_invalid_parameters(validation_error):
    for location, fielddata in validation_error.messages.items():
        for field, messages in fielddata.items():
            log.debug("bad value for '{}' [{}]: {}".format(field, location, messages))

Custom Error Handler Argument Names Changed

If you define a custom error handler via @parser.error_handler the function arguments are now keyword-only and status_code and headers have been renamed error_status_code and error_headers.

# webargs 5.x
@parser.error_handler
def custom_handle_error(error, req, schema, status_code, headers): ...


# webargs 6.x
@parser.error_handler
def custom_handle_error(error, req, schema, *, error_status_code, error_headers): ...

Some Functions Take Keyword-Only Arguments Now

The signature of several methods has changed to have keyword-only arguments. For the most part, this should not require any changes, but here’s a list of the changes.

parser.error_handler methods:

# webargs 5.x
def handle_error(error, req, schema, status_code, headers): ...


# webargs 6.x
def handle_error(error, req, schema, *, error_status_code, error_headers): ...

parser.__init__ methods:

# webargs 5.x
def __init__(self, location=None, error_handler=None, schema_class=None): ...


# webargs 6.x
def __init__(self, location=None, *, error_handler=None, schema_class=None): ...

parser.parse, parser.use_args, and parser.use_kwargs methods:

# webargs 5.x
def parse(
    self,
    argmap,
    req=None,
    location=None,
    validate=None,
    error_status_code=None,
    error_headers=None,
): ...


# webargs 6.x
def parse(
    self,
    argmap,
    req=None,
    *,
    location=None,
    validate=None,
    error_status_code=None,
    error_headers=None,
): ...


# webargs 5.x
def use_args(
    self,
    argmap,
    req=None,
    location=None,
    as_kwargs=False,
    validate=None,
    error_status_code=None,
    error_headers=None,
): ...


# webargs 6.x
def use_args(
    self,
    argmap,
    req=None,
    *,
    location=None,
    as_kwargs=False,
    validate=None,
    error_status_code=None,
    error_headers=None,
): ...


# use_kwargs is just an alias for use_args with as_kwargs=True

and finally, the dict2schema function:

# webargs 5.x
def dict2schema(dct, schema_class=ma.Schema): ...


# webargs 6.x
def dict2schema(dct, *, schema_class=ma.Schema): ...

PyramidParser Now Appends Arguments (Used To Prepend)

PyramidParser.use_args was not conformant with the other parsers in webargs. While all other parsers added new arguments to the end of the argument list of a decorated view function, the Pyramid implementation added them to the front of the argument list.

This has been corrected, but as a result pyramid views with use_args may need to be rewritten. The request object is always passed first in both versions, so the issue is only apparent with view functions taking other positional arguments.

For example, imagine code with a decorator for passing user information, pass_userinfo, like so:

# a decorator which gets information about the authenticated user
def pass_userinfo(f):
    def decorator(request, *args, **kwargs):
        return f(request, get_userinfo(), *args, **kwargs)

    return decorator

You will see a behavioral change if pass_userinfo is called on a function decorated with use_args. The difference between the two versions will be like so:

from webargs.pyramidparser import use_args


# webargs 5.x
# pass_userinfo is called first, webargs sees positional arguments of
#   (userinfo,)
# and changes it to
#   (request, args, userinfo)
@pass_userinfo
@use_args({"q": ma.fields.String()}, locations=("query",))
def viewfunc(request, args, userinfo):
    q = args.get("q")
    ...


# webargs 6.x
# pass_userinfo is called first, webargs sees positional arguments of
#   (userinfo,)
# and changes it to
#   (request, userinfo, args)
@pass_userinfo
@use_args({"q": ma.fields.String()}, location="query")
def viewfunc(request, userinfo, args):
    q = args.get("q")
    ...