Add a covariant `Mapping`-like type

[harahu]

What

I want a new protocol added to typing or typing_extensions, expressing the aspects of Mapping that are covariant with regards to the key type. That is, the Mapping type, but without the __getitem__ and get methods.

Something along the lines of:

class CovariantMappingLike(Protocol[KT_co, VT_co]):
    # A better name for this protocol might be needed
    def __eq__(self, other: object) -> bool: ...
    def __ne__(self, other: object) -> bool: ...
    def keys(self) -> KeysView[KT_co]: ...
    def items(self) -> ItemsView[KT_co, VT_co]: ...
    def values(self) -> ValuesView[VT_co]: ...

Why

Mappings are often, in practice, treated in ways that make them covariant with regards to key. A specific example is laid out in a comment of mine found here: python/typing#445 (comment)

The Mapping type, however, cannot be covariant w. regards to key, as it also implements methods that don't allow for it (__getitem__ and get) to be covariant.

This leaves me unable to express, in a simple and idiomatic way, that I intend to use a Mapping in a way that respects covariance.

Being able to write:

def foo(mapping: CovariantMappingLike[Hashable, Hashable]) -> int:
    ...

would allow me to express what I'm after. It would allow me to express: "Give me a mapping, but I promise to not be looking up keys on it."

This, again, makes it possible to have narrower key types for my Mapping objects, as I can pass a narrowly typed Mapping to foo, without casting:

my_map: Dict[Literal["foo", "bar", "baz"], str] = {}

# I want this to be fine
foo(my_mapping)

I was prompted by @srittau to create this issue in python/typing#445 (comment)

[srittau]

As I wrote in the original issue, I think this would be quite useful in addition to our tight protocols. Maybe we could have a look at the use of Mapping and the tighter protocols in typeshed to determine, which methods are best suited for such a protocol and add it to typing_extensions first to gain some experience.

[harahu]

The proposal I made, was intended to be "as broad as possible, but without breaking covariance". In light of that, I should probably inherit from Container as well. Unsure if there are more methods one could add, while maintaining covariance, than the ones I listed.

The reason I want a type that is "as broad as possible", has to do with usability. Tight protocols can, unfortunately, be cumbersome, in some situations, as long as we lack intersection types.

A concrete (but perhaps silly) example of a function in need of annotation:

def do_more_than_one_thing_with_a_mapping(mapping):
   # The exact methods called on the mapping are unimportant. 
   # The salient detail is that there's more than one method being called.
   return any(mapping.keys()), all(mapping.values())

And three ways of annotating it:

1. Not currently possible, but what I hope Python 3.13 will look like:

def silly_fn(mapping: SupportsKeys & SupportsItems) -> tuple[bool, bool]:
    ...

2. The most direct equivalent possible today:

class SupportsKeysAndItems(SupportsKeys[KT_co], SupportsItems[VT_co], Protocol[KT_co, VT_co]):
    # A full extra class definition, just to create an intersection type(!)
    ...

def silly_fn(mapping: SupportsKeysAndItems) -> tuple[bool, bool]:
    ...

3. Using a type that is broader in scope than it needs to be, but avoiding having to manually create the correct intersection type:

def silly_fn(mapping: CovariantMappingLike) -> tuple[bool, bool]:
    ...

My preference looks like: 1 > 3 > 2, so until we can have 1, I'd prefer having the option of 3 available when calling more than one method on the mapping.

Not trying to say that narrow protocols are bad. They can be really useful when annotating simple functions. So I want them around. It's just that the syntax for composing them is so bad that I'd like someone else to do it for me, neatly hidden away in some library.

[gvanrossum]

I feel this is too rarely needed to add here. You can just define a protocol in your own code.

[harahu]

I feel this is too rarely needed to add here. You can just define a protocol in your own code.

The need to interact co-variantly with mapping keys seems pretty common to me. See:

• Reconsider making Mapping covariant in the key type typing#445
• Make Mapping covariant in key mypy#1114

This type can be used for every function that interacts with a mapping without looking up values, i.e. covering all "safe" use-cases of what's suggested in the issues mentioned above. That's a pretty good compromise solution IMO.

The Mapping stubs in typeshed can also be defined based on this type, making for a neat simplification, and also isolates the part of the Mapping API that leads to invariance, which, I think, is a good idea.

[gvanrossum]

If we have to have it, maybe CovariantMapping?

[harahu]

If we have to have it, maybe CovariantMapping?

One objection to that name is that it can sound a bit like a subtype of Mapping. But in this case it's a supertype. Not sure how to convey that elegantly.

[gvanrossum]

I don't think you can get the perfect name, so I recommend not overthinking it.

[hmc-cs-mdrissi]

My main question is when would you choose covariant mapping vs mapping with a typevar for key? What's difference between,

def foo(mapping: CovariantMapping[Hashable, Hashable]):
  ...

vs

HashableT = TypeVar('HashableT', bound=Hashable)

def foo(mapping: Mapping[HashableT, Hashable]):
  ...

I occasionally use latter as a way to make an invariant type argument behave as if it were covariant.

[harahu]

I occasionally use latter as a way to make an invariant type argument behave as if it were covariant.

I have, and am using the same pattern. But it is both more verbose, and harder to understand for the reader.

Given that most documentation for TypeVars showcase their purpose as illustrating dependencies between types in more than one location, I think one need to expect the reader to have a pretty strong understanding of the concept before they'll be able to effortlessly read you second example.

That is, for the inexperienced reader, seeing: Mapping[HashableT, Hashable] without seeing HashableT appear a second time in the function definition, has the potential of creating confusion. This shouldn't be an issue with CovariantMapping[Hashable, Hashable]

[carljm]

I agree that the need for a mapping type that is covariant in its key does come up regularly, but a mapping protocol that doesn't implement __getitem__ at all seems too limited; is it really that common to use a mapping and never index into it? The common use cases would seem to be limited to when you are just iterating over the keys and values in the mapping, or comparing it to another mapping. Indexing seems pretty fundamental to what it means to be a mapping.

I think a more useful "covariant in keys" mapping type would be one that does provide __getitem__ but has it accept object instead of the key type, as originally proposed as option 2 in python/typing#445. Without typechecker special casing, this means you effectively give up type-checking of indices (so the type checker won't catch m[3] as an error if m: CovariantMapping[str, str] -- IMO this is not a big deal, as this is arguably not even a type error, just a key error), but it allows the mapping to be covariant in the key type and still permit indexing.

I think it's too late to make this change for Mapping, but if we do introduce a new CovariantMapping, I think it would be more useful this way.