Hi all,

I am not sure why mypy thinks this

gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible type "List[Dict[str, str]]"; expected
"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]" [arg-type]
rows, idx = run_rw_queries(link_obj = conn, queries = queries, return_data = True)
^~~~~~~

should be flagged. The intent is for "queries" to be

a list
of dicts
with keys of str
and values of
str OR
list of anything OR
dict with
keys of str
and values of anything

I'd have thunk list[dict[str,str]] matches that ?

This is on Python 3.11.2 with mypy 1.0.1 on Debian.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Fri, Dec 29, 2023 at 01:15:29PM +0100 schrieb Karsten Hilbert via Python-list:

I am not sure why mypy thinks this

gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible type "List[Dict[str, str]]"; expected
"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]" [arg-type]
rows, idx = run_rw_queries(link_obj = conn, queries = queries, return_data = True)
^~~~~~~

should be flagged. The intent is for "queries" to be

a list
of dicts
with keys of str
and values of
str OR
list of anything OR
dict with
keys of str
and values of anything

I'd have thunk list[dict[str,str]] matches that ?

This is on Python 3.11.2 with mypy 1.0.1 on Debian.

For completeness, this was the mypy call signature:

mypy --pretty --allow-redefinition --no-strict-optional --ignore-missing-imports --follow-imports silent --show-error-codes --warn-unused-ignores gmPG2.py

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 12/29/23 05:15, Karsten Hilbert via Python-list wrote:

Hi all,

I am not sure why mypy thinks this

gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible type "List[Dict[str, str]]"; expected
"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]" [arg-type]
rows, idx = run_rw_queries(link_obj = conn, queries = queries, return_data = True)
^~~~~~~

should be flagged. The intent is for "queries" to be

a list
of dicts
with keys of str
and values of
str OR
list of anything OR
dict with
keys of str
and values of anything

I'd have thunk list[dict[str,str]] matches that ?

Dict[str, str] means the key type and value type should both be strings,
but in your retelling above you indicate lots of possible value types... actually the mypy guess seems to be a pretty good recreation of your psuedo-code description.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Fri, Dec 29, 2023 at 07:49:17AM -0700 schrieb Mats Wichmann via Python-list:

I am not sure why mypy thinks this

gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible type "List[Dict[str, str]]"; expected
"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]" [arg-type]
rows, idx = run_rw_queries(link_obj = conn, queries = queries, return_data = True)
^~~~~~~

should be flagged. The intent is for "queries" to be

a list
of dicts
with keys of str
and values of
str OR
list of anything OR
dict with
keys of str
and values of anything

I'd have thunk list[dict[str,str]] matches that ?

Dict[str, str] means the key type and value type should both be strings,

Indeed, I know that much, list[dict[str, str]] is what is getting
passed in in this particular invocation of run_rw_queries().

For what it's worth here's the signature of that function:

def run_rw_queries (
link_obj:_TLnkObj=None,
queries:list[dict[str, str | list | dict[str, Any]]]=None,
end_tx:bool=False,
return_data:bool=None,
get_col_idx:bool=False,
verbose:bool=False
) -> tuple[list[dbapi.extras.DictRow], dict[str, int] | None]:

Given that I would have thought that passing in
list[dict[str, str]] for "queries" ought to be type safe.
Mypy indicates otherwise which I am not grokking as to why.

but in your
retelling above you indicate lots of possible value types... actually the mypy guess
seems to be a pretty good recreation of your psuedo-code description.

I agree that mypy's grasp of my intent from

queries:list[dict[str, str | list | dict[str, Any]]]=None,

into

"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]"

seems accurate. I just don't understand why list[dict[str,
str]] should not pass that construct.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 12/29/23 08:02, Karsten Hilbert via Python-list wrote:

Dict[str, str] means the key type and value type should both be strings,

Indeed, I know that much, list[dict[str, str]] is what is getting
passed in in this particular invocation of run_rw_queries().

For what it's worth here's the signature of that function:

def run_rw_queries (
link_obj:_TLnkObj=None,
queries:list[dict[str, str | list | dict[str, Any]]]=None,
end_tx:bool=False,
return_data:bool=None,
get_col_idx:bool=False,
verbose:bool=False
) -> tuple[list[dbapi.extras.DictRow], dict[str, int] | None]:

Given that I would have thought that passing in
list[dict[str, str]] for "queries" ought to be type safe.
Mypy indicates otherwise which I am not grokking as to why.

ah... didn't grok what you were asking, sorry - ignore my attempt then.
So you are passing something that has been typed more narrowly than the function parameter. Can you use a TypeGuard here?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Fri, Dec 29, 2023 at 11:04:59AM -0700 schrieb Mats Wichmann via Python-list:

For what it's worth here's the signature of that function:

def run_rw_queries (
link_obj:_TLnkObj=None,
queries:list[dict[str, str | list | dict[str, Any]]]=None,
end_tx:bool=False,
return_data:bool=None,
get_col_idx:bool=False,
verbose:bool=False
) -> tuple[list[dbapi.extras.DictRow], dict[str, int] | None]:

Given that I would have thought that passing in
list[dict[str, str]] for "queries" ought to be type safe.
Mypy indicates otherwise which I am not grokking as to why.

ah... didn't grok what you were asking, sorry - ignore my attempt then.

Never mind, the attempt to help is appreciated.

So you are passing something that has been typed more
narrowly than the function parameter.

That would then sort of skirt on violation of the Liskov
principle, of which I learned while trying to research this
mypy behaviour.

However, I would not think the above to be a narrowing-down
as it just *selects* one of the explicitely "legal" options.

list[dict[str, str | list | dict[str, Any]]]

should AFAICT expand to:

list[dict[str, dict[str, Any]]]

OR

list[dict[str, list]]

OR

list[dict[str, str]]

the last of which should provide coverage of

[{'some key': 'some value'}]

Can you use a TypeGuard here?

Not from what I understand about them...

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 30/12/23 4:02 am, Karsten Hilbert wrote:

def run_rw_queries (
link_obj:_TLnkObj=None,
queries:list[dict[str, str | list | dict[str, Any]]]=None,

Given that I would have thought that passing in
list[dict[str, str]] for "queries" ought to be type safe.

dict[str, str] is not a subtype of dict[str, str | something_else]
because you can assign a value of type something_else to the latter
but not the former.

In this case it happens to be okay because the function is (presumably) treating the dict passed in as immutable, but MyPy has no way to be sure
of that.

You could try declaring it as a collections.Mapping, which is immutable.

--
Greg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 12/29/2023 10:02 AM, Karsten Hilbert via Python-list wrote:

I agree that mypy's grasp of my intent from

queries:list[dict[str, str | list | dict[str, Any]]]=None,

into

"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]"

seems accurate. I just don't understand why list[dict[str,
str]] should not pass that construct.

I made a tiny test program with your type signature, and got this error
message from mypy:

c:\temp\python\typing_test.py:3: error: X | Y syntax for unions requires
Python 3.10 [syntax]

Aside from that, this variation causes no mypy error (you must use
Sequence instead of List), and is also more clear about what you are
trying to get:

from typing import Union, Sequence, Dict

DictType1 = Dict[str, str]
DictType2 = Dict[str, Sequence]
DictType3 = Dict[str, Dict]
QueryType = Sequence[Union[DictType1, DictType2, DictType3]]

def test_typing(queries:QueryType=None):
print(type(queries))

d1 = {'k1': 'v1', 'k2': 'v2'}
queries = [d1,]
test_typing(queries)

I'm not sure if this captures exactly what you want, but it avoids the
problem where mypy does not regard str and Union[str, list] as
equivalent types. I tested this using Python 3.12.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 30 Dec 2023, at 15:11, Karsten Hilbert via Python-list <python-list@python.org> wrote:

queries = [{'SQL': 'SELECT %(value)s', 'args': {'value': 1}}]

and

run_queries(conn, queries:list[str|dict[str, Any]]):

In cases like this I often use a wrapper class in place of a simple str.
If you have a class SqlString then your type becomes list[SqlString].

You may find that SqlString gains interesting methods over time.

Barry

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 12/29/2023 10:02 AM, Karsten Hilbert via Python-list wrote:

Am Fri, Dec 29, 2023 at 07:49:17AM -0700 schrieb Mats Wichmann via Python-list:

I am not sure why mypy thinks this

gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible type "List[Dict[str, str]]"; expected
"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]" [arg-type]
rows, idx = run_rw_queries(link_obj = conn, queries = queries, return_data = True)
^~~~~~~

should be flagged. The intent is for "queries" to be

a list
of dicts
with keys of str
and values of
str OR
list of anything OR
dict with
keys of str
and values of anything

I'd have thunk list[dict[str,str]] matches that ?

Dict[str, str] means the key type and value type should both be strings,

Indeed, I know that much, list[dict[str, str]] is what is getting
passed in in this particular invocation of run_rw_queries().

For what it's worth here's the signature of that function:

def run_rw_queries (
link_obj:_TLnkObj=None,
queries:list[dict[str, str | list | dict[str, Any]]]=None,
end_tx:bool=False,
return_data:bool=None,
get_col_idx:bool=False,
verbose:bool=False
) -> tuple[list[dbapi.extras.DictRow], dict[str, int] | None]:

Given that I would have thought that passing in
list[dict[str, str]] for "queries" ought to be type safe.
Mypy indicates otherwise which I am not grokking as to why.

but in your
retelling above you indicate lots of possible value types... actually the mypy guess
seems to be a pretty good recreation of your psuedo-code description.

I agree that mypy's grasp of my intent from

queries:list[dict[str, str | list | dict[str, Any]]]=None,

into

"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]"

seems accurate. I just don't understand why list[dict[str,
str]] should not pass that construct.

Maybe better to ask the mypy people directly.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 12/30/2023 9:14 AM, Thomas Passin via Python-list wrote:

On 12/29/2023 10:02 AM, Karsten Hilbert via Python-list wrote:

I agree that mypy's grasp of my intent from

    queries:list[dict[str, str | list | dict[str, Any]]]=None,

into

    "List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]"

seems accurate. I just don't understand why list[dict[str,
str]] should not pass that construct.

I made a tiny test program with your type signature, and got this error message from mypy:

c:\temp\python\typing_test.py:3: error: X | Y syntax for unions requires Python 3.10  [syntax]

Aside from that, this variation causes no mypy error (you must use
Sequence instead of List), and is also more clear about what you are
trying to get:

from typing import Union, Sequence, Dict

DictType1 = Dict[str, str]
DictType2 = Dict[str, Sequence]
DictType3 = Dict[str, Dict]
QueryType = Sequence[Union[DictType1, DictType2, DictType3]]

def test_typing(queries:QueryType=None):
    print(type(queries))

d1 = {'k1': 'v1', 'k2': 'v2'}
queries = [d1,]
test_typing(queries)

I'm not sure if this captures exactly what you want, but it avoids the problem where mypy does not regard str and Union[str, list] as
equivalent types.  I tested this using Python 3.12.

In doing more testing, I have learned that my suggestion above does
work, *except* that you cannot mix-and-match different DictTypex types
within the same Sequence - meaning within the same query argument. Any
of the Union types is OK but they all have to be the same in any instance.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 31/12/23 10:06 am, Thomas Passin wrote:

my suggestion above does
work, *except* that you cannot mix-and-match different DictTypex types

Have you tried declaring the argument as a Mapping instead of a dict?
Seeing as Thomas Passin's Sequence experiment worked, it seems like this
should work too.

--
Greg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Thanks to all. I ended up using Sequence for the list part
and Mapping for the dict part, which does require "import
typing" which I would rather have avoided.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Op 29/12/2023 om 16:02 schreef Karsten Hilbert via Python-list:

Am Fri, Dec 29, 2023 at 07:49:17AM -0700 schrieb Mats Wichmann via Python-list:

I am not sure why mypy thinks this

gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible type "List[Dict[str, str]]"; expected
"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]" [arg-type]
rows, idx = run_rw_queries(link_obj = conn, queries = queries, return_data = True)
^~~~~~~

should be flagged. The intent is for "queries" to be

a list
of dicts
with keys of str
and values of
str OR
list of anything OR
dict with
keys of str
and values of anything

I'd have thunk list[dict[str,str]] matches that ?

Dict[str, str] means the key type and value type should both be strings,

Indeed, I know that much, list[dict[str, str]] is what is getting
passed in in this particular invocation of run_rw_queries().

For what it's worth here's the signature of that function:

def run_rw_queries (
link_obj:_TLnkObj=None,
queries:list[dict[str, str | list | dict[str, Any]]]=None,
end_tx:bool=False,
return_data:bool=None,
get_col_idx:bool=False,
verbose:bool=False
) -> tuple[list[dbapi.extras.DictRow], dict[str, int] | None]:

Given that I would have thought that passing in
list[dict[str, str]] for "queries" ought to be type safe.
Mypy indicates otherwise which I am not grokking as to why.

but in your
retelling above you indicate lots of possible value types... actually the mypy guess
seems to be a pretty good recreation of your psuedo-code description.

I agree that mypy's grasp of my intent from

queries:list[dict[str, str | list | dict[str, Any]]]=None,

into

"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]"

seems accurate. I just don't understand why list[dict[str,
str]] should not pass that construct.

Sorry for the late reaction and may be I am missing something, but I was wondering if
your type hint for queries shouldn't be the following.

queries:list[dict[str,str]|dict[str,list]|dict[str,dict[str, dict[str, Ant]]]

My impression at this moment is that you are write something like: dict[str, str | int] as
as shorthand for dict[str, str] | dict[str, int]. But those two are different types.

--
Antoon Pardon.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Sat, Jan 13, 2024 at 09:20:00PM +0100 schrieb Karsten Hilbert via Python-list:

I was wondering if
your type hint for queries shouldn't be the following.

queries:list[dict[str,str]|dict[str,list]|dict[str,dict[str, dict[str, Ant]]]

Wait, not really. Let me give an example. Here's three times
the same query (as far as PostgreSQL is concerned, after
having been passed through psycopg2):

queries = [
{
'SQL': 'SELECT 1'
},
{
'SQL': 'SELECT %s',
'args': [1]
},
{
'SQL': 'SELECT %(value)s',
'args': {'value': 1}
}
]

The value for key "SQL" will always be str-like.

The value for "args" can be a list or a dict itself.

If "args" is a dict it will be of type [str, Any].

That's what I am trying to tell mypy.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Fri, Jan 12, 2024 at 02:23:43PM +0100 schrieb Antoon Pardon via Python-list:

queries:list[dict[str, str | list | dict[str, Any]]]=None,

into

"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]"

seems accurate. I just don't understand why list[dict[str,
str]] should not pass that construct.

Sorry for the late reaction

ne'er mind ya

I was wondering if
your type hint for queries shouldn't be the following.

queries:list[dict[str,str]|dict[str,list]|dict[str,dict[str, dict[str, Ant]]]

My impression at this moment is that you are write something like: dict[str, str | int] as
as shorthand for dict[str, str] | dict[str, int].

I do.

But those two are different types.

A-ha ! In what way ?

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)