conds = [ lambda msg: msg.hascode(z) for z in ("foo", "bar") ]
Hi list,
I've just encounted something that I found extremely unintuitive and
would like your feedback. This bit me *hard*, causing me to question my sanity for a moment. Consider this minimal example code (Py 3.10.4 on
Linux x64):
class Msg():
def hascode(self, value):
print("Check for", value)
return False
conds = [
lambda msg: msg.hascode("foo"),
lambda msg: msg.hascode("bar"),
]
msg = Msg()
print(conds[0](msg))
print(conds[1](msg))
It works perfectly and does exactly what it looks like. The output is:
Check for foo
False
Check for bar
False
But now consider what happens when we create the lambdas inside a list comprehension (in my original I used a generator expresison, but the
result is the same). Can you guess what happens when we create conds
like this?
conds = [ lambda msg: msg.hascode(z) for z in ("foo", "bar") ]
I certainly could not. Here's what it outputs:
Check for bar
False
Check for bar
False
I.e., the iteration variable "z" somehow gets bound inside the lambda
not by its value, but by its reference. All checks therefore refence
only the last variable.
This totally blew my mind. I can understand why it's happening, but is
this the behavior we would expect? And how can I create lambdas inside a generator expression and tell the expression to use the *value* and not
pass the "z" variable by reference?
Cheers,
Joe
funcs = []
for x in[ 0, 1, 2 ]: funcs.append( lambda: x )
|2x = 1; f1 = lambda: x
x = 2; f2 = lambda: x
f1()
|5f = lambda: a
a = 5
f()
|... return idef f():
|2i = 2
f()
Aha!
conds = [ lambda msg, z = z: msg.hascode(z) for z in ("foo", "bar") ]
Is what I was looking for to explicitly use the value of z. What a
caveat, didn't see that coming.
Learning something new every day.
Cheers,
Joe
Am 29.06.22 um 11:50 schrieb Johannes Bauer:
Hi list,
I've just encounted something that I found extremely unintuitive and
would like your feedback. This bit me *hard*, causing me to question my
sanity for a moment. Consider this minimal example code (Py 3.10.4 on
Linux x64):
class Msg():
def hascode(self, value):
print("Check for", value)
return False
conds = [
lambda msg: msg.hascode("foo"),
lambda msg: msg.hascode("bar"),
]
msg = Msg()
print(conds[0](msg))
print(conds[1](msg))
It works perfectly and does exactly what it looks like. The output is:
Check for foo
False
Check for bar
False
But now consider what happens when we create the lambdas inside a list
comprehension (in my original I used a generator expresison, but the
result is the same). Can you guess what happens when we create conds
like this?
conds = [ lambda msg: msg.hascode(z) for z in ("foo", "bar") ]
I certainly could not. Here's what it outputs:
Check for bar
False
Check for bar
False
I.e., the iteration variable "z" somehow gets bound inside the lambda
not by its value, but by its reference. All checks therefore refence
only the last variable.
This totally blew my mind. I can understand why it's happening, but is
this the behavior we would expect? And how can I create lambdas inside a
generator expression and tell the expression to use the *value* and not
pass the "z" variable by reference?
Cheers,
Joe
But now consider what happens when we create the lambdas inside a list comprehension (in my original I used a generator expresison, but the
result is the same). Can you guess what happens when we create conds
like this?
conds = [ lambda msg: msg.hascode(z) for z in ("foo", "bar") ]
I certainly could not. Here's what it outputs:
Check for bar
False
Check for bar
False
I.e., the iteration variable "z" somehow gets bound inside the lambda
not by its value, but by its reference. All checks therefore refence
only the last variable.
On Thu, 30 Jun 2022 at 02:49, Johannes Bauer <dfnsonfsduifb@gmx.de> wrote:
But now consider what happens when we create the lambdas inside a list
comprehension (in my original I used a generator expresison, but the
result is the same). Can you guess what happens when we create conds
like this?
conds = [ lambda msg: msg.hascode(z) for z in ("foo", "bar") ]
I certainly could not. Here's what it outputs:
Check for bar
False
Check for bar
False
I.e., the iteration variable "z" somehow gets bound inside the lambda
not by its value, but by its reference. All checks therefore refence
only the last variable.
Yep, that is the nature of closures. (Side point: This isn't actually
a generator expression, it's a list comprehension; current versions of
Python treat them broadly the same way, but there was previously a
difference in the way scoping worked.) What you're seeing is a
consequence of the way that closures work, and it is a very good thing
most of the time :)
The usual way to "snapshot" a variable is what you showed in your
followup: a default argument value.
def f(..., z=z):
... z has been snapshot
(As others have pointed out, this isn't unique to lambdas; any
function will behave that way.)
Antoon offered another variant, but written as a pair of lambda
functions, it's a little hard to see what's going on. Here's the same technique written as a factory function:
def does_it_have(z):
return lambda msg: msg.hascode(z)
conds = [does_it_have(z) for z in ("foo", "bar")]
Written like this, it's clear that the variable z in the comprehension
is completely different from the one inside does_it_have(), and they
could have different names if you wanted to. This is a fairly clean
way to snapshot too, and has the advantage that it doesn't pretend
that the function takes an extra parameter.
def __init__(self, msg):class Msg:
[False, True]conds = [partial(lambda z, msg: msg.hascode(z), z) for z in ("foo", "bar")]
[cond(Msg("barbaz")) for cond in conds]
[False, True]conds = [methodcaller("hascode", z) for z in ("foo", "bar")]
[cond(Msg("barbaz")) for cond in conds]
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 293 |
Nodes: | 16 (2 / 14) |
Uptime: | 227:27:11 |
Calls: | 6,624 |
Calls today: | 6 |
Files: | 12,171 |
Messages: | 5,318,856 |