• exec() an locals() puzzle

    From george trojan@21:1/5 to All on Wed Jul 20 16:56:02 2022
    I wish I could understand the following behaviour:

    1. This works as I expect it to work:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    exec('y *= 2')
    print('ok:', eval('y'))
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1}
    ok: 2

    2. I can access the value of y with eval() too:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    u = eval('y')
    print(u)
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1}
    1

    3. When I change variable name u -> y, somehow locals() in the body of
    the function loses an entry:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    y = eval('y')
    print(y)
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1}

    ---------------------------------------------------------------------------NameError
    Traceback (most recent call last)
    Input In [1], in <cell line: 10>() 7 print(y) 8 # y
    = eval('y') 9 #print('ok:', eval('y'))---> 10 f()

    Input In [1], in f() 4 exec('y = i; print(y); print(locals())')
    5 print(locals())----> 6 y = eval('y') 7 print(y)

    File <string>:1, in <module>
    NameError: name 'y' is not defined1.

    Another thing: within the first exec(), the print order seems
    reversed. What is going on?

    BTW, I am using python 3.8.13.

    George

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Eryk Sun@21:1/5 to george trojan on Wed Jul 20 13:54:06 2022
    On 7/20/22, george trojan <george.trojan@gmail.com> wrote:

    1. This works as I expect it to work:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    exec('y *= 2')
    print('ok:', eval('y'))
    f()

    In CPython, the locals of a function scope (as opposed to a class
    scope or module scope) are optimized by storing them in an array in
    the current frame. locals(), however, always returns the non-optimized
    locals mapping of the current frame, i.e. the value of sys._getframe(0).f_locals. In the case of optimized locals, the
    locals() call first updates this dict with a snapshot of the current
    values of local variables. This is useful for introspection, but
    modifying the snapshot cannot extend or modify the optimized local
    variables of the function scope.

    exec() and eval() default to using the global and local variable
    mappings that are returned by globals() and locals(). This allows them
    to access the snapshot of the containing function's optimized locals,
    but they can't extend or modify the function's optimized locals. At
    most they can add dynamic locals to the snapshot, as used by
    subsequent calls to locals(), exec() and eval().

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Martin Di Paola@21:1/5 to george trojan on Wed Jul 20 16:28:28 2022
    I did a few tests

    # test 1
    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    a = eval('y')
    print(locals())
    u = a
    print(u)
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1, 'a': 1}
    1

    # test 2
    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    a = eval('y')
    print(locals())
    y = a
    print(y)
    f()
    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1}
    Traceback (most recent call last):
    NameError: name 'y' is not defined


    So test 1 and 2 are the same except that the variable 'y' is not present/present in the f's code.

    When it is not present, exec() modifies the f's locals and adds an 'y'
    to it but when the variable 'y' is present in the code (even if not
    present in the locals()), exec() does not add any 'y' (and the next
    eval() then fails)

    The interesting part is that if the 'y' variable is in the f's code
    *and* it is defined in the f's locals, no error occur but once again the
    exec() does not modify f's locals:

    # test 3
    def f():
    i = 1
    y = 42
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    a = eval('y')
    print(locals())
    y = a
    print(y)
    f()
    {'i': 1, 'y': 42}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 42}
    {'i': 1, 'y': 42, 'a': 42}
    42

    Why does this happen? No idea.

    I may be related with this:

    # test 4
    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    print(y)
    f()
    Traceback (most recent call last):
    NameError: name 'y' is not defined

    Despite exec() adds the 'y' variable to f's locals, the variable is not accessible/visible from f's code.

    So, a few observations (by no means this is how the vm works):

    1) each function has a set of variables defined by the code (let's call
    this "code-defined locals" or "cdef-locals").
    2) each function also has a set of "runtime locals" accessible from
    locals().
    3) exec() can add variables to locals() (runtime) set but it cannot add
    any to cdef-locals.
    4) locals() may be a superset of cdef-locals (but entries in cdef-locals
    which value is still undefined are not shown in locals())
    5) due rule 4, exec() cannot add a variable to locals() if it is already
    present in the in cdef-locals.
    6) when eval() runs, it uses locals() set for lookup

    Perhaps rule 5 is to prevent exec() to modify any arbitrary variable of
    the caller...

    Anyways, nice food for our brains.

    On Wed, Jul 20, 2022 at 04:56:02PM +0000, george trojan wrote:
    I wish I could understand the following behaviour:

    1. This works as I expect it to work:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    exec('y *= 2')
    print('ok:', eval('y'))
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1}
    ok: 2

    2. I can access the value of y with eval() too:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    u = eval('y')
    print(u)
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1}
    1

    3. When I change variable name u -> y, somehow locals() in the body of
    the function loses an entry:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    y = eval('y')
    print(y)
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1}

    ---------------------------------------------------------------------------NameError
    Traceback (most recent call last)
    Input In [1], in <cell line: 10>() 7 print(y) 8 # y
    = eval('y') 9 #print('ok:', eval('y'))---> 10 f()

    Input In [1], in f() 4 exec('y = i; print(y); print(locals())')
    5 print(locals())----> 6 y = eval('y') 7 print(y)

    File <string>:1, in <module>
    NameError: name 'y' is not defined1.

    Another thing: within the first exec(), the print order seems
    reversed. What is going on?

    BTW, I am using python 3.8.13.

    George
    --
    https://mail.python.org/mailman/listinfo/python-list

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From george trojan@21:1/5 to All on Thu Jul 21 19:07:12 2022
    Thanks. That cdef-locals concept is consistent with the following example:

    def f():
    i = 1
    def g(): print('i' in globals(), 'i' in locals())
    def h(): print('i' in globals(), 'i' in locals()); i
    g()
    h()
    f()

    False False
    False True

    It is a mystery, which may be why the documentation for globals() and
    locals() is 2-line long.


    Le mer. 20 juill. 2022, à 19 h 31, Martin Di Paola < martinp.dipaola@gmail.com> a écrit :

    I did a few tests

    # test 1
    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    a = eval('y')
    print(locals())
    u = a
    print(u)
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1, 'a': 1}
    1

    # test 2
    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    a = eval('y')
    print(locals())
    y = a
    print(y)
    f()
    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1}
    Traceback (most recent call last):
    NameError: name 'y' is not defined


    So test 1 and 2 are the same except that the variable 'y' is not present/present in the f's code.

    When it is not present, exec() modifies the f's locals and adds an 'y'
    to it but when the variable 'y' is present in the code (even if not
    present in the locals()), exec() does not add any 'y' (and the next
    eval() then fails)

    The interesting part is that if the 'y' variable is in the f's code
    *and* it is defined in the f's locals, no error occur but once again the exec() does not modify f's locals:

    # test 3
    def f():
    i = 1
    y = 42
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    a = eval('y')
    print(locals())
    y = a
    print(y)
    f()
    {'i': 1, 'y': 42}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 42}
    {'i': 1, 'y': 42, 'a': 42}
    42

    Why does this happen? No idea.

    I may be related with this:

    # test 4
    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    print(y)
    f()
    Traceback (most recent call last):
    NameError: name 'y' is not defined

    Despite exec() adds the 'y' variable to f's locals, the variable is not accessible/visible from f's code.

    So, a few observations (by no means this is how the vm works):

    1) each function has a set of variables defined by the code (let's call
    this "code-defined locals" or "cdef-locals").
    2) each function also has a set of "runtime locals" accessible from
    locals().
    3) exec() can add variables to locals() (runtime) set but it cannot add
    any to cdef-locals.
    4) locals() may be a superset of cdef-locals (but entries in cdef-locals which value is still undefined are not shown in locals())
    5) due rule 4, exec() cannot add a variable to locals() if it is already
    present in the in cdef-locals.
    6) when eval() runs, it uses locals() set for lookup

    Perhaps rule 5 is to prevent exec() to modify any arbitrary variable of
    the caller...

    Anyways, nice food for our brains.

    On Wed, Jul 20, 2022 at 04:56:02PM +0000, george trojan wrote:
    I wish I could understand the following behaviour:

    1. This works as I expect it to work:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    exec('y *= 2')
    print('ok:', eval('y'))
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1}
    ok: 2

    2. I can access the value of y with eval() too:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    u = eval('y')
    print(u)
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1, 'y': 1}
    1

    3. When I change variable name u -> y, somehow locals() in the body of
    the function loses an entry:

    def f():
    i = 1
    print(locals())
    exec('y = i; print(y); print(locals())')
    print(locals())
    y = eval('y')
    print(y)
    f()

    {'i': 1}
    1
    {'i': 1, 'y': 1}
    {'i': 1}


    ---------------------------------------------------------------------------NameError
    Traceback (most recent call last)
    Input In [1], in <cell line: 10>() 7 print(y) 8 # y
    = eval('y') 9 #print('ok:', eval('y'))---> 10 f()

    Input In [1], in f() 4 exec('y = i; print(y); print(locals())')
    5 print(locals())----> 6 y = eval('y') 7 print(y)

    File <string>:1, in <module>
    NameError: name 'y' is not defined1.

    Another thing: within the first exec(), the print order seems
    reversed. What is going on?

    BTW, I am using python 3.8.13.

    George
    --
    https://mail.python.org/mailman/listinfo/python-list
    --
    https://mail.python.org/mailman/listinfo/python-list


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)