On Thu, 2 Mar 2023 at 06:37, <avi.e.gross@gmail.com> wrote:

If a workaround like itertools.count.__next__() is used because it will not be interrupted as it is implemented in C, then I have to ask if it would
make sense for Python to supply something similar in the standard library
for the sole purpose of a use in locks.

That's not lock-free :) The only way that it works is because it's
locked against other threads doing the same job. Lock-free ID
generation means that:

1) Two threads can request IDs simultaneously and will not block each other
2) No two "request an ID" calls will ever return the same value
3) Preferably (but not required), IDs are not wasted.

PostgreSQL has ways of doing this, and there are a few other ways, but
simply using a count object and relying on the GIL isn't going to
achieve the first (though it'll happily achieve the other two).

ChrisA

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Chris Angelico <rosuav@gmail.com> writes:

PostgreSQL has ways of doing this, and there are a few other ways, but
simply using a count object and relying on the GIL isn't going to
achieve the first (though it'll happily achieve the other two).

It seems like a big mistake to make the hazards of locks gone wild even
worse than they are in Python. So far I've been able to write almost
all my multi-threaded Python code in a fairly safe CSP (Erlang) style,
but there are performance costs to that. For example, ID generation in
that style would work by having an incrementing ID owned by a single
thread.

If there is a real appetite for shared memory concurrency in Python, I
think it would best be implemented by adding a transactional memory
library similar to the ones in Haskell and Clojure. That has its own performance cost in the contended case, but Python's spirit is it's a not-super-fast interpreted language designed for easy development.

Transactional memory has been compared to garbage collection in that it eliminates a programming pain point at the cost of some machine cycles.
That seems like a good match for Python.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

If a workaround like itertools.count.__next__() is used because it will not
be interrupted as it is implemented in C, then I have to ask if it would
make sense for Python to supply something similar in the standard library
for the sole purpose of a use in locks.

But realistically, this is one place the concept of an abstract python
language intersects aspects of what is bundled into a sort of core at or
soon after startup, as well as the reality that python can be implemented in many ways including some ways on some hardware that may not make guarantees
to behave this way.

Realistically, the history of computing is full of choices made that now
look less useful or obvious.

What would have happened if all processors had been required to have some
low level instruction that effectively did something in an atomic way that allowed a way for anyone using any language running on that machine a way to
do a simple thing like set a lock or check it?

Of course life has also turned out to be more complex. Some architectures
can now support a small number of operations and implement others as sort of streams of those operations liked together. You would need to be sure your program is very directly using the atomic operation directly.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On Behalf Of Dieter Maurer
Sent: Wednesday, March 1, 2023 1:43 PM
To: Chris Angelico <rosuav@gmail.com>
Cc: python-list@python.org
Subject: Look free ID genertion (was: Is there a more efficient threading lock?)

Chris Angelico wrote at 2023-3-1 12:58 +1100:

...
The
atomicity would be more useful in that context as it would give
lock-free ID generation, which doesn't work in Python.

I have seen `itertools.count` for that.
This works because its `__next__` is implemented in "C" and therefore will
not be interrupted by a thread switch.
--
https://mail.python.org/mailman/listinfo/python-list

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2023-03-01 at 14:35:35 -0500,
avi.e.gross@gmail.com wrote:

What would have happened if all processors had been required to have
some low level instruction that effectively did something in an atomic
way that allowed a way for anyone using any language running on that
machine a way to do a simple thing like set a lock or check it?

Have happened? I don't know about "required," but processors have
indeed had such instructions for decades; e.g., the MC68000 from the
early to mid 1980s (and used in the original Apple Macintosh, but I
digress) has/had a Test and Set instruction.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 2 Mar 2023 at 08:01, <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:

On 2023-03-01 at 14:35:35 -0500,
avi.e.gross@gmail.com wrote:

What would have happened if all processors had been required to have
some low level instruction that effectively did something in an atomic
way that allowed a way for anyone using any language running on that machine a way to do a simple thing like set a lock or check it?

Have happened? I don't know about "required," but processors have
indeed had such instructions for decades; e.g., the MC68000 from the
early to mid 1980s (and used in the original Apple Macintosh, but I
digress) has/had a Test and Set instruction.

As have all CPUs since; it's the only way to implement locks (push the
locking all the way down to the CPU level).

ChrisA

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2023-03-02, Chris Angelico <rosuav@gmail.com> wrote:

On Thu, 2 Mar 2023 at 08:01, <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:

On 2023-03-01 at 14:35:35 -0500,
avi.e.gross@gmail.com wrote:

What would have happened if all processors had been required to have
some low level instruction that effectively did something in an atomic
way that allowed a way for anyone using any language running on that
machine a way to do a simple thing like set a lock or check it?

Have happened? I don't know about "required," but processors have
indeed had such instructions for decades; e.g., the MC68000 from the
early to mid 1980s (and used in the original Apple Macintosh, but I
digress) has/had a Test and Set instruction.

As have all CPUs since; it's the only way to implement locks (push the locking all the way down to the CPU level).

Indeed, I remember thinking it was very fancy when they added the SWP instruction to the ARM processor.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

So I guess we know what would have happened.

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Python-list <python-list-bounces+gweatherby=uchc.edu@python.org> on behalf of Chris Angelico <rosuav@gmail.com>
Sent: Wednesday, March 1, 2023 8:45:50 PM
To: python-list@python.org <python-list@python.org>
Subject: Re: Look free ID genertion (was: Is there a more efficient threading lock?)

*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

On Thu, 2 Mar 2023 at 08:01, <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:

On 2023-03-01 at 14:35:35 -0500,
avi.e.gross@gmail.com wrote:

What would have happened if all processors had been required to have
some low level instruction that effectively did something in an atomic
way that allowed a way for anyone using any language running on that machine a way to do a simple thing like set a lock or check it?

Have happened? I don't know about "required," but processors have
indeed had such instructions for decades; e.g., the MC68000 from the
early to mid 1980s (and used in the original Apple Macintosh, but I
digress) has/had a Test and Set instruction.

As have all CPUs since; it's the only way to implement locks (push the
locking all the way down to the CPU level).

ChrisA
-- https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!g70S067RzF2oPCFUYpFFzUvPHRfS0AHIGEvVyww1Tlj7BCCrsU3DWIqCE9UBO_ex0ZVanquFLHGe1d2b$

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 2 Mar 2023 at 13:02, Weatherby,Gerard <gweatherby@uchc.edu> wrote:

So I guess we know what would have happened.

Yep. It's not what I was talking about, but it's also a very important concurrency management feature.

ChrisA

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 2 Mar 2023 12:45:50 +1100, Chris Angelico <rosuav@gmail.com>
declaimed the following:

As have all CPUs since; it's the only way to implement locks (push the >locking all the way down to the CPU level).

Xerox Sigma (circa 1970): Modify and Test (byte/halfword/word)

Granted, that was a "mainframe" system, not a microprocessor.

Looks like Intel didn't catch the boat until 1985 and the i386.



--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)