Fair point. However, I gave it a shot for the following reason:

I couldn’t find a way to make such performant function. Using python builtin components still ends up several times slower than builtin `all`. Cython or numba or similar is not an option as they do not support `truth` values. Or if they do, it ends up
slower than pure python variant.

So all what is left is writing a proper extension. Which I would prefer not to do for 1 function. I thought maybe `xor`, as in logical XOR functionality in its vanilla case could be compelling. And after doing a bit of search I see that very very few
languages have that and it seems for a good reason.

Some that do: R, thats all I could find. Although some (if not many) went through the proposal phase. And yes, none of them have a function that I am proposing.

So yes, you are right, not a good proposal.

But there still seems to be the need for short-circuiting performant implementations in python space. The issue is that there are many variants of what might be needed while there is no efficient solution to sourcing predicates from python to lower level
implementations. Someone mentioned that numpy experimented with such implementations in C, but they did not get anywhere with it.

The best I could come up with is cached numba for numpy problems, which does perform very well and more than worth it if function is re-used. It even ends up faster than cython or cffi extensions, however can’t have many of those due to JIT and AOT is
currently being deprecated (which wouldn’t solve anything anyway). However, as I mentioned earlier it does not apply to this case.

So it’s either:
a) Something very clever and flexible implemented that covers most of such needs and doesn’t require predicates.
b) I welcome any thoughts on this.

DG

On 14 Nov 2023, at 04:27, AVI GROSS via Python-list <python-list@python.org> wrote:

I was going to ask a dumb question. Has any other language you know of made something available that does what is being asked for and included it in the main program environment rather than an add-on?

A secondary mention here has been whether short-circuiting functions like "any" and "all" have been augmented with something like "has_n" that evaluates arguments till it has found n or perhaps n+1 of what it wants then skips the rest. Does any language supply something like that? What would
such a function return and does it have an "any" or an "all" side?

It sounds like if I asked if a list of integers has at least n prime numbers in "any" mode, it should ignore any that are not primes till it finds n primes or fails and returns true or false. If in "all" mode, I assume it would have to be the first n items without a failure.

Fine, but then someone may want to know WHERE you stopped or for you to return the sublist of the ones that made the match, or even return
everything that was skipped so you can later process that. Consider a long list of jurors you process to place a dozen that qualify on a jury and then later you want to choose from among the rest for another jury.

Human minds can come up with an amazing number of ideas including for "useful" functions or features but I find the vast majority would rarely be used as nobody remembers it is available and some fairly simple method using other functions can easily be cobbled together.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On Behalf Of Grant Edwards via Python-list
Sent: Monday, November 13, 2023 8:19 PM
To: python-list@python.org
Subject: Re: xor operator

On 2023-11-14, Dom Grigonis via Python-list <python-list@python.org> wrote:

Except the 'any' and 'all' builtins are _exactly_ the same as bitwise
or and and applided to many bits. To do something "in line" with that
using the 'xor' operator would return True for an odd number of True
values and False for an even Number of True values.

Fair point.

Have you ever encountered the need for xor for many bits (the one
that I am NOT referring to)? Would be interested in what sort of
case it could be useful.

Yes, it's used all the time in low-level communications protocols,
where it's often implemented in hardware. But, it is also not at all
unusual to implement it in software.

It's also not that unusual for the "count-ones" part of the function
you're asking for to be implemented in hardware by a CPU having an instruction that counts the number of 1 bits in a register.

GCC has a low-level builtins called __builtin_popcount() and __builtin-popcountl() that counts the number of 1's in an unsigned
(long) int.

--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Dom,

I hear you.

As you say, writing your own extension in something like C++ may not appeal to you even if it is faster.

I was wondering if using a generator or something similar in R might make sense.

I mean what happens if you write a function that includes a "yield" or two and does a part of what you want. It maintains some internal state between invocations. So you can call it once to setup things then call it repeatedly to keep processing the next
item. You stop calling it when you get a result you want, such as that it has seen what you want N times.

Since the code stays in memory, it may effectively run faster than some other kinds of functions calls. It can keep things in internal storage such as not just how many N you want but how many it has seen.

Your outer function can maintain a list of the items you want to XOR or generate a new one dynamically as needed. It can use functional programming techniques to create a new customized version of the iterator, such as with a value of N built in. You
would then call the outer function and let it use the inner function till the result is available or until the data in the iterator runs out or perhaps other tweaks involving two way communication of sorts between the functions.

I am NOT suggesting this approach is optimal or fast but merely wondering if something along these lines is worth trying that might speed things up even if not very fast. Such approaches can be even more effective if what you are working on need not all
be instantiated up front but can be dynamically calculated or incrementally read from files. With care, you can make multiple instantiations that each iterate over their own sets of data without interference.

Just a thought. In a sense, this can be a slightly decent substitute for the non-standard evaluation in R where you can arrange for lots of your data to not be interpreted till absolutely needed.



-----Original Message-----
From: Dom Grigonis <dom.grigonis@gmail.com>
Sent: Monday, November 13, 2023 10:12 PM
To: avi.e.gross@gmail.com
Cc: Grant Edwards <grant.b.edwards@gmail.com>; Python <python-list@python.org> Subject: Re: xor operator (DEPRECATED)

Fair point. However, I gave it a shot for the following reason:

I couldn’t find a way to make such performant function. Using python builtin components still ends up several times slower than builtin `all`. Cython or numba or similar is not an option as they do not support `truth` values. Or if they do, it ends up
slower than pure python variant.

So all what is left is writing a proper extension. Which I would prefer not to do for 1 function. I thought maybe `xor`, as in logical XOR functionality in its vanilla case could be compelling. And after doing a bit of search I see that very very few
languages have that and it seems for a good reason.

Some that do: R, thats all I could find. Although some (if not many) went through the proposal phase. And yes, none of them have a function that I am proposing.

So yes, you are right, not a good proposal.

But there still seems to be the need for short-circuiting performant implementations in python space. The issue is that there are many variants of what might be needed while there is no efficient solution to sourcing predicates from python to lower level
implementations. Someone mentioned that numpy experimented with such implementations in C, but they did not get anywhere with it.

The best I could come up with is cached numba for numpy problems, which does perform very well and more than worth it if function is re-used. It even ends up faster than cython or cffi extensions, however can’t have many of those due to JIT and AOT is
currently being deprecated (which wouldn’t solve anything anyway). However, as I mentioned earlier it does not apply to this case.

So it’s either:
a) Something very clever and flexible implemented that covers most of such needs and doesn’t require predicates.
b) I welcome any thoughts on this.

DG

On 14 Nov 2023, at 04:27, AVI GROSS via Python-list <python-list@python.org> wrote:

I was going to ask a dumb question. Has any other language you know of made something available that does what is being asked for and included it in the main program environment rather than an add-on?

A secondary mention here has been whether short-circuiting functions like "any" and "all" have been augmented with something like "has_n" that evaluates arguments till it has found n or perhaps n+1 of what it wants then skips the rest. Does any language supply something like that? What would
such a function return and does it have an "any" or an "all" side?

It sounds like if I asked if a list of integers has at least n prime numbers in "any" mode, it should ignore any that are not primes till it finds n primes or fails and returns true or false. If in "all" mode, I assume it would have to be the first n items without a failure.

Fine, but then someone may want to know WHERE you stopped or for you to return the sublist of the ones that made the match, or even return
everything that was skipped so you can later process that. Consider a long list of jurors you process to place a dozen that qualify on a jury and then later you want to choose from among the rest for another jury.

Human minds can come up with an amazing number of ideas including for "useful" functions or features but I find the vast majority would rarely be used as nobody remembers it is available and some fairly simple method using other functions can easily be cobbled together.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On Behalf Of Grant Edwards via Python-list
Sent: Monday, November 13, 2023 8:19 PM
To: python-list@python.org
Subject: Re: xor operator

On 2023-11-14, Dom Grigonis via Python-list <python-list@python.org> wrote:

Except the 'any' and 'all' builtins are _exactly_ the same as bitwise
or and and applided to many bits. To do something "in line" with that
using the 'xor' operator would return True for an odd number of True
values and False for an even Number of True values.

Fair point.

Have you ever encountered the need for xor for many bits (the one
that I am NOT referring to)? Would be interested in what sort of
case it could be useful.

Yes, it's used all the time in low-level communications protocols,
where it's often implemented in hardware. But, it is also not at all
unusual to implement it in software.

It's also not that unusual for the "count-ones" part of the function
you're asking for to be implemented in hardware by a CPU having an instruction that counts the number of 1 bits in a register.

GCC has a low-level builtins called __builtin_popcount() and __builtin-popcountl() that counts the number of 1's in an unsigned
(long) int.

--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 11/13/2023 11:44 PM, AVI GROSS via Python-list wrote:

Dom,

I hear you.

As you say, writing your own extension in something like C++ may not appeal to you even if it is faster.

I was wondering if using a generator or something similar in R might make sense.

I mean what happens if you write a function that includes a "yield" or two and does a part of what you want. It maintains some internal state between invocations. So you can call it once to setup things then call it repeatedly to keep processing the

next item. You stop calling it when you get a result you want, such as that it has seen what you want N times.

Since the code stays in memory, it may effectively run faster than some other kinds of functions calls. It can keep things in internal storage such as not just how many N you want but how many it has seen.

I'm inclined to just turn the iterable into a set to get the values,
then iterate through those values calling count() on a listified version
of the iterable. If the count >= target, return.

It may not be the fastest one could do but it's simple and probably
pretty fast for many uses. I suppose that for some iterables it would
be better not to turn them into lists, but one could figure out about
that after working out more carefully what cases need to be covered.

Your outer function can maintain a list of the items you want to XOR or generate a new one dynamically as needed. It can use functional programming techniques to create a new customized version of the iterator, such as with a value of N built in. You

would then call the outer function and let it use the inner function till the result is available or until the data in the iterator runs out or perhaps other tweaks involving two way communication of sorts between the functions.

I am NOT suggesting this approach is optimal or fast but merely wondering if something along these lines is worth trying that might speed things up even if not very fast. Such approaches can be even more effective if what you are working on need not

all be instantiated up front but can be dynamically calculated or incrementally read from files. With care, you can make multiple instantiations that each iterate over their own sets of data without interference.

Just a thought. In a sense, this can be a slightly decent substitute for the non-standard evaluation in R where you can arrange for lots of your data to not be interpreted till absolutely needed.

-----Original Message-----
From: Dom Grigonis <dom.grigonis@gmail.com>
Sent: Monday, November 13, 2023 10:12 PM
To: avi.e.gross@gmail.com
Cc: Grant Edwards <grant.b.edwards@gmail.com>; Python <python-list@python.org>
Subject: Re: xor operator (DEPRECATED)

Fair point. However, I gave it a shot for the following reason:

I couldn’t find a way to make such performant function. Using python builtin components still ends up several times slower than builtin `all`. Cython or numba or similar is not an option as they do not support `truth` values. Or if they do, it ends

up slower than pure python variant.

So all what is left is writing a proper extension. Which I would prefer not to do for 1 function. I thought maybe `xor`, as in logical XOR functionality in its vanilla case could be compelling. And after doing a bit of search I see that very very few

languages have that and it seems for a good reason.

Some that do: R, thats all I could find. Although some (if not many) went through the proposal phase. And yes, none of them have a function that I am proposing.

So yes, you are right, not a good proposal.

But there still seems to be the need for short-circuiting performant implementations in python space. The issue is that there are many variants of what might be needed while there is no efficient solution to sourcing predicates from python to lower

level implementations. Someone mentioned that numpy experimented with such implementations in C, but they did not get anywhere with it.

The best I could come up with is cached numba for numpy problems, which does perform very well and more than worth it if function is re-used. It even ends up faster than cython or cffi extensions, however can’t have many of those due to JIT and AOT

is currently being deprecated (which wouldn’t solve anything anyway). However, as I mentioned earlier it does not apply to this case.

So it’s either:
a) Something very clever and flexible implemented that covers most of such needs and doesn’t require predicates.
b) I welcome any thoughts on this.

DG

On 14 Nov 2023, at 04:27, AVI GROSS via Python-list <python-list@python.org> wrote:

I was going to ask a dumb question. Has any other language you know of made >> something available that does what is being asked for and included it in the >> main program environment rather than an add-on?

A secondary mention here has been whether short-circuiting functions like
"any" and "all" have been augmented with something like "has_n" that
evaluates arguments till it has found n or perhaps n+1 of what it wants then >> skips the rest. Does any language supply something like that? What would
such a function return and does it have an "any" or an "all" side?

It sounds like if I asked if a list of integers has at least n prime numbers >> in "any" mode, it should ignore any that are not primes till it finds n
primes or fails and returns true or false. If in "all" mode, I assume it
would have to be the first n items without a failure.

Fine, but then someone may want to know WHERE you stopped or for you to
return the sublist of the ones that made the match, or even return
everything that was skipped so you can later process that. Consider a long >> list of jurors you process to place a dozen that qualify on a jury and then >> later you want to choose from among the rest for another jury.

Human minds can come up with an amazing number of ideas including for
"useful" functions or features but I find the vast majority would rarely be >> used as nobody remembers it is available and some fairly simple method using >> other functions can easily be cobbled together.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On >> Behalf Of Grant Edwards via Python-list
Sent: Monday, November 13, 2023 8:19 PM
To: python-list@python.org
Subject: Re: xor operator

On 2023-11-14, Dom Grigonis via Python-list <python-list@python.org> wrote: >>>

Except the 'any' and 'all' builtins are _exactly_ the same as bitwise
or and and applided to many bits. To do something "in line" with that
using the 'xor' operator would return True for an odd number of True
values and False for an even Number of True values.

Fair point.

Have you ever encountered the need for xor for many bits (the one
that I am NOT referring to)? Would be interested in what sort of
case it could be useful.

Yes, it's used all the time in low-level communications protocols,
where it's often implemented in hardware. But, it is also not at all
unusual to implement it in software.

It's also not that unusual for the "count-ones" part of the function
you're asking for to be implemented in hardware by a CPU having an
instruction that counts the number of 1 bits in a register.

GCC has a low-level builtins called __builtin_popcount() and
__builtin-popcountl() that counts the number of 1's in an unsigned
(long) int.


--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Thank you, I have spent a fair bit of time on these and found optimal solutions (at least I think so), but they are still multiple times slower than python builtin function.

I am currently out of ideas, maybe will dig something out in time.

On 14 Nov 2023, at 07:23, Thomas Passin via Python-list <python-list@python.org> wrote:

On 11/13/2023 11:44 PM, AVI GROSS via Python-list wrote:

Dom,
I hear you.
As you say, writing your own extension in something like C++ may not appeal to you even if it is faster.
I was wondering if using a generator or something similar in R might make sense.
I mean what happens if you write a function that includes a "yield" or two and does a part of what you want. It maintains some internal state between invocations. So you can call it once to setup things then call it repeatedly to keep processing the

next item. You stop calling it when you get a result you want, such as that it has seen what you want N times.

Since the code stays in memory, it may effectively run faster than some other kinds of functions calls. It can keep things in internal storage such as not just how many N you want but how many it has seen.

I'm inclined to just turn the iterable into a set to get the values, then iterate through those values calling count() on a listified version of the iterable. If the count >= target, return.

It may not be the fastest one could do but it's simple and probably pretty fast for many uses. I suppose that for some iterables it would be better not to turn them into lists, but one could figure out about that after working out more carefully what

cases need to be covered.

Your outer function can maintain a list of the items you want to XOR or generate a new one dynamically as needed. It can use functional programming techniques to create a new customized version of the iterator, such as with a value of N built in. You

would then call the outer function and let it use the inner function till the result is available or until the data in the iterator runs out or perhaps other tweaks involving two way communication of sorts between the functions.

I am NOT suggesting this approach is optimal or fast but merely wondering if something along these lines is worth trying that might speed things up even if not very fast. Such approaches can be even more effective if what you are working on need not

all be instantiated up front but can be dynamically calculated or incrementally read from files. With care, you can make multiple instantiations that each iterate over their own sets of data without interference.

Just a thought. In a sense, this can be a slightly decent substitute for the non-standard evaluation in R where you can arrange for lots of your data to not be interpreted till absolutely needed.
-----Original Message-----
From: Dom Grigonis <dom.grigonis@gmail.com>
Sent: Monday, November 13, 2023 10:12 PM
To: avi.e.gross@gmail.com
Cc: Grant Edwards <grant.b.edwards@gmail.com>; Python <python-list@python.org>
Subject: Re: xor operator (DEPRECATED)
Fair point. However, I gave it a shot for the following reason:
I couldn’t find a way to make such performant function. Using python builtin components still ends up several times slower than builtin `all`. Cython or numba or similar is not an option as they do not support `truth` values. Or if they do, it ends

up slower than pure python variant.

So all what is left is writing a proper extension. Which I would prefer not to do for 1 function. I thought maybe `xor`, as in logical XOR functionality in its vanilla case could be compelling. And after doing a bit of search I see that very very few

languages have that and it seems for a good reason.

Some that do: R, thats all I could find. Although some (if not many) went through the proposal phase. And yes, none of them have a function that I am proposing.
So yes, you are right, not a good proposal.
But there still seems to be the need for short-circuiting performant implementations in python space. The issue is that there are many variants of what might be needed while there is no efficient solution to sourcing predicates from python to lower

level implementations. Someone mentioned that numpy experimented with such implementations in C, but they did not get anywhere with it.

The best I could come up with is cached numba for numpy problems, which does perform very well and more than worth it if function is re-used. It even ends up faster than cython or cffi extensions, however can’t have many of those due to JIT and AOT

is currently being deprecated (which wouldn’t solve anything anyway). However, as I mentioned earlier it does not apply to this case.

So it’s either:
a) Something very clever and flexible implemented that covers most of such needs and doesn’t require predicates.
b) I welcome any thoughts on this.
DG

On 14 Nov 2023, at 04:27, AVI GROSS via Python-list <python-list@python.org> wrote:

I was going to ask a dumb question. Has any other language you know of made >>> something available that does what is being asked for and included it in the
main program environment rather than an add-on?

A secondary mention here has been whether short-circuiting functions like >>> "any" and "all" have been augmented with something like "has_n" that
evaluates arguments till it has found n or perhaps n+1 of what it wants then
skips the rest. Does any language supply something like that? What would >>> such a function return and does it have an "any" or an "all" side?

It sounds like if I asked if a list of integers has at least n prime numbers
in "any" mode, it should ignore any that are not primes till it finds n
primes or fails and returns true or false. If in "all" mode, I assume it >>> would have to be the first n items without a failure.

Fine, but then someone may want to know WHERE you stopped or for you to
return the sublist of the ones that made the match, or even return
everything that was skipped so you can later process that. Consider a long >>> list of jurors you process to place a dozen that qualify on a jury and then >>> later you want to choose from among the rest for another jury.

Human minds can come up with an amazing number of ideas including for
"useful" functions or features but I find the vast majority would rarely be >>> used as nobody remembers it is available and some fairly simple method using
other functions can easily be cobbled together.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On >>> Behalf Of Grant Edwards via Python-list
Sent: Monday, November 13, 2023 8:19 PM
To: python-list@python.org
Subject: Re: xor operator

On 2023-11-14, Dom Grigonis via Python-list <python-list@python.org> wrote: >>>>

Except the 'any' and 'all' builtins are _exactly_ the same as bitwise >>>>> or and and applided to many bits. To do something "in line" with that >>>>> using the 'xor' operator would return True for an odd number of True >>>>> values and False for an even Number of True values.

Fair point.

Have you ever encountered the need for xor for many bits (the one
that I am NOT referring to)? Would be interested in what sort of
case it could be useful.

Yes, it's used all the time in low-level communications protocols,
where it's often implemented in hardware. But, it is also not at all
unusual to implement it in software.

It's also not that unusual for the "count-ones" part of the function
you're asking for to be implemented in hardware by a CPU having an
instruction that counts the number of 1 bits in a register.

GCC has a low-level builtins called __builtin_popcount() and
__builtin-popcountl() that counts the number of 1's in an unsigned
(long) int.


--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list <https://mail.python.org/mailman/listinfo/python-list>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)