I've been using python as a tool to solve competitive programming problems
for a while now and I've noticed a feature, python would benefit from
having.
Consider "reversed(enumerate(a))". This is a perfectly readable code,
except it's wrong in the current version of python. That's because
enumerate returns an iterator, but reversed can take only a sequence type.

The feature I am describing (and proposing) solves this.
Introducing an adapter type: this is an iterator, but it's items can be accessed out of order.
More formally it has to:
1. Have __getitem__ to allow access by index
2. Have __len__
3. Be immutable
(It is a lot like the sequence protocol)

An adapter can be converted to an iterator by accessing it from 0 to len(adapter). Which is done by iter(). (or by __iter__, I don't know which implementation would be "right")
```
iter(a)
#is equivalent to
(a[i] for i in range(len(a)))
```
For example any tuple is a valid adapter and any list can be easily
converted to one.

Built-in adapter-generators:
"map" function should really return an adapter.
```
#possible implementation
m=map(lambda x:x+1,[1,2,3,4,5])
#lambda is not called
print(m[3])# gives 5 by calling the lambda on list's 3rd element, which is 4 #simplified implementation
class map:
def __init__(self,f,a):
self.f=f
self.a=a
def __getitem__(self,idx):
return self.f(self.a[idx])
def __len__(self):
return len(self.a)
```
enumerate should really return an adapter
```
#simplified implementation
class enumerate:
def __init__(self,a):
self.a = a
def __getitem__(self,idx):
return idx,self.a[idx]
def __len__(self):
return len(self.a)
```
reversed should really return an adapter
```
#simplified implementation
class reversed:
def __init__(self,a):
self.a = a
self.length=len(a)
def __getitem__(self,idx):
return self.a[self.length-idx-1]
def __len__(self):
return self.length
```
zip should really return an adapter
range should really return an adapter
filter should *not* return an adapter

All of those functions return an adapter and take in an adapter. But some (excluding "reversed") should be able take an iterator and return an
iterator.
So the way I imagine a possible release version to work is that "reversed(enumerate(a))" works if a is an adapter and throws an exception
if not

Perhaps there should be even an adapter comprehension:
```
Add1 = (a+1 for a)
#is equivalent to
Add1 = lambda s: map((lambda a: a+1),s)
transformed = Add1([1,2,3])
print(transformed[1])# should be 3
```

This adapter feature also allows you to not use a key argument "key=" for certain functions (which could be non-existing). For example bisect.bisect functions before 3.10 didn't have a key= argument. Some third-party
libraries could do this with it's functions too.

(Subject to change)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/3/2023 5:45 AM, fedor tryfanau wrote:

I've been using python as a tool to solve competitive programming problems for a while now and I've noticed a feature, python would benefit from
having.
Consider "reversed(enumerate(a))". This is a perfectly readable code,
except it's wrong in the current version of python. That's because
enumerate returns an iterator, but reversed can take only a sequence type.

Depending on what you want to give and receive, enumerate(reversed(a))
will do the job here. Otherwise list() or tuple() can achieve some of
the same things.

The feature I am describing (and proposing) solves this.
Introducing an adapter type: this is an iterator, but it's items can be accessed out of order.
More formally it has to:
1. Have __getitem__ to allow access by index
2. Have __len__
3. Be immutable
(It is a lot like the sequence protocol)

An adapter can be converted to an iterator by accessing it from 0 to len(adapter). Which is done by iter(). (or by __iter__, I don't know which implementation would be "right")
```
iter(a)
#is equivalent to
(a[i] for i in range(len(a)))
```
For example any tuple is a valid adapter and any list can be easily
converted to one.

Built-in adapter-generators:
"map" function should really return an adapter.
```
#possible implementation
m=map(lambda x:x+1,[1,2,3,4,5])
#lambda is not called
print(m[3])# gives 5 by calling the lambda on list's 3rd element, which is 4 #simplified implementation
class map:
def __init__(self,f,a):
self.f=f
self.a=a
def __getitem__(self,idx):
return self.f(self.a[idx])
def __len__(self):
return len(self.a)
```
enumerate should really return an adapter
```
#simplified implementation
class enumerate:
def __init__(self,a):
self.a = a
def __getitem__(self,idx):
return idx,self.a[idx]
def __len__(self):
return len(self.a)
```
reversed should really return an adapter
```
#simplified implementation
class reversed:
def __init__(self,a):
self.a = a
self.length=len(a)
def __getitem__(self,idx):
return self.a[self.length-idx-1]
def __len__(self):
return self.length
```
zip should really return an adapter
range should really return an adapter
filter should *not* return an adapter

All of those functions return an adapter and take in an adapter. But some (excluding "reversed") should be able take an iterator and return an iterator.
So the way I imagine a possible release version to work is that "reversed(enumerate(a))" works if a is an adapter and throws an exception
if not

Perhaps there should be even an adapter comprehension:
```
Add1 = (a+1 for a)
#is equivalent to
Add1 = lambda s: map((lambda a: a+1),s)
transformed = Add1([1,2,3])
print(transformed[1])# should be 3
```

This adapter feature also allows you to not use a key argument "key=" for certain functions (which could be non-existing). For example bisect.bisect functions before 3.10 didn't have a key= argument. Some third-party
libraries could do this with it's functions too.

(Subject to change)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 3 May 2023 at 18:52, Thomas Passin <list1@tompassin.net> wrote:

On 5/3/2023 5:45 AM, fedor tryfanau wrote:

I've been using python as a tool to solve competitive programming problems for a while now and I've noticed a feature, python would benefit from having.
Consider "reversed(enumerate(a))". This is a perfectly readable code, except it's wrong in the current version of python. That's because enumerate returns an iterator, but reversed can take only a sequence type.

Depending on what you want to give and receive, enumerate(reversed(a))
will do the job here. Otherwise list() or tuple() can achieve some of
the same things.

I don't think that is equivalent to the intended behaviour:

reversed(enumerate(a)) # zip(reversed(range(len(a))), reversed(a)) enumerate(reversed(a)) # zip(range(len(a)), reversed(a))

In principle for a sequence input enumerate(a) could be something that
behaves like a sequence and therefore could be reiterated or reversed
etc. The enumerate(a).__reversed__ method could then delegate to
a.__reversed__ and a.__len__ if they exist. This could be confusing
though because the possible behaviour of enumerate(a) would be
different depending on the type of a.

--
Oscar

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/3/2023 3:46 PM, Oscar Benjamin wrote:

On Wed, 3 May 2023 at 18:52, Thomas Passin <list1@tompassin.net> wrote:

On 5/3/2023 5:45 AM, fedor tryfanau wrote:

I've been using python as a tool to solve competitive programming problems >>> for a while now and I've noticed a feature, python would benefit from
having.
Consider "reversed(enumerate(a))". This is a perfectly readable code,
except it's wrong in the current version of python. That's because
enumerate returns an iterator, but reversed can take only a sequence type. >>

Depending on what you want to give and receive, enumerate(reversed(a))
will do the job here. Otherwise list() or tuple() can achieve some of
the same things.

I don't think that is equivalent to the intended behaviour:

reversed(enumerate(a)) # zip(reversed(range(len(a))), reversed(a)) enumerate(reversed(a)) # zip(range(len(a)), reversed(a))

I don't think we know the intended behavior here. The OP did not say
what type of object should be returned. He only wanted an expression
that would run. Apparently the result should be an enumeration of
variable "a" but with "a" reversed. Is "a" supposed to be a sequence?
An iterator? Presumably the result was expected to be an enumeration,
which is to say an iterator, and enumerate(reversed(a)) would return an iterator.

Perhaps the OP wants all methods/functions that operate on either
sequences or iterators should be generalized to be able to operate on
both. That doesn't sound unreasonable on the face of it, but I think
that deeper study would uncover a lot of hard questions. There will also
be a lot of edge cases to handle and get right.

In principle for a sequence input enumerate(a) could be something that behaves like a sequence and therefore could be reiterated or reversed
etc. The enumerate(a).__reversed__ method could then delegate to a.__reversed__ and a.__len__ if they exist. This could be confusing
though because the possible behaviour of enumerate(a) would be
different depending on the type of a.

--
Oscar

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 4 May 2023 at 02:25, fedor tryfanau <fedor.tryfanau@gmail.com> wrote:

I've been using python as a tool to solve competitive programming problems for a while now and I've noticed a feature, python would benefit from
having.
Consider "reversed(enumerate(a))". This is a perfectly readable code,
except it's wrong in the current version of python. That's because
enumerate returns an iterator, but reversed can take only a sequence type.

The feature I am describing (and proposing) solves this.
Introducing an adapter type: this is an iterator, but it's items can be accessed out of order.
More formally it has to:
1. Have __getitem__ to allow access by index
2. Have __len__
3. Be immutable
(It is a lot like the sequence protocol)

An adapter can be converted to an iterator by accessing it from 0 to len(adapter). Which is done by iter(). (or by __iter__, I don't know which implementation would be "right")
```
iter(a)
#is equivalent to
(a[i] for i in range(len(a)))
```
For example any tuple is a valid adapter and any list can be easily
converted to one.

Built-in adapter-generators:
"map" function should really return an adapter.

The trouble with that is that map() already accepts any iterator. So
you're asking for map to be able to return an iterator if given an
iterator, or an adapter if given an adapter. That makes it quite
complicated to use and reason about.

In general, it's probably safest to just coalesce things to list when
you need to:

reversed(list(enumerate(a)))

because otherwise there are many MANY questions left open. For
example, in what order are the elements of a retrieved? with reversed(enumerate(a)) it's not clear, but if you force to list first,
it is (and you know exactly when they're accessed too).

ChrisA

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 4 May 2023 at 07:43, Thomas Passin <list1@tompassin.net> wrote:

On 5/3/2023 3:46 PM, Oscar Benjamin wrote:

On Wed, 3 May 2023 at 18:52, Thomas Passin <list1@tompassin.net> wrote:

On 5/3/2023 5:45 AM, fedor tryfanau wrote:

I've been using python as a tool to solve competitive programming problems
for a while now and I've noticed a feature, python would benefit from
having.
Consider "reversed(enumerate(a))". This is a perfectly readable code,
except it's wrong in the current version of python. That's because
enumerate returns an iterator, but reversed can take only a sequence type.

Depending on what you want to give and receive, enumerate(reversed(a))
will do the job here. Otherwise list() or tuple() can achieve some of
the same things.

I don't think that is equivalent to the intended behaviour:

reversed(enumerate(a)) # zip(reversed(range(len(a))), reversed(a)) enumerate(reversed(a)) # zip(range(len(a)), reversed(a))

I don't think we know the intended behavior here. The OP did not say
what type of object should be returned. He only wanted an expression
that would run. Apparently the result should be an enumeration of
variable "a" but with "a" reversed. Is "a" supposed to be a sequence?
An iterator? Presumably the result was expected to be an enumeration,
which is to say an iterator, and enumerate(reversed(a)) would return an iterator.

The part you're probably missing here can be easily shown:

a = [10, 20, 30, 40, 50]
for i, n in enumerate(reversed(a)):

... print("E-R", i, n)
...
E-R 0 50
E-R 1 40
E-R 2 30
E-R 3 20
E-R 4 10

for i, n in reversed(list(enumerate(a))):

... print("R-E", i, n)
...
R-E 4 50
R-E 3 40
R-E 2 30
R-E 1 20
R-E 0 10

So a closer equivalent would be:

for i, n in zip(reversed(range(len(a))), reversed(a)):

... print("Zip", i, n)
...
Zip 4 50
Zip 3 40
Zip 2 30
Zip 1 20
Zip 0 10

Note that it's reversing both the indices and the values.

ChrisA

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 4/05/23 9:29 am, Chris Angelico wrote:

So
you're asking for map to be able to return an iterator if given an
iterator, or an adapter if given an adapter. That makes it quite
complicated to use and reason about.

Also a bit slower, since it would need to inspect its argument
and decide what to do with it. Currently it can just get on
with its job and rely on duck typing to do the right thing.

Maybe there could be a parallel set of functions "enumerated",
"mapped", etc. that take sequences and return sequence views.

Although that naming convention would suggest that reversed()
itself should return a sequence view rather than an iterator.
That would require restricting it to working on sequences,
which would be an incompatible change.

--
Greg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

As others have mentioned features added like this need careful examination
not only at effects but costs.

As I see it, several somewhat different ideas were raised and one of them strikes me oddly. The whole point of an iterable is to AVOID calculating the next item till needed. Otherwise, you can just make something like a list.

To talk about random access to an iterable is a tad weird as it would mean
you need to get the first N items and store them and return the Nth item
value as well as maintain the remainder of the unused part of the iterable. Further requests already in the cache would be gotten from there and any
beyond it would require iterating more and adding more to the cache.

So say my iterator returns the first N primes or just those below 100. What should be the functionality if you request item 1,000?

As for reversing it, that requires you to basically do list(iterable) and
use it up. What if the iterable is infinite as in all the odd numbers?

If you really want an iterable that return something like prime numbers
below some level in reverse order, that can be done by changing the iterable
to create them going downward and that would be a different iterator. But
how easily could you make some iterators go backward? Fibonacci, maybe so. Other things perhaps not.

But again, as noted, anything already in a list can be set up as an iterator that returns one item at a time from that list, including in reverse. There won't be much savings as the data structure inside would likely be spread
out to take all the memory needed, albeit it may simplify the code to look
like it was being delivered just in time.

As with many things in python, rather than asking for a global solution that affects many others, sometimes in unexpected ways, it may be more reasonable
to make your own patches to your code and use them in ways you can control.
In the case being discussed, you simply need to create a generator function that accepts an iterator, converts it to a list in entirety, reverses the
list (or deals with it from the end) and enters a loop where it yields one value at a time till done. This should work with all kinds of iterators and return what looks like an iterator without any changes to the language.

Of course, I am likely to be missing something. And, certainly, there may already be modules doing things like the above or the opportunity for
someone to create a module such as the itertools module with nifty little functions including factory functions.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On Behalf Of Oscar Benjamin
Sent: Wednesday, May 3, 2023 3:47 PM
To: python-list@python.org
Subject: Re: An "adapter", superset of an iterator

On Wed, 3 May 2023 at 18:52, Thomas Passin <list1@tompassin.net> wrote:

On 5/3/2023 5:45 AM, fedor tryfanau wrote:

I've been using python as a tool to solve competitive programming

problems

for a while now and I've noticed a feature, python would benefit from having.
Consider "reversed(enumerate(a))". This is a perfectly readable code, except it's wrong in the current version of python. That's because enumerate returns an iterator, but reversed can take only a sequence

type.

Depending on what you want to give and receive, enumerate(reversed(a))
will do the job here. Otherwise list() or tuple() can achieve some of
the same things.

I don't think that is equivalent to the intended behaviour:

reversed(enumerate(a)) # zip(reversed(range(len(a))), reversed(a)) enumerate(reversed(a)) # zip(range(len(a)), reversed(a))

In principle for a sequence input enumerate(a) could be something that
behaves like a sequence and therefore could be reiterated or reversed
etc. The enumerate(a).__reversed__ method could then delegate to
a.__reversed__ and a.__len__ if they exist. This could be confusing
though because the possible behaviour of enumerate(a) would be
different depending on the type of a.

--
Oscar
--
https://mail.python.org/mailman/listinfo/python-list

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)