Hello,

pyastinterp is a tree-walking (aka meta-circular) interpreter for
Python. It was the most complete tree-walking interpreter as of a year
ago. As it was written as a personal "advent of code" project during
the winter holidays of 2019/2020.

I remembered about it this holiday season, and decided to post about
it, both as a holiday eyecandy, and as a challenge to other Python
hackers (such winter holidays challenges are known as "advent of
code" (of course, unless you believe there's only one of them)): can you
write a more complete implementation? Or maybe you did during this
year? Please post about it somewhere (perhaps, python-ideas mailing
list?).

https://github.com/pfalcon/pyastinterp


README with more details follows:

pyastinterp
===========

This is an AST interpreter (also known as tree-walking interpreter) for
the Python programming language, written in Python itself. It is thus
a meta-circular interpreter. While Python implementations of a Python VM
(i.e. bytecode interpreters) are relatively well known (starting with
PyPy), AST interpreters are much less common beyond those which implement
a very limited subset of the syntax.

Pyastinterp strives to fulfil 2 goals:

1. Be as complete as possible (and see how much effort that takes and
if there're any limits).
2. Yet be as minimal as possible, avoiding useless boilerplate and
reimplementing what may be already available.

First goal is pretty obvious: ideally, we'd like Pyastinterp to be able
to interpret (run) any application a native Python interpreter can.

Second goal is rooted in many reasons. To start with, Pyastinterp is
a spiritual part of the [Pycopy](https://github.com/pfalcon/pycopy)
project, whose goal is the development of minimalist Python-compatible ecosystem. Besides, "bloated" meta-circular interpreters are well out
there (the same PyPy) - there's lack on the opposite side of spectrum. Pyastinterp is also intended to be a "teaching" and "for-research"
project. Surely, it's much easier to learn and reuse/repurpose a small,
rather than large, project. These last requirements however also put
bounds on "minimality": `pyastinterp` tries to have a clear focus
(AST interpretation) and avoid unneeded boilerplate, but not be
obfuscated, and actually tries to be clear and easily hackable (provide
some reusable infrastructure), even at the expense of somewhat less optimality/performance.

To achieve the goals of minimality, `pyastinterp` does following:

1. Is concerned with syntax, not semantics (largely).
2. That means that it tries to interpret Python AST tree, but
relies on the underlying Python runtime to implement the actual
operations (semantics). In some cases `pyastinterp` actually
has to deal with runtime semantics, but the point is that it
should be done only when there's no other choice (i.e. rarely).
3. This in particular means that `pyastinterp` itself requires a
(pretty) full Python implementation to run, it's not the case
that it's written in "some subset of Python". (Contrast that
with e.g. PyPy's RPython, which is a Python subset language,
in which interpreters are written).
4. Another consequence is that there's no semantic separation
between "metalevel" (level of the interpreter) and target
application level. This is again unlike PyPy, where 2 are explicitly
separated. Lack of separation allows to save on "marshalling"
values between the 2 levels, but also make it possible to have
"leaks" between levels, e.g. unexpected and unhandled exception
in the interpreter to be delivered to the application, causing
havoc and confusion. Pyastinterp's response to this concern
is a pledge that there shouldn't be such unhandled interpreter-level
exceptions. Of course, catching all such cases may take quite
a lot of testing and debugging, and fixing them may affect code
minimality and clarity. We'll see how it goes.
5. However, due to choices described above, implementation of many
syntactic constructs is very clear, concise, and isomorphic
between the target and meta levels. See e.g. how "if" statement
is implemented: the implementation looks almost exactly how the
usage of the "if" statement is.


Supported Python subset
-----------------------

Pyastinterp should implement almost all Python3.5 features, stopping
at generators, which are not that easy to implement in a tree-walking interpreter. It also does not include import hook, so the imported
modules are interpreted by the underlying Python VM, not by
Pyastinterp.


Running
-------

Quick way to run something interesting:

wget https://raw.githubusercontent.com/python/cpython/3.6/Lib/test/pystone.py
python3 -m astinterp pystone.my 10000

You can pass any other script on the command line and see how it goes.


Credits and licensing
---------------------

Pyastinterp is (c) [Paul Sokolovsky](https://github.com/pfalcon) and
is released under the MIT license.



--
Best regards,
Paul mailto:pmiscml@gmail.com

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)