• Problem with macro pattern matching and iterating over macro arguments

    From alicetrillianosako@gmail.com@21:1/5 to All on Sun Jul 19 16:40:04 2020
    In going through the exercises in _Lisp In Small Pieces_ (using Guile 2.2.6 as the underlying implementation), I decided to add a utility macro (above and beyond those already used in the Ch. 1 evaluator) to simplify adding multiple primitive procedures
    of a given arity to the global namespace. However, the macro I've designed doesn't seem to match the intended patterns, and I am uncertain why. Also, I am concerned that the way in which I am iterating over the macro's arguments is invalid, or at least,
    non-optimal.

    The intended patterns are of the form:

    (define-primitives-by-arity <arity> <one or more primitive function names>)

    The macro I have developed so far is:

    (define-syntax define-primitives-by-arity
    (lambda (macro)
    (syntax-case macro (_)
    ((_ <arity> <primitive-name-0> <primitive-name-n> ...)
    (for-each (lambda (name)
    (define-primitive name #`<arity>))
    (list #`<primitive-name-0> #`(<primitive-name-n> ...)))))))

    The error message I get from Guile is:

    ;;; WARNING: compilation of /home/schol-r-lea/Documents/Programming/Projects/Lisp-In-Small-Pieces-walkthrough/basic-evaluator/basic-eval.scm failed:
    ;;; Syntax error:
    ;;; /home/schol-r-lea/Documents/Programming/Projects/Lisp-In-Small-Pieces-walkthrough/basic-evaluator/basic-eval.scm:64:0: source expression failed to match any pattern in form (define-primitives-by-arity 0 exit)
    ice-9/psyntax.scm:1585:32: In procedure expand-macro:
    Syntax error: /home/schol-r-lea/Documents/Programming/Projects/Lisp-In-Small-Pieces-walkthrough/basic-evaluator/basic-eval.scm:64:0: source expression failed to match any pattern in form (define-primitives-by-arity 0 exit)


    The previous version of the evaluator (with an earlier attempt at the macro, and the original, manual declarations of the primitives which are working as intended) can be viewed at https://github.com/Schol-R-LEA/Lisp-In-Small-Pieces-walkthrough should
    that be of any help.

    I am aware that I have asked many such questions before, under my previous GMail username (tied to a deadname email account), and I realize that the patience of some of you may be wearing thin with me. I seem to have had a lot of trouble grasping how
    Scheme macros are evaluated, and all I can say is that I am still working on learning them. Please bear with me a while longer.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alice Osako@21:1/5 to All on Sun Jul 19 21:16:26 2020
    OK, so, apparently I did something silly - I don't know when or why I inserted that underscore in the literal-keyword list, but that seems to have been part of the problem. However, after deleting it, I now have a different problem, in that the code will
    now run, but doesn't work as intended.

    The output after deleting the extraneous underscore is: -----------------------------------------------------
    ;;; WARNING: compilation of /home/schol-r-lea/Documents/Programming/Projects/Lisp-In-Small-Pieces-walkthrough/basic-evaluator/basic-eval.scm failed:
    ;;; Unbound variable: env.global
    Basic evaluator test REPL
    (cons 1 2)
    Backtrace:
    5 (primitive-load "/home/schol-r-lea/Documents/Programmin…")
    In ice-9/eval.scm:
    293:34 4 (_ #(#(#<directory (guile-user) 7f2e286ca140> #<va…>) #))
    159:9 3 (_ #(#(#(#<directory (guile-user) 7f2e286ca140>) # #) #))
    293:34 2 (_ #(#(#<directory (guile-user) 7f2e286ca140>) cons (…)))
    619:8 1 (_ #(#(#<directory (guile-user) 7f2e286ca140>) cons ()))
    In unknown file:
    0 (scm-error misc-error #f "~A ~S" ("LOOKUP - unbound…" …) …)

    ERROR: In procedure scm-error:
    LOOKUP - unbound variable cons -----------------------------------------------------

    Running the relevant code in the REPL and then checking the variable env.global shows the following:

    -----------------------------------------------------
    ((name . #<procedure 7f5fa4c042e0 at <unknown port>:54:19 (values)>) (name . #<procedure 7f5fa4c04300 at <unknown port>:54:19 (values)>) (name . #<procedure 7f5fa5a51de0 at <unknown port>:54:19 (values)>) (name . #<procedure 7f5fa5a51e00 at <unknown port>
    :54:19 (values)>) (name . #<procedure 7f5fa5fcb760 at <unknown port>:54:19 (values)>) (name . #<procedure 7f5fa5fcb780 at <unknown port>:54:19 (values)>) (nil . **null**) (f . **false**) (t . #t))
    -----------------------------------------------------

    In other words, the literal 'name' is being inserted rather than the procedure symbols.

    Thinking it might be the way I syntax-quoted the variables rather than the whole string, I tried changing the relevant line to:

    #`(define-primitive name <arity>))

    The result when running the whole is:

    -----------------------------------------------------
    Backtrace:
    9 (apply-smob/1 #<catch-closure 7f8254a01240>)
    In ice-9/boot-9.scm:
    705:2 8 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
    In ice-9/eval.scm:
    619:8 7 (_ #(#(#<directory (guile-user) 7f825467e140>)))
    In ice-9/boot-9.scm:
    2312:4 6 (save-module-excursion _)
    3832:12 5 (_)
    In /home/schol-r-lea/Documents/Programming/Projects/Lisp-In-Small-Pieces-walkthrough/basic-evaluator/basic-eval.scm:
    200:20 4 (basic:repl)
    95:24 3 (basic:eval (cons 1 2) ((nil . **null**) (f . #) (. #)))
    77:23 2 (basic:eval cons _)
    158:10 1 (lookup-env _ _)
    In unknown file:
    0 (scm-error misc-error #f "~A ~S" ("LOOKUP - unbound…" …) …) -----------------------------------------------------

    While running in the REPL shows that this version doesn't update env.global at all.

    I fell foolish for the earlier error which was in my initial post, but clearly I still don't have a clear idea of how syntax macros work.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Maciek Godek@21:1/5 to All on Mon Jul 20 00:56:31 2020
    W dniu poniedziałek, 20 lipca 2020 01:40:06 UTC+2 użytkownik alicetril...@gmail.com napisał:
    In going through the exercises in _Lisp In Small Pieces_ (using Guile 2.2.6 as the underlying implementation), I decided to add a utility macro (above and beyond those already used in the Ch. 1 evaluator) to simplify adding multiple primitive
    procedures of a given arity to the global namespace. However, the macro I've designed doesn't seem to match the intended patterns, and I am uncertain why. Also, I am concerned that the way in which I am iterating over the macro's arguments is invalid, or
    at least, non-optimal.

    The intended patterns are of the form:

    (define-primitives-by-arity <arity> <one or more primitive function names>)

    The macro I have developed so far is:

    (define-syntax define-primitives-by-arity
    (lambda (macro)
    (syntax-case macro (_)
    ((_ <arity> <primitive-name-0> <primitive-name-n> ...)
    (for-each (lambda (name)
    (define-primitive name #`<arity>))
    (list #`<primitive-name-0> #`(<primitive-name-n> ...)))))))

    The error message I get from Guile is:

    Hi Alice,
    It is unclear to me what is the goal of your macro.
    Could you give some example usages (and their expected expansions)?

    However, I am guessing that the definition you might be looking for is

    (define-syntax-rule (define-primitives-by-arity arity primitives ...)
    (begin
    (define-primitive primitives arity)
    ...))

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alice Osako@21:1/5 to Maciek Godek on Mon Jul 20 06:35:49 2020
    TL;DR - your version of the macro seems to do the trick. Thank you.

    On Monday, July 20, 2020 at 3:56:33 AM UTC-4, Maciek Godek wrote:

    It is unclear to me what is the goal of your macro.
    Could you give some example usages (and their expected expansions)?

    OK, the big picture first. The program in question is a sample meta-circular evaluator for a Scheme sub-set, from the first chapter of a book titled _Lisp in Small Pieces_ (the author's page for the book is at https://christian.queinnec.org/WWW/LiSP.html)
    . For didactic reasons, I am not copying the code exactly, as I want to be certain I've understood the ideas correctly.

    In the original code (which IIUC was written for R4RS, though I am using a variant form of R5RS right now), the program environments are represented as simple lists of key-value pairs. To add a primitive non-special form (a function in the underlying
    Scheme used for the implementation), it has a macro, (define-primitive), which in turn invokes another macro, (define-initial), that adds a key-value pair to the global environment.

    The main reason for (define-primitive), aside from simplifying the process of defining a form as opposed to a constant or global variable, is to wrap the value part (the procedure itself) in a lambda which tests whether the arity of a given procedure
    call is correct.

    My versions of these two macros are:

    --------------------------------------------------------
    (define-syntax define-initial
    (lambda (macro)
    (syntax-case macro (in)
    ((_ <name> <value> in <env>)
    (identifier? #'<name>)
    #`(begin
    (set! <env> (cons (cons '<name> <value>) <env>))
    '<name>))
    ((_ <name> <value>)
    #`(define-initial <name> <value> in env.global))
    ((_ <name> in <env>)
    #`(define-initial <name> void in <env>))
    ((_ <name>)
    #`(define-initial <name> void in env.global)))))

    (define-syntax define-primitive
    (lambda (macro)
    (syntax-case macro (in)
    ((_ <name> <primitive> <arity> in <env>)
    #`(define-initial <name>
    (lambda (values)
    (if (= <arity> (length values))
    (apply <primitive> values)
    (begin
    (display "PROCEDURE-APPLICATION: Incorrect arity for fn")
    (display <name>)
    (display ": expect ")
    (display <arity>)
    (display ", got ")
    (display (length values)))))
    in <env>))
    ((_ <name> <primitive> <arity>)
    #`(define-primitive <name> <primitive> <arity> in env.global))
    ((_ <primitive-name> <arity> in <env>)
    #`(define-primitive <primitive-name> <primitive-name> <arity> in <env>))
    ((_ <primitive-name> <arity>)
    #`(define-primitive <primitive-name> <arity> in env.global))))) --------------------------------------------------------


    My macro, (define-primitives-by-arity), is just syntactic sugar to simplify bulk definitions of basic forms such as plus, cons, car, cdr, etc. to be define all at once, based on their arity. For example:

    -------------------------------------------------------- (define-primitives-by-arity 0 exit)
    (define-primitives-by-arity 1
    null? procedure? pair? number? integer? symbol?
    car cdr)

    (define-primitives-by-arity 2 + - * / cons) --------------------------------------------------------

    The intent (as you seem to have guessed below) is to expand into a series of invocations of (define-primitive), in which the <arity> argument is constant for all of the newly-defined primitives. The result of that first expansion should be something like
    this:

    --------------------------------------------------------
    (define-primitive exit 0)
    (define-primitive null? 1)
    (define-primitive procedure? 1)
    (define-primitive pair? 1)
    [...]
    (define-primitive / 2)
    (define-primitive cons 2) --------------------------------------------------------

    As you can see, this would be trivial to do by hand anyway, so the macro is just syntactic sugar, but I expect that it would help later on down the road as I proceed with the textbook.


    However, I am guessing that the definition you might be looking for is

    (define-syntax-rule (define-primitives-by-arity arity primitives ...)
    (begin
    (define-primitive primitives arity)
    ...))

    Ah, thank you, this does exactly what I needed.

    I'm still unclear on why it works, and mine didn't, but I suspect I was just overthinking things. Part of the problem is that I'd set this project aside for about two years, and I am only just getting back to it now, so I don't exactly remember what my
    logic for it was.

    Also, I am used to (define-syntax-rule) always having a (syntax-rules) clause, whether or not it is used, but I can only assume from this that the clause is optional (only needed if the macro has more than one pattern to match, I take it). Am I
    understanding this correctly?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alice Osako@21:1/5 to All on Mon Jul 20 06:50:25 2020
    I have one semi-related question, though it is more about Emacs (and specifically Geiser) than Scheme per se. While the evaluator now works correctly when run by invoking Guile at the shell ('guile basic-eval.scm'), when running in the Guile REPL through
    the Geiser interface in Emacs, I get the following error:

    --------------------------------------------------------
    scheme@(guile-user)> (load "basic-eval.scm")
    Basic evaluator test REPL
    (+ 1 2)
    $29 = 3
    scheme@(guile-user) [3]> --------------------------------------------------------
    That is to say, it evaluates the form, but kicks me back out of the evaluator and into Guile's debugging menu at a fault depth of 3. The backtrace at level 1 is as follows:

    --------------------------------------------------------
    In ice-9/boot-9.scm:
    2312:4 6 (save-module-excursion #<procedure 7f1332c7bb10 at ice-9/boot-9.scm:3837:3 ()>)
    3832:12 5 (_)
    In /home/schol-r-lea/Documents/Programming/Projects/Lisp-In-Small-Pieces-walkthrough/basic-evaluator/basic-eval.scm:
    196:20 4 (basic:repl)
    91:24 3 (basic:eval (unquote geiser-eval) ((cons . #<procedure 7f133273d6c8 at /home/schol-r-lea/Documents/Programming/Projects/Lisp…>) …))
    73:23 2 (basic:eval unquote _)
    154:10 1 (lookup-env _ _)
    In unknown file:
    0 (scm-error misc-error #f "~A ~S" ("LOOKUP - unbound variable " unquote) #f)
    --------------------------------------------------------

    It looks as if Geiser is interpolating the form (unquote geiser-eval) into the input stream for some reason.

    Is anyone here familiar enough with Geiser to say why it would be doing this, and if there are any work-arounds or ways to prevent it? If not, does anyone have any recommendations for Emacs-friendly alternatives to Geiser?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Maciek Godek@21:1/5 to All on Tue Jul 21 02:39:24 2020
    W dniu poniedziałek, 20 lipca 2020 15:35:52 UTC+2 użytkownik Alice Osako napisał:

    However, I am guessing that the definition you might be looking for is

    (define-syntax-rule (define-primitives-by-arity arity primitives ...)
    (begin
    (define-primitive primitives arity)
    ...))

    Ah, thank you, this does exactly what I needed.

    I'm still unclear on why it works, and mine didn't, but I suspect I was just overthinking things.

    Essentially, a macro is some form ought to expand to some other form. 'syntax-case' macros allow you to evaluate some Scheme code, but this code needs to somehow construct a syntax object from its input.

    Your code contained a call to the "for-each" function (which discards the results of the function being passed as its first argument).

    The argument contains a call to the (define-primitive name #`<arity>), which - for things to make sense - would need to perform some side effect.

    If it did, it would be a nasty thing to do in a macro: a side-effect happening at macro expansion time!

    I could be guessing that what you wanted to do was to return a form, and construct the output from that form.


    Also, I am used to (define-syntax-rule) always having a (syntax-rules) clause, whether or not it is used, but I can only assume from this that the clause is optional (only needed if the macro has more than one pattern to match, I take it). Am I
    understanding this correctly?

    (define-syntax-rule (name . args) template)

    is equivalent to

    (define-syntax name
    (syntax-rules ()
    ((name . args) template)))

    'syntax-rules' is a simplified variant of 'syntax-case' which doesn't allow to do nasty things (such as the ones you did in your definitions). It can be defined in terms of syntax-case in the following way:

    (define-syntax syntax-rules
    (lambda (x)
    (syntax-case x ()
    ((_ (k ...) ((keyword . pattern) template) ...)
    #'(lambda (x)
    (syntax-case x (k ...)
    ((dummy . pattern) #'template)
    ...))))))

    I have copy-pasted this definition from Guile manual:

    https://www.gnu.org/software/guile/manual/html_node/Syntax-Case.html

    'syntax-case' is very difficult to use, and whenever possible, I'd recommend to use 'syntax-rules' instead.

    https://www.gnu.org/software/guile/manual/html_node/Syntax-Rules.html

    Why the variant I wrote above works is because of the treatment of ellipses in syntax-case/syntax-rules macros: The ... operator behaves a bit like the "map" function in Scheme, and a bit like "unzip" function.

    So, for example, this definition of the "let" form works:

    (define-syntax-rule (let ((variable value) ...) . body)
    ((lambda (variable ...) . body) value ...))

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)