• Announcing Ox release 1.10.1

    From Tom Shields@21:1/5 to All on Wed Feb 2 13:27:18 2022
    Ox version 1.10.1 is now available on SourceForge (sourceforge.net/projects/ox-attribute-grammar-compiler/).

    Ox generalizes the function of Yacc in the way that an attribute grammar generalizes a context-free grammar. Ordinary Yacc and Lex specifications may be augmented with definitions of synthesized and inherited attributes written in C/C++ syntax. Ox checks these specifications for consistency and completeness, and generates from them a program that builds and decorates attributed parse trees. The user may specify post-decoration traversals for easy ordering of side effects, such as code generation. Ox handles the
    tedious and error-prone details of writing code for parse-tree management, so its use eases problems of security and maintainability associated with that aspect of translator development.


    Ox NEWS

    * Noteworthy changes in release 1.10.1 (1 Feb 2022)

    ** Internals

    This release continues the upgrade of the Ox source from C99 to C++11.

    The Ox internal 'stringTable []' implementation was changed to use the 'std::unordered_map' container. The generation of 'yyyStringTab [] from the contents of 'stringTable []' was changed to eliminate empty entries.

    The code generation architecture was restructured:
    - The C 'typedef union target * targetT' pun (aka hack) was replaced with a class hierarchy, with abstract base class 'outMgr'.
    - The use of C global 'FILE *' variables and C standard library file
    management functions was replaced by a 'std::iostream *' member in class 'outMgr' and the respective member functions for output.
    - Classes derived from 'outMgr' unify output to files (using a 'std::fstream
    *' member) and in-memory buffers (using a 'std::stringstream *' member).
    - The generation of '#line' directives was simplified, replacing the 'targetT' pun's set of 'lex* ()' and ‘gen* ()' text output functions with a set of overloaded 'put ()' text output functions.
    - Calls to 'put ()' are preceded by a call to 'lex ()' (for input file '#line' directives) or 'gen ()' (for output file '#line' directives), as required.
    - The deferred blank line ring buffer complexity was simplified to just count blank lines submitted for output, but not yet written.
    - Blank lines (up to a maximum) are actually written to the output when switching between writing non-blank *as-is* input text and generated text,
    with '#line' directives written, as required.
    - Other code generation function name prefixes 'gen' and 'print' were replaced with 'put' for unifority.

    The lexer specifications are now processed by 'flex++':
    - The use of C global 'FILE *' variables and C standard library file
    management functions was replaced with 'std::ifstream' objects and the respecitive member functions for input.
    - The custom 'YY_INPUT ()' macro used with 'flex'-generated lexer source was replaced with an overriden 'LexerInput ()' implementation to handle the
    Windows "\r\n" EOL sequence.
    - The custom input buffer stack management was replaced with the 'flex' standard input buffer stack management.

    The following additional changes were made to the lexer specifications:
    - A number of individual pattern rules with the same action were combined into a single pattern rule using the '|' pattern alternative operator, for
    clarity.
    - The use of 'unput ()' was replaced with 'yyless ()', for efficiency.
    - A number of trailing context pattern rules were rewritten to eliminate the trailing context, for efficiency.


    ** Bug fixes

    Occurrences of unmatched ` and ' characters in // comments, which cause 'flex++' version 2.6.2 to run off the end of the specification file, were either removed or paired to avoid the issue.

    The error message string created for failure to erase a file was left in a global 'std::stringstream' buffer if the erase operation succeeded.
    Subsequent use of that global buffer for an error message contained the erase failure message text.

    The Y-file lexer was corrected to return the 'RBRACE' token for a '}'
    character in the Yacc definition section (not associated with a "%<directive>
    { ... }" C code block, which would normally never occur); it was incorrectly returning the 'LBRACE' token.

    The tree visualization skeleton was missing "#include <cassert>" (for a C++ compile) or "#include <assert.h>" (for a C compile).

    If Ox is configured for a single target parser (e.g., with the option "--enable-target=bison"), only that corresponding target parser header file (e.g., '#include "agc-bison.hh"') is now processed when compiling './source/main.cc'.

    A typographical error in an attribute evaluation expression (a + operator in the expression should have been a - operator) was corrected in the '*calc*' test cases.

    The test suite cleanup configuration where the 'bison' version is < 3.5.4 was corrected.

    The './configure' script check for the presence of the C++ header
    'FlexLexer.h' in the MinGW environment was corrected to look in the '/usr/include' system directory. If 'flex++' is used in the MinGW
    environment, the 'AM_CPPFLAGS' Automake variable is augmented with "-idirafter /usr/include" so the MinGW C++ compilers can find 'FlexLexer.h' at compile time.

    The './configure' script was corrected to extract the 'flex' version number regardless of the executable's name.


    ** Build system

    The C++ header 'FlexLexer.h' from the 'flex' distribution used to generated
    the distributed lexer C++ source was added to the './source' distribution directory. The 'FlexLexer.h' file installed in other environments may not match the distributed source, causing C++ compile failure.

    The './configure' script check for C++ compiler support for the "-std=gnu++11|14|17" option was changed to "-std=c++11|14|17".

    The Automake 'BUILT_SOURCES' variable is now only set if Ox is configured with the "--enable-maintainer-mode" option.

    The lexer generation Make rule is now only enabled if Ox is configured with
    the "--enable-maintainer-mode" option.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tom Shields@21:1/5 to but so far no one has on Mon Feb 7 20:46:10 2022
    Howdy, Roger, responses to your questions below:
    On Feb 7, 2022, at 1:21 PM, Roger L Costello <costello@mitre.org> wrote:

    Hello Tom,

    I have a few questions about Ox:

    1. Is there a complete, up-to-date, easy-to-read Ox manual?

    Kurt Bischoff, the original (primary?) Ox developer at Iowa State University, wrote a tutorial (dtd 5 Nov 1993) and a reference manual (dtd 14 Nov 1993).

    I have attempted to keep both documents current as I’ve expanded on the original Ox functionality - the reference manual more extensively than the tutorial. Both documents are in the ‘./docs’ distribution directory, in both PDF and in LaTeX source form.

    I believe that both documents are current, although it is certainly possible that I’ve missed something. To date, this is a one-man labor of love ;-)

    As to whether the documents are “easy-to-read”, you will have to be the judge of that.

    I suspect that the reference manual might be considered a bit cryptic if one isn’t already familiar with attribute grammars. Furthermore, the
    limitations of the current Ox translator (in particular, allowing arbitrary C/C++ code in an attribute's definition), make it easy to create hidden attribute dependencies that will randomly break the Ox-generated code.

    I’m open to constructive comments from users, but so far no one has said anything to me directly.


    2. Are there any published books on Ox?

    No … do you want to write one?

    The original Ox distribution includes an example compiler for a small programming language, GPPL, built using Ox (and originally Yacc and Lex). The compiler generates C source code. I have maintained the GPPL compiler, but only to the extent required to enable it to compile, link & execute.

    Kurt Bischoff wrote a report describing the compiler: "GPPL: A Small Block-Structured Imperative Programming Language Implemented using Ox”, Iowa State University TR#92-32, December, 1992. A Postscript file of that document from Kurt’s last known Ox distribution is in the ‘./gppl’ distribution directory.


    3. Is there an Ox group on Stack Overflow?

    Not that I am aware of.

    I did a search just now on the Stack Overflow web and only found 2 questions that were obviously about Ox, both from the same user in April & May of 2021.
    I guess I should investigate how to set up a trigger to get notified of future Ox questions.


    4. Are there any commercial products that were developed using Ox?

    Not to my knowledge. I don’t get much user feedback, save for the
    infrequent bug report.

    According to the SourceForge server, there have been a total of 472 downloads of the various Ox distribution files since December 2018, which was when I first registered the project. That tells me is there is a non-trivial level
    of interest, so I’ve kept at it.

    Regards,

    Tom Shields

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)