Preamble
Very early in my long career as a computer programmer, I appreciated the source code editors that highlight source code syntax. Some of these code editors know how to adapt to the programming language used.
Later, by writing articles about programming, on blogs, then on my websites, I used GesHi, a powerful but complex library.
The problem with GesHi is that it does not process the FORTH code. I did well to create scripts adapted to the FORTH language for GesHi, but these scripts had still some flaws.
After analysis, GesHi processes the FORTH code character by character and considers the characters . and - as code operators.
GesHi represents more than 4000 lines of code in PHP. A complex code and little factored. We feel the touch of several programmers who have each added layers.
Complete code is available here:
https://github.com/MPETREMANN11/php/blob/main/Forth.php
A new code analyzer for FORTH
I was faced with several choices:
- leave FORTH GEsHi jobs as is and accept errors from GesHi
- choose another code analyzer and fall back on similar adaptation problems
- spending dozens of hours tracing and trying to improve Geshi....
It turns out that I'm very good at programming in PHP. So I started with do a very simple test: explode a FORTH code with PHP:
/**
* Explode FORTH source code in array of keywords
* @param type $code
*/
private function explodeSource($code) {
$lines = explode( "\n", str_replace("\r", "", $code));
$this->keySource = array();
foreach ($lines AS $line) {
$this->keySource[] = explode(" ", $line);
}
}
The result was just what I was looking for. This function explodeSource splits each row into an array of rows. The second loop slices the contents of each row to table using the space character as a separator. The result is returned in an array $this->
keySource.
This is what made me decide to create my own FORTH code analyzer in PHP. I made a FORTH code highlighter in just two evenings....
Define FORTH wordlists
On my website, the FORTH words are stored in a database. Here's how it's done injecting FORTH wordlists into a $this->keywords array:
/**
* fetch all forth words from database
*/
private function getKeywords() {
$Glossaire = new Application_Model_Glossaire;
$this->keywords[0] = $Glossaire->getListeMotsOrdinaires();
$this->keywords[1] = $Glossaire->getListeMotsDefinition();
$this->keywords[2] = $Glossaire->getListeMotsStructure();
$this->keywords[3] = $Glossaire->getVariablesConstantes();
$this->keywords[4] = $Glossaire->getListeMotsVocabulaires();
//$motsDefered = $Glossaire->getListeMotsDefered();
$Registers = new Application_Model_Registers;
$this->keywords[5] = $Registers->getListeRegistres();
}
On your website, you can define your lists in this way if you don't want not operate a database:
/**
* fetch all forth words from database
*/
private function getKeywords() {
$Glossaire = new Application_Model_Glossaire;
$this->keywords[0] = array('drop', 'dup', 'rot', 'over', 'swap');
$this->keywords[1] = array(':', ';', 'constant', 'variable');
$this->keywords[2] = array('if','else','then');
$this->keywords[3] = array('base');
$this->keywords[4] = array('forth');
}
With this method, different categories of FORTH words are defined. this allows to highlight the FORTH code with different styles.
Definition of styles
We define another table containing the different styles to highlight FORTH code:
/**
* define all styles for decoration
* @var array
*/
var $outStyle = array(
0 => 'color: #0000ff;', // normal words
1 => 'color: red; font-weight: bold;', // definitions words
2 => 'color: black; background-color: yellow;', // structure words
3 => 'color: #ff00ff; font-weight: bold;', // constants
4 => 'color: black; font-weight: bold; background-color: #d7d7ff', // vocs
'bin' => 'color: #7f00dd;', // binaries values
'dec' => 'color: #0000dd;', // decimal values
'hex' => 'color: #003ffd;', // hexadecimal values
);
In this code we have three keys bin dec and hex. These keys are used to decorate numeric values in binary, decimal or hexadecimal.
Decoration of a word FORTH
The decoration of a FORTH word works in successive stages:
if the word is in a comment, we don't decorate it
if the word is a binary or decimal or hexadecimal number, we decorate it. It's the role of regular expressions analyzing the structure of the word. For example, 10010011 is parsed as binary value;
if we encounter the word \ we activate the flag marking the comment and we return the decoration marking this comment;
finally, we test whether the word appears in the table of FORTH words to decorate. This is the role of the last loop foreach. If it's a word to decorate, we decorate it.
At each step, exit the function with return. If no condition is met we return the word without decoration, followed by a space character.
/**
* highlight FORTH code
* @param string $word
*/
private function styliseWord($word) {
if ($this->commentFlag == TRUE) {
return $word ." ";
}
// test if $word is a binary value
$re = '/^[0-1]{8}/is';
preg_match($re, $word, $matches, PREG_OFFSET_CAPTURE, 0);
if (!empty($matches)) {
return '<abbr style="' . $this->outStyle['bin'] . '" title="binary byte">'
. $word . '</abbr> ';
}
// test if $word is a decimal value
$re = '/^\d{1,}/is';
preg_match($re, $word, $matches, PREG_OFFSET_CAPTURE, 0);
if (!empty($matches)) {
return '<abbr style="' . $this->outStyle['dec'] . '" title="number">'
. $word . '</abbr> ';
}
// test if $word is a hexadecimal value
$re = '/^\$[a-f0-9]{1,}/is';
preg_match($re, $word, $matches, PREG_OFFSET_CAPTURE, 0);
if (!empty($matches)) {
return '<abbr style="' . $this->outStyle['hex']
. '" title="hexadecimal value">' . $word . '</abbr> ';
}
// if it's the word \ tag comment
if (ord($word) == 92) {
$this->commentFlag = TRUE;
return '<span style="color: #808080; font-style: italic;">' . $word ." ";
}
// test if $word is a FORTH word
foreach ($this->keywords AS $level => $listeMots) {
if (in_array(strtoupper($word), $listeMots)) {
return $this->linkifyWord($word, $level);
}
}
return $word ." ";
}
The decoration of a FORTH word that is defined in the FORTH word list is done this way in PHP:
/**
* array of external links
* @var array
*/
var $outlink = array(
0 => 'help/index-esp32/word/',
1 => 'help/index-esp32/word/',
2 => 'help/index-esp32/word/',
3 => 'help/index-esp32/word/',
4 => 'help/index-esp32/word/',
5 => 'help/reg-esp32/word/',
);
/**
* include FORTH word in an external link
* @param string $word
* @param mixed $level
* @return string
*/
private function linkifyWord($word, $level) {
if ($this->enable_keyword_links) {
return '<a href="' . $this->outlink[$level] . base64_encode($word)
. '" style="' . $this->outStyle[$level] . '">' . $word . '</a> ';
}
return '<abbr style="' . $this->outStyle[$level] . '">' . $word . '</abbr> ';
}
We define a list of links in the $outlink array. Here are internal links specific to my website. It's easy to set links to external sites:
/**
* array of external links
* @var array
*/
var $outlink = array(
0 => '
https://forth-standard.org/standard/core/DUP',
1 => '
https://forth-standard.org/standard/core/DUP',
2 => '
https://forth-standard.org/standard/core/DUP',
3 => '
https://forth-standard.org/standard/core/DUP',
4 => '
https://forth-standard.org/standard/core/DUP',
);
Reconstruction of the decorated FORTH code
To reconstruct our decorated FORTH code, we start from the $this->keySource array. This array was populated by explodeSource. This table is read item by item. If we encounter an empty item, we concatenate a single space. Otherwise, we decorate the item
according to the conditions of the styliseWord function.
/**
* Generate highlighted FORTH source code
* @return string
*/
private function generateOutsource() {
$outSource = "";
foreach ($this->keySource AS $line) {
foreach ($line AS $word) {
if (strlen($word)==0) {
$outSource .= " ";
} else {
$outSource .= $this->styliseWord($word);
}
}
if ($this->commentFlag == TRUE) {
$this->commentFlag = FALSE;
$outSource .= "</span>";
}
$outSource .= "\n";
}
return $outSource;
}
Here is how to use PHP code to highlight FORTH code. the PHP code must be loaded beforehand by include or the PHP autoload function:
<?php $forthStr = <<<EOT
DEFINED? --random [if] forget --random [then]
create --random
\ Random number data
$3FF75144 constant RNG_DATA_REG \ Read Only ESP32 register
\ get 32 bits random b=number
: rnd ( -- x )
RNG_DATA_REG L@
;
EOT;
echo $Forth->sourceForView($forthStr); ?>
The result is the expected one:
DEFINED? --random [if] forget --random [then]
create --random
\ Random number data
$3FF75144 constant RNG_DATA_REG \ Read Only ESP32 register
\ get 32 bits random b=number
: rnd ( -- x )
RNG_DATA_REG L@
;
Conclusion
If we refer only to the content of the GesHi code, the code highlighting seems an arduous task. And it is indeed one if we build a gas plant.
Here, with less than 200 lines of code, in PHP, I developed, in a few hours, a FORTH code highlighter that works much better than the gesHi code on which I spent dozens of days trying to understand all the subtleties.
There remains the question of the interest of highlighting the FORTH code...
What interested me, with GesHi, is that natively, if you highlight PHP code, it is possible to activate external documentation links.
It is this aspect of GesHi that seems to me to be of enormous interest to FORTH. Because you will have no matter how well documenting a program, nothing beats direct access to explanations of any FORTH vocabulary word.
In the source codes disseminated here, all the words of ESP32forth that are documented are displayed with syntax highlighting and a hyperlink giving direct access to the documentation of this word.
Because, if we want FORTH to interest programmers, it seems essential to me to facilitate the understanding the code. And, unless I'm mistaken, no other FORTH developer website has yet given such easy access to FORTH code documentation through syntax
highlighting.
A word for Peter FORTH:
Peter.
You blocked me from your FORTH2020 group when I presented this method of FORTH code highlighting.
You also blocked me from other bands as soon as I intervened to present FORTH. Your ESP32FORTH blog is needy. Badly built. It would gain in readability to exploit my PHP code.
So put your ego aside. If you really think you are working in the interest of the FORTH language, Stop this childish attitude towards me.
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)