• The rise, fall, and resurrection of APL\3000

    From Doug White@21:1/5 to Gavin Scott on Tue Dec 22 02:12:52 2020
    I realize the original post is a bit old, but I just saw it recently when I visited the newsgroup after a lenghty absence. I thought I would chime in
    with my experiences with APL/3000.

    I worked as a circuit designer at HP Santa Clara between 1978 and 1982. I
    used APL when I was a student at MIT, and was delighted to have it
    available at my firt job. The HP3000 was used by everyone in the facility (which was quite large). The accountants used it, the secretaries used it
    for word processing, and the engineers used it. It tracked inventory &
    sales as well. I forget the details, but the APL was supposed to "psudo- compile" functions the first tiem it ran them so that it didn't have to re- interpret them the next they were called. The marketing hype was that it
    was supposed to be lightning fast,

    I designed an analog (inductor & capacitor) filter for a new instrument. I needed to make sure that it would meet specification using moderate
    precision components without requiring any tuning in production. To check this, I wrote a dedicated circuit simulation function, and set up a Monte
    Carlo analysis that fed it randomized component values. The circuit
    analysis wasn't especially computationally demanding, but running the Monte Carlo simulations with a reasonmable number of trials took a while.

    After several short test runs, I fired it up one day with a larger number
    of trials. Within about 10 or 15 minutes, the computer people had been
    flooded with complaints that the 3000 was suddenly crawling. They tracked
    me down and demanded to know what I was doing that had brought the entire Division's mainframe to its knees.

    I quickly switched to coming in on nights & weekends to do my simulations.
    It wasn't exactly speedy, but it was a lot faster with no one else on the system.

    In discussing this with other engineers, there was something about the
    HP3000 computer architecture that swapped code & data in & out of memory
    when it switched tasks. If you ran anything memory intensive, that tended
    to bog things down. Supposedly the performance of APL3000 in particular
    was so bad that they actually lost a law suit when it failed to come close
    to its claimed performance. I think part of APL's demise at HP was the resulting black eye it earned in the minds of management.

    The one other thing I remember was that the APL3000 language manual was wonderful. They had beautiful color graphics that showed how some of the
    more complex matrix/array manipulation primatives worked. Long after I
    left HP, I still kept the manual for reference.

    Doug White

    Gavin Scott <gavin@learn.bio> wrote in news:b62e0d7f-09a1-4740-b634-293c59805b40n@googlegroups.com:

    (If anyone can add to this story or knows if any of the HP people
    involved are still around I would love to hear it!)

    Back at the beginning of the 1970s, Hewlett-Packard developed a new general-purpose timesharing commercial data processing computer system
    called the HP 3000. It was a stack-based CISC system that ran an
    operating system called MPE (Multi-Programming Executive). This series
    of computer systems became legendary among its loyal user community
    and had a long life across multiple hardware architectures before
    being unceremoniously axed in 2001 (there are still a few of the last generations in use today of course).

    Perhaps its least-known optional subsystem product among COBOL/3000, BASIC/3000, FORTRAN/3000, RPG/3000, etc. was an APL implementation
    called APL\3000. As one could perhaps guess by the backslash in its
    name, HP seems to have been inspired by APL\360's success as a
    timesharing product, and it implements almost exactly the IBM APL.SV
    feature set.

    While at first glance it appears as a clone of the IBM product, the HP implementation included a number of advanced features including a (the first?) APL compiler, some very advanced optimizations, APLGOL, and a virtually (pun possibly intended) unlimited workspace and expression
    result size through a clever implementation of a flat 32-bit address
    space on a machine where every other language was limited to 64KiB of directly addressable memory. HP also produced a variant of their
    (several thousand dollar, 50lb) HP2645 terminal, the HP2641A, which
    offered the APL character set and overstrike character entry and tight integration with the APL\3000 product. They also supported simple
    ASCII terminals through a three-character replacement for the missing
    special characters so if you wanted to enter a quad you would type
    "QD. Iota was "IO, rho "RO etc. Ugly, but it meant you could also
    write a batch job using plain ASCII files that could utilize APL.

    In order to get the 32-bit address space needed to have an unlimited workspace and result size, they developed ten new CPU instructions
    whose microcode came on a set of PROM chips when you bought the
    $15,000 product. This would ultimately spell the doom for APL\3000
    when HP lost interest in it at the end of the 1970s and failed to
    implement those required instructions on any later HP 3000 CPUs. Thus APL\3000 was only ever supported on their Series II and Series III
    models of HP 3000s.

    The implementation was actually pretty clever. The APL system would
    open the workspace file and set up a series of virtual memory "pages"
    in the stack in the form of a linked-list whose address was stored in
    a memory location known to the firmware. The new microcoded
    instructions provided virtual 32-bit address loads and stores and
    moves to and from the regular part of the stack, as well as a virtual
    to virtual move and some special instructions to accelerate the
    execution of their compiled APL code which used a virtual machine they created called the "E Machine". The microcode used the pages in the
    stack as a cache for the much larger workspace file and would search
    through the linked list of pages for a particular virtual address, maintaining the list in most-recently-accessed order as it went, and
    would call back to APL if an address could not be found at which point
    the APL system would write out the least-recently-accessed page if it
    was dirty and replace it by reading in the appropriate chunk of the
    workspace file after which the faulting instruction would be restarted
    by the microcode.

    The default setup was something like 24 pages of 512 or 1024 bytes
    each, and a system variable []VM was provided to interrogate and
    reconfigure the VM page number and size and the search method used by
    the microcode.

    As a result, one can evaluate things like 10000000?20000000 that
    likely were impossible on any other APL implementation at the time
    (ok, well, that takes about ten minutes on a modern PC under
    simulation and likely would have taken a few days to complete on a
    real system, but it DID work!).

    So if this APL implementation was so amazing, with unlimited result
    size, unlimited workspace size, etc. why has almost nobody heard of
    it? Well, when it came out, performance apparently sucked. The
    developers had fallen back on a virtual machine target for their
    compiler after deciding there was no efficient way to emit native code
    due to an enforced separation of code and data. So every APL
    expression went through compilation into an 8-bit byte-code for the
    virtual E Machine (though they were very clever about determining when
    they could avoid recompilation, and IF it was required they would
    generate a different style of code the second time that was more
    generic and was less likely to require further recompilation) which
    then had to be executed by their virtual E Machine implementation.
    They spent the next three years working to improve things, but by 1979
    no magic bullet solution was in sight and even if it had been, the HP
    sales force and systems engineers had almost universally forgotten
    about the product's existence.

    So when the last HP 3000 Series III system with APL installed was shut
    down sometime in the 1980s, APL\3000 passed into legend. There were
    those who whispered about its existence, and I think the first
    discussion I had about the possibility of resurrecting it was in the
    early 1980s. At the time the thought was that a real Series III would
    be needed, and I knew someone who had a set of the firmware PROMs, but
    it never happened and slowly the legends faded but were never
    forgotten.

    A month or so ago I started playing with J. David Bryan's amazing
    classic Series III HP 3000 hardware simulator with the goal of
    building up the most complete system possible, utilizing all the old
    bits that our lovable retro-computing aficionados have saved over the
    years like the incurable packrats they are. A friend had squirreled
    away a tape with APL on it and I became interested again in the idea
    of bringing it back to life. Since we now had a software simulator of
    the hardware, the idea of implementing the mythical (and undocumented)
    APL instructions needed to run it seemed within reach, and I started
    trying to figure out the behavior of the missing instructions, and
    after a week or so of hacking on it we now have APL\3000 back from the
    dead.

    :APL
    APL\3000 HP32105A.01.05 (C)HEWLETT-PACKARD CO. 1979
    05:50 08/29/92
    2+2
    4

    The ironic thing is that I ended up implementing the instructions in
    MPE's own unimplemented instruction trap handler and made no changes
    to the hardware simulator itself, so this technique could have been
    used back in 1980 when we first started talking about getting APL
    going somehow. Oh well.

    Is this good for anything? Probably not really, though it's always fun
    to have a chance to play with a piece of software that probably has
    not been executed in 35 years. And it holds an important place in the
    history of language development, having been the "first" at a few
    things. Unfortunately, it was perhaps just a bit too far ahead of its
    time and failed to survive until machines got big and fast and wide
    enough for it to really shine.

    Due to the shadowy license state (or lack thereof) of all this
    forty-year-old software, I probably won't be putting it up for public download, but if anyone is interested you can hit me up and I can
    probably set you up fam.

    Gavin



    --
    This email has been checked for viruses by AVG.
    https://www.avg.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Gavin Scott@21:1/5 to Doug White on Mon Dec 21 19:02:37 2020
    On Monday, December 21, 2020 at 8:12:54 PM UTC-6, Doug White wrote:
    I realize the original post is a bit old, but I just saw it recently when I visited the newsgroup after a lenghty absence. I thought I would chime in with my experiences with APL/3000.

    Thanks, Doug, it's always great to hear from actual APL\3000 users!

    Cheers,

    Gavin

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)