• Architecture and best practise questions

    From stk@21:1/5 to All on Tue Sep 5 22:41:35 2023
    Hello there and thanks for reading in advance,

    Long post... Sorry.

    long time Tcl'er here (since about 1992, professionally and privately).

    Context: a few years ago, I wrote a kind of a picture- and blogging-module using Rivet for Apache-HTTP. Didn't want to use WordPress or the like, since I have some specific features in mind and also just for the fun of implementing it myself...
    I used it extensively periodically over years (since about 2014) and it got a few new features over time - it "grew"... What started as a quick hack, got a little messy over time and it would be hard to add a few serious new features the way it is now.
    So I'm refactoring, reimplementing it and now I'm considering some design-decisions for the architecture, where I'd like to ask the expert-community about your opinion.

    *Documentation:*
    I'd like to use Doxygen, but not sure, if this is still a good option, since Tcl is not supported anymore. Whats your preference? My goal is, to document the API of my project in source and generate it into HTML automatically. Previously, Doxygen was my
    tool of choice here. - Another option would be to make the system self-documenting, since it will have parser-plugins and the ability to create document-trees. But why inventing the wheel, when it's already there (even, if it's a flat tyre nowadays...)? -
    Are there better ways to create API-docs from sources for Tcl?

    *Tcl-packages:*
    The first implementations (there are three at this time...) used Tcl-packages to split the whole thing into architecture components. A main package that loaded all the others in its init-part and then provided the Rivet-Apache-plugin as package which was
    used in a small Rivet-script that just passed on the HTTPS-request to my package.

    The whole package hierarchy did not start at global namespace "::" but as "myPackage::mod1::...". The idea was, that this way it would be possible to use "require package" in a namespace and thus use the whole thing not in global namespace but where
    convenient.
    For refactoring I'm now considering, if starting globally as "::myPackage::mod1::..." is better or not? Is there a best practice regarding that?

    *Rivet vs. Standalone:*
    This is almost decided, but maybe you have some good input here: I think, I will get rid of using Rivet-apache-module. Although I really like it and use it for many small scripts still, I feel that while it's not really dead, it's also not really alive
    anymore... Let's face it, Tcl-support is dying out in general, sadly. But it is still by far my favourite scripting-lanuage.

    So, I'm considering going standalone: Write my application as a server-demon and only use a very small CGI or plugin to pass on requests to it and receive the results.

    Advantage would be, that it's independent of Rivet, allowing to use NGinx or other HTTP-servers or even run it locally as a static HTML-creator and Document-generator (can output several target formats from the created document-tree). Also data/state-
    persistence is easier in a standalone-server-application, compared to Rivet-modules, where I only have limited control over when the application gets terminated/restarted depending on Apache-load and other factors.

    Disadvantages: Handling of data access in httpd-context. E.g. if a generated page refers to an image, I can handover the handling and delivery of the image or other files to Apache/Rivet. A standalone-service needs some clever mechanism to put file
    references into the HTML-output (or other formats), that are then resolved by the httpd, including access-rights-management, which is currently completely handled by Apache.

    Still, going standalone looks more tempting to me atm, because it allows fine grained access control, different httpd-environments and local application mode for other purposes. Any thoughts?

    *Procs in namespaces vs. namespace ensembles:*
    Ensembles feel kind of elegant to me, but by default, I mostly just write procs in namespaces, although related procs could go into an ensemble - so much for clean APIs... But then I'm not sure about the name resolving performance of ensemble commands vs.
    direct proc call. Just kind of fuzzy thoughts about the pros and cons here. - Happy about useful input...

    Thank you very much for reading, if you made it to this point... Appreciate any helpful inputs and thought.

    Best regards,
    Stephan

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to stk on Wed Sep 6 12:51:36 2023
    stk <skuhagen@web.de> wrote:
    *Documentation:*
    I'd like to use Doxygen, but not sure, if this is still a good
    option, since Tcl is not supported anymore. Whats your preference?
    My goal is, to document the API of my project in source and generate
    it into HTML automatically.

    Personal opinion here, but in every case I can think of where doxygen
    was used to "autodocument the source" the resulting docs were no more
    valuable than simply reading the source (in fact, often less valuable).
    The doxygen html would tell you that there were these twelve C
    functions you could call, and that their parameters were X, Y, or Z,
    but leave off what the function did (I guess the author's though their
    names were meaningful enough) and more often than not, leave off any description of what was expected of the parameters.

    Now, this might have been a failure of the author to insert enough
    "doxygen tags" into the source to generate proper docs, but as I saw
    the pattern with *every* one I remember, it felt more like doxygen
    existed merely to satisify some corportate code review checkbox for "auto-generated code documentation" where the code review checkbox
    omitted a second checkbox of "and the output documentation is useful to
    someone for the purpose of learning/using the API).

    *Tcl-packages:*
    The first implementations (there are three at this time...) used
    Tcl-packages to split the whole thing into architecture components.
    A main package that loaded all the others in its init-part and then
    provided the Rivet-Apache-plugin as package which was used in a small Rivet-script that just passed on the HTTPS-request to my package.

    The whole package hierarchy did not start at global namespace "::"
    but as "myPackage::mod1::...". The idea was, that this way it would
    be possible to use "require package" in a namespace and thus use the
    whole thing not in global namespace but where convenient. For
    refactoring I'm now considering, if starting globally as "::myPackage::mod1::..." is better or not? Is there a best practice regarding that?

    I don't think this will matter much. Most scripts do their package
    requires at the top level of the initial script and so they are "at
    global level" anyway. Not forcing global level makes the package
    slightly more flexible, but perhaps a documentation note of the
    fexibility would be in order.

    *Rivet vs. Standalone:*
    This is almost decided, but maybe you have some good input here: I
    think, I will get rid of using Rivet-apache-module.

    I feel that if you do so you will find yourself eventually adding back,
    as custom code you have to write yourself, much of what Rivet provides
    to you if you go this route.

    If that is what you want, then go for it. But if your focus is the
    blog system, not the underlying web plumbing, then keeping Rivet may
    prove better in the end.

    *Procs in namespaces vs. namespace ensembles:*
    Ensembles feel kind of elegant to me, but by default, I mostly just
    write procs in namespaces, although related procs could go into an
    ensemble - so much for clean APIs... But then I'm not sure about the
    name resolving performance of ensemble commands vs. direct proc
    call.

    Tcl has a very useful [time] command. Test, on your hardware and with
    your code the performance of both and see what you get. Then you won't
    be guessing, but will have actual data.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ted Nolan @21:1/5 to rich@example.invalid on Wed Sep 6 14:44:06 2023
    In article <ud9sko$2gkmn$1@dont-email.me>, Rich <rich@example.invalid> wrote: >stk <skuhagen@web.de> wrote:
    *Documentation:*
    I'd like to use Doxygen, but not sure, if this is still a good
    option, since Tcl is not supported anymore. Whats your preference?
    My goal is, to document the API of my project in source and generate
    it into HTML automatically.

    Personal opinion here, but in every case I can think of where doxygen
    was used to "autodocument the source" the resulting docs were no more >valuable than simply reading the source (in fact, often less valuable).
    The doxygen html would tell you that there were these twelve C
    functions you could call, and that their parameters were X, Y, or Z,
    but leave off what the function did (I guess the author's though their
    names were meaningful enough) and more often than not, leave off any >description of what was expected of the parameters.

    Now, this might have been a failure of the author to insert enough
    "doxygen tags" into the source to generate proper docs, but as I saw
    the pattern with *every* one I remember, it felt more like doxygen
    existed merely to satisify some corportate code review checkbox for >"auto-generated code documentation" where the code review checkbox
    omitted a second checkbox of "and the output documentation is useful to >someone for the purpose of learning/using the API).


    I find the Doxygen reports for Tcl projects very useful. You do need
    to use enough tags to provide the information -- nothing will provide
    it for you if it's not there. I anticipate being able to build the
    versions supporting Tcl for many years to come.
    --
    columbiaclosings.com
    What's not in Columbia anymore..

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From stk@21:1/5 to Rich on Wed Sep 6 21:50:04 2023
    Thanks for your inputs!

    Rich schrieb am Mittwoch, 6. September 2023 um 14:51:41 UTC+2:
    stk wrote:
    Personal opinion here, but in every case I can think of where doxygen
    was used to "autodocument the source" the resulting docs were no more valuable than simply reading the source

    Well, yes... Of course I was referring to a use case where I really do the documentation and provide the necessary tags to document my sources. As also pointed out by Ted Nolan, only when using and and populating tags with useful contents, the result is
    good. But that is, what I'm doing. I'm the kind of programmer, who writes documentation first, then the code... Habit from working a few decades in safety critical areas.

    *Tcl-packages:*
    ...
    The whole package hierarchy did not start at global namespace "::"
    but as "myPackage::mod1::...". The idea was, that this way it would
    be possible to use "require package" in a namespace and thus use the
    whole thing not in global namespace but where convenient. For
    refactoring I'm now considering, if starting globally as "::myPackage::mod1::..." is better or not? Is there a best practice regarding that?
    I don't think this will matter much. Most scripts do their package
    requires at the top level of the initial script and so they are "at
    global level" anyway. Not forcing global level makes the package
    slightly more flexible, but perhaps a documentation note of the
    fexibility would be in order.

    Maybe you're right here. I may overthinking this, because I see so much cluttered global namespaces in different programming languages by unexperienced programmers and wonder, what the good Tcl-gurus do with their packages usually, or what is considered
    good practice in the community.

    *Rivet vs. Standalone:*
    This is almost decided, but maybe you have some good input here: I
    think, I will get rid of using Rivet-apache-module.
    I feel that if you do so you will find yourself eventually adding back,
    as custom code you have to write yourself, much of what Rivet provides
    to you if you go this route.

    True, but it makes the system usable as generic static document-generator. It currently can only produce pure text, LaTeX and HTML from different markdown-flavours, but that might change.

    If that is what you want, then go for it. But if your focus is the
    blog system, not the underlying web plumbing, then keeping Rivet may
    prove better in the end.

    Rivet will definitely better be integrated into Apache, that's for sure and safe a lot of effort. But then I'm also tied to Apache and have bigger effort making it work with other httpd.

    Tcl has a very useful [time] command. Test, on your hardware and with
    your code the performance of both and see what you get. Then you won't
    be guessing, but will have actual data.

    True, of course. I will measure a few alternative designs as I did in the first implementation as well. It was a more a broader consideration. Timing might be one aspect, but also elegance of the design and resulting API.

    Thanks again for your thoughts.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From stk@21:1/5 to All on Wed Sep 6 21:32:48 2023
    Thanks for the answers so far!

    Ted Nolan <tednolan> schrieb am Mittwoch, 6. September 2023 um 16:44:12 UTC+2:
    I find the Doxygen reports for Tcl projects very useful. You do need
    to use enough tags to provide the information -- nothing will provide
    it for you if it's not there.

    I agree with that, which is why I'm still considering it. It is so far the best design documentation tool, that I know. I usually recommend it to younger developers and projects in the company where I work to document the C-stuff, they do. With well
    selected sets and a few followed style-guide rules, that are to be followed, the generated documentation is a lot better than reading just the code. Especially as it gives a good overview of the connections between architecture components, modules and
    functions.

    Besides using my document/blogging thingy to document itself, I consideres also to use emacs org-mode and do literate programming. Tried this many years ago (and it feels to me, that the idea came out of fashion at least 20 years ago anyway, but that
    really requires a lot of discipline and doxygen looks like the easiest and most feature rich solution still.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)