• Re: Defensive programming

    From Stefan Ram@21:1/5 to Stefan Ram on Fri Sep 23 18:06:03 2022
    ram@zedat.fu-berlin.de (Stefan Ram) writes:
    infrequent that I happen to see that dialog! Reasons might include
    problems with the character encoding, full disks, or recently,

    I forgot: The Python script depends on a library I wrote.
    A common source of errors also is that I change something
    with the library, but then forget to adjust the script to
    this change. That change in the library will render the
    script inoperable. But the VBA check then will detect and
    report this problem.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefan Ram@21:1/5 to All on Fri Sep 23 17:57:13 2022
    I wrote a small Python script to append a line to the end
    of a log file. This script is invoked from a VBA subroutine
    to append a line to that log file.

    In the same VBA subroutine, I then read the last line from
    the same log file and compare it with the line to be written.
    If they do not agree, the VBA subroutine opens a new message
    box informing the user.

    This might sound overly defensive, but in fact it is not
    infrequent that I happen to see that dialog! Reasons might include
    problems with the character encoding, full disks, or recently,
    a possible spurious error message for reasons (yet) unknown
    (maybe due to an overload of the system, which led to timeouts).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefan Ram@21:1/5 to Dmitry A. Kazakov on Fri Sep 23 18:58:57 2022
    "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
    You should simply flush the log file periodically (keeping it open of >course).

    This log file gets infrequent writes. Not more than 1000 per day.
    So to append a line, I always open the file for appending, write
    the line, and close it immediately.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dmitry A. Kazakov@21:1/5 to Stefan Ram on Fri Sep 23 20:51:49 2022
    On 2022-09-23 19:57, Stefan Ram wrote:
    I wrote a small Python script to append a line to the end
    of a log file. This script is invoked from a VBA subroutine
    to append a line to that log file.

    In the same VBA subroutine, I then read the last line from
    the same log file and compare it with the line to be written.
    If they do not agree, the VBA subroutine opens a new message
    box informing the user.

    That is an awful idea IMO. With disastrous performance as it would block
    the logging file for a long time.

    The key point is that logging is secondary to functionality. Thus
    whatever might happen with the logs, that shall not disrupt normal
    operation.

    For example, in a real-time application logger queues messages instead
    of writing them. Then a lower priority task flushes the log messages
    queue to the disk in background.

    This might sound overly defensive, but in fact it is not
    infrequent that I happen to see that dialog! Reasons might include
    problems with the character encoding, full disks, or recently,
    a possible spurious error message for reasons (yet) unknown
    (maybe due to an overload of the system, which led to timeouts).

    You should simply flush the log file periodically (keeping it open of
    course).

    Modern file systems do caching, mirroring, journaling, "never-overwrite-same-block" stuff. You cannot do any better from a VBA application! You even cannot know if the stuff you read back was
    actually physically written down to the physical medium and not sitting
    in some N-th order cache of some M-th controller in between... (Mission critical logger solutions like blackboxes may use special hardware
    write-once mediums etc)

    --
    Regards,
    Dmitry A. Kazakov
    http://www.dmitry-kazakov.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dmitry A. Kazakov@21:1/5 to Stefan Ram on Fri Sep 23 21:25:35 2022
    On 2022-09-23 20:58, Stefan Ram wrote:
    "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
    You should simply flush the log file periodically (keeping it open of
    course).

    This log file gets infrequent writes. Not more than 1000 per day.
    So to append a line, I always open the file for appending, write
    the line, and close it immediately.

    Keep it open. Flush after each write since it is infrequent.

    If the file was opened for shared access then after flush you should be
    able to read the file from outside, e.g. from Notepad++ etc.

    --
    Regards,
    Dmitry A. Kazakov
    http://www.dmitry-kazakov.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefan Ram@21:1/5 to Stefan Ram on Fri Sep 23 19:21:30 2022
    ram@zedat.fu-berlin.de (Stefan Ram) writes:
    This log file gets infrequent writes. Not more than 1000 per day.
    So to append a line, I always open the file for appending, write
    the line, and close it immediately.

    But several different processes could append to that log file.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefan Ram@21:1/5 to Dmitry A. Kazakov on Fri Sep 23 19:30:26 2022
    "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
    Keep it open. Flush after each write since it is infrequent.

    What's the problem this is supposed to be the solution to?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefan Ram@21:1/5 to Stefan Ram on Fri Sep 23 19:29:07 2022
    ram@zedat.fu-berlin.de (Stefan Ram) writes:
    But several different processes could append to that log file.

    Including batch files using ">>".

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dmitry A. Kazakov@21:1/5 to Stefan Ram on Fri Sep 23 21:31:53 2022
    On 2022-09-23 21:21, Stefan Ram wrote:
    ram@zedat.fu-berlin.de (Stefan Ram) writes:
    This log file gets infrequent writes. Not more than 1000 per day.
    So to append a line, I always open the file for appending, write
    the line, and close it immediately.

    But several different processes could append to that log file.

    Then you need something like a database to manage transactions to the
    log in order to keep in consistent and properly interlock. If the file
    is opened for shared write, you will get a mess. If not, you will fail
    to open a locked file. The behavior will depend on the OS. E.g. Linux
    does not even have a proper locking mechanism.

    P.S. It is a bad idea anyway. Each process should have its own log file.
    Supply messages with a time stamp if you want to be able to generate
    some combined report later.

    --
    Regards,
    Dmitry A. Kazakov
    http://www.dmitry-kazakov.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefan Ram@21:1/5 to Dmitry A. Kazakov on Fri Sep 23 19:41:47 2022
    "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
    "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
    Keep it open. Flush after each write since it is infrequent.
    What's the problem this is supposed to be the solution to?
    That upon system crash you won't lose all file, since it is never closed.

    That's why I always keep it open as short as possible;
    I immediately close it after every append. I think, ">>"
    does this too. It might be open for append access few
    seconds per day.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dmitry A. Kazakov@21:1/5 to Stefan Ram on Fri Sep 23 21:50:13 2022
    On 2022-09-23 21:41, Stefan Ram wrote:
    "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
    "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
    Keep it open. Flush after each write since it is infrequent.
    What's the problem this is supposed to be the solution to?
    That upon system crash you won't lose all file, since it is never closed.

    That's why I always keep it open as short as possible;
    I immediately close it after every append. I think, ">>"
    does this too. It might be open for append access few
    seconds per day.

    This explains well the problems you experience.

    --
    Regards,
    Dmitry A. Kazakov
    http://www.dmitry-kazakov.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dmitry A. Kazakov@21:1/5 to Stefan Ram on Fri Sep 23 21:33:27 2022
    On 2022-09-23 21:30, Stefan Ram wrote:
    "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
    Keep it open. Flush after each write since it is infrequent.

    What's the problem this is supposed to be the solution to?

    That upon system crash you won't lose all file, since it is never closed.

    --
    Regards,
    Dmitry A. Kazakov
    http://www.dmitry-kazakov.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dmitry A. Kazakov@21:1/5 to Stefan Ram on Fri Sep 23 21:35:52 2022
    On 2022-09-23 21:29, Stefan Ram wrote:
    ram@zedat.fu-berlin.de (Stefan Ram) writes:
    But several different processes could append to that log file.

    Including batch files using ">>".

    Then mess is guaranteed! Batch files are perfect at this... (:-))

    --
    Regards,
    Dmitry A. Kazakov
    http://www.dmitry-kazakov.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)