• Recycle semaphore?

    From Va7aqd@1:103/705 to All on Thu Mar 21 12:15:58 2019
    Howdy!

    I'm running current CVS on Linux (Debian 64-bit) here. I've noticed that the QWKnet call-out semaphores are working, the recycle
    semaphore is not. Is there a change in how things get reloaded?

    -rw-r--r-- 1 sbbs sbbs 0 Mar 20 16:08 ctrl/recycle
    sbbs@home:~$ date
    Thu Mar 21 12:13:18 PDT 2019

    That's awhile for the semaphore to sit around, it seems?

    Thanks in advance.

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Va7aqd on Thu Mar 21 13:38:06 2019
    Re: Recycle semaphore?
    By: Va7aqd to All on Thu Mar 21 2019 12:15 pm

    Howdy!

    I'm running current CVS on Linux (Debian 64-bit) here. I've noticed that the QWKnet call-out semaphores are working, the recycle
    semaphore is not. Is there a change in how things get reloaded?

    Not that I can think of.

    -rw-r--r-- 1 sbbs sbbs 0 Mar 20 16:08 ctrl/recycle
    sbbs@home:~$ date
    Thu Mar 21 12:13:18 PDT 2019

    That's awhile for the semaphore to sit around, it seems?

    That's normal. The recycle semaphore file is a "timestamp semaphore file" where
    only the timestamp is checked of the existing file and the file is not deleted:
    http://wiki.synchro.net/config:semfiles#timestamp_semaphore_files

    If the file is touched/updated and the BBS doesn't recycle, it could be a number of things, but the log output should reflect the detection of the semaphore file unless the "NO_RECYCLE" option flag is set in the [BBS] section of your ctrl/sbbs.ini file.

    digital man

    This Is Spinal Tap quote #14:
    The Boston gig has been cancelled. [Don't] worry, it's not a big college town. Norco, CA WX: 53.2øF, 81.0% humidity, 0 mph ENE wind, 0.69 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Thu Mar 21 15:49:09 2019
    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Thu Mar 21 2019 01:38 pm

    If the file is touched/updated and the BBS doesn't recycle, it could be a number of things, but the log output should reflect the detection of the

    Ah, got it, thanks. I will keep my eye on it but didn't catch it reloading when I thought I was watching yesterday or the day before. I
    just tested again now and it did exactly what it was supposed to.

    Thanks!

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Mon Mar 25 13:01:17 2019
    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Thu Mar 21 2019 01:38 pm

    If the file is touched/updated and the BBS doesn't recycle, it could be a number of things, but the log output should reflect the detection of the semaphore file unless the "NO_RECYCLE" option flag is set in the [BBS] section of your ctrl/sbbs.ini file.

    After playing around with things for a few more days, I seem to be having an issue with the terminal service
    recycling. The logging shows detection of the recycle semaphore and the Web and FTP services recycle, but the
    terminal services do not, and I am guessing a large portion of SBBS itself doesn't, as things like newly added
    message bases or external programs, etc., are not showing up until I completely restart SBBS.

    There aren't any errors being thrown in to the logs, things just don't appear to be happening.

    If this is not the expected behaviour, what kind of information could I supply on this?

    Thanks!

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Va7aqd on Mon Mar 25 18:47:04 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Mon Mar 25 2019 01:01 pm

    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Thu Mar 21 2019 01:38 pm

    If the file is touched/updated and the BBS doesn't recycle, it could be number of things, but the log output should reflect the detection of the semaphore file unless the "NO_RECYCLE" option flag is set in the [BBS] section of your ctrl/sbbs.ini file.

    After playing around with things for a few more days, I seem to be having
    an
    issue with the terminal service
    recycling. The logging shows detection of the recycle semaphore and the
    Web
    and FTP services recycle, but the
    terminal services do not, and I am guessing a large portion of SBBS itself doesn't, as things like newly added
    message bases or external programs, etc., are not showing up until I completely restart SBBS.

    There aren't any errors being thrown in to the logs, things just don't appear to be happening.

    If this is not the expected behaviour, what kind of information could I supply on this?

    The terminal server cannot recycle until there are:
    1. no nodes in use
    2. no foreground events (e.g. timed events) running in the event thread

    Your log output will tell you what's happening.

    digital man

    This Is Spinal Tap quote #2:
    Nigel Tufnel: Well, this piece is called "Lick My Love Pump".
    Norco, CA WX: 66.3øF, 61.0% humidity, 11 mph E wind, 0.00 inches rain/24hrs
    --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Tue Mar 26 10:28:58 2019
    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Mon Mar 25 2019 06:47 pm

    The terminal server cannot recycle until there are:
    1. no nodes in use
    2. no foreground events (e.g. timed events) running in the event thread

    Your log output will tell you what's happening.

    Unfortunately, the logs aren't being terribly helpful, from what I can see. Currently the
    terminal server seems to be waiting to recycle - all nodes have an [R] status - and will have
    been that way overnight. As far as the log goes, it's not reporting on any event doing any
    work aside from the events that are coming up and finishing as things come along here.

    I could post the logs somewhere if that would be useful - or if there's something in
    particular I should be seeing in the logs that would explain this, I'd be greatful to know
    what, exactly. It sounds like if there were a stuck event I should see some fairly constant
    logging about it?

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From mark lewis@1:3634/12.73 to Va7aqd on Tue Mar 26 14:14:26 2019

    On 2019 Mar 26 10:28:58, you wrote to Digital Man:

    It sounds like if there were a stuck event I should see some fairly constant logging about it?

    unless i'm mistaken, if there was a stuck event, no other events would run... just going by the logging and not looking at the code, it looks like they are in a FIFO queue so they are executed in order of entry...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... I may be dumb.....but I'm NOT stupid!! ;*)
    ---
    * Origin: (1:3634/12.73)
  • From Digital Man@1:103/705 to Va7aqd on Tue Mar 26 16:05:01 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Tue Mar 26 2019 10:28 am

    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Mon Mar 25 2019 06:47 pm

    The terminal server cannot recycle until there are:
    1. no nodes in use
    2. no foreground events (e.g. timed events) running in the event thread

    Your log output will tell you what's happening.

    Unfortunately, the logs aren't being terribly helpful, from what I can see.

    Perhaps it's useful to compare your log output with what a successful recycle looks like (as far terminal server log messages). This is a successful start-up
    and recycle:

    3/26 03:59:02p Synchronet Terminal Server Version 3.17 Revision C Debug
    3/26 03:59:02p Compiled Mar 24 2019 03:30:33 with MSC 1800
    3/26 03:59:02p Initializing on Tue Mar 26 15:59:02 2019 with options: 800070aa
    3/26 03:59:02p Loading configuration files from s:\sbbs\ctrl\
    3/26 03:59:03p Verifying/creating data directories
    3/26 03:59:03p Verifying/creating node directories
    3/26 03:59:03p Telnet Server listening on socket 71.95.196.34 port 23
    3/26 03:59:03p Telnet Server listening on socket ::1 port 23
    3/26 03:59:03p Telnet Server listening on socket 127.0.0.1 port 23
    3/26 03:59:03p Telnet Server listening on socket 71.95.196.34 port 64
    3/26 03:59:03p Telnet Server listening on socket 71.95.196.34 port 128
    3/26 03:59:03p RLogin Server listening on socket 71.95.196.34 port 513
    3/26 03:59:03p RLogin Server listening on socket :: port 513
    3/26 03:59:04p SSH Server listening on socket 71.95.196.34 port 22
    3/26 03:59:04p SSH Server listening on socket ::1 port 22
    3/26 03:59:04p SSH Server listening on socket 127.0.0.1 port 22
    3/26 03:59:04p Terminal Server thread started for nodes 1 through 4
    3/26 03:59:12p Recycle semaphore signaled
    3/26 03:59:12p Waiting for events thread to terminate...
    3/26 03:59:13p Done waiting for events thread to terminate
    3/26 03:59:13p Terminal Server thread terminating
    3/26 03:59:13p 1408 Telnet Server closing socket 71.95.196.34 port 23
    3/26 03:59:13p 1500 Telnet Server closing socket ::1 port 23
    3/26 03:59:13p 1508 Telnet Server closing socket 127.0.0.1 port 23
    3/26 03:59:13p 1512 Telnet Server closing socket 71.95.196.34 port 64
    3/26 03:59:13p 1524 Telnet Server closing socket 71.95.196.34 port 128
    3/26 03:59:13p 1248 RLogin Server closing socket 71.95.196.34 port 513
    3/26 03:59:13p 1532 RLogin Server closing socket :: port 513
    3/26 03:59:13p 1328 SSH Server closing socket 71.95.196.34 port 22
    3/26 03:59:13p 1280 SSH Server closing socket ::1 port 22
    3/26 03:59:13p 1480 SSH Server closing socket 127.0.0.1 port 22
    3/26 03:59:13p Recycling server...
    3/26 03:59:15p Reading s:\sbbs\ctrl\sbbs.vert.ini
    3/26 03:59:15p Synchronet Terminal Server Version 3.17 Revision C Debug
    3/26 03:59:15p Compiled Mar 24 2019 03:30:33 with MSC 1800
    3/26 03:59:15p Initializing on Tue Mar 26 15:59:15 2019 with options: 800070aa
    3/26 03:59:15p Loading configuration files from s:\sbbs\ctrl\
    3/26 03:59:16p Verifying/creating data directories
    3/26 03:59:16p Verifying/creating node directories
    3/26 03:59:16p Telnet Server listening on socket 71.95.196.34 port 23
    3/26 03:59:16p Telnet Server listening on socket ::1 port 23
    3/26 03:59:16p Telnet Server listening on socket 127.0.0.1 port 23
    3/26 03:59:16p Telnet Server listening on socket 71.95.196.34 port 64
    3/26 03:59:16p Telnet Server listening on socket 71.95.196.34 port 128
    3/26 03:59:16p RLogin Server listening on socket 71.95.196.34 port 513
    3/26 03:59:16p RLogin Server listening on socket :: port 513
    3/26 03:59:16p SSH Server listening on socket 71.95.196.34 port 22
    3/26 03:59:16p SSH Server listening on socket ::1 port 22
    3/26 03:59:16p SSH Server listening on socket 127.0.0.1 port 22
    3/26 03:59:17p Terminal Server thread started for nodes 1 through 4

    Currently the terminal server seems to be waiting to recycle - all nodes have an [R] status - and will have been that way overnight.

    How many nodes is "all nodes"? What do you have set for the FirstNode and LastNode values in the [bbs] section of you ctrl/sbbs.ini file?

    As far as the
    log goes, it's not reporting on any event doing any work aside from the events that are coming up and finishing as things come along here.

    So long as you can see when the last event completed and it's still executing new events, then the event thread isn't the problem.

    I could post the logs somewhere if that would be useful - or if there's something in particular I should be seeing in the logs that would explain this, I'd be greatful to know what, exactly. It sounds like if there were stuck event I should see some fairly constant logging about it?

    No, but the last event log message would say something to the effect of "Running timed event: suchandsuch" and then no more log messages.

    What OS are you running? If *nix, then whether or not you run as root and change user-IDs (configured in the [unix] section of sbbs.ini) and if Linux, what version, all these things play a role in whether or not recycling is supported.

    How are you initiating the recycle?

    digital man

    Synchronet "Real Fact" #14:
    SBBSecho was originally written by Allen Christiansen (King Drafus) in 1994. Norco, CA WX: 73.4øF, 36.0% humidity, 4 mph ENE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Wed Mar 27 13:06:24 2019
    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Tue Mar 26 2019 04:05 pm

    Perhaps it's useful to compare your log output with what a successful recycle looks like (as far terminal server log messages). This is a

    Thank you for providing that log output - mine is definitely very different, with just web & FTP services
    noticing the recycle and recycling.

    How many nodes is "all nodes"? What do you have set for the FirstNode and LastNode values in the [bbs] section of you ctrl/sbbs.ini file?

    7 nodes total, FirstNode = 1, LastNode = 7

    So long as you can see when the last event completed and it's still executing new events, then the event thread isn't the problem.

    That's good to know, I believe there's no issue with the event thread then.

    What OS are you running? If *nix, then whether or not you run as root and change user-IDs (configured in the [unix] section of sbbs.ini) and if
    Linux,
    what version, all these things play a role in whether or not recycling is supported.

    Debian stretch. SBBS is started as a service via systemd and is set to use the sbbs user & group in sbbs.ini
    as well as in the sbbs.service systemd services unit file. The information from Synchro's wiki was used and
    systemd runs the setcap service binding fix on /sbbs/src/sbbs3/gcc.linux.x64.exe.release/sbbs on each run.

    How are you initiating the recycle?

    scfg seems to trigger it itself, and the same behaviour is seen when I simply 'touch ctrl/recycle'.

    So, yes, just doing it again right now I get the "Recycle semaphore file (/sbbs/ctrl/recycle) detected" log
    lines from the ftp and web services only.

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Va7aqd on Thu Mar 28 00:58:02 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Wed Mar 27 2019 01:06 pm

    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Tue Mar 26 2019 04:05 pm

    Perhaps it's useful to compare your log output with what a successful recycle looks like (as far terminal server log messages). This is a

    Thank you for providing that log output - mine is definitely very
    different,
    with just web & FTP services noticing the recycle and recycling.

    The example I supplied was for sbbsctrl/Windows and I was just clicking the "recycle" button, so I would expect it to appear somewhat different.

    How many nodes is "all nodes"? What do you have set for the FirstNode
    and
    LastNode values in the [bbs] section of you ctrl/sbbs.ini file?

    7 nodes total, FirstNode = 1, LastNode = 7

    Are any of the nodes in use at that time? Perhaps run "/sbbs/exec/node list" to
    be sure that all the nodes are in a recycleable state.

    So long as you can see when the last event completed and it's still executing new events, then the event thread isn't the problem.

    That's good to know, I believe there's no issue with the event thread then.

    What OS are you running? If *nix, then whether or not you run as root
    and
    change user-IDs (configured in the [unix] section of sbbs.ini) and if Linux,
    what version, all these things play a role in whether or not recycling
    is
    supported.

    Debian stretch. SBBS is started as a service via systemd and is set to use the sbbs user & group in sbbs.ini as well as in the sbbs.service systemd services unit file. The information from Synchro's wiki was used and systemd runs the setcap service binding fix on /sbbs/src/sbbs3/gcc.linux.x64.exe.release/sbbs on each run.

    How are you initiating the recycle?

    scfg seems to trigger it itself, and the same behaviour is seen when I simply 'touch ctrl/recycle'.

    Well they shouldn't be exactly the same (in regards to log output, node status).

    So, yes, just doing it again right now I get the "Recycle semaphore file (/sbbs/ctrl/recycle) detected" log lines from the ftp and web services
    only.

    That's unexpected. Is it possible you have the NO_RECYCLE option flag set for one or more sections of your ctrl/sbbs.ini file?

    digital man

    This Is Spinal Tap quote #35:
    Jeanine Pettibone: You don't do heavy metal in Dubly, you know.
    Norco, CA WX: 54.9øF, 91.0% humidity, 3 mph ESE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Thu Mar 28 12:23:03 2019
    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Thu Mar 28 2019 12:58 am

    The example I supplied was for sbbsctrl/Windows and I was just clicking the "recycle" button, so I would expect it to appear somewhat different.

    Sure, that makes sense. The parts that I mean are vastly different is the terminal services restarting, which
    doesn't happen here on my node.

    Are any of the nodes in use at that time? Perhaps run "/sbbs/exec/node
    list"
    to be sure that all the nodes are in a recycleable state.

    "Waiting for connection" on all is currently by far the most common state. At this point I'm the only user on
    my system.

    scfg seems to trigger it itself, and the same behaviour is seen when I simply 'touch ctrl/recycle'.
    Well they shouldn't be exactly the same (in regards to log output, node status).

    I will post the log output for each of these.

    So, yes, just doing it again right now I get the "Recycle semaphore file (/sbbs/ctrl/recycle) detected" log lines from the ftp and web services only.

    That's unexpected. Is it possible you have the NO_RECYCLE option flag set for one or more sections of your ctrl/sbbs.ini file?

    Definitely not:

    sbbs@home:~$ grep -i RECYC ctrl/sbbs.ini
    ; NO_RECYCLE
    ; NO_RECYCLE - Don't allow server recycles
    ; NO_RECYCLE
    ; NO_RECYCLE

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Thu Mar 28 12:29:54 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Thu Mar 28 2019 12:23 pm

    When making a change via SCFG and nobody on the system (I've included a couple of lines prior to the semaphore
    detection so you can see that the first semaphore detected line is the first logged line of the recycle
    semaphore being updated):

    Mar 28 12:23:34 home synchronet: srvc 0071 ActiveUser-UDP connection accepted from: 104.237.137.254 port 38507
    Mar 28 12:23:34 home synchronet: srvc 0071 ActiveUser-UDP service thread terminated (0 clients remain, 0
    total, 1730 served)
    Mar 28 12:23:42 home synchronet: ftp 0000 Recycle semaphore file (/sbbs/ctrl/recycle) detected
    Mar 28 12:23:42 home synchronet: ftp 0010 FTP Server closing socket 0.0.0.0 port 2121
    Mar 28 12:23:42 home synchronet: ftp 0015 FTP Server closing socket :: port 2121
    Mar 28 12:23:42 home synchronet: ftp Recycling server...
    Mar 28 12:23:43 home synchronet: web Recycle semaphore file (/sbbs/ctrl/recycle) detected
    Mar 28 12:23:43 home synchronet: web HTTP logging thread terminated
    Mar 28 12:23:43 home synchronet: web 0046 Web Server closing socket 0.0.0.0 port 8008
    Mar 28 12:23:43 home synchronet: web 0047 Web Server closing socket :: port 8008
    Mar 28 12:23:43 home synchronet: web Recycling server...
    Mar 28 12:23:44 home synchronet: Reading /sbbs/ctrl/sbbs.ini
    Mar 28 12:23:44 home synchronet: ftp Synchronet FTP Server Revision 1.488
    Mar 28 12:23:44 home synchronet: ftp Compiled Mar 14 2019 16:44:53 with GCC 6.3.0
    Mar 28 12:23:44 home synchronet: ftp Initializing on Thu Mar 28 12:23:44 2019 with options: 114
    Mar 28 12:23:44 home synchronet: ftp Loading configuration files from /sbbs/ctrl
    Mar 28 12:23:44 home synchronet: ftp FTP Server listening on socket 0.0.0.0 port 2121
    Mar 28 12:23:44 home synchronet: ftp FTP Server listening on socket :: port 2121
    Mar 28 12:23:44 home synchronet: ftp FTP Server thread started
    Mar 28 12:23:45 home synchronet: Reading /sbbs/ctrl/sbbs.ini
    Mar 28 12:23:45 home synchronet: web Synchronet Web Server Revision 1.678
    Mar 28 12:23:45 home synchronet: web Compiled Mar 14 2019 16:45:00 with GCC 6.3.0
    Mar 28 12:23:45 home synchronet: web Initializing on Thu Mar 28 12:23:45 2019 with options: 8e0
    Mar 28 12:23:45 home synchronet: web Loading configuration files from /sbbs/ctrl
    Mar 28 12:23:45 home synchronet: web Web Server listening on socket 0.0.0.0 port 8008
    Mar 28 12:23:45 home synchronet: web Web Server listening on socket :: port 8008
    Mar 28 12:23:45 home synchronet: web Secure Web Server listening on socket 0.0.0.0 port 8443
    Mar 28 12:23:45 home synchronet: web Secure Web Server listening on socket :: port 8443
    Mar 28 12:23:45 home synchronet: web Web Server thread started
    Mar 28 12:23:45 home synchronet: web HTTP logging thread started

    When I do a:
    sbbs@home:~$ touch ctrl/recycle

    The log output is:

    Mar 28 12:25:36 home synchronet: srvc 0074 ActiveUser-UDP connection accepted from: 104.237.137.254 port 38507
    Mar 28 12:25:36 home synchronet: srvc 0074 ActiveUser-UDP service thread terminated (0 clients remain, 0
    total, 1740 served)
    Mar 28 12:26:02 home synchronet: ftp 0000 Recycle semaphore file (/sbbs/ctrl/recycle) detected
    Mar 28 12:26:02 home synchronet: ftp 0010 FTP Server closing socket 0.0.0.0 port 2121
    Mar 28 12:26:02 home synchronet: ftp 0015 FTP Server closing socket :: port 2121
    Mar 28 12:26:02 home synchronet: ftp Recycling server...
    Mar 28 12:26:03 home synchronet: web Recycle semaphore file (/sbbs/ctrl/recycle) detected
    Mar 28 12:26:03 home synchronet: web HTTP logging thread terminated
    Mar 28 12:26:03 home synchronet: web 0046 Web Server closing socket 0.0.0.0 port 8008
    Mar 28 12:26:03 home synchronet: web 0047 Web Server closing socket :: port 8008
    Mar 28 12:26:03 home synchronet: web 0071 Secure Web Server closing socket 0.0.0.0 port 8443
    Mar 28 12:26:03 home synchronet: web 0073 Secure Web Server closing socket :: port 8443
    Mar 28 12:26:03 home synchronet: web Recycling server...
    Mar 28 12:26:04 home synchronet: srvc 0010 ActiveUser-UDP connection accepted from: 95.248.172.13 port 53634
    Mar 28 12:26:04 home synchronet: srvc 0010 ActiveUser-UDP service thread terminated (0 clients remain, 0
    total, 1741 served)
    Mar 28 12:26:04 home synchronet: Reading /sbbs/ctrl/sbbs.ini
    Mar 28 12:26:04 home synchronet: ftp Synchronet FTP Server Revision 1.488
    Mar 28 12:26:04 home synchronet: ftp Compiled Mar 14 2019 16:44:53 with GCC 6.3.0
    Mar 28 12:26:04 home synchronet: ftp Initializing on Thu Mar 28 12:26:04 2019 with options: 114
    Mar 28 12:26:04 home synchronet: ftp Loading configuration files from /sbbs/ctrl
    Mar 28 12:26:04 home synchronet: ftp FTP Server listening on socket 0.0.0.0 port 2121
    Mar 28 12:26:04 home synchronet: ftp FTP Server listening on socket :: port 2121
    Mar 28 12:26:04 home synchronet: ftp FTP Server thread started
    Mar 28 12:26:05 home synchronet: Reading /sbbs/ctrl/sbbs.ini
    Mar 28 12:26:05 home synchronet: web Synchronet Web Server Revision 1.678
    Mar 28 12:26:05 home synchronet: web Compiled Mar 14 2019 16:45:00 with GCC 6.3.0
    Mar 28 12:26:05 home synchronet: web Initializing on Thu Mar 28 12:26:05 2019 with options: 8e0
    Mar 28 12:26:05 home synchronet: web Loading configuration files from /sbbs/ctrl
    Mar 28 12:26:05 home synchronet: web Web Server listening on socket 0.0.0.0 port 8008
    Mar 28 12:26:05 home synchronet: web Web Server listening on socket :: port 8008
    Mar 28 12:26:05 home synchronet: web Secure Web Server listening on socket 0.0.0.0 port 8443
    Mar 28 12:26:05 home synchronet: web Secure Web Server listening on socket :: port 8443
    Mar 28 12:26:05 home synchronet: web Web Server thread started
    Mar 28 12:26:05 home synchronet: web HTTP logging thread started

    There's nobody on the system:

    sbbs@home:~$ exec/node list

    Synchronet Node Display/Control Utility v1.31

    Node 1: Waiting for connection [R]
    Node 2: Waiting for connection [R]
    Node 3: Waiting for connection [R]
    Node 4: Waiting for connection [R]
    Node 5: Waiting for connection [R]
    Node 6: Waiting for connection [R]
    Node 7: Waiting for connection [R]

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Va7aqd on Sat Mar 30 21:15:40 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Thu Mar 28 2019 12:23 pm

    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Thu Mar 28 2019 12:58 am

    The example I supplied was for sbbsctrl/Windows and I was just clicking the
    "recycle" button, so I would expect it to appear somewhat different.

    Sure, that makes sense. The parts that I mean are vastly different is the terminal services restarting, which doesn't happen here on my node.

    Are any of the nodes in use at that time? Perhaps run "/sbbs/exec/node list"
    to be sure that all the nodes are in a recycleable state.

    "Waiting for connection" on all is currently by far the most common state. At this point I'm the only user on my system.

    scfg seems to trigger it itself, and the same behaviour is seen when simply 'touch ctrl/recycle'.
    Well they shouldn't be exactly the same (in regards to log output, node status).

    I will post the log output for each of these.

    So, yes, just doing it again right now I get the "Recycle semaphore file
    (/sbbs/ctrl/recycle) detected" log lines from the ftp and web
    services
    only.

    That's unexpected. Is it possible you have the NO_RECYCLE option flag
    set
    for one or more sections of your ctrl/sbbs.ini file?

    Definitely not:

    sbbs@home:~$ grep -i RECYC ctrl/sbbs.ini
    ; NO_RECYCLE
    ; NO_RECYCLE - Don't allow server recycles
    ; NO_RECYCLE
    ; NO_RECYCLE

    It's option value to be set automatically by sbbs. What is "Initializing on ...
    with options" value from the log of your various servers (during startup)?

    If bit 27 is set, that means "No recycling is supported".

    digital man

    Synchronet "Real Fact" #98:
    The Synchronet Wiki (wiki.synchro.net) went online in April of 2010.
    Norco, CA WX: 65.2øF, 41.0% humidity, 5 mph ESE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Dmxrob@1:103/705 to Va7aqd on Sun Mar 31 07:45:34 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Mon
    Mar 25 2019 01:01 pm

    After playing around with things for a few more days, I seem to be having
    an
    issue with the terminal service
    recycling. The logging shows detection of the recycle semaphore and the
    Web

    I'm having a similar issue, in my case nothing is recognizing the recycle file has been touched.

    Are you running on CentOS by chance?

    ---
    þ Synchronet þ Gateway to the West | bbs.homelabber.net
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Sun Mar 31 11:56:30 2019
    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Sat Mar 30 2019 09:15 pm

    It's option value to be set automatically by sbbs. What is "Initializing on ... with options" value from the log of your various servers (during startup)?

    If bit 27 is set, that means "No recycling is supported".

    OK, that's definitely it - Is there a way I can discover why that might be getting set? I
    upped the log level to debug and didn't see anything interesting explaining this particular
    item.

    Mar 29 10:49:03 home synchronet: mail Initializing on Fri Mar 29 10:49:03 2019 with options:
    68804624
    Mar 29 10:49:03 home synchronet: srvc Initializing on Fri Mar 29 10:49:03 2019 with options:
    8000800
    Mar 29 10:49:03 home synchronet: term Initializing on Fri Mar 29 10:49:03 2019 with options:
    8001022

    Those seem to be the 3 services that have bit 27 set.

    Thanks!

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Dmxrob on Sun Mar 31 11:59:29 2019
    Re: Recycle semaphore?
    By: Dmxrob to Va7aqd on Sun Mar 31 2019 07:45 am

    Are you running on CentOS by chance?

    No sir.. Debian Stretch - it looks like 3 of sbbs' services are detecting that they
    can't/won't recycle during their start time.

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Va7aqd on Mon Apr 1 01:15:08 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Sun Mar 31 2019 11:56 am

    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Sat Mar 30 2019 09:15 pm

    It's option value to be set automatically by sbbs. What is "Initializing on
    ... with options" value from the log of your various servers (during startup)?

    If bit 27 is set, that means "No recycling is supported".

    OK, that's definitely it - Is there a way I can discover why that might be getting set? I upped the log level to debug and didn't see anything interesting explaining this particular item.

    Mar 29 10:49:03 home synchronet: mail Initializing on Fri Mar 29 10:49:03 2019 with options: 68804624
    Mar 29 10:49:03 home synchronet: srvc Initializing on Fri Mar 29 10:49:03 2019 with options: 8000800
    Mar 29 10:49:03 home synchronet: term Initializing on Fri Mar 29 10:49:03 2019 with options: 8001022

    Those seem to be the 3 services that have bit 27 set.

    In order for sbbs to set that option flag (sbbscon.c, around line 2050), you'd have to not have linux "capabilities" installed. Do you have the file /usr/include/sys/capability.h ?

    digital man

    This Is Spinal Tap quote #23:
    David St. Hubbins: I envy us.
    Norco, CA WX: 63.7øF, 36.0% humidity, 0 mph SSW wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Mon Apr 1 11:23:48 2019
    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Mon Apr 01 2019 01:15 am

    In order for sbbs to set that option flag (sbbscon.c, around line 2050), you'd have to not have linux "capabilities" installed. Do you have the
    file
    /usr/include/sys/capability.h ?

    OK, getting closer - I believe when I installed I realised there wasn't currently a Debian
    package "libcap2-dev" so continued on to see how things would go. I see it's now called
    "libcap-dev", so that's installed and I now have /usr/include/sys/capability.h

    I did a CVS update and have rebuilt pretty much everything, but recycling is still being
    disabled and I am still not quite sure why - this is what's showing in the logs now, though:

    Apr 1 11:14:57 home synchronet: The process 12989 was given capabilities = cap_net_bind_service+ep
    Apr 1 11:14:57 home synchronet: linux_initialprivs() FAILED
    Apr 1 11:14:57 home synchronet: Verify the following kernel module is loaded [See insmod(8)]:
    capability
    Apr 1 11:14:57 home synchronet: Operation not permitted
    Apr 1 11:14:57 home synchronet: Current uids: ruid - 2006, euid - 2006, suid - 2006
    Apr 1 11:14:57 home synchronet: Current gids: rgid - 2006, egid - 2006, sgid - 2006
    Apr 1 11:14:57 home synchronet: The process 12989 was given capabilities = cap_net_bind_service+ep
    Apr 1 11:14:57 home synchronet: Disabling Terminal Server recycle support
    Apr 1 11:14:57 home synchronet: Disabling Mail Server recycle support
    Apr 1 11:14:57 home synchronet: Disabling Services recycle support

    There doesn't seem to be a 'capability' kernel module for either the 3.16 or 4.9 kernels on
    the system.

    So.. I suspect that either I need to do something further on the re-compiling, or add
    something on to the system?

    Thanks!

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Va7aqd on Mon Apr 1 16:14:30 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Mon Apr 01 2019 11:23 am

    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Mon Apr 01 2019 01:15 am

    In order for sbbs to set that option flag (sbbscon.c, around line 2050), you'd have to not have linux "capabilities" installed. Do you have the file /usr/include/sys/capability.h ?

    OK, getting closer - I believe when I installed I realised there wasn't currently a Debian package "libcap2-dev" so continued on to see how things would go. I see it's now called "libcap-dev", so that's installed and I
    now
    have /usr/include/sys/capability.h

    I did a CVS update and have rebuilt pretty much everything, but recycling
    is
    still being disabled and I am still not quite sure why - this is what's showing in the logs now, though:

    Apr 1 11:14:57 home synchronet: The process 12989 was given capabilities = cap_net_bind_service+ep
    Apr 1 11:14:57 home synchronet: linux_initialprivs() FAILED
    Apr 1 11:14:57 home synchronet: Verify the following kernel module is loaded [See insmod(8)]: capability

    There doesn't seem to be a 'capability' kernel module for either the 3.16
    or
    4.9 kernels on the system.

    So.. I suspect that either I need to do something further on the re-compiling, or add something on to the system?

    It sounds like you're missing some kernel module associated with capabilities. Sorry, I don't know more than that (I didn't write this portion of the code, Deuce did). I don't seem to have any modules with "cap" in the name installed/running on my Debian systems, yet the capabilites work fine.

    digital man

    Synchronet/BBS Terminology Definition #26:
    FF = Form Feed (ASCII 12, Ctrl-L)
    Norco, CA WX: 80.0øF, 24.0% humidity, 10 mph NE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Mon Apr 1 19:49:42 2019
    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Mon Apr 01 2019 04:14 pm

    It sounds like you're missing some kernel module associated with capabilities. Sorry, I don't know more than that (I didn't write this portion of the code, Deuce did). I don't seem to have any modules with
    "cap"
    in the name installed/running on my Debian systems, yet the capabilites
    work
    fine.

    OK, I think I have things sorted, but much of what was wrong seems to be with the suggestions
    in the documentation (and that's no complaint about the docs at all - there's tons of great
    info in the wiki). Specifically, it looks like SBBS should be started as
    root as it drops
    privileges and runs as the user defined in ctrl/sbbs.ini appropriately. However, in the wiki
    docs, and I would point the finger mostly at the systemd page, which gives an example with
    systemd starting it as a non-root user & group.

    Now I suspect it's possible to set some extra capabilities on the sbbs binary to give it what
    it needs, but it appears that "setcap 'cap_net_bind_service=ep'" isn't enough. When doing an
    strace on sbbs when starting (and complaining about it's inability to recycle),
    it looks like
    it's checking other capabilities (though I'm not familiar with granting capabilities this way,
    so I'm just guessing on what this means):

    write(1, "Setting initial privileges\n", 27) = 27 capset({version=_LINUX_CAPABILITY_VERSION_1, pid=0}, {effective=1<<CAP_DAC_READ _SEARCH|1<<CAP_SETGID|1<<CAP_SETUID|1<<CAP_NET_BIND_SERVICE|1<<CAP_S YS_RESOURCE, permitted=1<<CAP_DAC_READ_SEARCH|1<<CAP_SETGID|1<<CAP_SETUID|1<<CA P_NET_BIND_SERVICE|1<<CAP_SY
    S_RESOURCE, inheritable=0}) = -1 EPERM (Operation not permitted)
    write(1, "\r\rlinux_initialprivs() FAILED\n", 30) = 30

    The only one the linux non-root user wiki page references is cap_net_bind_service.

    So, I was just working on doing things as securely as possible with the least amount of
    privileges, but really SBBS seems to do things like any smart system daemon does - it starts
    as root and drops down to user privileges so it can do things like run on privileged ports
    (without having to monkey around with specific capabilities settings on a binary).

    Thanks for all the assistance, I hope some of this pain is helpful.

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Va7aqd on Mon Apr 1 23:57:49 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Mon Apr 01 2019 07:49 pm

    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Mon Apr 01 2019 04:14 pm

    It sounds like you're missing some kernel module associated with capabilities. Sorry, I don't know more than that (I didn't write this portion of the code, Deuce did). I don't seem to have any modules with "cap"
    in the name installed/running on my Debian systems, yet the capabilites work
    fine.

    OK, I think I have things sorted, but much of what was wrong seems to be with
    the suggestions
    in the documentation (and that's no complaint about the docs at all - there's tons of great info in the wiki). Specifically, it looks like SBBS should be started as
    root as it drops
    privileges and runs as the user defined in ctrl/sbbs.ini appropriately.

    That's how I run it.

    However, in the wiki
    docs, and I would point the finger mostly at the systemd page, which gives an
    example with
    systemd starting it as a non-root user & group.

    Someone contributed that wiki page. You can certainly update it to clarify your
    findings.

    When doing an
    strace on sbbs when starting (and complaining about it's inability to recycle),
    it looks like
    it's checking other capabilities (though I'm not familiar with granting capabilities this way,
    so I'm just guessing on what this means):

    The source for this is in src/sbbs3/sbbscon.c

    It looks to me like it tries to use the following privs before binding ports:

    caps |= (1 << CAP_NET_BIND_SERVICE);
    caps |= (1 << CAP_SETUID);
    caps |= (1 << CAP_SETGID);
    caps |= (1 << CAP_DAC_READ_SEARCH);
    caps |= (1 << CAP_SYS_RESOURCE);

    And then after binding:

    caps |= (1 << CAP_NET_BIND_SERVICE);
    caps |= (1 << CAP_SYS_RESOURCE);

    it needs, but it appears that "setcap 'cap_net_bind_service=ep'" isn't enough.

    It seems to work for people. I don't personally use that method, so I can't vouch for it.

    digital man

    Synchronet/BBS Terminology Definition #66:
    SyncEdit = A defunct 3rd party full-screen editor written for Synchronet
    Norco, CA WX: 62.4øF, 44.0% humidity, 0 mph SW wind, 0.00 inches rain/24hrs
    --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Va7aqd@1:103/705 to Digital Man on Tue Apr 2 17:34:57 2019
    Re: Recycle semaphore?
    By: Digital Man to Va7aqd on Mon Apr 01 2019 11:57 pm

    Someone contributed that wiki page. You can certainly update it to clarify your findings.

    It seems to work for people. I don't personally use that method, so I can't vouch for it.

    OK, after going through the steps here to tinker with what's running what under what user, it
    appears that the documentation just needs some clarification. There's only the one capability
    required when starting the service as non-root, but it's then imperative one doesn't set
    User/Group in sbbs.ini, otherwise sbbs will try to change to that user & group, even if it's
    the same user/group it has been run as.

    How does one get an account to edit the wiki? Does it authenticate against Vertrauen?

    Thanks!

    ---
    þ Synchronet þ VA7AQD's Tavern - bbs.isurf.ca
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From mark lewis@1:3634/12.73 to Va7aqd on Tue Apr 2 22:44:26 2019

    On 2019 Apr 02 17:34:56, you wrote to Digital Man:

    How does one get an account to edit the wiki? Does it authenticate
    against
    Vertrauen?

    yes so if you have a personal account there, sign in with that name and password...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... Never put off until tomorrow, what you can forget about forever.
    ---
    * Origin: (1:3634/12.73)
  • From Digital Man@1:103/705 to Va7aqd on Wed Apr 3 02:50:08 2019
    Re: Recycle semaphore?
    By: Va7aqd to Digital Man on Tue Apr 02 2019 05:34 pm

    How does one get an account to edit the wiki? Does it authenticate against Vertrauen?

    Right. Corrections / clarifications welcome!

    digital man

    This Is Spinal Tap quote #19:
    Oh then, maybe it's not green. Anyway this is what I sleep in sometimes.
    Norco, CA WX: 53.5øF, 84.0% humidity, 0 mph SSW wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)