XPost: alt.os.linux

I have the following piece of code creating an rsync process launched in
the early hours of every morning to sync a NAS from another upstream. I
have only included the business part of the file, the beginning of it is
to do with setting up the variable values used in these lines. As you
will note, there are two debugging statements trapping the output of ps ...

# Debug Line 1:
ps | grep -v grep | grep tutsync.sh > $HD/rsync.log
echo "Syncing the following shares with rsync: ${SHARES}" >> $HD/rsync.log
for SHARE in ${SHARES}
do
# Debug Line 2:
ps | grep -v grep | grep tutsync.sh >> $HD/rsync.log
echo "nice -n ${NICE} rsync ${OPTIONS}
${SOURCE}/${SHARE}/ ${BU}/${SHARE}/"
nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/ ${BU}/${SHARE}/
done | grep -v '/$' >> $HD/rsync.log 2>&1


... because, when run, the process appears to duplicate itself within
the for loop - not only does the tutsync.sh process itself get
duplicated, but also the rsync process it launches as well, as can be
proven by launching with a trailing & and running a similar command from
the shell:

~ # cat $HD/rsync.log
32453 root 2720 S /bin/sh /opt/share/bin/tutsync.sh
Syncing the following shares with rsync: A List Of Shares
32453 root 2720 S /bin/sh /opt/share/bin/tutsync.sh
32458 root 2720 S /bin/sh /opt/share/bin/tutsync.sh
receiving incremental file list
etc

~ # pf rsync
9680 root 1972 S /opt/bin/rsync --ipv4 --daemon --config=/etc/rsyncd.conf
32467 root 2908 R N rsync -Drltugpv --exclude=*.flv
--exclude=*.partial.* --exclude=*.part --exclude=**/.streams
32468 root 49772 S N rsync -Drltugpv --exclude=*.flv
--exclude=*.partial.* --exclude=*.part --exclude=**/.streams

What is going on, why are the processes duplicating themselves?

--

Fake news kills!

I may be contacted via the contact address given on my website:
www.macfh.co.uk

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: alt.os.linux

On Sat, 24 Jul 2021 14:40:08 +0100, Java Jive wrote:

I have the following piece of code creating an rsync process launched in
the early hours of every morning to sync a NAS from another upstream. I
have only included the business part of the file, the beginning of it is
to do with setting up the variable values used in these lines. As you
will note, there are two debugging statements trapping the output of ps
...

# Debug Line 1:
ps | grep -v grep | grep tutsync.sh > $HD/rsync.log echo "Syncing the following shares with rsync: ${SHARES}" >> $HD/rsync.log for SHARE in ${SHARES}
do
# Debug Line 2:
ps | grep -v grep | grep tutsync.sh >> $HD/rsync.log
echo "nice -n ${NICE} rsync ${OPTIONS}
${SOURCE}/${SHARE}/ ${BU}/${SHARE}/"
nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/ ${BU}/${SHARE}/
done | grep -v '/$' >> $HD/rsync.log 2>&1

... because, when run, the process appears to duplicate itself within
the for loop - not only does the tutsync.sh process itself get
duplicated, but also the rsync process it launches as well, as can be
proven by launching with a trailing & and running a similar command from
the shell:

~ # cat $HD/rsync.log 32453 root 2720 S /bin/sh /opt/share/bin/tutsync.sh Syncing the following shares with rsync: A
List Of Shares 32453 root 2720 S /bin/sh
/opt/share/bin/tutsync.sh 32458 root 2720 S /bin/sh /opt/share/bin/tutsync.sh receiving incremental file list etc

~ # pf rsync
9680 root 1972 S /opt/bin/rsync --ipv4 --daemon --config=/etc/rsyncd.conf 32467 root 2908 R N rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part
--exclude=**/.streams 32468 root 49772 S N rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part
--exclude=**/.streams

What is going on, why are the processes duplicating themselves?

FWIW my weekly backup process is run from the host with the backup disk
mounted on it and uses rsync to back up itself as well as other hosts on
my LAN. Backups are run from a menu with each backup being triggered
manually. I've watched this run using 'top' and seen duplicate copies of
rsync get spawned from time to time: the duplicated rsync processes are
not present in all backup runs and and can come and go during a backup.

I have no idea why the duplicate processes are created, but the backups
always complete without errors reported and seem to not duplicate,
corrupt or omit any files, so I just accept that this is what rsync does.

For an explanation, look at this (which I've just found): https://serverfault.com/questions/547165/why-does-rsync-spawn-multiple- processes-for-me#547194



--
Martin | martin at
Gregorie | gregorie dot org

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: alt.os.linux

Java Jive <java@evij.com.invalid> writes:

I have the following piece of code creating an rsync process launched
in the early hours of every morning to sync a NAS from another
upstream. I have only included the business part of the file, the
beginning of it is to do with setting up the variable values used in
these lines. As you will note, there are two debugging statements
trapping the output of ps ...

# Debug Line 1:
ps | grep -v grep | grep tutsync.sh > $HD/rsync.log
echo "Syncing the following shares with rsync: ${SHARES}" >> $HD/rsync.log for SHARE in ${SHARES}
do
# Debug Line 2:
ps | grep -v grep | grep tutsync.sh >> $HD/rsync.log
echo "nice -n ${NICE} rsync ${OPTIONS}
${SOURCE}/${SHARE}/ ${BU}/${SHARE}/"
nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/ ${BU}/${SHARE}/
done | grep -v '/$' >> $HD/rsync.log 2>&1

... because, when run, the process appears to duplicate itself within
the for loop - not only does the tutsync.sh process itself get
duplicated, but also the rsync process it launches as well, as can be
proven by launching with a trailing & and running a similar command
from the shell:

~ # cat $HD/rsync.log
32453 root 2720 S /bin/sh /opt/share/bin/tutsync.sh
Syncing the following shares with rsync: A List Of Shares
32453 root 2720 S /bin/sh /opt/share/bin/tutsync.sh
32458 root 2720 S /bin/sh /opt/share/bin/tutsync.sh
receiving incremental file list
etc

~ # pf rsync
9680 root 1972 S /opt/bin/rsync --ipv4 --daemon --config=/etc/rsyncd.conf
32467 root 2908 R N rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part --exclude=**/.streams
32468 root 49772 S N rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part --exclude=**/.streams

What is going on, why are the processes duplicating themselves?

The first half of the main pipeline (for ... done) runs in a separate
process.

rsync uses multiple processes as part of its normal operation.

--
https://www.greenend.org.uk/rjk/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, 24 Jul 2021 14:40:08 +0100, Java Jive wrote:

I have the following piece of code creating an rsync process launched in
the early hours of every morning to sync a NAS from another upstream. I
have only included the business part of the file, the beginning of it is
to do with setting up the variable values used in these lines. As you
will note, there are two debugging statements trapping the output of ps
...

# Debug Line 1:
ps | grep -v grep | grep tutsync.sh > $HD/rsync.log echo "Syncing the following shares with rsync: ${SHARES}" >> $HD/rsync.log for SHARE in ${SHARES}
do
# Debug Line 2:
ps | grep -v grep | grep tutsync.sh >> $HD/rsync.log
echo "nice -n ${NICE} rsync ${OPTIONS}
${SOURCE}/${SHARE}/ ${BU}/${SHARE}/"
nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/ ${BU}/${SHARE}/
done | grep -v '/$' >> $HD/rsync.log 2>&1

... because, when run, the process appears to duplicate itself within
the for loop - not only does the tutsync.sh process itself get
duplicated, but also the rsync process it launches as well, as can be
proven by launching with a trailing & and running a similar command from
the shell:

~ # cat $HD/rsync.log 32453 root 2720 S /bin/sh /opt/share/bin/tutsync.sh Syncing the following shares with rsync: A
List Of Shares 32453 root 2720 S /bin/sh
/opt/share/bin/tutsync.sh 32458 root 2720 S /bin/sh /opt/share/bin/tutsync.sh receiving incremental file list etc

~ # pf rsync
9680 root 1972 S /opt/bin/rsync --ipv4 --daemon --config=/etc/rsyncd.conf 32467 root 2908 R N rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part
--exclude=**/.streams 32468 root 49772 S N rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part
--exclude=**/.streams

What is going on, why are the processes duplicating themselves?

FWIW my weekly backup process is run from the host with the backup disk
mounted on it and uses rsync to back up itself as well as other hosts on
my LAN. Backups are run from a menu with each backup being triggered
manually. I've watched this run using 'top' and seen duplicate copies of
rsync get spawned from time to time: the duplicated rsync processes are
not present in all backup runs and and can come and go during a backup.

I have no idea why the duplicate processes are created, but the backups
always complete without errors reported and seem to not duplicate,
corrupt or omit any files, so I just accept that this is what rsync does.

For an explanation, look at this (which I've just found): https://serverfault.com/questions/547165/why-does-rsync-spawn-multiple- processes-for-me#547194



--
Martin | martin at
Gregorie | gregorie dot org

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: alt.os.linux

On 24/07/2021 14:40, Java Jive wrote:

I have the following piece of code creating an rsync process launched in
the early hours of every morning to sync a NAS from another upstream.  I have only included the business part of the file, the beginning of it is
to do with setting up the variable values used in these lines.  As you
will note, there are two debugging statements trapping the output of ps ...

# Debug Line 1:
ps | grep -v grep | grep tutsync.sh > $HD/rsync.log
echo "Syncing the following shares with rsync: ${SHARES}" >> $HD/rsync.log for SHARE in ${SHARES}
        do
        # Debug Line 2:
                ps | grep -v grep | grep tutsync.sh >> $HD/rsync.log
                echo "nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/ ${BU}/${SHARE}/"
                nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/
${BU}/${SHARE}/
        done | grep -v '/$' >> $HD/rsync.log 2>&1

... because, when run, the process appears to duplicate itself within
the for loop  -  not only does the tutsync.sh process itself get duplicated, but also the rsync process it launches as well, as can be
proven by launching with a trailing & and running a similar command from
the shell:

~ # cat $HD/rsync.log
32453 root      2720 S    /bin/sh /opt/share/bin/tutsync.sh
Syncing the following shares with rsync: A List Of Shares
32453 root      2720 S    /bin/sh /opt/share/bin/tutsync.sh
32458 root      2720 S    /bin/sh /opt/share/bin/tutsync.sh
receiving incremental file list
etc

~ # pf rsync
 9680 root      1972 S    /opt/bin/rsync --ipv4 --daemon --config=/etc/rsyncd.conf
32467 root      2908 R N  rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part --exclude=**/.streams
32468 root     49772 S N  rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part --exclude=**/.streams

What is going on, why are the processes duplicating themselves?

Thanks for both the replies. It seems there's nothing to worry about,
and the script seems to have been working aright anyhow.

--

Fake news kills!

I may be contacted via the contact address given on my website:
www.macfh.co.uk

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: alt.os.linux

On 24/07/2021 15.40, Java Jive wrote:

I have the following piece of code creating an rsync process launched in
the early hours of every morning to sync a NAS from another upstream.  I have only included the business part of the file, the beginning of it is
to do with setting up the variable values used in these lines.  As you
will note, there are two debugging statements trapping the output of ps ...

# Debug Line 1:
ps | grep -v grep | grep tutsync.sh > $HD/rsync.log
echo "Syncing the following shares with rsync: ${SHARES}" >> $HD/rsync.log for SHARE in ${SHARES}
        do
        # Debug Line 2:
                ps | grep -v grep | grep tutsync.sh >> $HD/rsync.log
                echo "nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/ ${BU}/${SHARE}/"
                nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/
${BU}/${SHARE}/
        done | grep -v '/$' >> $HD/rsync.log 2>&1

... because, when run, the process appears to duplicate itself within
the for loop  -  not only does the tutsync.sh process itself get duplicated, but also the rsync process it launches as well, as can be
proven by launching with a trailing & and running a similar command from
the shell:

~ # cat $HD/rsync.log
32453 root      2720 S    /bin/sh /opt/share/bin/tutsync.sh
Syncing the following shares with rsync: A List Of Shares
32453 root      2720 S    /bin/sh /opt/share/bin/tutsync.sh
32458 root      2720 S    /bin/sh /opt/share/bin/tutsync.sh
receiving incremental file list
etc

~ # pf rsync
 9680 root      1972 S    /opt/bin/rsync --ipv4 --daemon --config=/etc/rsyncd.conf
32467 root      2908 R N  rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part --exclude=**/.streams
32468 root     49772 S N  rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part --exclude=**/.streams

What is going on, why are the processes duplicating themselves?

Running ps with the f option will give you a clearer picture of process relations

5467 ? Ss 0:02 sshd: /usr/sbin/sshd -f /etc/ssh/sshd_config
13958 ? Ss 0:00 \_ sshd: user1 [priv]
13960 ? S 0:01 \_ sshd: user1@pts/4
13962 pts/4 Ss 0:00 \_ -bash
18569 pts/4 S 0:00 \_ su -
18571 pts/4 S 0:00 \_ -su
20700 pts/4 R+ 0:00 \_ ps axf

--

//Aho

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: alt.os.linux

On 25/07/2021 10:08, J.O. Aho wrote:

Running ps with the f option will give you a clearer picture of process relations

 5467 ?        Ss     0:02 sshd: /usr/sbin/sshd -f /etc/ssh/sshd_config
13958 ?        Ss     0:00  \_ sshd: user1 [priv]
13960 ?        S      0:01      \_ sshd: user1@pts/4
13962 pts/4    Ss     0:00          \_ -bash
18569 pts/4    S      0:00              \_ su -
18571 pts/4    S      0:00                  \_ -su 20700 pts/4    R+     0:00                      \_ ps axf

That's a useful general tip which I didn't know about before, thanks.
However, in this particular instance, this is an embedded device, and
the busybox version of ps doesn't support that option, more's the pity.

--

Fake news kills!

I may be contacted via the contact address given on my website:
www.macfh.co.uk

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 24/07/2021 15.40, Java Jive wrote:

I have the following piece of code creating an rsync process launched in
the early hours of every morning to sync a NAS from another upstream.  I have only included the business part of the file, the beginning of it is
to do with setting up the variable values used in these lines.  As you
will note, there are two debugging statements trapping the output of ps ...

# Debug Line 1:
ps | grep -v grep | grep tutsync.sh > $HD/rsync.log
echo "Syncing the following shares with rsync: ${SHARES}" >> $HD/rsync.log for SHARE in ${SHARES}
        do
        # Debug Line 2:
                ps | grep -v grep | grep tutsync.sh >> $HD/rsync.log
                echo "nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/ ${BU}/${SHARE}/"
                nice -n ${NICE} rsync ${OPTIONS} ${SOURCE}/${SHARE}/
${BU}/${SHARE}/
        done | grep -v '/$' >> $HD/rsync.log 2>&1

... because, when run, the process appears to duplicate itself within
the for loop  -  not only does the tutsync.sh process itself get duplicated, but also the rsync process it launches as well, as can be
proven by launching with a trailing & and running a similar command from
the shell:

~ # cat $HD/rsync.log
32453 root      2720 S    /bin/sh /opt/share/bin/tutsync.sh
Syncing the following shares with rsync: A List Of Shares
32453 root      2720 S    /bin/sh /opt/share/bin/tutsync.sh
32458 root      2720 S    /bin/sh /opt/share/bin/tutsync.sh
receiving incremental file list
etc

~ # pf rsync
 9680 root      1972 S    /opt/bin/rsync --ipv4 --daemon --config=/etc/rsyncd.conf
32467 root      2908 R N  rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part --exclude=**/.streams
32468 root     49772 S N  rsync -Drltugpv --exclude=*.flv --exclude=*.partial.* --exclude=*.part --exclude=**/.streams

What is going on, why are the processes duplicating themselves?

Running ps with the f option will give you a clearer picture of process relations

5467 ? Ss 0:02 sshd: /usr/sbin/sshd -f /etc/ssh/sshd_config
13958 ? Ss 0:00 \_ sshd: user1 [priv]
13960 ? S 0:01 \_ sshd: user1@pts/4
13962 pts/4 Ss 0:00 \_ -bash
18569 pts/4 S 0:00 \_ su -
18571 pts/4 S 0:00 \_ -su
20700 pts/4 R+ 0:00 \_ ps axf

--

//Aho

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)