• Try to understand complex sed command logic instructions.

    From hongyi.zhao@gmail.com@21:1/5 to All on Sat Dec 11 03:30:22 2021
    I try to understand the following complex sed command logic instructions given here [1]:

    If your ~/.profile sources ~/.bashrc (Debian, Ubuntu, Mint):

    # the sed invocation inserts the lines at the start of the file
    # after any initial comment lines
    sed -Ei -e '/^([^#]|$)/ {a \
    export PYENV_ROOT="$HOME/.pyenv"
    a \
    export PATH="$PYENV_ROOT/bin:$PATH"
    a \
    ' -e ':a' -e '$!{n;ba};}' ~/.profile

    I tried the above code, and found that the following lines will be inserted into `~/.profile' after the shebang line:

    $ git diff
    diff --git a/.profile b/.profile
    index c99e6c3e..5b33339f 100644
    --- a/.profile
    +++ b/.profile
    @@ -1,5 +1,8 @@
    #!/usr/bin/env bash

    +export PYENV_ROOT="$HOME/.pyenv"
    +export PATH="$PYENV_ROOT/bin:$PATH"
    +
    # To do:
    #bootstrap based on this file.

    [1] https://github.com/pyenv/pyenv#basic-github-checkout

    But the logic contained in the sed code snippet above is still obscure to me. Any hints?

    OTOH, the final result achieved by the sed workflow does seem so trivial. Why do they still insist on providing such an ancient sed-based solution?

    Regards,
    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to hongy...@gmail.com on Sat Dec 11 13:23:03 2021
    On 11.12.2021 12:30, hongy...@gmail.com wrote:
    I try to understand the following complex sed command logic
    instructions given here [1]:
    [...]

    But the logic contained in the sed code snippet above is still
    obscure to me. Any hints?

    Get the sed manual and identify the used commands. Or formulate
    the simple algorithm yourself informally and try to identify the
    building blocks in the sed code.


    OTOH, the final result achieved by the sed workflow does seem so
    trivial. Why do they still insist on providing such an ancient
    sed-based solution?

    Maybe it's code from the 1980's?
    Maybe the programmer is used to that tool?
    Maybe the programmer doesn't know any better tool?
    Who knows?
    But I see no one insisting in anything.

    Though, for simple tasks you can always write your own version,
    and use a language that is or appears to you as less cryptic.

    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    (or replace the 1 by {print} to make it less cryptic). For many
    folks that's easier to read because syntax is similar to the C
    language and its language family.

    But the code is just a quick shot; you may want your program to
    also handle empty files, or files that have just comments and
    no data lines (that triggers the condition), and maybe use GNU
    Awk with the in-place editing feature that is also used in the
    sed call.

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Janis Papanagnou on Sat Dec 11 16:42:46 2021
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    On 11.12.2021 12:30, hongy...@gmail.com wrote:
    I try to understand the following complex sed command logic
    instructions given here [1]:
    [...]

    But the logic contained in the sed code snippet above is still
    obscure to me. Any hints?
    Get the sed manual and identify the used commands. Or formulate
    the simple algorithm yourself informally and try to identify the
    building blocks in the sed code.

    OTOH, the final result achieved by the sed workflow does seem so
    trivial. Why do they still insist on providing such an ancient
    sed-based solution?
    Maybe it's code from the 1980's?
    Maybe the programmer is used to that tool?
    Maybe the programmer doesn't know any better tool?
    Who knows?
    But I see no one insisting in anything.

    Though, for simple tasks you can always write your own version,
    and use a language that is or appears to you as less cryptic.

    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    I try to debug/understand your above code as follows:

    $ awk '!/^#/ && !f++ { print "..." } ; {print $0, f, f++}' .profile | head -3 #!/usr/bin/env bash 0
    2 2
    # To do: 3 3

    $ awk '!/^#/ && !f++ { print "..." } ; {print $0, f}' .profile | head -3 #!/usr/bin/env bash
    ...
    1

    As you can see, when I try to print the value of `f++`, the desired modification results will disappear. This should mean that the variable was altered as it was printed, which makes debugging in this form impossible.


    (or replace the 1 by {print} to make it less cryptic). For many
    folks that's easier to read because syntax is similar to the C
    language and its language family.

    But the code is just a quick shot; you may want your program to
    also handle empty files, or files that have just comments and
    no data lines (that triggers the condition), and maybe use GNU
    Awk with the in-place editing feature that is also used in the
    sed call.

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Janis Papanagnou on Sat Dec 11 18:43:44 2021
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    On 11.12.2021 12:30, hongy...@gmail.com wrote:
    I try to understand the following complex sed command logic
    instructions given here [1]:
    [...]

    But the logic contained in the sed code snippet above is still
    obscure to me. Any hints?
    Get the sed manual and identify the used commands. Or formulate
    the simple algorithm yourself informally and try to identify the
    building blocks in the sed code.

    I try to decrypt it as follows, please correct me if I'm wrong:

    sed -Ei

    use extended regular expressions in the script
    and
    edit files in place

    -e

    add the script to the commands to be executed

    '
    Begin the script


    /^([^#]|$)/

    Anchor lines that is empty or does not start with #

    {

    Begin a block of commands

    a \
    export PYENV_ROOT="$HOME/.pyenv"
    a \
    export PATH="$PYENV_ROOT/bin:$PATH"

    Append text, which has each embedded newline preceded by a backslash.

    a \
    ' -e ':a' -e '$!{n;ba};

    Failed to figure out the above obscure code block.

    }

    End a block of commands

    '
    End the script

    ~/.profile

    The input file to be manipulated.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Janis Papanagnou on Sat Dec 11 22:32:16 2021
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    The above code will add the stuff at the very beginning of the script if occasionally the shebang line is not the first line:

    $ cat test.sh

    #!/usr/bin/env bash
    echo 'hello world.'

    $ ./test.sh
    hello world.

    $ awk '!/^#/ && !f++ { print "..."} ; {print $0}' test.sh
    ...

    #!/usr/bin/env bash
    echo 'hello world.'

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Sat Dec 11 22:36:12 2021
    On Sunday, December 12, 2021 at 10:43:47 AM UTC+8, hongy...@gmail.com wrote:
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    On 11.12.2021 12:30, hongy...@gmail.com wrote:
    I try to understand the following complex sed command logic
    instructions given here [1]:
    [...]

    But the logic contained in the sed code snippet above is still
    obscure to me. Any hints?
    Get the sed manual and identify the used commands. Or formulate
    the simple algorithm yourself informally and try to identify the
    building blocks in the sed code.
    I try to decrypt it as follows, please correct me if I'm wrong:

    sed -Ei

    use extended regular expressions in the script
    and
    edit files in place

    -e

    add the script to the commands to be executed

    '
    Begin the script


    /^([^#]|$)/

    Anchor lines that is empty or does not start with #

    {

    Begin a block of commands
    a \
    export PYENV_ROOT="$HOME/.pyenv"
    a \
    export PATH="$PYENV_ROOT/bin:$PATH"
    Append text, which has each embedded newline preceded by a backslash.
    a \
    ' -e ':a' -e '$!{n;ba};
    Failed to figure out the above obscure code block.

    Especially, there are two '-e' constructs embedded in outer '-e' construct, which is very confusing for me.

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to hongy...@gmail.com on Sun Dec 12 12:57:21 2021
    On 12.12.2021 01:42, hongy...@gmail.com wrote:
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    On 11.12.2021 12:30, hongy...@gmail.com wrote:
    I try to understand the following complex sed command logic
    instructions given here [1]:
    [...]

    But the logic contained in the sed code snippet above is still
    obscure to me. Any hints?
    Get the sed manual and identify the used commands. Or formulate
    the simple algorithm yourself informally and try to identify the
    building blocks in the sed code.

    OTOH, the final result achieved by the sed workflow does seem so
    trivial. Why do they still insist on providing such an ancient
    sed-based solution?
    Maybe it's code from the 1980's?
    Maybe the programmer is used to that tool?
    Maybe the programmer doesn't know any better tool?
    Who knows?
    But I see no one insisting in anything.

    Though, for simple tasks you can always write your own version,
    and use a language that is or appears to you as less cryptic.

    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    I try to debug/understand your above code as follows:

    Understand how f++ works - in *any* of the many languages that support
    it - before you try to "debug".

    GNU Awk has a debugger that makes it unnecessary in practice to add own [possibly or likely wrong] code.

    Janis


    $ awk '!/^#/ && !f++ { print "..." } ; {print $0, f, f++}' .profile | head -3 #!/usr/bin/env bash 0
    2 2
    # To do: 3 3

    $ awk '!/^#/ && !f++ { print "..." } ; {print $0, f}' .profile | head -3 #!/usr/bin/env bash
    ...
    1

    As you can see, when I try to print the value of `f++`, the desired modification results will disappear. This should mean that the variable was altered as it was printed, which makes debugging in this form impossible.


    (or replace the 1 by {print} to make it less cryptic). For many
    folks that's easier to read because syntax is similar to the C
    language and its language family.

    But the code is just a quick shot; you may want your program to
    also handle empty files, or files that have just comments and
    no data lines (that triggers the condition), and maybe use GNU
    Awk with the in-place editing feature that is also used in the
    sed call.

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to hongy...@gmail.com on Sun Dec 12 12:51:56 2021
    On 12.12.2021 07:32, hongy...@gmail.com wrote:
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    The above code will add the stuff at the very beginning of the
    script if occasionally the shebang line is not the first line:

    1. The posted requirements had been
    # the sed invocation inserts the lines at the start of the file
    # after any initial comment lines

    2. What do you expect from a shebang line that is _ineffective_;
    because it is *not* on the first line?

    If you happen to have other requirements adjust the condition.
    (Don't expect that we can read your mind from that distance.)

    You want to consider [initial] blank lines similar to comments,
    for example?

    !(/^#/||!NF) && !f++ ...


    Janis


    $ cat test.sh

    #!/usr/bin/env bash
    echo 'hello world.'

    $ ./test.sh
    hello world.

    $ awk '!/^#/ && !f++ { print "..."} ; {print $0}' test.sh
    ...

    #!/usr/bin/env bash
    echo 'hello world.'

    HZ


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Janis Papanagnou on Sun Dec 12 04:15:21 2021
    On Sunday, December 12, 2021 at 7:52:02 PM UTC+8, Janis Papanagnou wrote:
    On 12.12.2021 07:32, hongy...@gmail.com wrote:
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    The above code will add the stuff at the very beginning of the
    script if occasionally the shebang line is not the first line:
    1. The posted requirements had been
    # the sed invocation inserts the lines at the start of the file
    # after any initial comment lines
    2. What do you expect from a shebang line that is _ineffective_;
    because it is *not* on the first line?

    But as you can see, even if shebang is not the first line, such as the empty line preceding it, the script still works as expected.

    If you happen to have other requirements adjust the condition.
    (Don't expect that we can read your mind from that distance.)

    You want to consider [initial] blank lines similar to comments,
    for example?

    !(/^#/||!NF) && !f++ ...

    I see. Thank you again

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Janis Papanagnou on Sun Dec 12 04:18:13 2021
    On Sunday, December 12, 2021 at 7:57:26 PM UTC+8, Janis Papanagnou wrote:
    On 12.12.2021 01:42, hongy...@gmail.com wrote:
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    On 11.12.2021 12:30, hongy...@gmail.com wrote:
    I try to understand the following complex sed command logic
    instructions given here [1]:
    [...]

    But the logic contained in the sed code snippet above is still
    obscure to me. Any hints?
    Get the sed manual and identify the used commands. Or formulate
    the simple algorithm yourself informally and try to identify the
    building blocks in the sed code.

    OTOH, the final result achieved by the sed workflow does seem so
    trivial. Why do they still insist on providing such an ancient
    sed-based solution?
    Maybe it's code from the 1980's?
    Maybe the programmer is used to that tool?
    Maybe the programmer doesn't know any better tool?
    Who knows?
    But I see no one insisting in anything.

    Though, for simple tasks you can always write your own version,
    and use a language that is or appears to you as less cryptic.

    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    I try to debug/understand your above code as follows:
    Understand how f++ works - in *any* of the many languages that support
    it - before you try to "debug".

    GNU Awk has a debugger that makes it unnecessary in practice to add own [possibly or likely wrong] code.

    But this way it only supports running script from file, instead of the in-line ones from stdin:

    $ awk --help |grep -i debug
    -D[file] --debug[=file]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to hongy...@gmail.com on Sun Dec 12 16:11:47 2021
    On 12.12.2021 13:18, hongy...@gmail.com wrote:
    On Sunday, December 12, 2021 at 7:57:26 PM UTC+8, Janis Papanagnou wrote:

    GNU Awk has a debugger that makes it unnecessary in practice to add own
    [possibly or likely wrong] code.

    But this way it only supports running script from file, instead of the in-line ones from stdin:

    And your problem with that is what?

    You want to understand what the program does, don't you?
    So put it in a file to use the debugger. What is your problem.

    Janis


    $ awk --help |grep -i debug
    -D[file] --debug[=file]


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to hongy...@gmail.com on Sun Dec 12 16:17:57 2021
    On 12.12.2021 13:15, hongy...@gmail.com wrote:
    On Sunday, December 12, 2021 at 7:52:02 PM UTC+8, Janis Papanagnou wrote:
    On 12.12.2021 07:32, hongy...@gmail.com wrote:
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote: >>>> I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    The above code will add the stuff at the very beginning of the
    script if occasionally the shebang line is not the first line:
    1. The posted requirements had been
    # the sed invocation inserts the lines at the start of the file
    # after any initial comment lines
    2. What do you expect from a shebang line that is _ineffective_;
    because it is *not* on the first line?

    But as you can see, even if shebang is not the first line, such as the empty line preceding it, the script still works as expected.

    I have the impression that you still don't understand the mechanics.

    Try this code (first line empty) and you see that the #! is ineffective. ---snip---

    #!/usr/bin/awk -f
    BEGIN { print "Hi" }
    ---snip---

    Or if you run a standard shell script with #!/usr/bin/bash with
    some non-standard shell environment you will also get errors.

    There are a couple conditions that must be fulfilled for a shebang
    script to get that line become active as an interpreter directive.
    The one we have been talking about here is that it must be the first
    line in the file (no empty lines or anything).

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From John-Paul Stewart@21:1/5 to hongy...@gmail.com on Sun Dec 12 15:19:02 2021
    On 2021-12-12 01:36, hongy...@gmail.com wrote:
    {

    Begin a block of commands
    a \
    export PYENV_ROOT="$HOME/.pyenv"
    a \
    export PATH="$PYENV_ROOT/bin:$PATH"
    Append text, which has each embedded newline preceded by a backslash.
    a \
    ' -e ':a' -e '$!{n;ba};
    Failed to figure out the above obscure code block.

    Especially, there are two '-e' constructs embedded in outer '-e'
    construct, which is very confusing for me.

    No, there is no "embedded" -e construct. Re-read the original post:

    sed -Ei -e '/^([^#]|$)/ {a \
    export PYENV_ROOT="$HOME/.pyenv"
    a \
    export PATH="$PYENV_ROOT/bin:$PATH"
    a \
    ' -e ':a' -e '$!{n;ba};}' ~/.profile

    The first -e is followed by an opening single quote ' so everything from
    there to the following closing single quote ' on the last line is one
    "script" to be executed. Then there is the second -e ':a' after that,
    and finally the third -e script (also in single quotes). None are
    embedded in the other. There are three consecutive script fragments.

    Little details such as the placement of those single quotes are
    crucially important in shell/sed/awk scripting. You must learn to
    notice them and understand them!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Keith Thompson@21:1/5 to hongy...@gmail.com on Sun Dec 12 14:05:57 2021
    "hongy...@gmail.com" <hongyi.zhao@gmail.com> writes:
    On Sunday, December 12, 2021 at 7:52:02 PM UTC+8, Janis Papanagnou wrote:
    [...]
    2. What do you expect from a shebang line that is _ineffective_;
    because it is *not* on the first line?

    But as you can see, even if shebang is not the first line, such as the
    empty line preceding it, the script still works as expected.

    No, a shebang line must be on the first line, and the "#!" must be the
    first two characters of the first line, or it will be ignored (which,
    depending on the interpreter, usually means it's treated as a comment).

    If a script has no shebang, the way it's executed depends on how you
    invoke it. If you invoke it from bash, it executes it as a bash script.
    If you invoke it by calling system(script_name), it uses /bin/sh.

    --
    Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
    Working, but not speaking, for Philips
    void Void(void) { Void(); } /* The recursive call of the void */

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Janis Papanagnou on Sun Dec 12 17:42:09 2021
    On Sunday, December 12, 2021 at 11:11:52 PM UTC+8, Janis Papanagnou wrote:
    You want to understand what the program does, don't you?
    So put it in a file to use the debugger. What is your problem.

    I mean, it would be more convenient if I could debug it by running the content of the script from the terminal.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Keith Thompson on Sun Dec 12 17:51:12 2021
    On Monday, December 13, 2021 at 6:06:03 AM UTC+8, Keith Thompson wrote:
    "hongy...@gmail.com" <hongy...@gmail.com> writes:
    On Sunday, December 12, 2021 at 7:52:02 PM UTC+8, Janis Papanagnou wrote:
    [...]
    2. What do you expect from a shebang line that is _ineffective_;
    because it is *not* on the first line?

    But as you can see, even if shebang is not the first line, such as the empty line preceding it, the script still works as expected.
    No, a shebang line must be on the first line, and the "#!" must be the
    first two characters of the first line, or it will be ignored (which, depending on the interpreter, usually means it's treated as a comment).

    Where is this specification clearly stated?

    If a script has no shebang, the way it's executed depends on how you
    invoke it. If you invoke it from bash, it executes it as a bash script.
    If you invoke it by calling system(script_name), it uses /bin/sh.

    Thank you for pointing this out.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Sun Dec 12 18:05:38 2021
    On Monday, December 13, 2021 at 9:49:17 AM UTC+8, hongy...@gmail.com wrote:
    On Sunday, December 12, 2021 at 11:18:02 PM UTC+8, Janis Papanagnou wrote:
    On 12.12.2021 13:15, hongy...@gmail.com wrote:
    On Sunday, December 12, 2021 at 7:52:02 PM UTC+8, Janis Papanagnou wrote:
    On 12.12.2021 07:32, hongy...@gmail.com wrote:
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    The above code will add the stuff at the very beginning of the
    script if occasionally the shebang line is not the first line:
    1. The posted requirements had been
    # the sed invocation inserts the lines at the start of the file
    # after any initial comment lines
    2. What do you expect from a shebang line that is _ineffective_;
    because it is *not* on the first line?

    But as you can see, even if shebang is not the first line, such as the empty line preceding it, the script still works as expected.
    I have the impression that you still don't understand the mechanics.

    Try this code (first line empty) and you see that the #! is ineffective. ---snip---

    #!/usr/bin/awk -f
    BEGIN { print "Hi" }
    ---snip---
    Yes. The above code will fail as follows:

    $ ./test.awk
    ./test.awk: line 3: BEGIN: command not found


    But I still don't understand why the following example works:

    $ cat test.sh

    #!/bin/bash
    echo 'Hello.'

    $ ./test.sh
    Hello.

    And strace can't trace the system calls and signals of this script:
    $ strace ./test.sh
    execve("./test.sh", ["./test.sh"], 0x7ffc7d8136c0 /* 102 vars */) = -1 ENOEXEC (Exec format error)
    strace: exec: Exec format error
    +++ exited with 1 +++

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Janis Papanagnou on Sun Dec 12 17:49:14 2021
    On Sunday, December 12, 2021 at 11:18:02 PM UTC+8, Janis Papanagnou wrote:
    On 12.12.2021 13:15, hongy...@gmail.com wrote:
    On Sunday, December 12, 2021 at 7:52:02 PM UTC+8, Janis Papanagnou wrote:
    On 12.12.2021 07:32, hongy...@gmail.com wrote:
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    The above code will add the stuff at the very beginning of the
    script if occasionally the shebang line is not the first line:
    1. The posted requirements had been
    # the sed invocation inserts the lines at the start of the file
    # after any initial comment lines
    2. What do you expect from a shebang line that is _ineffective_;
    because it is *not* on the first line?

    But as you can see, even if shebang is not the first line, such as the empty line preceding it, the script still works as expected.
    I have the impression that you still don't understand the mechanics.

    Try this code (first line empty) and you see that the #! is ineffective. ---snip---

    #!/usr/bin/awk -f
    BEGIN { print "Hi" }
    ---snip---

    Yes. The above code will fail as follows:

    $ ./test.awk
    ./test.awk: line 3: BEGIN: command not found


    But I still don't understand why the following example works:

    $ cat test.sh

    #!/bin/bash
    echo 'Hello.'

    $ ./test.sh
    Hello.



    Or if you run a standard shell script with #!/usr/bin/bash with
    some non-standard shell environment you will also get errors.

    There are a couple conditions that must be fulfilled for a shebang
    script to get that line become active as an interpreter directive.
    The one we have been talking about here is that it must be the first
    line in the file (no empty lines or anything).

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From David W. Hodgins@21:1/5 to hongy...@gmail.com on Sun Dec 12 21:14:54 2021
    On Sun, 12 Dec 2021 20:51:12 -0500, hongy...@gmail.com <hongyi.zhao@gmail.com> wrote:

    On Monday, December 13, 2021 at 6:06:03 AM UTC+8, Keith Thompson wrote:
    "hongy...@gmail.com" <hongy...@gmail.com> writes:
    On Sunday, December 12, 2021 at 7:52:02 PM UTC+8, Janis Papanagnou wrote: >> [...]
    2. What do you expect from a shebang line that is _ineffective_;
    because it is *not* on the first line?

    But as you can see, even if shebang is not the first line, such as the
    empty line preceding it, the script still works as expected.
    No, a shebang line must be on the first line, and the "#!" must be the
    first two characters of the first line, or it will be ignored (which,
    depending on the interpreter, usually means it's treated as a comment).

    Where is this specification clearly stated?

    In the kernel source, specifically the program loader.

    From "man execve" ...
    Interpreter scripts
    An interpreter script is a text file that has execute permission enabled and whose first line is of the form:
    #!interpreter [optional-arg]

    Regards, Dave Hodgins

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to John-Paul Stewart on Sun Dec 12 18:25:05 2021
    On Monday, December 13, 2021 at 4:19:08 AM UTC+8, John-Paul Stewart wrote:
    No, there is no "embedded" -e construct. Re-read the original post:

    sed -Ei -e '/^([^#]|$)/ {a \
    export PYENV_ROOT="$HOME/.pyenv"
    a \
    export PATH="$PYENV_ROOT/bin:$PATH"
    a \
    ' -e ':a' -e '$!{n;ba};}' ~/.profile

    The first -e is followed by an opening single quote ' so everything from there to the following closing single quote ' on the last line is one "script" to be executed. Then there is the second -e ':a' after that,
    and finally the third -e script (also in single quotes). None are
    embedded in the other. There are three consecutive script fragments.

    Ok. Let me try to understand the last two script fragments:

    -e ':a' -e '$!{n;ba};}'

    :a Make label a for b command.
    $ Match the last line.
    ! After the address (or address-range), and before the command, a ! may be inserted, which specifies
    that the command shall only be executed if the address (or address-range) does not match.

    { Begin a block of commands (end with a }).

    n N Read/append the next line of input into the pattern space.

    b label
    Branch to label; if label is omitted, branch to end of script.

    ; Is used to separate multiple sed commands, IIRC.

    By putting all the above knowledge together, I still haven't figured out any clues
    pertaining to the goal to be achieved here.

    Little details such as the placement of those single quotes are
    crucially important in shell/sed/awk scripting. You must learn to
    notice them and understand them!

    Thank you for pointing this out.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to David W. Hodgins on Sun Dec 12 18:27:49 2021
    On Monday, December 13, 2021 at 10:15:08 AM UTC+8, David W. Hodgins wrote:
    From "man execve" ...
    Interpreter scripts
    An interpreter script is a text file that has execute permission enabled and whose first line is of the form:
    #!interpreter [optional-arg]

    Thank you for letting me know this.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From William Unruh@21:1/5 to hongy...@gmail.com on Mon Dec 13 05:09:22 2021
    On 2021-12-13, hongy...@gmail.com <hongyi.zhao@gmail.com> wrote:
    On Sunday, December 12, 2021 at 11:18:02 PM UTC+8, Janis Papanagnou wrote:
    On 12.12.2021 13:15, hongy...@gmail.com wrote:
    On Sunday, December 12, 2021 at 7:52:02 PM UTC+8, Janis Papanagnou wrote: >> >> On 12.12.2021 07:32, hongy...@gmail.com wrote:
    On Saturday, December 11, 2021 at 8:23:09 PM UTC+8, Janis Papanagnou wrote:
    I'd use Awk, for example, probably something like

    awk '!/^#/ && !f++ { print "..." } ; 1'

    The above code will add the stuff at the very beginning of the
    script if occasionally the shebang line is not the first line:
    1. The posted requirements had been
    # the sed invocation inserts the lines at the start of the file
    # after any initial comment lines
    2. What do you expect from a shebang line that is _ineffective_;
    because it is *not* on the first line?

    But as you can see, even if shebang is not the first line, such as the empty line preceding it, the script still works as expected.
    I have the impression that you still don't understand the mechanics.

    Try this code (first line empty) and you see that the #! is ineffective.
    ---snip---

    #!/usr/bin/awk -f
    BEGIN { print "Hi" }
    ---snip---

    Yes. The above code will fail as follows:

    $ ./test.awk
    ./test.awk: line 3: BEGIN: command not found




    But I still don't understand why the following example works:
    Becase the system runs this file with bash, with the shebang interpreted
    as a comment, and the rest of the lines ar valid bash commands.



    $ cat test.sh

    #!/bin/bash
    echo 'Hello.'

    $ ./test.sh
    Hello.



    Or if you run a standard shell script with #!/usr/bin/bash with
    some non-standard shell environment you will also get errors.

    You needed toread and understand this sentence.



    There are a couple conditions that must be fulfilled for a shebang
    script to get that line become active as an interpreter directive.
    The one we have been talking about here is that it must be the first
    line in the file (no empty lines or anything).

    It would really help if you did not treat your own prejudices as if they
    were revelations from God, and tried to understand what others were
    trying to say.



    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to hongy...@gmail.com on Mon Dec 13 07:20:48 2021
    On 13.12.2021 02:49, hongy...@gmail.com wrote:
    On Sunday, December 12, 2021 at 11:18:02 PM UTC+8, Janis Papanagnou wrote:

    Try this code (first line empty) and you see that the #! is ineffective.
    ---snip---

    #!/usr/bin/awk -f
    BEGIN { print "Hi" }
    ---snip---

    Yes. The above code will fail as follows:

    $ ./test.awk
    ./test.awk: line 3: BEGIN: command not found


    But I still don't understand why the following example works:

    For the same reason why that program also works without the #! line.


    $ cat test.sh

    #!/bin/bash
    echo 'Hello.'

    $ ./test.sh
    Hello.


    You may want to inform yourself more thoroughly by reading articles
    about #!. Wikipedia - https://en.wikipedia.org/wiki/Shebang_(Unix) -
    is a good start and incidentally answers also your question. If you
    have specialized questions or want to know about some gory details
    have a look into https://www.in-ulm.de/~mascheck/various/shebang/ .

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Janis Papanagnou on Sun Dec 12 23:45:27 2021
    On Monday, December 13, 2021 at 2:20:55 PM UTC+8, Janis Papanagnou wrote:
    On 13.12.2021 02:49, hongy...@gmail.com wrote:
    On Sunday, December 12, 2021 at 11:18:02 PM UTC+8, Janis Papanagnou wrote:

    Try this code (first line empty) and you see that the #! is ineffective. >> ---snip---

    #!/usr/bin/awk -f
    BEGIN { print "Hi" }
    ---snip---

    Yes. The above code will fail as follows:

    $ ./test.awk
    ./test.awk: line 3: BEGIN: command not found


    But I still don't understand why the following example works:

    For the same reason why that program also works without the #! line.

    Wonderful clue/hint.


    $ cat test.sh

    #!/bin/bash
    echo 'Hello.'

    $ ./test.sh
    Hello.

    You may want to inform yourself more thoroughly by reading articles
    about #!. Wikipedia - https://en.wikipedia.org/wiki/Shebang_(Unix) -
    is a good start and incidentally answers also your question. If you
    have specialized questions or want to know about some gory details
    have a look into https://www.in-ulm.de/~mascheck/various/shebang/ .

    I really don't know this website until now. It also has many other priceless notes, such as the following:

    https://www.in-ulm.de/~mascheck/various/bourne_args https://www.in-ulm.de/~mascheck/various/echo+printf/ https://www.in-ulm.de/~mascheck/various/find/#xargs

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Tue Dec 14 00:37:10 2021
    On Monday, December 13, 2021 at 10:25:08 AM UTC+8, hongy...@gmail.com wrote:
    On Monday, December 13, 2021 at 4:19:08 AM UTC+8, John-Paul Stewart wrote:
    No, there is no "embedded" -e construct. Re-read the original post:

    sed -Ei -e '/^([^#]|$)/ {a \
    export PYENV_ROOT="$HOME/.pyenv"
    a \
    export PATH="$PYENV_ROOT/bin:$PATH"
    a \
    ' -e ':a' -e '$!{n;ba};}' ~/.profile

    The first -e is followed by an opening single quote ' so everything from there to the following closing single quote ' on the last line is one "script" to be executed. Then there is the second -e ':a' after that,
    and finally the third -e script (also in single quotes). None are
    embedded in the other. There are three consecutive script fragments.
    Ok. Let me try to understand the last two script fragments:

    -e ':a' -e '$!{n;ba};}'

    Here [1] I find the correlated example:

    $ sed -e '/foo/ {s//bar/; ' -e ':a' -e '$!{n;ba' -e '};}' <<<$'1st foo\nUnrelated\n2nd foo\n3rd foo'
    1st bar
    Unrelated
    2nd foo
    3rd foo

    [1] https://stackoverflow.com/a/33416489


    :a Make label a for b command.
    $ Match the last line.
    ! After the address (or address-range), and before the command, a ! may be inserted, which specifies
    that the command shall only be executed if the address (or address-range) does not match.

    { Begin a block of commands (end with a }).

    n N Read/append the next line of input into the pattern space.

    b label
    Branch to label; if label is omitted, branch to end of script.

    ; Is used to separate multiple sed commands, IIRC.

    By putting all the above knowledge together, I still haven't figured out any clues
    pertaining to the goal to be achieved here.
    Little details such as the placement of those single quotes are
    crucially important in shell/sed/awk scripting. You must learn to
    notice them and understand them!
    Thank you for pointing this out.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)