• sed replace strings with single quotes and brackets

    From Harry@21:1/5 to All on Tue Nov 9 12:59:42 2021
    I am trying to modify a whole bunch of text files (*.sh)
    with a small shell script.

    $ cat yy
    SRC="new_time(tx.create_dt_tm,'GMT','TIMEZONE')" DEST="cast(from_tz(cast(tx.create_dt_tm as timestamp), 'Etc/GMT0') at time zone 'Canada/Pacific' as date)"
    for i in *.sh
    do
    echo modifying $i ...
    sed -i "s/$SRC/$DEST/g" $i
    done

    I got errors like these ... what am I missing ?

    modifying xxx.sh ...
    sed: -e expression #1, char 100: unknown option to `s'

    Note: both $SRC and $DEST have no double quotes in the string contents.

    TIA

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From David W. Hodgins@21:1/5 to Harry on Tue Nov 9 16:14:24 2021
    On Tue, 09 Nov 2021 15:59:42 -0500, Harry <harryooopotter@hotmail.com> wrote:

    I am trying to modify a whole bunch of text files (*.sh)
    with a small shell script.

    $ cat yy
    SRC="new_time(tx.create_dt_tm,'GMT','TIMEZONE')" DEST="cast(from_tz(cast(tx.create_dt_tm as timestamp), 'Etc/GMT0') at time zone 'Canada/Pacific' as date)"
    for i in *.sh
    do
    echo modifying $i ...
    sed -i "s/$SRC/$DEST/g" $i
    done

    I got errors like these ... what am I missing ?

    modifying xxx.sh ...
    sed: -e expression #1, char 100: unknown option to `s'

    Note: both $SRC and $DEST have no double quotes in the string contents.

    Run it with "bash -x -v yy"

    + for i in *.sh
    + echo modifying newcerts.sh ...
    modifying newcerts.sh ...
    + echo sed -i 's/new_time(tx.create_dt_tm,'\''GMT'\'','\''TIMEZONE'\'')/cast(from_tz(cast(tx.create_dt_tm as timestamp), '\''Etc/GMT0'\'') at time zone '\''Canada/Pacific'\'' as date)/g' newcerts.sh
    sed -i s/new_time(tx.create_dt_tm,'GMT','TIMEZONE')/cast(from_tz(cast(tx.create_dt_tm as timestamp), 'Etc/GMT0') at time zone 'Canada/Pacific' as date)/g newcerts.sh

    Notice how the generated sed command contains too many slash characters.

    Replace the delimiter slash characters with some other character not present in the
    strings.

    Regards, Dave Hodgins

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Harry@21:1/5 to David W. Hodgins on Tue Nov 9 14:25:25 2021
    On Tuesday, November 9, 2021 at 1:14:37 PM UTC-8, David W. Hodgins wrote:

    Run it with "bash -x -v yy"

    + for i in *.sh
    + echo modifying newcerts.sh ...
    modifying newcerts.sh ...
    + echo sed -i 's/new_time(tx.create_dt_tm,'\''GMT'\'','\''TIMEZONE'\'')/cast(from_tz(cast(tx.create_dt_tm as timestamp), '\''Etc/GMT0'\'') at time zone '\''Canada/Pacific'\'' as date)/g' newcerts.sh
    sed -i s/new_time(tx.create_dt_tm,'GMT','TIMEZONE')/cast(from_tz(cast(tx.create_dt_tm as timestamp), 'Etc/GMT0') at time zone 'Canada/Pacific' as date)/g newcerts.sh

    Notice how the generated sed command contains too many slash characters.

    Replace the delimiter slash characters with some other character not present in the
    strings.

    Thanks, I replace / with # in the sed cmd and it works now.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From applemcg@21:1/5 to Harry on Sat Nov 13 07:35:26 2021
    On Tuesday, November 9, 2021 at 5:25:28 PM UTC-5, Harry wrote:
    On Tuesday, November 9, 2021 at 1:14:37 PM UTC-8, David W. Hodgins wrote:

    Run it with "bash -x -v yy"

    + for i in *.sh
    + echo modifying newcerts.sh ...
    modifying newcerts.sh ...
    + echo sed -i 's/new_time(tx.create_dt_tm,'\''GMT'\'','\''TIMEZONE'\'')/cast(from_tz(cast(tx.create_dt_tm as timestamp), '\''Etc/GMT0'\'') at time zone '\''Canada/Pacific'\'' as date)/g' newcerts.sh
    sed -i s/new_time(tx.create_dt_tm,'GMT','TIMEZONE')/cast(from_tz(cast(tx.create_dt_tm as timestamp), 'Etc/GMT0') at time zone 'Canada/Pacific' as date)/g newcerts.sh

    Notice how the generated sed command contains too many slash characters.

    Replace the delimiter slash characters with some other character not present in the
    strings.
    Thanks, I replace / with # in the sed cmd and it works now.


    anytime there's a potential file path involved mine looks like

    sed "s=$this=$that="

    suggesting a function:

    psed () { : path-protected sed; sed "s=$1=$2=g" ; }

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to applemcg on Sat Nov 13 17:47:49 2021
    On 13.11.2021 16:35, applemcg wrote:
    On Tuesday, November 9, 2021 at 5:25:28 PM UTC-5, Harry wrote:
    On Tuesday, November 9, 2021 at 1:14:37 PM UTC-8, David W. Hodgins wrote: >>> [ slash characters in sed substitution and as sed separators ]
    Notice how the generated sed command contains too many slash characters. >>>
    Replace the delimiter slash characters with some other character not present in the
    strings.
    Thanks, I replace / with # in the sed cmd and it works now.


    anytime there's a potential file path involved mine looks like

    sed "s=$this=$that="

    suggesting a function:

    psed () { : path-protected sed; sed "s=$1=$2=g" ; }


    Functions should be suggested if they are generally applicable.
    Your code shows the same issue with '=' characters that the OP
    had with '/' characters. (Consider path-component data with '='
    and $this or $that containing a '='.)

    Taking another rare character might delay the failure further but
    not solve the task generally.

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From applemcg@21:1/5 to Janis Papanagnou on Sat Nov 13 09:19:29 2021
    On Saturday, November 13, 2021 at 11:47:55 AM UTC-5, Janis Papanagnou wrote:
    On 13.11.2021 16:35, applemcg wrote:
    On Tuesday, November 9, 2021 at 5:25:28 PM UTC-5, Harry wrote:
    On Tuesday, November 9, 2021 at 1:14:37 PM UTC-8, David W. Hodgins wrote: >>> [ slash characters in sed substitution and as sed separators ]
    Notice how the generated sed command contains too many slash characters. >>>
    Replace the delimiter slash characters with some other character not present in the
    strings.
    Thanks, I replace / with # in the sed cmd and it works now.


    anytime there's a potential file path involved mine looks like

    sed "s=$this=$that="

    suggesting a function:

    psed () { : path-protected sed; sed "s=$1=$2=g" ; }

    Functions should be suggested if they are generally applicable.
    Your code shows the same issue with '=' characters that the OP
    had with '/' characters. (Consider path-component data with '='
    and $this or $that containing a '='.)

    Taking another rare character might delay the failure further but
    not solve the task generally.

    Janis

    of course; one might defend oneself against a range of unlikely characters using a CASE statement.

    my last gainful employ was as trainer at a wall st fin svcs company. our
    help desk, operating on Windows, trying to read log files with date stamps in their
    name were relieved to see a file system where the HH:MM:SS's were linked with LN(1),
    to an HH_MM_SS named file sometimes the fix lies elsewhere.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to applemcg on Sat Nov 13 19:04:52 2021
    On 13.11.2021 18:19, applemcg wrote:
    [ slash characters in sed substitution and as sed separators ]

    Taking another rare character might delay the failure further but
    not solve the task generally.

    of course; one might defend oneself against a range of unlikely characters using a CASE statement.

    Or dynamically checking the $this and $that values in the function.
    Even using binary separator values might fail in pathological cases,
    though using something like sed s$'\1'"$this"$'\1'"$that"$'\1'
    certainly fails much rarer.


    my last gainful employ was as trainer at a wall st fin svcs company. [...]

    (I cannot decipher "st fin svcs". - Ah, wait! Now I've got it.)

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stonebridge Mens Club@21:1/5 to Janis Papanagnou on Mon Nov 15 07:39:24 2021
    On Saturday, November 13, 2021 at 1:04:57 PM UTC-5, Janis Papanagnou wrote:
    On 13.11.2021 18:19, applemcg wrote:
    [ slash characters in sed substitution and as sed separators ]

    Taking another rare character might delay the failure further but
    not solve the task generally.

    of course; one might defend oneself against a range of unlikely characters using a CASE statement.
    Or dynamically checking the $this and $that values in the function.
    Even using binary separator values might fail in pathological cases,
    though using something like sed s$'\1'"$this"$'\1'"$that"$'\1'
    certainly fails much rarer.


    my last gainful employ was as trainer at a wall st fin svcs company. [...]

    (I cannot decipher "st fin svcs". - Ah, wait! Now I've got it.)

    Janis

    Janis,

    If I may, I'll disagree slightly with the idea that "Functions should
    be suggested if they are generally applicable"

    You have noticed a practice of mine: precede a command name with a
    single letter to isolate a specific use of an idea, in this case psed.

    I've a small family of awk wrappers: cawk, pawk, and tawk. a clue to
    their usage:

    tawk () { awk -F'\t' "$@"; }

    "pawk" got quite a bit of use when i was at the T (at&~), since
    pipe-separated tables were widely in use there.

    "cawk" _never_ gets used. There seems to be no standard for the CSV
    file. i.e. what if a field has a comma in it?

    "tawk" has no such problems. there will never be a TAB in data field.
    IBM saw to that with the 3270 terminal, since the TAB character (on
    that machine) was designed to take the user to the next data input
    field. And ...

    I met Rod Manis, one of three authors of "Unix Relational Data Base Management", which produced the text-only data base management system
    /rdb. He gave me a draft of their book, ~ '86, and I produced a KSH
    version. Where awk is the engine. Manis was dogmatic about little:

    "There will never be a TAB character in a data field"

    I'm at last producing a bash version -- annotated -- since a great
    deal of the book is either obsolescent or application-specific, with
    too much to do with accounting.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to Stonebridge Mens Club on Mon Nov 15 17:45:22 2021
    On 15.11.2021 16:39, Stonebridge Mens Club wrote:

    If I may, I'll disagree slightly with the idea that "Functions should
    be suggested if they are generally applicable"

    The point in this thread was that the function was written to solve a
    task but did that in an inherently error prone way. Please read this
    statement in that context.

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lew Pitcher@21:1/5 to Keith Thompson on Mon Nov 15 20:31:56 2021
    On Mon, 15 Nov 2021 12:16:36 -0800, Keith Thompson wrote:

    Stonebridge Mens Club <sbmc08831@gmail.com> writes: [...]
    "cawk" _never_ gets used. There seems to be no standard for the CSV
    file. i.e. what if a field has a comma in it?

    As I recall there are multiple standards for CSV files.

    The most nearly definitive is probably RFC 4180.

    https://www.ietf.org/rfc/rfc4180.txt https://datatracker.ietf.org/doc/html/rfc4180

    Caveat from https://www.ietf.org/rfc/rfc4180.txt:

    Status of This Memo

    This memo provides information for the Internet community. It does
    not specify an Internet standard of any kind. Distribution of this
    memo is unlimited.


    --
    Lew Pitcher
    "In Skills, We Trust"

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Keith Thompson@21:1/5 to Stonebridge Mens Club on Mon Nov 15 12:16:36 2021
    Stonebridge Mens Club <sbmc08831@gmail.com> writes:
    [...]
    "cawk" _never_ gets used. There seems to be no standard for the CSV
    file. i.e. what if a field has a comma in it?

    As I recall there are multiple standards for CSV files.

    The most nearly definitive is probably RFC 4180.

    https://www.ietf.org/rfc/rfc4180.txt https://datatracker.ietf.org/doc/html/rfc4180

    --
    Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
    Working, but not speaking, for Philips
    void Void(void) { Void(); } /* The recursive call of the void */

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kaz Kylheku@21:1/5 to Stonebridge Mens Club on Mon Nov 15 22:47:50 2021
    On 2021-11-15, Stonebridge Mens Club <sbmc08831@gmail.com> wrote:
    I've a small family of awk wrappers: cawk, pawk, and tawk. a clue to
    their usage:

    tawk () { awk -F'\t' "$@"; }

    "pawk" got quite a bit of use when i was at the T (at&~), since pipe-separated tables were widely in use there.

    "cawk" _never_ gets used.

    You tawk the tawk, but don't get to pawk your cawk?

    That's more than I want to know about the affairs of the
    Stonebridge Mens Club.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)