See the following example bash script:
#!/usr/bin/env bash
sudo apt install -y pkg_1 pkg_2 ... pkg_N \
# There are some comments here.
pkg_N+1 ...
The above script will fail due to the there are comments on the
next line of the line continuation symbol (\). Is there a
workaround/trick which can let me write bash script this way?
"hongy...@gmail.com" <hongyi.zhao@gmail.com> writes:
See the following example bash script:
```
#!/usr/bin/env bash
sudo apt install -y pkg_1 pkg_2 ... pkg_N \
# There are some comments here.
pkg_N+1 ...
```
The above script will fail due to the there are comments on the
next line of the line continuation symbol (\). Is there a
workaround/trick which can let me write bash script this way?
Backslash the end of line on the comment.
The backslash character at the end of the line on the commentwill be treated and discarded as part of the comment. No line
Thank you and all others for the explanation given here. I figured
out the following solution based on your above suggestion:
------------------= begin =--------------
#!/usr/bin/env bash
# Here's that comment for packages related to a.
a="pkga_1 pkga_2 ... pkga_n \
pkga_n+1"
# Here's that comment for packages related to b.
b="pkgb_1 pkgb_2 ... pkgb_n \
pkgb_n+1"
sudo apt install -y $a $b
------------------= end =--------------
On Sunday, December 26, 2021 at 2:59:36 PM UTC+8, Helmut Waitzmann
wrote:
It only works if neither of the package names contains white
space.
For Debian derivative systems, the package name specification is
described here [1]:
--------
This manual page documents the dpkg-name program which provides an
easy way to rename Debian packages into their full package names. A
full package name consists of
package_version_architecture.package-type as specified in the
control file of the package. The version part of the filename
consists of the upstream version information optionally followed by
a hyphen and the revision information. The package-type part comes
from that field if present or fallbacks to deb.
--------
[1]
https://manpages.debian.org/bullseye/dpkg-dev/dpkg-name.1.en.html
As you can see, the package names don't contain white space.
As for other operating systems, I'm not sure, but I believe it's
not a good practice to use whitespace in the naming of software
packages on a system.
On Tuesday, December 28, 2021 at 8:55:13 AM UTC+8, Helmut Waitzmann wrote:
"hongy...@gmail.com" <hongy...@gmail.com>:
As far as I understand, the cited text does not say anything
about the package, version, architecture, and package-type
components of a full package name other than that they are
specified in the control file (of unknown contents) of the
package, and that the version part is a concatenation of the
upstream version information (of unknown contents) followed by a
hyphen and the revision information (of unknown contents). And
finally these parts are concatenated by underscores.
If any of these parts contains some white space, then that white
space will be in the full package name, as well.
Sorry. I should have shown you, but I haven't found it till now,
the following description given in "Debian RFC822 control data
format" [1]:
I should have shown you, but I didn't find the following
description given in "Debian RFC822 Control Data Formats" until
now:
$ man deb822 | egrep -A3 -i 'whitespace must not'
Whitespace must not appear inside names (of packages,
architectures, files or anything else) or version numbers,
or between the characters of multi-character version
relationships.
[1] https://man7.org/linux/man-pages/man5/deb822.5.html
The best practice with regard to shell programming: Don't ever
misuse a (non‐array) shell variable v as an array variable by
merely gluing together the array elements with white space and
then using the value of the variable by unquoted references ($v)
to have it be split at white space.
With Bash and other modern shells you have got even true array
variables. So you could use them in this case. (But as you might
know, I prefer (if at all possible) to use solutions which work
with any shell conforming to the POSIX standard.)
But this may lead to the loss of some convenience. There is no best
of both worlds in the world.
"hongy...@gmail.com" <hongy...@gmail.com>:
On Tuesday, December 28, 2021 at 8:55:13 AM UTC+8, Helmut Waitzmann wrote:
"hongy...@gmail.com" <hongy...@gmail.com>:
As far as I understand, the cited text does not say anything
about the package, version, architecture, and package-type
components of a full package name other than that they are
specified in the control file (of unknown contents) of the
package, and that the version part is a concatenation of the
upstream version information (of unknown contents) followed by a
hyphen and the revision information (of unknown contents). And
finally these parts are concatenated by underscores.
If any of these parts contains some white space, then that white
space will be in the full package name, as well.
Sorry. I should have shown you, but I haven't found it till now,
the following description given in "Debian RFC822 control data
format" [1]:
I should have shown you, but I didn't find the following
description given in "Debian RFC822 Control Data Formats" until
now:
$ man deb822 | egrep -A3 -i 'whitespace must not'
Whitespace must not appear inside names (of packages,
architectures, files or anything else) or version numbers,
or between the characters of multi-character version
relationships.
[1] https://man7.org/linux/man-pages/man5/deb822.5.html
Thank you. So we are lucky, and in this case it's not necessary to
do what is described below.
The best practice with regard to shell programming: Don't ever
misuse a (non‐array) shell variable v as an array variable by
merely gluing together the array elements with white space and
then using the value of the variable by unquoted references ($v)
to have it be split at white space.
With Bash and other modern shells you have got even true array
variables. So you could use them in this case. (But as you might
know, I prefer (if at all possible) to use solutions which work
with any shell conforming to the POSIX standard.)
But this may lead to the loss of some convenience. There is no bestCompared with the disatrous consequences of an exploit I'd clearly
of both worlds in the world.
prefer array variables with Bash and other modern shells.
And even if I want to stay compatible with the POSIX standard, I
consider an exploit far more inconvenient than the increased
complexity in programming for the shell.
Simulating an array variable using the POSIX standard could be done
like this:
Define a shell function "quote_words" (see below), which, when
given zero or more invocation arguments, computes (a part of) a
shell command line (of a simple command) resembling a command line
to invoke those arguments and writes it to standard output, where
it can be put in a simple shell variable by command substitution,
like this:
v="$(quote_words printf '%s\n' 'Hello, world!')"
It can then invoked using the "eval" shell builtin command:
eval "$v"
Try the following command to see, what that is. The commands
v="$(quote_words printf '%s\n' 'Hello, world!')" &&
printf '%s\n' "$v" &&
eval "$v"
will output
'printf' '%s\n' 'Hello, world!'
Hello, world!
Now, this content of the variable v, which is a (part of a) shell
command line of a simple command, can even be glued together with
more parts of a command line by putting a blank between each of the
command line parts to be glued together and stored in a simple
shell variable. Finally, this shell variable can be fed to the
"eval" shell builtin command to let it be invoked as a command
line.
In the example of the OP this would look like these shell commands:
v="$(quote_words sudo apt install -y)" &&
# Here's that comment for packages related to a.
#
v="$v $(quote_words 'pkg a 1' 'pkg a 2' … 'pkg a n+1')" &&
# Here's that comment for packages related to b.
#
v="$v $(quote_words 'pkg b 1' 'pkg b 2' … 'pkg b n+1')" &&
# Finally, let the "eval" command parse and invoke it:
#
eval "$v" &&
That quote_words function might look like this:
### The function quote_words starts here.
quote_words()
(
# Outputs a (part of) a command line resembling a 'simple
# command' to be used by a POSIX-compliant shell.
#
# Usage:
#
# quote_words words to be quoted...
#
# exit code:
# 0, if the command line has been successfully constructed,
# !=0, if the function failed to construct the command line.
#
# Examples:
#
# Put a word list resembling a command invocation into a command
# line:
#
# if command_line="$(
# quote_words program with parameters
# )"
# then
# # "eval" it:
# eval " $command_line"
#
# # or pass it to a new shell:
# sh -c -- "$command_line" sh
#
# # or pass it to "su":
# su -- - a_user -c -- "$command_line" -su
#
# # or use it remotely:
# ssh us...@remote.host.example " $command_line"
#
# else
# # failed to compute the command line.
# fi
#
#
# Store the shell positional parameters in a variable and
# restore them later:
#
# if args="$(quote_words "$@")"
# then
# # the positional parameters may be changed for any purpose
# # ...
#
# # restore the positional parameters:
#
# eval "set -- $args"
# else
# # failed to save the positional parameters.
# fi
# "wordsep" contains a separator used for the list of the quoted
# words. The first need not to be preceded by a separater:
wordsep=
for word
do
# "$wordsep" separates each word (except the first one) from
# its precedessor:
printf '%s' "$wordsep"
if test -z "$word"
then
# The word is empty. Then the result is "''":
printf '%s' "''"
else
# The word is not empty.
while
{
prefix="${word%%\'*}"
word="${word#"$prefix"}"
# "$prefix" is the longest prefix of "$word", that does
# not contain any apostrophes; it is removed from "$word".
# Conclusion: "$word" is either empty or starts with an
# apostrophe.
if test -n "$prefix"
then
# "$prefix" consists of one or more characters. Put
# them in between of two apostrophes:
printf '%s' \'"${prefix}"\'
fi
test -n "$word" &&
{
# "$word" is not empty. Conclusion: "$word" starts with
# one or more apostrophes.
apostr="${word%%[!\']*}"
# "$apostr" ist the longest beginning part of "$word",
# that contains apostrophes only.
if test -n "${apostr#"'"}"
then
# There are at least 2 apostrophes: Put it in between
# of two double quotes:
printf '"%s"' "${apostr}"
else
# There is 1 apostrophe: Precede it by a backslash:
printf '\\%s' "${apostr}"
fi
# Remove the beginning apostrophes from "$word":
word="${word#"$apostr"}"
# If nothing is left, we are done:
${word:+:} false
}
}
do
:
done
fi
# All following words (except the first one) have to be
# separated from their precedessor with a blank:
wordsep=' '
done
printf '\n'
)
### The function quote_words ends here.
Of course I won't want to write this function from scratch in every
shell script where it is to be used. Therefore I put it in a file,
say "~/shell-lib/quote_words.sh". Then, if I like to use it in a
shell script, I just do
. ~/shell-lib/quote_words.sh
in that shell script to have that function defined. It's far more convenient than to be hit by any exploit.
Look again at the example above, annotated by some comments on the
value of the variable v:
v="$(quote_words sudo apt install -y)" &&
#
# Now the variable v has got the contents
#
# 'sudo' 'apt' 'install' '-y'
#
# Note, how the 4 parameters have been quoted. (That will be
# needed by the "eval" command, below.)
# Here's that comment for packages related to a.
#
# Note, how the package names are glued with a blank character
# to the end of the existing value of the variable v:
# v="$v $(quote_words …)"
#
v="$v $(quote_words 'pkg a 1' 'pkg a 2' … 'pkg a n+1')" &&
#
# Now the variable v has got the contents:
#
# 'sudo' 'apt' 'install' '-y' 'pkg a 1' 'pkg a 2' … 'pkg a n+1'
# Here's that comment for packages related to b.
#
v="$v $(quote_words 'pkg b 1' 'pkg b 2' … 'pkg b n+1')" &&
#
# Now the variable v has got the contents (shown wrapped for
# better readability):
#
# 'sudo' 'apt' 'install' '-y' \
# 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
# 'pkg b 1' 'pkg b 2' … 'pkg b n+1'
#
# Note, how the parameters have been quoted. (Thus they will
# be protected from being broken apart by the "eval" command.)
eval "$v" &&
#
# This command line has got the contents (shown wrapped
# because of its length):
#
# ''\''sudo'\'' '\''apt'\'' '\''install'\'' '\''-y'\'' \
# '\''pkg a 1'\'' '\''pkg a 2'\'' … '\''pkg a n+1'\'' \
# '\''pkg b 1'\'' '\''pkg b 2'\'' … '\''pkg b n+1'\'
#
# Yes, that's terrible for humans but perfect for the shell.
#
# The shell will parse that command line, thus invoking the
# "eval" builtin command with the following single literally
# parameter (shown wrapped because of its length):
#
# 'sudo' 'apt' 'install' '-y' \
# 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
# 'pkg b 1' 'pkg b 2' … 'pkg b n+1'
#
# The "eval" builtin command will parse its parameter like
# a shell command line -- that's the purpose of the "eval"
# command -- and finally invoke the equivalent of the following
# command line:
#
# 'sudo' 'apt' 'install' '-y' \
# 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
# 'pkg b 1' 'pkg b 2' … 'pkg b n+1'
This is the difference between using the "quote_words" function and
merely gluing parameters together with white space: "quote_words"
will proper quote each of the arguments before gluing them
together. That allows and mandates the resulting string to be
parsed by the "eval" builtin command. And of course that string is
not to be split at white space prior to be passed to the "eval"
command, therefore it shall be quoted. In the example above, that
is
eval "$v"
Because the argument to the "eval" command has been quoted, it will
not be broken apart at white space as would be the case with an
unquoted reference to the variable.
On Wednesday, December 29, 2021 at 6:42:40 AM UTC+8, Helmut Waitzmann
wrote:
[ 325 lines of quoted text and code deleted ]
I read through your long reply and didn't quite understand it.
But I think for a problem with a small probability of occurrence,
is it necessary to do such complex processing and consideration?
HZ
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 293 |
Nodes: | 16 (2 / 14) |
Uptime: | 223:15:36 |
Calls: | 6,623 |
Calls today: | 5 |
Files: | 12,171 |
Messages: | 5,318,361 |