The traditional DEC concept of a “command line” is a simple string/array
of characters. For example, you see this in the “foreign” command buffer
in VMS, which can be passed from one program to another.

VMS really wants you to use a DCL-style syntax for your command lines. It
does weird things like uppercasing text in the command buffer, except for
parts in quotes, which are left as-is, but then the quotes are also left
in place, so command-line parsers have to worry about dealing with text
that might or might not be quoted.

And also implicit in all this is that the command will be executed by
sending a request to some CLI/shell.

Compare the Unix concept of a command line: it is not a “line” at all, but an array of strings. One program can directly execute another without
having to go through a shell: it can construct and pass this argument
array without having to worry about “special” characters that might be (mis)interpreted by some CLI/shell intermediary -- unless you choose to go through such an intermediary.

One might argue that DCL-style syntax is much more sophisticated than what
is supported by your typical *nix command-line program. But then again,
the sheer wealth of *nix command-line tools, compared to what is available
on VMS, shows that the *nix-style command line is far more versatile. In
*nix, most of the sophistication is delegated to the shell language that
is used to invoke these programs, while the programs themselves can
usually parse their arguments using a fairly modest amount of code, with
little or no need for some more sophisticated command-line-handling
library.

And just to add to the fun, CP/M, MS-DOS and then Windows NT all copied
the basic DEC-style command line architecture, while neglecting the
crucial part of a built-in command parser to handle more sophisticated
syntax. So Microsoft’s recent rediscovery that “the command line is cool” remains hampered by this brain-dead underlying architecture.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

And just to add to the fun, CP/M, MS-DOS and then Windows NT all copied
the basic DEC-style command line architecture, while neglecting the
crucial part of a built-in command parser to handle more sophisticated >syntax. So Microsoft’s recent rediscovery that “the command line is cool”
remains hampered by this brain-dead underlying architecture.

To give them credit, Microsoft finally realized that command.com was
hopeless and started from first principles with Powershell. Powershell
is easily as good as anything DEC had in 1974 and as such is a huge
advance for the desktop.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/2024 6:19 AM, Scott Dorsey wrote:

Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

And just to add to the fun, CP/M, MS-DOS and then Windows NT all copied
the basic DEC-style command line architecture, while neglecting the
crucial part of a built-in command parser to handle more sophisticated
syntax. So Microsoft’s recent rediscovery that “the command line is cool”
remains hampered by this brain-dead underlying architecture.

To give them credit, Microsoft finally realized that command.com was
hopeless and started from first principles with Powershell. Powershell
is easily as good as anything DEC had in 1974 and as such is a huge
advance for the desktop.

Just so that everyone know the history:

command.com - 1980
cmd.exe - 1987 (backwards compatible with command.com, but significantly extended functionality, often via obscure syntax though)
Windows Script Host / VBScript - 1995 (not interactive, scripts only,
but very powerful via WMI)
PowerShell - 2006

Not super new.

:-)

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 May 2024 09:28:16 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

On 5/23/2024 6:19 AM, Scott Dorsey wrote:

Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

And just to add to the fun, CP/M, MS-DOS and then Windows NT all
copied the basic DEC-style command line architecture, while
neglecting the crucial part of a built-in command parser to handle
more sophisticated syntax. So Microsoft’s recent rediscovery that
“the command line is cool” remains hampered by this brain-dead
underlying architecture.

To give them credit, Microsoft finally realized that command.com was hopeless and started from first principles with Powershell.
Powershell is easily as good as anything DEC had in 1974 and as
such is a huge advance for the desktop.

Just so that everyone know the history:

command.com - 1980
cmd.exe - 1987 (backwards compatible with command.com, but
significantly extended functionality, often via obscure syntax though)

Thanks.
Until reading your post I didn't realize that cmd.exe existed before Nt
3.1 (1993).
Personally I often feel that cmd.exe is an improvement on it's
successor.

Windows Script Host / VBScript - 1995 (not interactive, scripts only,
but very powerful via WMI)
PowerShell - 2006

Not super new.

:-)

Arne

BTW, I tried to figure out what "command line architecture" Lawrence is
talking about, but didn't quite succeed.
As far as Win32 CreateProcess() function is concerned, command line
is just a string. If caller omits applicationName parameter then
system does small amount of parsing to find application name in command
line. But when caller provides applicationName parameter, which is a
most logical thing to do in any non-interactive/non-shell scenario,
then OS does not care at all about content of command line. In both
cases OS does not modify command line and passes it to child
process as is.
So, the only difference between *nix and Windows is array of
ASCIIZ strings (today, of UTF8 strings) vs single, possibly very long, null-terminated ASCII or UTF16 string.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/2024 12:02 PM, Michael S wrote:

On Thu, 23 May 2024 09:28:16 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

On 5/23/2024 6:19 AM, Scott Dorsey wrote:

Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

And just to add to the fun, CP/M, MS-DOS and then Windows NT all
copied the basic DEC-style command line architecture, while
neglecting the crucial part of a built-in command parser to handle
more sophisticated syntax. So Microsoft’s recent rediscovery that
“the command line is cool” remains hampered by this brain-dead
underlying architecture.

To give them credit, Microsoft finally realized that command.com was
hopeless and started from first principles with Powershell.
Powershell is easily as good as anything DEC had in 1974 and as
such is a huge advance for the desktop.

Just so that everyone know the history:

command.com - 1980
cmd.exe - 1987 (backwards compatible with command.com, but
significantly extended functionality, often via obscure syntax though)

Thanks.
Until reading your post I didn't realize that cmd.exe existed before Nt
3.1 (1993).
Personally I often feel that cmd.exe is an improvement on it's
successor.

Windows Script Host / VBScript - 1995 (not interactive, scripts only,
but very powerful via WMI)
PowerShell - 2006

Not super new.

BTW, I tried to figure out what "command line architecture" Lawrence is talking about, but didn't quite succeed.

Neither did I.

As far as Win32 CreateProcess() function is concerned, command line
is just a string. If caller omits applicationName parameter then
system does small amount of parsing to find application name in command
line. But when caller provides applicationName parameter, which is a
most logical thing to do in any non-interactive/non-shell scenario,
then OS does not care at all about content of command line. In both
cases OS does not modify command line and passes it to child
process as is.

The concept is pretty common.

VMS LIB$SPAWN and C system also use full command line.

So, the only difference between *nix and Windows is array of
ASCIIZ strings (today, of UTF8 strings) vs single, possibly very long, null-terminated ASCII or UTF16 string.

In most practical usages it does not matter at all.

But it can matter if one needs to construct a single command
line from multiple arguments and those arguments contain
spaces or shell operators.

Example:
https://nvd.nist.gov/vuln/detail/CVE-2024-24576

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/24 17:41, Arne Vajhøj wrote:

On 5/23/2024 12:02 PM, Michael S wrote:

On Thu, 23 May 2024 09:28:16 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

On 5/23/2024 6:19 AM, Scott Dorsey wrote:

Lawrence D'Oliveiro  <ldo@nz.invalid> wrote:

And just to add to the fun, CP/M, MS-DOS and then Windows NT all
copied the basic DEC-style command line architecture, while
neglecting the crucial part of a built-in command parser to handle
more sophisticated syntax. So Microsoft’s recent rediscovery that
“the command line is cool” remains hampered by this brain-dead
underlying architecture.

To give them credit, Microsoft finally realized that command.com was
hopeless and started from first principles with Powershell.
Powershell is easily as good as anything DEC had in 1974 and as
such is a huge advance for the desktop.

Just so that everyone know the history:

command.com - 1980
cmd.exe - 1987 (backwards compatible with command.com, but
significantly extended functionality, often via obscure syntax though)

Thanks.
Until reading your post I didn't realize that cmd.exe existed before Nt
3.1 (1993).
Personally I often feel that cmd.exe is an improvement on it's
successor.

Windows Script Host / VBScript - 1995 (not interactive, scripts only,
but very powerful via WMI)
PowerShell - 2006

Not super new.

BTW, I tried to figure out what "command line architecture" Lawrence is
talking about, but didn't quite succeed.

Neither did I.

As far as Win32 CreateProcess() function is concerned, command line
is just a string. If caller omits applicationName parameter then
system does small amount of parsing to find application name in command
line. But when caller provides applicationName parameter, which is a
most logical thing to do in any non-interactive/non-shell scenario,
then OS does not care at all about content of command line. In both
cases OS does not modify command line and passes it to child
process as is.

The concept is pretty common.

VMS LIB$SPAWN and C system also use full command line.

So, the only difference between *nix and Windows is array of
ASCIIZ strings (today, of UTF8 strings) vs single, possibly very long,
null-terminated ASCII or UTF16 string.

In most practical usages it does not matter at all.

But it can matter if one needs to construct a single command
line from multiple arguments and those arguments contain
spaces or shell operators.

Example:
  https://nvd.nist.gov/vuln/detail/CVE-2024-24576

Arne

Unix shell just uses single quotes to enclose spaced arguments, which
are then removed by the shell, before storing the argument as pointer
to a null terminated array of char. Neat, clean and simple.

At parser level, there has to be some agreed token(s) to delimit the
command line arguments and spaces or tabs are still the simplest way
to handle that. Things like spaces in filenames are a bit of an
abomination, but thank uSoft for that. How would you handle it ?...

Chris

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/2024 12:16 AM, Lawrence D'Oliveiro wrote:

VMS really wants you to use a DCL-style syntax for your command lines. It does weird things like uppercasing text in the command buffer, except for parts in quotes, which are left as-is, but then the quotes are also left
in place, so command-line parsers have to worry about dealing with text
that might or might not be quoted.

That is an oversimplified view on a complex issue.

real: ABC def "GIH" "jkl"
parse=traditional:
CLI : ABC DEF GIH jkl
FC : ABC DEF "GIH" "jkl"
Mix : ABC DEF GIH jkl
C : abc def GIH jkl
DCL : ABC DEF GIH jkl
parse=extended:
CLI : ABC DEF GIH jkl
FC : ABC def "GIH" "jkl"
Mix : ABC DEF GIH jkl
C : ABC def GIH jkl
DCL : ABC DEF GIH jkl

For details see below.

Arne

$ type cli.for
program cli
integer*4 p1len,p2len,p3len,p4len
character*80 p1,p2,p3,p4
call cli$get_value('p1',p1,p1len)
call cli$get_value('p2',p2,p2len)
call cli$get_value('p3',p3,p3len)
call cli$get_value('p4',p4,p4len)
write(*,*) 'CLI : '//p1(1:p1len)//' '//p2(1:p2len)//' '
+ //p3(1:p3len)//' '//p4(1:p4len)
end
$ type cli.cld
define verb cli
image "sys$disk:[]cli.exe"
parameter p1
parameter p2
parameter p3
parameter p4
$ for cli
$ link cli
$ set comm cli.cld
$ type mix.for
program mix
integer*4 cmdlen
character*80 cmd
integer*4 p1len,p2len,p3len,p4len
character*80 p1,p2,p3,p4
external mixcld
call lib$get_foreign(cmd,,cmdlen)
call cli$dcl_parse('cli '//cmd(1:cmdlen),mixcld,,,)
call cli$get_value('p1',p1,p1len)
call cli$get_value('p2',p2,p2len)
call cli$get_value('p3',p3,p3len)
call cli$get_value('p4',p4,p4len)
write(*,*) 'Mix : '//p1(1:p1len)//' '//p2(1:p2len)//' '
+ //p3(1:p3len)//' '//p4(1:p4len)
end
$ type mixcld.cld
module mixcld
define verb cli
parameter p1
parameter p2
parameter p3
parameter p4
$ for mix
$ set command/object mixcld
$ link mix + mixcld
$ type fc.for
program fc
integer*4 cmdlen
character*80 cmd
call lib$get_foreign(cmd,,cmdlen)
write(*,*) 'FC : '//cmd(1:cmdlen)
end
$ for fc
$ link fc
$ type c.c
#include <stdio.h>

int main(int argc, char *argv[])
{
printf("C :");
for(int i = 1; i < argc; i++) printf(" %s", argv[i]);
printf("\n");
return 0;
}
$ cc c
$ link c
$ type dcl.com
$ write sys$output "DCL : ''p1' ''p2' ''p3' ''p4'"

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/2024 12:16 AM, Lawrence D'Oliveiro wrote:

And also implicit in all this is that the command will be executed by
sending a request to some CLI/shell.

You can start a process with no CLI. SYS$CREPC or
some $ RUN commands.

But yes if you want to use command line arguments then
you need a command line interface present.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 May 2024 18:37:35 +0100
chrisq <devzero@nospam.com> wrote:

On 5/23/24 17:41, Arne Vajhøj wrote:

On 5/23/2024 12:02 PM, Michael S wrote:

On Thu, 23 May 2024 09:28:16 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

On 5/23/2024 6:19 AM, Scott Dorsey wrote:

Lawrence D'Oliveiro  <ldo@nz.invalid> wrote:

And just to add to the fun, CP/M, MS-DOS and then Windows NT all
copied the basic DEC-style command line architecture, while
neglecting the crucial part of a built-in command parser to
handle more sophisticated syntax. So Microsoft’s recent
rediscovery that “the command line is cool” remains hampered by >>>>> this brain-dead underlying architecture.

To give them credit, Microsoft finally realized that command.com
was hopeless and started from first principles with Powershell.
Powershell is easily as good as anything DEC had in 1974 and as
such is a huge advance for the desktop.

Just so that everyone know the history:

command.com - 1980
cmd.exe - 1987 (backwards compatible with command.com, but
significantly extended functionality, often via obscure syntax
though)

Thanks.
Until reading your post I didn't realize that cmd.exe existed
before Nt 3.1 (1993).
Personally I often feel that cmd.exe is an improvement on it's
successor.

Windows Script Host / VBScript - 1995 (not interactive, scripts
only, but very powerful via WMI)
PowerShell - 2006

Not super new.

BTW, I tried to figure out what "command line architecture"
Lawrence is talking about, but didn't quite succeed.

Neither did I.

As far as Win32 CreateProcess() function is concerned, command line
is just a string. If caller omits applicationName parameter then
system does small amount of parsing to find application name in
command line. But when caller provides applicationName parameter,
which is a most logical thing to do in any
non-interactive/non-shell scenario, then OS does not care at all
about content of command line. In both cases OS does not modify
command line and passes it to child process as is.

The concept is pretty common.

VMS LIB$SPAWN and C system also use full command line.

So, the only difference between *nix and Windows is array of
ASCIIZ strings (today, of UTF8 strings) vs single, possibly very
long, null-terminated ASCII or UTF16 string.

In most practical usages it does not matter at all.

But it can matter if one needs to construct a single command
line from multiple arguments and those arguments contain
spaces or shell operators.

Example:
  https://nvd.nist.gov/vuln/detail/CVE-2024-24576

Arne

Unix shell just uses single quotes to enclose spaced arguments, which
are then removed by the shell, before storing the argument as pointer
to a null terminated array of char. Neat, clean and simple.

Moving pointers between address spaces is never very simple. Moving variable-length arrays of pointers adds another level of complexity on
top of it.
Not a rocket science, but not as simple as copying a single monolithic
string.

At parser level, there has to be some agreed token(s) to delimit the
command line arguments and spaces or tabs are still the simplest way
to handle that. Things like spaces in filenames are a bit of an
abomination, but thank uSoft for that. How would you handle it ?...

Chris

Microsoft's own conventions are documented here: https://learn.microsoft.com/en-us/cpp/c-language/parsing-c-command-line-arguments?view=msvc-170
They provide default parser https://learn.microsoft.com/en-us/windows/win32/api/shellapi/nf-shellapi-commandlinetoargvw
And, of course, they have it built-in in C start-up library code.

But when you have control over both parent and child processes, you do
not have to follow Microsoft's conventions if you prefer some different
way.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 May 2024 18:37:35 +0100, chrisq wrote:

Unix shell just uses single quotes to enclose spaced arguments, which
are then removed by the shell, before storing the argument as pointer to
a null terminated array of char. Neat, clean and simple.

Actually, single quotes OR double quotes.

The difference is that single quotes prevent further interpretation of the arguments, and double quotes don't (so that you can have shell variables
in there, for instance)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 May 2024 15:05:42 -0400, Arne Vajhøj wrote:

On 5/23/2024 12:16 AM, Lawrence D'Oliveiro wrote:

VMS really wants you to use a DCL-style syntax for your command lines.
It does weird things like uppercasing text in the command buffer,
except for parts in quotes, which are left as-is, but then the quotes
are also left in place, so command-line parsers have to worry about
dealing with text that might or might not be quoted.

That is an oversimplified view on a complex issue.

What you have done is rely on library routines to impose a commonality on
the underlying raw command buffer contents. And those library routines are CLI-specific, as I recall; notwithstanding their “cli$” name prefix, they are part of DCL itself.

Like I said, on *nix systems, there is no automatic assumption that you
need the intervention of a CLI/shell just for one program to execute
another.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 May 2024 19:02:39 +0300, Michael S wrote:

As far as Win32 CreateProcess() function is concerned, command line is
just a string.

Which is what I said.

If caller omits applicationName parameter then system
does small amount of parsing to find application name in command line.

This is already starting to set off alarm bells ...

But when caller provides applicationName parameter, which is a most
logical thing to do in any non-interactive/non-shell scenario, then OS
does not care at all about content of command line.

There is an issue if the program being executed is CMD.EXE itself. On
*nix, the same array-of-strings rule applies to a shell as to any other program, but on Windows, the shell has to do its own parsing of the
command line. There was a security vulnerability recently that was related
to this: the vulnerability was present only on Windows, because of this (mis)feature, not on *nix.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/24 2:05 PM, Arne Vajhøj wrote:

On 5/23/2024 12:16 AM, Lawrence D'Oliveiro wrote:

VMS really wants you to use a DCL-style syntax for your command lines. It
does weird things like uppercasing text in the command buffer, except for
parts in quotes, which are left as-is, but then the quotes are also left
in place, so command-line parsers have to worry about dealing with text
that might or might not be quoted.

That is an oversimplified view on a complex issue.

real: ABC def "GIH" "jkl"
parse=traditional:
CLI : ABC DEF GIH jkl
FC  : ABC DEF "GIH" "jkl"
Mix : ABC DEF GIH jkl
C   : abc def GIH jkl
DCL : ABC DEF GIH jkl
parse=extended:
CLI : ABC DEF GIH jkl
FC  : ABC def "GIH" "jkl"
Mix : ABC DEF GIH jkl
C   : ABC def GIH jkl
DCL : ABC DEF GIH jkl

For details see below.

Arne

$ type cli.for
      program cli
      integer*4 p1len,p2len,p3len,p4len
      character*80 p1,p2,p3,p4
      call cli$get_value('p1',p1,p1len)
      call cli$get_value('p2',p2,p2len)
      call cli$get_value('p3',p3,p3len)
      call cli$get_value('p4',p4,p4len)
      write(*,*) 'CLI : '//p1(1:p1len)//' '//p2(1:p2len)//' '
     +                   //p3(1:p3len)//' '//p4(1:p4len)
      end
$ type cli.cld
define verb cli
   image "sys$disk:[]cli.exe"
   parameter p1
   parameter p2
   parameter p3
   parameter p4
$ for cli
$ link cli
$ set comm cli.cld
$ type mix.for
      program mix
      integer*4 cmdlen
      character*80 cmd
      integer*4 p1len,p2len,p3len,p4len
      character*80 p1,p2,p3,p4
      external mixcld
      call lib$get_foreign(cmd,,cmdlen)
      call cli$dcl_parse('cli '//cmd(1:cmdlen),mixcld,,,)
      call cli$get_value('p1',p1,p1len)
      call cli$get_value('p2',p2,p2len)
      call cli$get_value('p3',p3,p3len)
      call cli$get_value('p4',p4,p4len)
      write(*,*) 'Mix : '//p1(1:p1len)//' '//p2(1:p2len)//' '
     +                   //p3(1:p3len)//' '//p4(1:p4len)
      end
$ type mixcld.cld
module mixcld
define verb cli
   parameter p1
   parameter p2
   parameter p3
   parameter p4
$ for mix
$ set command/object mixcld
$ link mix + mixcld
$ type fc.for
      program fc
      integer*4 cmdlen
      character*80 cmd
      call lib$get_foreign(cmd,,cmdlen)
      write(*,*) 'FC  : '//cmd(1:cmdlen)
      end
$ for fc
$ link fc
$ type c.c
#include <stdio.h>

int main(int argc, char *argv[])
{
    printf("C   :");
    for(int i = 1; i < argc; i++) printf(" %s", argv[i]);
    printf("\n");
    return 0;
}
$ cc c
$ link c
$ type dcl.com
$ write sys$output "DCL : ''p1' ''p2' ''p3' ''p4'"

Good summary. Did you have DECC$ARGV_PARSE_STYLE defined for the C example?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 May 2024 18:37:35 +0100, chrisq wrote:

Unix shell just uses single quotes to enclose spaced arguments, which
are then removed by the shell, before storing the argument as pointer to
a null terminated array of char. Neat, clean and simple.

Actually, there’s a bit more to it than that, if you want to avoid the pitfalls.

Which is why it is often so much simpler just to execute a program
directly, without going through a shell, if you don’t need the
capabilities of the shell language as such.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 23 May 2024 21:28:23 GMT, Bob Eager wrote:

The difference is that single quotes prevent further interpretation of
the arguments, and double quotes don't (so that you can have shell
variables in there, for instance)

And what happens if you have no quotes at all?

Then the variable value is subject to word-splitting, which it is not if
it is within double quotes.

The nice thing is, that’s all there is to the variable-substitution rules. “Special” characters in variable values have no special significance.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 May 2024 12:41:24 -0400, Arne Vajhøj wrote:

Example:
https://nvd.nist.gov/vuln/detail/CVE-2024-24576

Yup, that sounds like the one. Got a full 10 out of 10 vulnerability
score, as I recall.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 May 2024 15:14:36 -0400, Arne Vajhøj wrote:

But yes if you want to use command line arguments then you need a
command line interface present.

That leads to its own share of bugs, because you need to make sure to
properly escape (or avoid, if you cannot escape) characts that might have special meaning to the CLI.

And failure to do this can lead to security vulnerabilities.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/2024 6:32 PM, Craig A. Berry wrote:

On 5/23/24 2:05 PM, Arne Vajhøj wrote:

On 5/23/2024 12:16 AM, Lawrence D'Oliveiro wrote:

VMS really wants you to use a DCL-style syntax for your command
lines. It
does weird things like uppercasing text in the command buffer, except
for
parts in quotes, which are left as-is, but then the quotes are also left >>> in place, so command-line parsers have to worry about dealing with text
that might or might not be quoted.

That is an oversimplified view on a complex issue.

real: ABC def "GIH" "jkl"
parse=traditional:
CLI : ABC DEF GIH jkl
FC  : ABC DEF "GIH" "jkl"
Mix : ABC DEF GIH jkl
C   : abc def GIH jkl
DCL : ABC DEF GIH jkl
parse=extended:
CLI : ABC DEF GIH jkl
FC  : ABC def "GIH" "jkl"
Mix : ABC DEF GIH jkl
C   : ABC def GIH jkl
DCL : ABC DEF GIH jkl

Good summary.  Did you have DECC$ARGV_PARSE_STYLE defined for the C
example?

Looks that way:

$ sh log *pars*

(LNM$PROCESS_TABLE)

"DECC$ARGV_PARSE_STYLE" = "enable"

Note that I have not set it explicit - something
has set it for me.

A little investigation reveals that it is Java:

$ sear disk0:[sys0.syscommon.openjdk$80.com]java$setup.com decc
$ define'defopt' decc$fd_locking true
$ define'defopt' decc$efs_case_preserve enable ! Enable ODS-5 names
$ define'defopt' decc$readdir_dropdotnotype enable
$ define'defopt' decc$efs_charset enable
$ define'defopt' decc$argv_parse_style enable

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/2024 6:36 PM, Lawrence D'Oliveiro wrote:

On Thu, 23 May 2024 15:05:42 -0400, Arne Vajhøj wrote:

On 5/23/2024 12:16 AM, Lawrence D'Oliveiro wrote:

VMS really wants you to use a DCL-style syntax for your command lines.
It does weird things like uppercasing text in the command buffer,
except for parts in quotes, which are left as-is, but then the quotes
are also left in place, so command-line parsers have to worry about
dealing with text that might or might not be quoted.

That is an oversimplified view on a complex issue.

What you have done is rely on library routines to impose a commonality on
the underlying raw command buffer contents.

Then you have not read it careful enough. The point is that different approaches have different behavior - the exact opposite of impose
commonality.

And those library routines are CLI-specific,

I doubt that. I would expect any proper CLI on VMS to work with them

as I recall; notwithstanding their “cli$” name prefix, they are part of DCL itself.

No. They are utility routines.

And they reside in STARLET.OLB module CLI$INTERFACE.

Like I said, on *nix systems, there is no automatic assumption that you
need the intervention of a CLI/shell just for one program to execute
another.

On *nix there is little difference between a shell and another
program. Huge difference on VMS.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/2024 7:36 PM, Arne Vajhøj wrote:

On 5/23/2024 6:36 PM, Lawrence D'Oliveiro wrote:

                                              And those library
routines are
CLI-specific,

I doubt that. I would expect any proper CLI on VMS to work with them

Anyone running VMS 4.x could verify by logging in
with /CLI=MCR and try running a program calling
CLI$ routines.

as I recall; notwithstanding their “cli$” name prefix, they
are part of DCL itself.

No. They are utility routines.

And they reside in STARLET.OLB module CLI$INTERFACE.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 24/05/2024 00:22, Arne Vajhøj wrote:

On 5/23/2024 6:32 PM, Craig A. Berry wrote:

On 5/23/24 2:05 PM, Arne Vajhøj wrote:

<snip>

Arne

Why do you all feed the troll/Village Idiot?

--
Chris

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 May 2024 19:36:49 -0400, Arne Vajhøj wrote:

On 5/23/2024 6:36 PM, Lawrence D'Oliveiro wrote:

as I recall; notwithstanding their “cli$” name prefix, they
are part of DCL itself.

No. They are utility routines.

And they reside in STARLET.OLB module CLI$INTERFACE.

I can’t see them in the VMSRTL map files, at least as of VMS 4.0 <http://bitsavers.trailing-edge.com/pdf/dec/vax/microfiche/vms-source-listings/AH-BT13A-SE__VAX-VMS_V4.0_SRC_LST_MCRF/AH-BT13A-SE__VAX-VMS_V4.0_SRC_LST_MCRF/109__VMSRTL/>.

I’m pretty sure you’ll find that the symbols, in whatever libraries
you find them in, are calls into DCL.

On *nix there is little difference between a shell and another program.
Huge difference on VMS.

Which is part of my point.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/24 6:22 PM, Arne Vajhøj wrote:

On 5/23/2024 6:32 PM, Craig A. Berry wrote:

On 5/23/24 2:05 PM, Arne Vajhøj wrote:

real: ABC def "GIH" "jkl"
parse=extended:
C   : ABC def GIH jkl

Good summary.  Did you have DECC$ARGV_PARSE_STYLE defined for the C
example?

Looks that way:

$ sh log *pars*

(LNM$PROCESS_TABLE)

  "DECC$ARGV_PARSE_STYLE" = "enable"

Note that I have not set it explicit - something
has set it for me.

I think if you deassign it you'll get abc not ABC.

A little investigation reveals that it is Java:

$ sear disk0:[sys0.syscommon.openjdk$80.com]java$setup.com decc
$ define'defopt' decc$fd_locking true
$ define'defopt' decc$efs_case_preserve enable  ! Enable ODS-5 names
$ define'defopt' decc$readdir_dropdotnotype enable
$ define'defopt' decc$efs_charset enable
$ define'defopt' decc$argv_parse_style enable

Yeah, I believe this has been fixed so OpenJDK now sets what it needs in
in a LIB$INITIALIZE routine, but they left the startup command
procedures as-is.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/2024 7:38 AM, chrisq wrote:

Perhaps, but from a programmers point of view, the elegance of the shell command line method is that on entry to main(), both the arg count and
args are separated and ready to verify and use. All the background
legwork is done for you.

If you just hand the whole command line to a program, you have to
parse and separate the args, for every program you write, very bug
prone and added effort. Whole idea of good system design is to free
the programmer from repetition, and mundane stuff that the system
should deal with...

I don't think it is a big problem on the consuming side - the
problem is on the producing side.

A lot of the most popular programming languages provides
split arguments to the program also on operating systems where
command line is considered a single string.

And for other languages the operating system likely provide
functionality to do the split. On VMS it is the CLI$
routines. Michael linked to CommandLineToArgv for
Windows.

Of course developers can still fuck it up with
a DIY solution, but if people choose DIY over what
is already provided for them, then they chose.

It is the other direction that is more tricky. You have
N separated arguments and need to produce a command
line that will actually be split into the same N
arguments. Not a super common requirement. But tricky
to get 100% correct.

As illustrated by the Rust issue.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/24 20:38, Michael S wrote:

<snipped>

Moving pointers between address spaces is never very simple. Moving variable-length arrays of pointers adds another level of complexity on
top of it.
Not a rocket science, but not as simple as copying a single monolithic string.

Perhaps, but from a programmers point of view, the elegance of the shell command line method is that on entry to main(), both the arg count and
args are separated and ready to verify and use. All the background
legwork is done for you.

If you just hand the whole command line to a program, you have to
parse and separate the args, for every program you write, very bug
prone and added effort. Whole idea of good system design is to free
the programmer from repetition, and mundane stuff that the system
should deal with...

Chris

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/23/2024 9:46 PM, Craig A. Berry wrote:

On 5/23/24 6:22 PM, Arne Vajhøj wrote:

On 5/23/2024 6:32 PM, Craig A. Berry wrote:

On 5/23/24 2:05 PM, Arne Vajhøj wrote:

real: ABC def "GIH" "jkl"
parse=extended:
C   : ABC def GIH jkl

Good summary.  Did you have DECC$ARGV_PARSE_STYLE defined for the C
example?

Looks that way:

$ sh log *pars*

(LNM$PROCESS_TABLE)

   "DECC$ARGV_PARSE_STYLE" = "enable"

I think if you deassign it you'll get abc not ABC.

Confirmed.

real: ABC def "GIH" "jkl"
parse=traditional:
CLI : ABC DEF GIH jkl
FC : ABC DEF "GIH" "jkl"
Mix : ABC DEF GIH jkl
C : abc def GIH jkl (DECC$ARGV_PARSE_STYLE = (null))
C : abc def GIH jkl (DECC$ARGV_PARSE_STYLE = enable)
DCL : ABC DEF GIH jkl
parse=extended:
CLI : ABC DEF GIH jkl
FC : ABC def "GIH" "jkl"
Mix : ABC DEF GIH jkl
C : abc def GIH jkl (DECC$ARGV_PARSE_STYLE = (null))
C : ABC def GIH jkl (DECC$ARGV_PARSE_STYLE = enable)
DCL : ABC DEF GIH jkl

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
printf("C :");
for(int i = 1; i < argc; i++) printf(" %s", argv[i]);
printf(" (DECC$ARGV_PARSE_STYLE = %s)\n", getenv("DECC$ARGV_PARSE_STYLE"));
return 0;
}

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 24 May 2024 08:13:04 -0400
Arne Vajh�j <arne@vajhoej.dk> wrote:

As illustrated by the Rust issue.

I was not able to figure out what exactly Rust guys were trying to
achieve. Feeding cmd.exe with command line from untrusted source and
expecting no harm sounds like mission impossible.
That is, impossible when you run cmd.exe under privileged account.
It is possible when you run it under sufficiently deprived account, but
that is orthogonal to parsing of command line.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/2024 9:32 AM, Michael S wrote:

On Fri, 24 May 2024 08:13:04 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

As illustrated by the Rust issue.

I was not able to figure out what exactly Rust guys were trying to
achieve. Feeding cmd.exe with command line from untrusted source and expecting no harm sounds like mission impossible.
That is, impossible when you run cmd.exe under privileged account.
It is possible when you run it under sufficiently deprived account, but
that is orthogonal to parsing of command line.

To my very limited understanding then the problem was that:

Command::new("program").arg("a1").arg("a2")

ended up as:

program a1 a2

not:

program "a1" "a2"

which is fine but is also meant that:

Command::new("program").arg("a1").arg("a2 x y z")

ended up as:

program a1 a2 x y z

not:

program "a1" "a2 x y z"

which is not so fine.

It is definitely a functional problem.

And if the security depends on program treating the two
arguments securely, then it is also a security problem.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 24 May 2024 09:42:49 -0400
Arne Vajh�j <arne@vajhoej.dk> wrote:

On 5/24/2024 9:32 AM, Michael S wrote:

On Fri, 24 May 2024 08:13:04 -0400
Arne Vajh�j <arne@vajhoej.dk> wrote:

As illustrated by the Rust issue.

I was not able to figure out what exactly Rust guys were trying to
achieve. Feeding cmd.exe with command line from untrusted source and expecting no harm sounds like mission impossible.
That is, impossible when you run cmd.exe under privileged account.
It is possible when you run it under sufficiently deprived account,
but that is orthogonal to parsing of command line.

To my very limited understanding then the problem was that:

Command::new("program").arg("a1").arg("a2")

ended up as:

program a1 a2

not:

program "a1" "a2"

which is fine but is also meant that:

Command::new("program").arg("a1").arg("a2 x y z")

ended up as:

program a1 a2 x y z

not:

program "a1" "a2 x y z"

which is not so fine.

It is definitely a functional problem.

And if the security depends on program treating the two
arguments securely, then it is also a security problem.

Arne

It seems like the simplest solution is to not try to run batch files by
means of spawn("cmd.exe", ...) or CreateProcess("cmd.exe", ...).
They could have use more specialized function: system() from C RTL or ShellExecuteEx() from Win32 API. The former is easier to use, the later
works as expected in wider range of host console environments, most importantly, it works from mintty.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/2024 11:11 AM, Michael S wrote:

On Fri, 24 May 2024 09:42:49 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

On 5/24/2024 9:32 AM, Michael S wrote:

On Fri, 24 May 2024 08:13:04 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

As illustrated by the Rust issue.

I was not able to figure out what exactly Rust guys were trying to
achieve. Feeding cmd.exe with command line from untrusted source and
expecting no harm sounds like mission impossible.
That is, impossible when you run cmd.exe under privileged account.
It is possible when you run it under sufficiently deprived account,
but that is orthogonal to parsing of command line.

To my very limited understanding then the problem was that:

Command::new("program").arg("a1").arg("a2")

ended up as:

program a1 a2

not:

program "a1" "a2"

which is fine but is also meant that:

Command::new("program").arg("a1").arg("a2 x y z")

ended up as:

program a1 a2 x y z

not:

program "a1" "a2 x y z"

which is not so fine.

It is definitely a functional problem.

And if the security depends on program treating the two
arguments securely, then it is also a security problem.

It seems like the simplest solution is to not try to run batch files by
means of spawn("cmd.exe", ...) or CreateProcess("cmd.exe", ...).
They could have use more specialized function: system() from C RTL or ShellExecuteEx() from Win32 API. The former is easier to use, the later
works as expected in wider range of host console environments, most importantly, it works from mintty.

Both system and ShellExecuteEx still take all parameters as a single
string, which require some non-trivial conversion from array of
parameters to that string.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 24 May 2024 11:28:05 -0400
Arne Vajh�j <arne@vajhoej.dk> wrote:

On 5/24/2024 11:11 AM, Michael S wrote:

On Fri, 24 May 2024 09:42:49 -0400
Arne Vajh�j <arne@vajhoej.dk> wrote:

On 5/24/2024 9:32 AM, Michael S wrote:

On Fri, 24 May 2024 08:13:04 -0400
Arne Vajh�j <arne@vajhoej.dk> wrote:

As illustrated by the Rust issue.

I was not able to figure out what exactly Rust guys were trying to
achieve. Feeding cmd.exe with command line from untrusted source
and expecting no harm sounds like mission impossible.
That is, impossible when you run cmd.exe under privileged account.
It is possible when you run it under sufficiently deprived
account, but that is orthogonal to parsing of command line.

To my very limited understanding then the problem was that:

Command::new("program").arg("a1").arg("a2")

ended up as:

program a1 a2

not:

program "a1" "a2"

which is fine but is also meant that:

Command::new("program").arg("a1").arg("a2 x y z")

ended up as:

program a1 a2 x y z

not:

program "a1" "a2 x y z"

which is not so fine.

It is definitely a functional problem.

And if the security depends on program treating the two
arguments securely, then it is also a security problem.

It seems like the simplest solution is to not try to run batch
files by means of spawn("cmd.exe", ...) or CreateProcess("cmd.exe",
...). They could have use more specialized function: system() from
C RTL or ShellExecuteEx() from Win32 API. The former is easier to
use, the later works as expected in wider range of host console environments, most importantly, it works from mintty.

Both system and ShellExecuteEx still take all parameters as a single
string, which require some non-trivial conversion from array of
parameters to that string.

Arne

Conversion is actually trivial - dumb double-quoting.
There is an opposite problem - the child will get parameters together
with double quotes.
But at least the biggest of their problems (complicated undocumented
rules of cmd.exe) goes away. The rule, applied by CommandLineToArgv()
are non-complicated and documented.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/24 10:28 AM, Arne Vajhøj wrote:

On 5/24/2024 11:11 AM, Michael S wrote:

On Fri, 24 May 2024 09:42:49 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

On 5/24/2024 9:32 AM, Michael S wrote:

On Fri, 24 May 2024 08:13:04 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

As illustrated by the Rust issue.

I was not able to figure out what exactly Rust guys were trying to
achieve. Feeding cmd.exe with command line from untrusted source and
expecting no harm sounds like mission impossible.
That is, impossible when you run cmd.exe under privileged account.
It is possible when you run it under sufficiently deprived account,
but that is orthogonal to parsing of command line.

To my very limited understanding then the problem was that:

Command::new("program").arg("a1").arg("a2")

ended up as:

program a1 a2

not:

program "a1" "a2"

which is fine but is also meant that:

Command::new("program").arg("a1").arg("a2 x y z")

ended up as:

program a1 a2 x y z

not:

program "a1" "a2 x y z"

which is not so fine.

It is definitely a functional problem.

And if the security depends on program treating the two
arguments securely, then it is also a security problem.

It seems like the simplest solution is to not try to run batch files by
means of spawn("cmd.exe", ...) or CreateProcess("cmd.exe", ...).
They could have use more specialized function: system() from C RTL or
ShellExecuteEx() from Win32 API. The former is easier to use, the later
works as expected in wider range of host console environments, most
importantly, it works from mintty.

Both system and ShellExecuteEx still take all parameters as a single
string, which require some non-trivial conversion from array of
parameters to that string.

The Windows CRT has _spawn() [1] which looks pretty similar to
posix_spawn() [2]. With either one you pass arguments or an array of
arguments rather than a complete command line. From the docs it sounds
like on Windows the arguments get concatenated under the hood, but at
least "someone else" is doing that rather than each program having to
take responsibility for it.

There was a mention sometime in the last couple of years that
posix_spawn() is being added to the VMS CRTL.

[1] https://learn.microsoft.com/en-us/cpp/c-runtime-library/spawn-wspawn-functions?view=msvc-170

[2] https://pubs.opengroup.org/onlinepubs/9699919799.2013edition/functions/posix_spawn.html

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 24 May 2024 18:11:25 +0300, Michael S wrote:

It seems like the simplest solution is to not try to run batch files ...

That was part of the problem. Remember that, in Windows, if you pass a
command with no extension, it tries appending things like .BAT and .EXE,
to try to find a match. So you can’t really be sure beforehand that the command will *not* run a batch file.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 24 May 2024 16:32:25 +0300, Michael S wrote:

I was not able to figure out what exactly Rust guys were trying to
achieve.

They were trying to achieve the same result you get on a *nix system with essentially no work at all.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/2024 12:05 PM, Michael S wrote:

On Fri, 24 May 2024 11:28:05 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

On 5/24/2024 11:11 AM, Michael S wrote:

It seems like the simplest solution is to not try to run batch
files by means of spawn("cmd.exe", ...) or CreateProcess("cmd.exe",
...). They could have use more specialized function: system() from
C RTL or ShellExecuteEx() from Win32 API. The former is easier to
use, the later works as expected in wider range of host console
environments, most importantly, it works from mintty.

Both system and ShellExecuteEx still take all parameters as a single
string, which require some non-trivial conversion from array of
parameters to that string.

Conversion is actually trivial - dumb double-quoting.

Maybe.

There is an opposite problem - the child will get parameters together
with double quotes.

They should be stripped by the shell.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/2024 12:45 PM, Craig A. Berry wrote:

On 5/24/24 10:28 AM, Arne Vajhøj wrote:

On 5/24/2024 11:11 AM, Michael S wrote:

It seems like the simplest solution is to not try to run batch files by
means of spawn("cmd.exe", ...) or CreateProcess("cmd.exe", ...).
They could have use more specialized function: system() from C RTL or
ShellExecuteEx() from Win32 API. The former is easier to use, the later
works as expected in wider range of host console environments, most
importantly, it works from mintty.

Both system and ShellExecuteEx still take all parameters as a single
string, which require some non-trivial conversion from array of
parameters to that string.

The Windows CRT has _spawn() [1] which looks pretty similar to
posix_spawn() [2].  With either one you pass arguments or an array of arguments rather than a complete command line.  From the docs it sounds
like on Windows the arguments get concatenated under the hood, but at
least "someone else" is doing that rather than each program having to
take responsibility for it.

[1]

https://learn.microsoft.com/en-us/cpp/c-runtime-library/spawn-wspawn-functions?view=msvc-170

That looked very promising.

But a test did keep the optimism.

parmdump.c:

#include <stdio.h>

int main(int argc, char *argv[])
{
for(int i = 0; i < argc; i++)
{
printf("%d : %s\n", i, argv[i]);
}
return 0;
}

demo.c:

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <process.h>

void bad(const char *a1, const char *a2)
{
printf("Bad\n");
char cmd[1000];
snprintf(cmd, sizeof(cmd), "parmdump %s %s", a1, a2);
system(cmd);
}

void mystrcat(char *to, const char *from)
{
int ix = strlen(to);
for(int i = 0; i <= strlen(from); i++)
{
if(from[i] == '"')
{
to[ix] = from[i];
ix++;
to[ix] = from[i];
ix++;
}
else
{
to[ix] = from[i];
ix++;
}
}
}

void hack(const char *a1, const char *a2)
{
char *pto, *pfrom;
printf("Hack\n");
char cmd[1000];
strcpy(cmd, "parmdump \"");
mystrcat(cmd, a1);
strcat(cmd, "\" \"");
mystrcat(cmd, a2);
strcat(cmd, "\"");
system(cmd);
}

void latest_l(const char *a1, const char *a2)
{
printf("Latest (l)\n");
_spawnl(_P_WAIT, "parmdump", a1, a2, NULL);
}

void latest_v(const char *a1, const char *a2)
{
const char *argv[] = { a1, a2, NULL };
printf("Latest (v)\n");
_spawnv(_P_WAIT, "parmdump", argv);
}

void test(const char *a1, const char *a2)
{
bad(a1, a2);
hack(a1, a2);
latest_l(a1, a2);
latest_v(a1, a2);
}

int main()
{
test("a", "b");
test("a", "b c d");
test("a", "b\"c");
return 0;
}

Output:

Bad
0 : parmdump
1 : a
2 : b
Hack
0 : parmdump
1 : a
2 : b
Latest (l)
0 : a
1 : b
Latest (v)
0 : a
1 : b
Bad
0 : parmdump
1 : a
2 : b
3 : c
4 : d
Hack
0 : parmdump
1 : a
2 : b c d
Latest (l)
0 : a
1 : b
2 : c
3 : d
Latest (v)
0 : a
1 : b
2 : c
3 : d
Bad
0 : parmdump
1 : a
2 : bc
Hack
0 : parmdump
1 : a
2 : b"c
Latest (l)
0 : a
1 : bc
Latest (v)
0 : a
1 : bc

Unless I am doing something wrong then these _spawn functions still
expect the caller to handle the proper quoting to avoid problems.

There was a mention sometime in the last couple of years that
posix_spawn() is being added to the VMS CRTL.

[2] https://pubs.opengroup.org/onlinepubs/9699919799.2013edition/functions/posix_spawn.html

I don't see it in 9.2-2.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/2024 8:20 PM, Lawrence D'Oliveiro wrote:

On Fri, 24 May 2024 19:05:40 +0300, Michael S wrote:

Conversion is actually trivial - dumb double-quoting.

What do you do if the value has quotes?

1 double-quote -> 2 double-quotes

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 24 May 2024 19:05:40 +0300, Michael S wrote:

Conversion is actually trivial - dumb double-quoting.

What do you do if the value has quotes?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 24 May 2024 20:27:38 -0400, Arne Vajhøj wrote:

On 5/24/2024 8:20 PM, Lawrence D'Oliveiro wrote:

On Fri, 24 May 2024 19:05:40 +0300, Michael S wrote:

Conversion is actually trivial - dumb double-quoting.

What do you do if the value has quotes?

1 double-quote -> 2 double-quotes

Assuming the receiving end implements the same convention. Otherwise you
have trouble.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/2024 8:42 PM, Lawrence D'Oliveiro wrote:

On Fri, 24 May 2024 20:27:38 -0400, Arne Vajhøj wrote:

On 5/24/2024 8:20 PM, Lawrence D'Oliveiro wrote:

On Fri, 24 May 2024 19:05:40 +0300, Michael S wrote:

Conversion is actually trivial - dumb double-quoting.

What do you do if the value has quotes?

1 double-quote -> 2 double-quotes

Assuming the receiving end implements the same convention. Otherwise you
have trouble.

????

The receiving end does not see any of the added double quotes.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/2024 8:58 PM, Arne Vajhøj wrote:

On 5/24/2024 8:42 PM, Lawrence D'Oliveiro wrote:

On Fri, 24 May 2024 20:27:38 -0400, Arne Vajhøj wrote:

On 5/24/2024 8:20 PM, Lawrence D'Oliveiro wrote:

On Fri, 24 May 2024 19:05:40 +0300, Michael S wrote:

Conversion is actually trivial - dumb double-quoting.

What do you do if the value has quotes?

1 double-quote -> 2 double-quotes

Assuming the receiving end implements the same convention. Otherwise you
have trouble.

????

The receiving end does not see any of the added double quotes.

Windows use ' -> "" and Linux (bash) use " -> \".

But receiver does not see any of it.

Windows:

C:\Work\Python>type arg.py
from sys import argv

for i in range(len(argv)):
print('%d : %s' % (i, argv[i]))

C:\Work\Python>python arg.py a "b c" "d""e"
0 : arg.py
1 : a
2 : b c
3 : d"e

Linux:

arne@arnepc6:~$ cat arg.py
from sys import argv

for i in range(len(argv)):
print('%d : %s' % (i, argv[i]))
arne@arnepc6:~$ python3 arg.py a "b c" "d\"e"
0 : arg.py
1 : a
2 : b c
3 : d"e

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/24/2024 9:11 PM, Arne Vajhøj wrote:

Windows use ' -> "" and Linux (bash) use " -> \".

Windows use " -> "" and Linux (bash) use " -> \".

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 24 May 2024 20:58:23 -0400, Arne Vajhøj wrote:

On 5/24/2024 8:42 PM, Lawrence D'Oliveiro wrote:

On Fri, 24 May 2024 20:27:38 -0400, Arne Vajhøj wrote:

On 5/24/2024 8:20 PM, Lawrence D'Oliveiro wrote:

On Fri, 24 May 2024 19:05:40 +0300, Michael S wrote:

Conversion is actually trivial - dumb double-quoting.

What do you do if the value has quotes?

1 double-quote -> 2 double-quotes

Assuming the receiving end implements the same convention. Otherwise
you have trouble.

????

The receiving end does not see any of the added double quotes.

It does on VMS, and I think on Windows too, at the raw-buffer level.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 24 May 2024 20:02:46 -0400
Arne Vajh�j <arne@vajhoej.dk> wrote:

On 5/24/2024 12:05 PM, Michael S wrote:

On Fri, 24 May 2024 11:28:05 -0400
Arne Vajh�j <arne@vajhoej.dk> wrote:

On 5/24/2024 11:11 AM, Michael S wrote:

It seems like the simplest solution is to not try to run batch
files by means of spawn("cmd.exe", ...) or
CreateProcess("cmd.exe", ...). They could have use more
specialized function: system() from C RTL or ShellExecuteEx()
from Win32 API. The former is easier to use, the later works as
expected in wider range of host console environments, most
importantly, it works from mintty.

Both system and ShellExecuteEx still take all parameters as a
single string, which require some non-trivial conversion from
array of parameters to that string.

Conversion is actually trivial - dumb double-quoting.

Maybe.

There is an opposite problem - the child will get parameters
together with double quotes.

They should be stripped by the shell.

It does not work like that.
For programs that use CommandLineToArgv() either directly or via C-alike sturtup code, you'll get double quotes stripped. But internal commands
in batch file, e.g echo, will see the double quotes. It's probably
the most harmful for batch files that use 'IF %N==string' logic.

Arne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <20240523223853.00007d91@yahoo.com>,
Michael S <already5chosen@yahoo.com> wrote:

On Thu, 23 May 2024 18:37:35 +0100
chrisq <devzero@nospam.com> wrote:

On 5/23/24 17:41, Arne Vajhøj wrote:

On 5/23/2024 12:02 PM, Michael S wrote:

On Thu, 23 May 2024 09:28:16 -0400
Arne Vajhøj <arne@vajhoej.dk> wrote:

On 5/23/2024 6:19 AM, Scott Dorsey wrote:

Lawrence D'Oliveiro  <ldo@nz.invalid> wrote:

And just to add to the fun, CP/M, MS-DOS and then Windows NT all
copied the basic DEC-style command line architecture, while
neglecting the crucial part of a built-in command parser to
handle more sophisticated syntax. So Microsoft’s recent
rediscovery that “the command line is cool” remains hampered by
this brain-dead underlying architecture.

To give them credit, Microsoft finally realized that command.com
was hopeless and started from first principles with Powershell.
Powershell is easily as good as anything DEC had in 1974 and as
such is a huge advance for the desktop.

Just so that everyone know the history:

command.com - 1980
cmd.exe - 1987 (backwards compatible with command.com, but
significantly extended functionality, often via obscure syntax
though)

Thanks.
Until reading your post I didn't realize that cmd.exe existed
before Nt 3.1 (1993).
Personally I often feel that cmd.exe is an improvement on it's
successor.

Windows Script Host / VBScript - 1995 (not interactive, scripts
only, but very powerful via WMI)
PowerShell - 2006

Not super new.

BTW, I tried to figure out what "command line architecture"
Lawrence is talking about, but didn't quite succeed.

Neither did I.

As far as Win32 CreateProcess() function is concerned, command line
is just a string. If caller omits applicationName parameter then
system does small amount of parsing to find application name in
command line. But when caller provides applicationName parameter,
which is a most logical thing to do in any
non-interactive/non-shell scenario, then OS does not care at all
about content of command line. In both cases OS does not modify
command line and passes it to child process as is.

The concept is pretty common.

VMS LIB$SPAWN and C system also use full command line.

So, the only difference between *nix and Windows is array of
ASCIIZ strings (today, of UTF8 strings) vs single, possibly very
long, null-terminated ASCII or UTF16 string.

In most practical usages it does not matter at all.

But it can matter if one needs to construct a single command
line from multiple arguments and those arguments contain
spaces or shell operators.

Example:
  https://nvd.nist.gov/vuln/detail/CVE-2024-24576

Unix shell just uses single quotes to enclose spaced arguments, which
are then removed by the shell, before storing the argument as pointer
to a null terminated array of char. Neat, clean and simple.

Moving pointers between address spaces is never very simple. Moving >variable-length arrays of pointers adds another level of complexity on
top of it.

I'm not sure how that's particularly relevant; moving argument
strings between address spaces in Unix is handled by the kernel,
not userspace programs (which call a library routine to invoke
the relevant system call). The above quote was about
tokenization, which in Unix shells tends to be very simple.

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)