io.popen Run command with spaces in argument filename, and get the result

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

io.popen Run command with spaces in argument filename, and get the result

Robin Stern
Hi there,

Am new to lua, and coming here from Python land. One thing that I have been not able to get right after browsing internet for 2 hours (and trying solutions) is run a simple command from within lua, and get the stdout, stderr for it. Here's the command:

otfinfo -p /path/to/file name/ with spaces.otf

It will be great if someone can give a complete solution. The path to filename is coming from a list that does not escape spaces, so spaces are just ' ' characters.

Thanks!
v
Reply | Threaded
Open this post in threaded view
|

Re: io.popen Run command with spaces in argument filename, and get the result

v
For your case it would be

io.popen('otfinfo -p "/path/to/file name/ with spaces.otf"', 'r')

Escape it like you would in usual shell, that's the general rule.

The deeper problem here is that you can't reliably escape user input
(well, you theoretically can, but good luck with that), and all ways
Lua allows to call a command require you to escape inputs (due to
portability requirements, it's the only way ANSI C provides). You also
can only have one I/O stream (stdin/stdout/stderr) open per child
process, but not more. You probably want to take a look at lua-
subprocess or luaposix libraries that allow to bypass both of those
limitations.

On Sun, 2021-02-07 at 00:30 -0800, Robin Stern wrote:

> Hi there,
> Am new to lua, and coming here from Python land. One thing that I
> have been not able to get right after browsing internet for 2 hours
> (and trying solutions) is run a simple command from within lua, and
> get the stdout, stderr for it. Here's the command:
>
> otfinfo -p /path/to/file name/ with spaces.otf
>
> It will be great if someone can give a complete solution. The path to
> filename is coming from a list that does not escape spaces, so spaces
> are just ' ' characters.
>
> Thanks!

--
v <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: io.popen Run command with spaces in argument filename, and get the result

Spar
On windows I have to quote both argument and the whole command

"lua "folder one/file.lua""
On 7 Feb 2021, 15:00 +0300, v <[hidden email]>, wrote:
For your case it would be

io.popen('otfinfo -p "/path/to/file name/ with spaces.otf"', 'r')

Escape it like you would in usual shell, that's the general rule.

The deeper problem here is that you can't reliably escape user input
(well, you theoretically can, but good luck with that), and all ways
Lua allows to call a command require you to escape inputs (due to
portability requirements, it's the only way ANSI C provides). You also
can only have one I/O stream (stdin/stdout/stderr) open per child
process, but not more. You probably want to take a look at lua-
subprocess or luaposix libraries that allow to bypass both of those
limitations.

On Sun, 2021-02-07 at 00:30 -0800, Robin Stern wrote:
Hi there,
Am new to lua, and coming here from Python land. One thing that I
have been not able to get right after browsing internet for 2 hours
(and trying solutions) is run a simple command from within lua, and
get the stdout, stderr for it. Here's the command:

otfinfo -p /path/to/file name/ with spaces.otf

It will be great if someone can give a complete solution. The path to
filename is coming from a list that does not escape spaces, so spaces
are just ' ' characters.

Thanks!

--
v <[hidden email]>
v
Reply | Threaded
Open this post in threaded view
|

Re: io.popen Run command with spaces in argument filename, and get the result

v
On Sun, 2021-02-07 at 15:05 +0300, Spar wrote:
> On windows I have to quote both argument and the whole command
>
> "lua "folder one/file.lua""

I'm not sure what are you talking about. In my example I've quoted both
command (to mark it as string to Lua) and filename (to mark it as a
single argument to shell Lua initiates), using single quotes for first
and double quotes for second. Yours would confuse lua because it
doesn't know that you wanted to nest quotes instead of creating two
strings.
--
v <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: io.popen Run command with spaces in argument filename, and get the result

Andrew Gierth
In reply to this post by Robin Stern
>>>>> "Robin" == Robin Stern <[hidden email]> writes:

 Robin> Hi there,

 Robin> Am new to lua, and coming here from Python land. One thing that
 Robin> I have been not able to get right after browsing internet for 2
 Robin> hours (and trying solutions) is run a simple command from within
 Robin> lua, and get the stdout, stderr for it. Here's the command:

 Robin> otfinfo -p /path/to/file name/ with spaces.otf

The Lua standard library only supports a thin wrapper around the OS's
popen() function, which means you (a) have to deal with shell behavior,
which in turn may be OS-dependent on non-POSIX systems, and (b) you can
only get the stdout (the subprocess stderr will go to wherever stderr is
going in the Lua program, unless you put redirections in the command).

Assuming a POSIX-like system (i.e. that popen() invokes a POSIX shell),
something like this should do, provided that the value of LC_CTYPE (if
set) does not specify an "unsafe" encoding:

function myprog(filename)
  assert(not filename:find("\0", 1, true), "NUL not allowed in filenames")
  local file_esc = filename:gsub([[']], [['\'']])
  return io.popen(("otfinfo -p '%s'"):format(file_esc))
end

The logic here is that '...' in POSIX shell quotes every character
except ' itself, without allowing any form of escape, so we replace ' by
the sequence '\'' which closes the existing quote, adds an escaped '
character, and opens another quote. There are other ways to do shell
quoting but this one is the safest.

If you want the code to work on Windows, the required quoting will be
different, and I don't know what they are.

The other option is to use a module specifically for spawning
subprocesses.

(The only encoding which I know of which is "unsafe" for this purpose is
the Johab encoding of Korean, which allows ' to be the second byte of a
two-byte sequence. Other encodings which are unsafe in other contexts,
such as Shift-JIS, Big5, GB18030, which have the property of allowing
second bytes in the range 0x40 upwards (including the \ character), are
not unsafe in this context.)

--
Andrew.
v
Reply | Threaded
Open this post in threaded view
|

Re: io.popen Run command with spaces in argument filename, and get the result

v
On Sun, 2021-02-07 at 17:41 +0000, Andrew Gierth wrote:

> > > > >
> function myprog(filename)
>   assert(not filename:find("\0", 1, true), "NUL not allowed in
> filenames")
>   local file_esc = filename:gsub([[']], [['\'']])
>   return io.popen(("otfinfo -p '%s'"):format(file_esc))
> end
>
> The logic here is that '...' in POSIX shell quotes every character
> except ' itself, without allowing any form of escape, so we replace '
> by
> the sequence '\'' which closes the existing quote, adds an escaped '
> character, and opens another quote. There are other ways to do shell
> quoting but this one is the safest.

Until someone passes \' in as input filename, which makes it break
horribly. E.g.

somefile\';rm -rf /*;echo \'

turns into

otfinfo -p 'somefile\\';rm -rf /*;echo \\''

Match quotes and backslashes to see why it is such a bad idea.
--
v <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: io.popen Run command with spaces in argument filename, and get the result

Andrew Gierth
>>>>> "v" == v  <[hidden email]> writes:

 >> The logic here is that '...' in POSIX shell quotes every character
 >> except ' itself, without allowing any form of escape, so we replace
 >> ' by the sequence '\'' which closes the existing quote, adds an
 >> escaped ' character, and opens another quote. There are other ways
 >> to do shell quoting but this one is the safest.

 v> Until someone passes \' in as input filename, which makes it break
 v> horribly. E.g.

 v> somefile\';rm -rf /*;echo \'

 v> turns into

 v> otfinfo -p 'somefile\\';rm -rf /*;echo \\''

No it doesn't, it turns into:

otfinfo -p 'somefile\'\'';rm -rf /*;echo \'\'''

Note that \ is not an escape character inside '...' so this is correct.

Perhaps you misunderstood what the [[ ]] do, you certainly didn't try
running the actual code.

--
Andrew.
v
Reply | Threaded
Open this post in threaded view
|

Re: io.popen Run command with spaces in argument filename, and get the result

v
On Sun, 2021-02-07 at 23:45 +0000, Andrew Gierth wrote:

> > > > >
> No it doesn't, it turns into:
>
> otfinfo -p 'somefile\'\'';rm -rf /*;echo \'\'''
>
> Note that \ is not an escape character inside '...' so this is
> correct.
>
> Perhaps you misunderstood what the [[ ]] do, you certainly didn't try
> running the actual code.

Oops, my bad - I've misunderstood how shell matches tick braces (and
still a little confused about that as of right now, actually).
--
v <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: io.popen Run command with spaces in argument filename, and get the result

Andrew Gierth
>>>>> "v" == v  <[hidden email]> writes:

 >> No it doesn't, it turns into:
 >>
 >> otfinfo -p 'somefile\'\'';rm -rf /*;echo \'\'''
 >>
 >> Note that \ is not an escape character inside '...' so this is
 >> correct.
 >>
 >> Perhaps you misunderstood what the [[ ]] do, you certainly didn't try
 >> running the actual code.

 v> Oops, my bad - I've misunderstood how shell matches tick braces (and
 v> still a little confused about that as of right now, actually).

The reason for using '...' here is exactly because it has the _simplest_
quoting rules: the character ' itself may not appear within '...' (i.e.
any ' character terminates the quote) and every other character, without
exception, is quoted (there are no escape characters). So in this
example the input breaks down into the following 5 sequences (with no
characters between them, so they are concatenated into one single word):

'somefile\'
\'
';rm -rf /*;echo \'
\'
''

Since \' outside of any quoting construct is itself just a quoted '
character, then when these are reassembled with the quoting removed we
get a word with:

somefile\';rm -rf /*;echo \'

which was the original string.

BTW, you can use a command like  printf '[%s]' blahblah  or similar to
test argument splitting/dequoting in the shell.

--
Andrew.