Problems with lposix on win32

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Problems with lposix on win32

Chris Marrin

I have been trying to port lposix to compile and link under MSDev. The
problem is that MSDev does not support many of the Posix capabilities.
For instance there is no fork() function in Win32 and MSDev has no
wrapper for it. This is true for many other Posix functions.

The natural solution is to use Cygwin. I can build lposix with Cygwin
just fine. The problem comes in linking. The rest of my app
(http://emma3d.org) builds Lua as a dll, so all the various components
can share the interpreter. This is important because Emma is an
extensible system. You can build an extension dll which can access the
interpreter. So the interpreter needs to be a DLL.

The problem is that Cygwin can't link with a DLL produced by MSDev, so I
can use GCC to link lposix. And MSDev can't link with an object produced
by GCC (or more properly MSDev can't link with the Cygwin libraries that
object needs). I can't build liblua.dll using Cygwin because then I
would have to build every subsystem that links with that dll in Cygwin,
and that is not acceptable for our app.

So I would like to suggest that we go back to the plan to make the
'osex' package small and have it grow. We can still start with lposix,
but toss the problematic function calls. To be honest, I'm not sure how
practical fork() and some of the permission related calls are in many
systems anyway.

Here is what lposix has today:

"access", "chdir", "chmod", "chown", "ctermid", "dir", "errno",
"exec", "files", "fork", "getcwd", "getenv", "getgroup",
"getlogin", "getpasswd", "getprocessid", "kill", "link",
"mkdir", "mkfifo", "pathconf", "putenv", "readlink", "rmdir",
"setgid", "setuid", "sleep", "stat", "symlink", "sysconf",
"times", "ttyname", "umask", "uname", "unlink", "utime", "wait"

Here is what I think we should take from it:

"access", "chdir", "chmod", "errno", "exec", "getcwd",
"getenv", "mkdir", "putenv", "rmdir", "sleep", "stat",
"umask"

Of the things left out, the only thing I think missing is "dir" and
"files". These are the directory traversal functions. I left them out
because Win32 does this sort of thing very differently. I can implement
it pretty easily, but with a totally different stream of code. So I will
do it later. I know there are other ideas out there about how to do
directory traversal, so we should probably discuss and agree on a design.

I have ported all the above to Win32, using MSDev 7.1. Issues in porting
to Win32:

1) Windows has the concept of permissions, but not for
user/group/others. So I made it so that setting permissions for any of
these sets the Win32 notion of permission. For instance, if you set read
permission for user or group or other, I set read permission. (Note:
cygwin seems to support full permissions for win32 files, looking at the
'ls -l' command in the cygwin shell. How do they do that?)

2) Windows has no notion of an executable bit, so I just ignore it

3) Windows supports the notion of regular files, directories and
character files, but not links, block files, fifos or sockets. So I
ignore them.

4) Since Win32 has a Sleep function that takes milliseconds, I decided
to change the implementation of sleep to take a double seconds value
rather than an integer. On Win32 this means you get resolution down to
1ms. And I used nanosleep() for the Linux implementation which gives
resolution down to 1ns (dependent on the machine resolution, of course).
If nanosleep does not exist elsewhere, we can always use something else,
like sleep(), and just have 1sec resolution.

Comments?

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
         ,.`           ,b`    ,`                            , 1$'
      ,|`             mP    ,`                              :$$'     ,mm
    ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
   m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
  b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
  ```"```'"    `"`         `""`        ""`                          ,P`
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Wim Couwenberg-2
> "access", "chdir", "chmod", "errno", "exec", "getcwd",
> "getenv", "mkdir", "putenv", "rmdir", "sleep", "stat",
> "umask"

> I have ported all the above to Win32, using MSDev 7.1. Issues in porting
> to Win32:

There's no equivalent for the posix exec call in Win32.  How did you
implement it?

--
Wim


Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Dolan, Ryanne Thomas (UMR-Student)
You can use ShellExecute.  http://msdn.microsoft.com/library/default.asp?url=/library/en-us/shellcc/platform/shell/reference/functions/shellexecute.asp

ex:
ShellExecute(NULL, NULL, <filename>, <parameters>, <path>, SW_SHOWNORMAL);

On Sat, 2006-01-14 at 19:02 +0100, Wim Couwenberg wrote:
> "access", "chdir", "chmod", "errno", "exec", "getcwd",
> "getenv", "mkdir", "putenv", "rmdir", "sleep", "stat",
> "umask"

> I have ported all the above to Win32, using MSDev 7.1. Issues in porting
> to Win32:

There's no equivalent for the posix exec call in Win32.  How did you
implement it?

--
Wim



Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Wim Couwenberg-2
>  You can use ShellExecute. 

ShellExecute (and CreateProcess) create a *new* process (if file is an
executable that is.)  The posix exec family replaces the executable
image of the running process.  Very different models...

BTW If you want to spawn a new process on Win32 then I'd suggest to
use CreateProcess since that does not depend on shell32.dll

--
Wim


Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Rici Lake-2

On 14-Jan-06, at 1:34 PM, Wim Couwenberg wrote:

>>  You can use ShellExecute. 
>
> ShellExecute (and CreateProcess) create a *new* process (if file is an
> executable that is.)  The posix exec family replaces the executable
> image of the running process.  Very different models...


Also, the command parsing model is different. With exec(), you specify
the argv array directly, so there is less need to worry about  
metacharacters (it is still possible that the utility will misinterpret
an argument starting with a '-', though.) With ShellExecute and
CreateProcess, you provide an unparsed command line, which is much
riskier if you're constructing it from user input.

(In checking the MSDN Library, I also discovered this interesting
tidbit:

   The Unicode version of this function, CreateProcessW,
   can modify the contents of this string. Therefore, this
   parameter cannot be a pointer to read-only memory (such
   as a const variable or a literal string) or the function
   could cause an access violation.

Or, in the case of a Lua binding, modify a Lua string with potentially
disastrous consequences.

Leaving all that aside, though, wouldn't it be possible to implement a
higher-level interface which did fork+exec on Unix and CreateProcess on
Windows? That's a fairly commmon situation, anyway. CreateProcess seems
to be able to specify stdin/stdout/stderr for the newly created
process, which is what would normally done in the gap between the
fork() and the exec() in Unix, so there is some hope for an interface
which would work on both. Although I'm not sure how I would handle
quoting of "shell" metacharacters.
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
In reply to this post by Chris Marrin
My understanding is that cygwin and MSDev coexistence gets
more difficult as the MS version numbers get higher.

Some points to note:

 - I have had most of the functions you mention implmeneted for some
time. Let me know if you want code.
 -  errno() should call GetLastError() rather than use errno.
 - I would add to your list the following (I have them implemented),
  dir, exec, files, getpid, getppid, kill and unlink
 Note that the unlink direct equivalent is something like -

#ifdef _WIN32
static int l_unlink(lua_State  *L)
{
    const char* s = fixpath(L, 1);
        HANDLE h = CreateFile(s,GENERIC_READ, FILE_SHARE_READ|FILE_SHARE_WRITE,
          (LPSECURITY_ATTRIBUTES)0, // address of security descriptor
          OPEN_EXISTING,
          FILE_FLAG_DELETE_ON_CLOSE|FILE_ATTRIBUTE_NORMAL,
          (HANDLE)0);   // handle of file with attributes to copy

        if (h!=INVALID_HANDLE_VALUE) {
          CloseHandle(h);
          lua_pushboolean(L,1);
          return 1;
        }
        return pusherrstr(L, GetLastError());
}

#else

This simulates the *nix unlink semantics.

- In luasocket Diego has a very good sleep implementation that
sounds similar to what you have done. Check it out.

-

On 1/15/06, Chris Marrin <[hidden email]> wrote:
>
> I have been trying to port lposix to compile and link under MSDev. ...
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
In reply to this post by Rici Lake-2
Hi Rici, Wim,

I consider myself a bit of a "CreateProcess" expert. There are lots
of issues here and exec/spawn probably deserves a thread of
its own.

1) I would suggest that posix spawn() be used rather fork()/exec().
posix_spawn() is avalable under linux and some others. Th spawn()
behaviour is implementable under win32.

2) ShellExecute (and relatives) vs CreateProcess. As you point out
they are very different beasts. I have implemented different Lua
functions for both of these. I believe that ShellExecute does not
have a *nix equivalent.

3) The fun starts when one allows the remapping of stdin/stderr/stdout.
I have done this (without threads) using named pipes and overlapped
io. It works ok.

4) Both the A and W versions of CreateProcess modify the command
string. Note that this "feature" is also a security hole.
"c:\program files param1"
Windows parses the string so that it attempts to execute c:\program.
It may expand or reduce the length of the command string.


DB

On 1/15/06, Rici Lake <[hidden email]> wrote:

>
> On 14-Jan-06, at 1:34 PM, Wim Couwenberg wrote:
>
> >>  You can use ShellExecute.
> >
> > ShellExecute (and CreateProcess) create a *new* process (if file is an
> > executable that is.)  The posix exec family replaces the executable
> > image of the running process.  Very different models...
>
>
> Also, the command parsing model is different. With exec(), you specify
> the argv array directly, so there is less need to worry about
> metacharacters (it is still possible that the utility will misinterpret
> an argument starting with a '-', though.) With ShellExecute and
> CreateProcess, you provide an unparsed command line, which is much
> riskier if you're constructing it from user input.
>
> (In checking the MSDN Library, I also discovered this interesting
> tidbit:
>
>    The Unicode version of this function, CreateProcessW,
>    can modify the contents of this string. Therefore, this
>    parameter cannot be a pointer to read-only memory (such
>    as a const variable or a literal string) or the function
>    could cause an access violation.
>
> Or, in the case of a Lua binding, modify a Lua string with potentially
> disastrous consequences.
>
> Leaving all that aside, though, wouldn't it be possible to implement a
> higher-level interface which did fork+exec on Unix and CreateProcess on
> Windows? That's a fairly commmon situation, anyway. CreateProcess seems
> to be able to specify stdin/stdout/stderr for the newly created
> process, which is what would normally done in the gap between the
> fork() and the exec() in Unix, so there is some hope for an interface
> which would work on both. Although I'm not sure how I would handle
> quoting of "shell" metacharacters.
>
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Rici Lake-2

On 14-Jan-06, at 6:27 PM, D Burgess wrote:

> Hi Rici, Wim,
>
> I consider myself a bit of a "CreateProcess" expert.

Cool. I only read the description in the MSDN library, so
I'm perfectly happy to defer to your practical knowledge.

> There are lots
> of issues here and exec/spawn probably deserves a thread of
> its own.

:)

> 1) I would suggest that posix spawn() be used rather fork()/exec().
> posix_spawn() is avalable under linux and some others. Th spawn()
> behaviour is implementable under win32.
>

I don't know how widely posix_spawn is implemented; it's not supported
by FreeBSD afaik, and I doubt whether it's supported by Mac OS X.
However, it can be implemented in terms of fork() and execve(). I
wouldn't suggest a complete binding to posix_spawn() (or the Windows
equivalent); rather a simplified (and extensible, perhaps) version
which covered common cases.

> 2) ShellExecute (and relatives) vs CreateProcess. As you point out
> they are very different beasts. I have implemented different Lua
> functions for both of these. I believe that ShellExecute does not
> have a *nix equivalent.

You could fork() and execve() /bin/sh, with more or less the same
semantics, as far as I can see. I might be missing something, of
course. I don't think that the Unix and Windows concepts of what a
"shell" is are very well aligned.
>
> 3) The fun starts when one allows the remapping of stdin/stderr/stdout.
> I have done this (without threads) using named pipes and overlapped
> io. It works ok.

Maybe I misunderstood the MSDN writeup. A quick skim seemed to indicate
that you could just hand a file handle over to the new process as its
stdin handle, which would be the sort of thing you'd want to do in a
standard "forking" server architecture. I'm aware that forking servers
are considered declassé these days, and that event-driven
single-process collaboration is sexier. On the other hand, forking
servers are perfectly suitable, and a lot less subject to random
resource leaks.

Anyway, it would be sufficient for most purposes if there were a way of
doing the remapping of the three standard streams, however it was done.
Lua only defines the three streams, so there would be no need to go
beyond that in a basic implementation.

> 4) Both the A and W versions of CreateProcess modify the command
> string. Note that this "feature" is also a security hole.
> "c:\program files param1"
> Windows parses the string so that it attempts to execute c:\program.
> It may expand or reduce the length of the command string.

As I read the docs, you have the option of providing a specific file
and a command line, or a commmand line from which Windows will extract
the filename, but no option to provide a pre-chunked argv vector. I
regard this as an error in API design, but as I said, the Unix version
is not bullet-proof either. It's just easier to fix; if you're about to
put a filename into an argv list, you pretty much only have to check to
see if the first character is '-' or not. Unfortunately, the convention
that -- terminates options parsing is only a convention, not enforced.

I quoted the section on the string being modified not because of the
security issues involved with parsing metacharacters in the command
line (which are a separate issue) but because it might be relevant to
someone doing a naive binding; passing tostring(L, 3) to CreateProcess
would be a Really Bad Idea if CreateProcess treats its argument as a
mutable char* rather than a const char*. Of course, this is a general
rule for binding functions which take character string arguments; if
you don't know for sure that the parameter is const char* (and is not
overridden), then you must make a copy of the string before passing it.

On the other hand, I don't offhand see an interface which abstracts
command-line chunking. You can't insert escape characters into an argv
vector without changing the semantics, so a note saying that
"metacharacters may or may not be interpreted" would make the interface
unusable.

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
In reply to this post by Wim Couwenberg-2
Wim Couwenberg wrote:

>>"access", "chdir", "chmod", "errno", "exec", "getcwd",
>>"getenv", "mkdir", "putenv", "rmdir", "sleep", "stat",
>>"umask"
>
>
>>I have ported all the above to Win32, using MSDev 7.1. Issues in porting
>>to Win32:
>
>
> There's no equivalent for the posix exec call in Win32.  How did you
> implement it?

Win32 implements the full array of exec related calls, including
execvp(), which lposix uses. In fact, I have found these calls to be
surprisingly full featured, relative to their Unix counterparts!

--
chris marrin
[hidden email]
"As a general rule,don't solve puzzles that open portals to Hell"
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
In reply to this post by D Burgess-4
D Burgess wrote:

> Hi Rici, Wim,
>
> I consider myself a bit of a "CreateProcess" expert. There are lots
> of issues here and exec/spawn probably deserves a thread of
> its own.

Ok, I have looked at the descriptions of execvp() in win32 docs and the
Posix spec. Win32 says it creates a "child process" while Posix says it
"replaces the currently running process". I have trouble understanding
this. Does the calling process exit cleanly, or just cease to exist?

I thought both did the same thing, meaning that I always thought the
Unix version did not gun down the calling process. I guess I have been
using Windows for too long, which is sad.

Anyway, generally, I think the goal of this "osex" package should not be
to throw in the kitchen sink. This is why I felt that fork() should be
left out. Nice, (albeit old fashioned), but its arcane process semantics
  has little chance of running on smaller systems. The same is true of
spawn(), kill() and many of the other process control functions.

I would like it to contain a small number of useful file-system and
environment calls. So I think the current list:

"access", "chdir", "chmod", "errno", "exec", "getcwd",
"getenv", "mkdir", "putenv", "rmdir", "sleep", "stat",
"umask"

should lose exec() altogether, and gain dir() and/or files().

On that subject, it seems wasteful to have a forms of directory
discovery which return an array of string names (dir) and an iterator
which will return each string name in turn. It seems like we only really
need files(). How does that sound?

--
chris marrin
[hidden email]
"As a general rule,don't solve puzzles that open portals to Hell"
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
I would agree with leaving exec() for another day.
The only addition to the list that I would add is maybe unlink().

On 1/15/06, Chris Marrin <[hidden email]> wrote:

> Ok, I have looked at the descriptions of execvp() in win32 docs and the
> Posix spec. Win32 says it creates a "child process" while Posix says it
> "replaces the currently running process". I have trouble understanding
> this. Does the calling process exit cleanly, or just cease to exist?
On Windows it exits cleanly. If that is the question. While the
exec and spawn functions are OK, they have a few limitations with
GUI proceses.

> This is why I felt that fork() should be
> left out.
> should lose exec() altogether, and gain dir() and/or files().
agreed (for now).

>
> On that subject, it seems wasteful to have a forms of directory
> discovery which return an array of string names (dir) and an iterator
> which will return each string name in turn. It seems like we only really
> need files(). How does that sound?
the iterator version should be fine.
I like files(dir, [regex]) where dir is the directory name and regex is an
optional string.find compatible regex.

DB
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
D Burgess wrote:
> I would agree with leaving exec() for another day.
> The only addition to the list that I would add is maybe unlink().

What does unlink() do that the existing os.remove() does not?

Posix has both a remove() and an unlink() function. The former follows
the clib definition, i.e., it causes "the file ... to be no longer
accessible by that name". This is the abstract equivalent of unlink() on
a unix system, which has stated semantics that are specific to the
various flavors of unix file system. Seems like remove() is the better
function for cross-platform support.

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
         ,.`           ,b`    ,`                            , 1$'
      ,|`             mP    ,`                              :$$'     ,mm
    ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
   m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
  b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
  ```"```'"    `"`         `""`        ""`                          ,P`
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
I understand, someone correct me if I am wrong that on
Unix unlink will remove the file when the last user of it
closes the file. THis equals the FILEFLAG_DELETE_ON_CLOSE
on windows.

On windows remove and unlink require exclusive access.

DB

On 1/15/06, Chris Marrin <[hidden email]> wrote:

> D Burgess wrote:
> > I would agree with leaving exec() for another day.
> > The only addition to the list that I would add is maybe unlink().
>
> What does unlink() do that the existing os.remove() does not?
>
> Posix has both a remove() and an unlink() function. The former follows
> the clib definition, i.e., it causes "the file ... to be no longer
> accessible by that name". This is the abstract equivalent of unlink() on
> a unix system, which has stated semantics that are specific to the
> various flavors of unix file system. Seems like remove() is the better
> function for cross-platform support.
>
> --
> chris marrin              ,""$, "As a general rule,don't solve puzzles
> [hidden email]        b`    $  that open portals to Hell" ,,.
>          ,.`           ,b`    ,`                            , 1$'
>       ,|`             mP    ,`                              :$$'     ,mm
>     ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
>    m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
>   b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
> b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
> P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
>   ```"```'"    `"`         `""`        ""`                          ,P`
>
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
D Burgess wrote:
> I understand, someone correct me if I am wrong that on
> Unix unlink will remove the file when the last user of it
> closes the file. THis equals the FILEFLAG_DELETE_ON_CLOSE
> on windows.

I think the clib remove() function is intentionally non-specific. Remove
a file with a given name and no one can access it by that name again. It
says nothing about what capabilities the caller must have or if the
action actually frees any disk space. I think this is a good spec for a
cross-platform function.

Ok, so maybe clib is not so arbitrary :-)

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
         ,.`           ,b`    ,`                            , 1$'
      ,|`             mP    ,`                              :$$'     ,mm
    ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
   m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
  b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
  ```"```'"    `"`         `""`        ""`                          ,P`
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Edgar Toernig
In reply to this post by D Burgess-4
D Burgess wrote:
>
> I understand, someone correct me if I am wrong that on
> Unix unlink will remove the file when the last user of it
> closes the file.

The name is removed immediately but the contents of the
file may stay and is removed only when the last user closes
the file.

The difference between unlink and remove is, that unlink won't
delete directories.  remove does as long as the directory is
empty.  On older system which lack remove you had to call rmdir
the delete directories.


> THis equals the FILEFLAG_DELETE_ON_CLOSE on windows.

Sure?  Will this example work?  It will give you ten
/different/ files:

    int fd[10];
    for (int i = 0; i < 10; i++) {
       fd[i] = open("foo", O_CREAT|O_TRUNC|O_RDWR, 0666);
       unlink("foo");
    }

> On windows remove and unlink require exclusive access.

Posix allows this restriction (errno=EBUSY) but it's unusual
and most Unixes let you delete busy files/directories.

Ciao, ET.
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
Edgar Toernig wrote:

> D Burgess wrote:
>
>>I understand, someone correct me if I am wrong that on
>>Unix unlink will remove the file when the last user of it
>>closes the file.
>
>
> The name is removed immediately but the contents of the
> file may stay and is removed only when the last user closes
> the file.
>
> The difference between unlink and remove is, that unlink won't
> delete directories.  remove does as long as the directory is
> empty.  On older system which lack remove you had to call rmdir
> the delete directories.

Yes, I see that the posix spec says this explicitly. Furthermore it says
that you cannot remove non-empty directories. So maybe we should get rid
of rmdir() from osex, since remove() in os can do this functionality.
And you can discover if a file is a regular file or directory using
osex.stat().

I'd also like to argue that we toss osex.getenv() since this function is
in os as well. I originally thought we should keep it for completeness.
But now I think we should minimize the footprint of osex. I would also
like to replace putenv() with a setenv(name[,value]) function. This
avoids the need to construct the putenv string. And it allows the name
to be removed from the environment by omitting value. The original
lposix package had this, but it had a third 'overwrite' param. I don't
think this is needed, since you can see if a variable exists with
os.getenv() and then avoid calling osex.setenv() if so.

I have also added a files() function, so here is the current proposed list:

access(path,[mode]) test file accessibility
chdir(path) make the given path the cwd
chmod(path,mode) change permissions on the path to the mode str
errno() return the last error string,number
files([path]) return an iterator for the passed path
getcwd() return the cwd
mkdir(path) create a directory at the passed path
setenv(name[,value]) set a value in the environment
sleep(sec) sleep for the passed number of seconds
stat(path,[selector]) return status for the file at the passed path
umask([mode]) set the mask for future file creations

I think that's a pretty complete set for starters. I have implemented
this set for Win32 using MSDev 7.1. I will send the code as soon as I
have tested it.

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
         ,.`           ,b`    ,`                            , 1$'
      ,|`             mP    ,`                              :$$'     ,mm
    ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
   m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
  b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
  ```"```'"    `"`         `""`        ""`                          ,P`
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Edgar Toernig
Chris Marrin wrote:
>
> [...] so here is the current proposed list:
>
> errno() return the last error string,number

errno is pointless.  Unlike C, Lua has multiple return values.
And I would suggest to use strings like "ENOTEMPTY" instead of
errno numbers.  The numbers are totally useless.

> mkdir(path) create a directory at the passed path

mkdir(path [, mode])

Ciao, ET.


PS: I don't like the name osex.  I'd simply put them into os.
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Rici Lake-2
In reply to this post by Chris Marrin
A few things:

On 15-Jan-06, at 12:43 PM, Chris Marrin wrote:

> The original lposix package had this, but it had a third 'overwrite'
> param. I don't think this is needed, since you can see if a variable
> exists with os.getenv() and then avoid calling osex.setenv() if so.

setenv with overwrite (should be) atomic, which you cannot achieve with
getenv/setenv. However, I don't think anyone actually uses this feature
as a locking mechanism, so I don't see any need to reproduce it.

Given that, it seems like a table interface to getenv/setenv is much
more natural than a function call interface. Moreover, it's faster. It
also makes it easy for a sandbox to provide a fake environment. So
personally, I'd prefer to scrap both getenv and setenv, and just
provide an os[ex].env table.

putenv has the odd characteristic that the string you provide is
mutable; changing the string is supposed to change the environment
variable. ("This is the only way to guarantee that changing the
environment doesn't leak memory.") There doesn't seem to be any obvious
way to implement this in Lua; it does not conform to Windows env
variable semantics, and so I agree that it should simply be scrapped.

Almost all of your remaining functions have to do with the filesystem,
not the operating system; the exceptions are sleep() and possibly
errno(). I think sleep() is uncontroversial; the remaining
filesystem-related calls should go into a filesystem library, not an os
library (imho).

errno() is very difficult to use from a scripting language; it is not
easy (or possibly even possible) for the scripting language to know
that there have not been any intervening calls which may change errno.
In addition, POSIX semantics require errno to be unchanged by calls
which do not return an error value. (However, iirc it also used to
require that floating point errors be reported in errno.) Managing
errno is therefore replete with subtle bugs. Finally, there are
environments in which different "OS-like objects" use different
errno-like globals.

On the whole, I think that errors should be retained with OS-like
objects, which is essentially the semantics of standard C i/o, for
example, where each FILE has its own error indicator (which can be both
read and cleared.) That would be possible to implement, and I think it
would be easier to implement that in an OS-independent way. In
addition, it would be easier to use. Say, for example, that both files
and sockets had a lasterror() method (or a lasterror key). That would
make it much easier to write loops which respected error conditions:

   for line in f:lines() do
     ...
   end
   if f:lasterror() then   -- or possibly "if not f.eof"
     ... report the error
   end

I personally think that open() should be a member function on a
directory object, rather than a global. An implementation should
provide both a root directory object (sort of the analogy to stdin) and
(if possible) a cwd() function; regardless, there would be some object
onto which open() errors could be attached. This would simplify
programming of directory traversal functions as well.



Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
In reply to this post by Edgar Toernig
I agree with this.

On 1/16/06, Edgar Toernig <[hidden email]> wrote:
> PS: I don't like the name osex.  I'd simply put them into os.
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
In reply to this post by Edgar Toernig
Edgar Toernig wrote:

> Chris Marrin wrote:
>
>>[...] so here is the current proposed list:
>>
>>errno() return the last error string,number
>
>
> errno is pointless.  Unlike C, Lua has multiple return values.
> And I would suggest to use strings like "ENOTEMPTY" instead of
> errno numbers.  The numbers are totally useless.

It's no more pointless than in C, where you can compare it against
constants for useful results. But I agree that Lua does not have
switch() or enums making it difficult to use integer return values. I
will cobble together an initial (small) set of string return values. I
am already returning multiple values (string,number) because that is
what lposix did. The string is the strerror() string. Perhaps a better
approach would be to return a string value for the error number along
with a numeric version and add a separate strerror() function. This
would allow lookup of error codes that do not come from these functions.
If errno is a value for which I don't have a symbolic constant, I would
return nil for the string and still return the number. That gives us
fully general functionality.

How does that sound?

>
>
>>mkdir(path) create a directory at the passed path
>
>
> mkdir(path [, mode])

Sounds good. Done. But mode will have no effect on win32 because mkdir()
on that platform does not take a mode param and I don't think win32
obeys the umask when creating directories. Anyone have info to the contrary?

>
> Ciao, ET.
>
>
> PS: I don't like the name osex.  I'd simply put them into os.

I don't think that's possible using the 5.1 package model. I suppose it
is possible to make:

     require "osex"

add things to the 'os' table, assuming it exists. But that seems, at
least, poor practice.

But I did just notice that this reads like "o-sex", so maybe "exos"
would be better? :-)

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
         ,.`           ,b`    ,`                            , 1$'
      ,|`             mP    ,`                              :$$'     ,mm
    ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
   m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
  b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
  ```"```'"    `"`         `""`        ""`                          ,P`
123