Problems with lposix on win32

classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Problems with lposix on win32

Chris Marrin

I have been trying to port lposix to compile and link under MSDev. The problem is that MSDev does not support many of the Posix capabilities. For instance there is no fork() function in Win32 and MSDev has no wrapper for it. This is true for many other Posix functions.

The natural solution is to use Cygwin. I can build lposix with Cygwin just fine. The problem comes in linking. The rest of my app (http://emma3d.org) builds Lua as a dll, so all the various components can share the interpreter. This is important because Emma is an extensible system. You can build an extension dll which can access the interpreter. So the interpreter needs to be a DLL.

The problem is that Cygwin can't link with a DLL produced by MSDev, so I can use GCC to link lposix. And MSDev can't link with an object produced by GCC (or more properly MSDev can't link with the Cygwin libraries that object needs). I can't build liblua.dll using Cygwin because then I would have to build every subsystem that links with that dll in Cygwin, and that is not acceptable for our app.

So I would like to suggest that we go back to the plan to make the 'osex' package small and have it grow. We can still start with lposix, but toss the problematic function calls. To be honest, I'm not sure how practical fork() and some of the permission related calls are in many systems anyway.

Here is what lposix has today:

"access", "chdir", "chmod", "chown", "ctermid", "dir", "errno",
"exec", "files", "fork", "getcwd", "getenv", "getgroup",
"getlogin", "getpasswd", "getprocessid", "kill", "link",
"mkdir", "mkfifo", "pathconf", "putenv", "readlink", "rmdir",
"setgid", "setuid", "sleep", "stat", "symlink", "sysconf",
"times", "ttyname", "umask", "uname", "unlink", "utime", "wait"

Here is what I think we should take from it:

"access", "chdir", "chmod", "errno", "exec", "getcwd",
"getenv", "mkdir", "putenv", "rmdir", "sleep", "stat",
"umask"

Of the things left out, the only thing I think missing is "dir" and "files". These are the directory traversal functions. I left them out because Win32 does this sort of thing very differently. I can implement it pretty easily, but with a totally different stream of code. So I will do it later. I know there are other ideas out there about how to do directory traversal, so we should probably discuss and agree on a design.

I have ported all the above to Win32, using MSDev 7.1. Issues in porting to Win32:

1) Windows has the concept of permissions, but not for user/group/others. So I made it so that setting permissions for any of these sets the Win32 notion of permission. For instance, if you set read permission for user or group or other, I set read permission. (Note: cygwin seems to support full permissions for win32 files, looking at the 'ls -l' command in the cygwin shell. How do they do that?)

2) Windows has no notion of an executable bit, so I just ignore it

3) Windows supports the notion of regular files, directories and character files, but not links, block files, fifos or sockets. So I ignore them.

4) Since Win32 has a Sleep function that takes milliseconds, I decided to change the implementation of sleep to take a double seconds value rather than an integer. On Win32 this means you get resolution down to 1ms. And I used nanosleep() for the Linux implementation which gives resolution down to 1ns (dependent on the machine resolution, of course). If nanosleep does not exist elsewhere, we can always use something else, like sleep(), and just have 1sec resolution.

Comments?

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
        ,.`           ,b`    ,`                            , 1$'
     ,|`             mP    ,`                              :$$'     ,mm
   ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
  m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
 b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
 ```"```'"    `"`         `""`        ""`                          ,P`

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Wim Couwenberg-2
> "access", "chdir", "chmod", "errno", "exec", "getcwd",
> "getenv", "mkdir", "putenv", "rmdir", "sleep", "stat",
> "umask"

> I have ported all the above to Win32, using MSDev 7.1. Issues in porting
> to Win32:

There's no equivalent for the posix exec call in Win32.  How did you
implement it?

--
Wim



Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Dolan, Ryanne Thomas (UMR-Student)
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Wim Couwenberg-2
>  You can use ShellExecute. 

ShellExecute (and CreateProcess) create a *new* process (if file is an
executable that is.)  The posix exec family replaces the executable
image of the running process.  Very different models...

BTW If you want to spawn a new process on Win32 then I'd suggest to
use CreateProcess since that does not depend on shell32.dll

--
Wim




Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Rici Lake-2

On 14-Jan-06, at 1:34 PM, Wim Couwenberg wrote:

 You can use ShellExecute. 

ShellExecute (and CreateProcess) create a *new* process (if file is an
executable that is.)  The posix exec family replaces the executable
image of the running process.  Very different models...


Also, the command parsing model is different. With exec(), you specify the argv array directly, so there is less need to worry about metacharacters (it is still possible that the utility will misinterpret an argument starting with a '-', though.) With ShellExecute and CreateProcess, you provide an unparsed command line, which is much riskier if you're constructing it from user input.

(In checking the MSDN Library, I also discovered this interesting tidbit:

  The Unicode version of this function, CreateProcessW,
  can modify the contents of this string. Therefore, this
  parameter cannot be a pointer to read-only memory (such
  as a const variable or a literal string) or the function
  could cause an access violation.

Or, in the case of a Lua binding, modify a Lua string with potentially disastrous consequences.

Leaving all that aside, though, wouldn't it be possible to implement a higher-level interface which did fork+exec on Unix and CreateProcess on Windows? That's a fairly commmon situation, anyway. CreateProcess seems to be able to specify stdin/stdout/stderr for the newly created process, which is what would normally done in the gap between the fork() and the exec() in Unix, so there is some hope for an interface which would work on both. Although I'm not sure how I would handle quoting of "shell" metacharacters.


Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
In reply to this post by Chris Marrin
My understanding is that cygwin and MSDev coexistence gets
more difficult as the MS version numbers get higher.

Some points to note:

 - I have had most of the functions you mention implmeneted for some
time. Let me know if you want code.
 -  errno() should call GetLastError() rather than use errno.
 - I would add to your list the following (I have them implemented),
  dir, exec, files, getpid, getppid, kill and unlink
 Note that the unlink direct equivalent is something like -

#ifdef _WIN32
static int l_unlink(lua_State  *L)
{
    const char* s = fixpath(L, 1);
        HANDLE h = CreateFile(s,GENERIC_READ, FILE_SHARE_READ|FILE_SHARE_WRITE,
          (LPSECURITY_ATTRIBUTES)0,	// address of security descriptor
          OPEN_EXISTING,
          FILE_FLAG_DELETE_ON_CLOSE|FILE_ATTRIBUTE_NORMAL,
          (HANDLE)0);				 	   // handle of file with attributes to copy

        if (h!=INVALID_HANDLE_VALUE) {
          CloseHandle(h);
          lua_pushboolean(L,1);
          return 1;
        }
        return pusherrstr(L, GetLastError());
}

#else

This simulates the *nix unlink semantics.

- In luasocket Diego has a very good sleep implementation that
sounds similar to what you have done. Check it out.

-

On 1/15/06, Chris Marrin <[hidden email]> wrote:
>
> I have been trying to port lposix to compile and link under MSDev. ...


Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
In reply to this post by Rici Lake-2
Hi Rici, Wim,

I consider myself a bit of a "CreateProcess" expert. There are lots
of issues here and exec/spawn probably deserves a thread of
its own.

1) I would suggest that posix spawn() be used rather fork()/exec().
posix_spawn() is avalable under linux and some others. Th spawn()
behaviour is implementable under win32.

2) ShellExecute (and relatives) vs CreateProcess. As you point out
they are very different beasts. I have implemented different Lua
functions for both of these. I believe that ShellExecute does not
have a *nix equivalent.

3) The fun starts when one allows the remapping of stdin/stderr/stdout.
I have done this (without threads) using named pipes and overlapped
io. It works ok.

4) Both the A and W versions of CreateProcess modify the command
string. Note that this "feature" is also a security hole.
"c:\program files param1"
Windows parses the string so that it attempts to execute c:\program.
It may expand or reduce the length of the command string.


DB

On 1/15/06, Rici Lake <[hidden email]> wrote:
>
> On 14-Jan-06, at 1:34 PM, Wim Couwenberg wrote:
>
> >>  You can use ShellExecute.
> >
> > ShellExecute (and CreateProcess) create a *new* process (if file is an
> > executable that is.)  The posix exec family replaces the executable
> > image of the running process.  Very different models...
>
>
> Also, the command parsing model is different. With exec(), you specify
> the argv array directly, so there is less need to worry about
> metacharacters (it is still possible that the utility will misinterpret
> an argument starting with a '-', though.) With ShellExecute and
> CreateProcess, you provide an unparsed command line, which is much
> riskier if you're constructing it from user input.
>
> (In checking the MSDN Library, I also discovered this interesting
> tidbit:
>
>    The Unicode version of this function, CreateProcessW,
>    can modify the contents of this string. Therefore, this
>    parameter cannot be a pointer to read-only memory (such
>    as a const variable or a literal string) or the function
>    could cause an access violation.
>
> Or, in the case of a Lua binding, modify a Lua string with potentially
> disastrous consequences.
>
> Leaving all that aside, though, wouldn't it be possible to implement a
> higher-level interface which did fork+exec on Unix and CreateProcess on
> Windows? That's a fairly commmon situation, anyway. CreateProcess seems
> to be able to specify stdin/stdout/stderr for the newly created
> process, which is what would normally done in the gap between the
> fork() and the exec() in Unix, so there is some hope for an interface
> which would work on both. Although I'm not sure how I would handle
> quoting of "shell" metacharacters.
>


Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Rici Lake-2

On 14-Jan-06, at 6:27 PM, D Burgess wrote:

Hi Rici, Wim,

I consider myself a bit of a "CreateProcess" expert.

Cool. I only read the description in the MSDN library, so
I'm perfectly happy to defer to your practical knowledge.

There are lots
of issues here and exec/spawn probably deserves a thread of
its own.

:)

1) I would suggest that posix spawn() be used rather fork()/exec().
posix_spawn() is avalable under linux and some others. Th spawn()
behaviour is implementable under win32.


I don't know how widely posix_spawn is implemented; it's not supported by FreeBSD afaik, and I doubt whether it's supported by Mac OS X. However, it can be implemented in terms of fork() and execve(). I wouldn't suggest a complete binding to posix_spawn() (or the Windows equivalent); rather a simplified (and extensible, perhaps) version which covered common cases.

2) ShellExecute (and relatives) vs CreateProcess. As you point out
they are very different beasts. I have implemented different Lua
functions for both of these. I believe that ShellExecute does not
have a *nix equivalent.

You could fork() and execve() /bin/sh, with more or less the same semantics, as far as I can see. I might be missing something, of course. I don't think that the Unix and Windows concepts of what a "shell" is are very well aligned.

3) The fun starts when one allows the remapping of stdin/stderr/stdout.
I have done this (without threads) using named pipes and overlapped
io. It works ok.

Maybe I misunderstood the MSDN writeup. A quick skim seemed to indicate that you could just hand a file handle over to the new process as its stdin handle, which would be the sort of thing you'd want to do in a standard "forking" server architecture. I'm aware that forking servers are considered declassé these days, and that event-driven single-process collaboration is sexier. On the other hand, forking servers are perfectly suitable, and a lot less subject to random resource leaks.

Anyway, it would be sufficient for most purposes if there were a way of doing the remapping of the three standard streams, however it was done. Lua only defines the three streams, so there would be no need to go beyond that in a basic implementation.

4) Both the A and W versions of CreateProcess modify the command
string. Note that this "feature" is also a security hole.
"c:\program files param1"
Windows parses the string so that it attempts to execute c:\program.
It may expand or reduce the length of the command string.

As I read the docs, you have the option of providing a specific file and a command line, or a commmand line from which Windows will extract the filename, but no option to provide a pre-chunked argv vector. I regard this as an error in API design, but as I said, the Unix version is not bullet-proof either. It's just easier to fix; if you're about to put a filename into an argv list, you pretty much only have to check to see if the first character is '-' or not. Unfortunately, the convention that -- terminates options parsing is only a convention, not enforced.

I quoted the section on the string being modified not because of the security issues involved with parsing metacharacters in the command line (which are a separate issue) but because it might be relevant to someone doing a naive binding; passing tostring(L, 3) to CreateProcess would be a Really Bad Idea if CreateProcess treats its argument as a mutable char* rather than a const char*. Of course, this is a general rule for binding functions which take character string arguments; if you don't know for sure that the parameter is const char* (and is not overridden), then you must make a copy of the string before passing it.

On the other hand, I don't offhand see an interface which abstracts command-line chunking. You can't insert escape characters into an argv vector without changing the semantics, so a note saying that "metacharacters may or may not be interpreted" would make the interface unusable.



Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
In reply to this post by Wim Couwenberg-2
Wim Couwenberg wrote:
"access", "chdir", "chmod", "errno", "exec", "getcwd",
"getenv", "mkdir", "putenv", "rmdir", "sleep", "stat",
"umask"


I have ported all the above to Win32, using MSDev 7.1. Issues in porting
to Win32:


There's no equivalent for the posix exec call in Win32.  How did you
implement it?

Win32 implements the full array of exec related calls, including execvp(), which lposix uses. In fact, I have found these calls to be surprisingly full featured, relative to their Unix counterparts!

--
chris marrin
[hidden email]
"As a general rule,don't solve puzzles that open portals to Hell"

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
In reply to this post by D Burgess-4
D Burgess wrote:

Hi Rici, Wim,

I consider myself a bit of a "CreateProcess" expert. There are lots
of issues here and exec/spawn probably deserves a thread of
its own.

Ok, I have looked at the descriptions of execvp() in win32 docs and the Posix spec. Win32 says it creates a "child process" while Posix says it "replaces the currently running process". I have trouble understanding this. Does the calling process exit cleanly, or just cease to exist?

I thought both did the same thing, meaning that I always thought the Unix version did not gun down the calling process. I guess I have been using Windows for too long, which is sad.

Anyway, generally, I think the goal of this "osex" package should not be to throw in the kitchen sink. This is why I felt that fork() should be left out. Nice, (albeit old fashioned), but its arcane process semantics has little chance of running on smaller systems. The same is true of spawn(), kill() and many of the other process control functions.

I would like it to contain a small number of useful file-system and environment calls. So I think the current list:

"access", "chdir", "chmod", "errno", "exec", "getcwd",
"getenv", "mkdir", "putenv", "rmdir", "sleep", "stat",
"umask"

should lose exec() altogether, and gain dir() and/or files().

On that subject, it seems wasteful to have a forms of directory discovery which return an array of string names (dir) and an iterator which will return each string name in turn. It seems like we only really need files(). How does that sound?

--
chris marrin
[hidden email]
"As a general rule,don't solve puzzles that open portals to Hell"

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
I would agree with leaving exec() for another day.
The only addition to the list that I would add is maybe unlink().

On 1/15/06, Chris Marrin <[hidden email]> wrote:

> Ok, I have looked at the descriptions of execvp() in win32 docs and the
> Posix spec. Win32 says it creates a "child process" while Posix says it
> "replaces the currently running process". I have trouble understanding
> this. Does the calling process exit cleanly, or just cease to exist?
On Windows it exits cleanly. If that is the question. While the
exec and spawn functions are OK, they have a few limitations with
GUI proceses.

> This is why I felt that fork() should be
> left out.
> should lose exec() altogether, and gain dir() and/or files().
agreed (for now).

>
> On that subject, it seems wasteful to have a forms of directory
> discovery which return an array of string names (dir) and an iterator
> which will return each string name in turn. It seems like we only really
> need files(). How does that sound?
the iterator version should be fine.
I like files(dir, [regex]) where dir is the directory name and regex is an
optional string.find compatible regex.

DB


Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
D Burgess wrote:
I would agree with leaving exec() for another day.
The only addition to the list that I would add is maybe unlink().

What does unlink() do that the existing os.remove() does not?

Posix has both a remove() and an unlink() function. The former follows the clib definition, i.e., it causes "the file ... to be no longer accessible by that name". This is the abstract equivalent of unlink() on a unix system, which has stated semantics that are specific to the various flavors of unix file system. Seems like remove() is the better function for cross-platform support.

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
        ,.`           ,b`    ,`                            , 1$'
     ,|`             mP    ,`                              :$$'     ,mm
   ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
  m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
 b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
 ```"```'"    `"`         `""`        ""`                          ,P`

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
I understand, someone correct me if I am wrong that on
Unix unlink will remove the file when the last user of it
closes the file. THis equals the FILEFLAG_DELETE_ON_CLOSE
on windows.

On windows remove and unlink require exclusive access.

DB

On 1/15/06, Chris Marrin <[hidden email]> wrote:
> D Burgess wrote:
> > I would agree with leaving exec() for another day.
> > The only addition to the list that I would add is maybe unlink().
>
> What does unlink() do that the existing os.remove() does not?
>
> Posix has both a remove() and an unlink() function. The former follows
> the clib definition, i.e., it causes "the file ... to be no longer
> accessible by that name". This is the abstract equivalent of unlink() on
> a unix system, which has stated semantics that are specific to the
> various flavors of unix file system. Seems like remove() is the better
> function for cross-platform support.
>
> --
> chris marrin              ,""$, "As a general rule,don't solve puzzles
> [hidden email]        b`    $  that open portals to Hell" ,,.
>          ,.`           ,b`    ,`                            , 1$'
>       ,|`             mP    ,`                              :$$'     ,mm
>     ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
>    m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
>   b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
> b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
> P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
>   ```"```'"    `"`         `""`        ""`                          ,P`
>


Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
D Burgess wrote:
I understand, someone correct me if I am wrong that on
Unix unlink will remove the file when the last user of it
closes the file. THis equals the FILEFLAG_DELETE_ON_CLOSE
on windows.

I think the clib remove() function is intentionally non-specific. Remove a file with a given name and no one can access it by that name again. It says nothing about what capabilities the caller must have or if the action actually frees any disk space. I think this is a good spec for a cross-platform function.

Ok, so maybe clib is not so arbitrary :-)

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
        ,.`           ,b`    ,`                            , 1$'
     ,|`             mP    ,`                              :$$'     ,mm
   ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
  m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
 b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
 ```"```'"    `"`         `""`        ""`                          ,P`

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Edgar Toernig
In reply to this post by D Burgess-4
D Burgess wrote:
>
> I understand, someone correct me if I am wrong that on
> Unix unlink will remove the file when the last user of it
> closes the file.

The name is removed immediately but the contents of the
file may stay and is removed only when the last user closes
the file.

The difference between unlink and remove is, that unlink won't
delete directories.  remove does as long as the directory is
empty.  On older system which lack remove you had to call rmdir
the delete directories.


> THis equals the FILEFLAG_DELETE_ON_CLOSE on windows.

Sure?  Will this example work?  It will give you ten
/different/ files:

    int fd[10];
    for (int i = 0; i < 10; i++) {
       fd[i] = open("foo", O_CREAT|O_TRUNC|O_RDWR, 0666);
       unlink("foo");
    }

> On windows remove and unlink require exclusive access.

Posix allows this restriction (errno=EBUSY) but it's unusual
and most Unixes let you delete busy files/directories. 

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
Edgar Toernig wrote:
D Burgess wrote:

I understand, someone correct me if I am wrong that on
Unix unlink will remove the file when the last user of it
closes the file.


The name is removed immediately but the contents of the
file may stay and is removed only when the last user closes
the file.

The difference between unlink and remove is, that unlink won't
delete directories.  remove does as long as the directory is
empty.  On older system which lack remove you had to call rmdir
the delete directories.

Yes, I see that the posix spec says this explicitly. Furthermore it says that you cannot remove non-empty directories. So maybe we should get rid of rmdir() from osex, since remove() in os can do this functionality. And you can discover if a file is a regular file or directory using osex.stat().

I'd also like to argue that we toss osex.getenv() since this function is in os as well. I originally thought we should keep it for completeness. But now I think we should minimize the footprint of osex. I would also like to replace putenv() with a setenv(name[,value]) function. This avoids the need to construct the putenv string. And it allows the name to be removed from the environment by omitting value. The original lposix package had this, but it had a third 'overwrite' param. I don't think this is needed, since you can see if a variable exists with os.getenv() and then avoid calling osex.setenv() if so.

I have also added a files() function, so here is the current proposed list:

access(path,[mode])	test file accessibility
chdir(path)		make the given path the cwd
chmod(path,mode)	change permissions on the path to the mode str
errno()			return the last error string,number
files([path])		return an iterator for the passed path
getcwd()		return the cwd
mkdir(path)		create a directory at the passed path
setenv(name[,value])	set a value in the environment
sleep(sec)		sleep for the passed number of seconds
stat(path,[selector])	return status for the file at the passed path
umask([mode])		set the mask for future file creations

I think that's a pretty complete set for starters. I have implemented this set for Win32 using MSDev 7.1. I will send the code as soon as I have tested it.

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
        ,.`           ,b`    ,`                            , 1$'
     ,|`             mP    ,`                              :$$'     ,mm
   ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
  m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
 b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
 ```"```'"    `"`         `""`        ""`                          ,P`

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Edgar Toernig
Chris Marrin wrote:
>
> [...] so here is the current proposed list:
>
> errno()		return the last error string,number

errno is pointless.  Unlike C, Lua has multiple return values.
And I would suggest to use strings like "ENOTEMPTY" instead of
errno numbers.  The numbers are totally useless.

> mkdir(path)		create a directory at the passed path

mkdir(path [, mode])

Ciao, ET.


PS: I don't like the name osex.  I'd simply put them into os.

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Rici Lake-2
In reply to this post by Chris Marrin
A few things:

On 15-Jan-06, at 12:43 PM, Chris Marrin wrote:

The original lposix package had this, but it had a third 'overwrite' param. I don't think this is needed, since you can see if a variable exists with os.getenv() and then avoid calling osex.setenv() if so.

setenv with overwrite (should be) atomic, which you cannot achieve with getenv/setenv. However, I don't think anyone actually uses this feature as a locking mechanism, so I don't see any need to reproduce it.

Given that, it seems like a table interface to getenv/setenv is much more natural than a function call interface. Moreover, it's faster. It also makes it easy for a sandbox to provide a fake environment. So personally, I'd prefer to scrap both getenv and setenv, and just provide an os[ex].env table.

putenv has the odd characteristic that the string you provide is mutable; changing the string is supposed to change the environment variable. ("This is the only way to guarantee that changing the environment doesn't leak memory.") There doesn't seem to be any obvious way to implement this in Lua; it does not conform to Windows env variable semantics, and so I agree that it should simply be scrapped.

Almost all of your remaining functions have to do with the filesystem, not the operating system; the exceptions are sleep() and possibly errno(). I think sleep() is uncontroversial; the remaining filesystem-related calls should go into a filesystem library, not an os library (imho).

errno() is very difficult to use from a scripting language; it is not easy (or possibly even possible) for the scripting language to know that there have not been any intervening calls which may change errno. In addition, POSIX semantics require errno to be unchanged by calls which do not return an error value. (However, iirc it also used to require that floating point errors be reported in errno.) Managing errno is therefore replete with subtle bugs. Finally, there are environments in which different "OS-like objects" use different errno-like globals.

On the whole, I think that errors should be retained with OS-like objects, which is essentially the semantics of standard C i/o, for example, where each FILE has its own error indicator (which can be both read and cleared.) That would be possible to implement, and I think it would be easier to implement that in an OS-independent way. In addition, it would be easier to use. Say, for example, that both files and sockets had a lasterror() method (or a lasterror key). That would make it much easier to write loops which respected error conditions:

  for line in f:lines() do
    ...
  end
  if f:lasterror() then   -- or possibly "if not f.eof"
    ... report the error
  end

I personally think that open() should be a member function on a directory object, rather than a global. An implementation should provide both a root directory object (sort of the analogy to stdin) and (if possible) a cwd() function; regardless, there would be some object onto which open() errors could be attached. This would simplify programming of directory traversal functions as well.




Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

D Burgess-4
In reply to this post by Edgar Toernig
I agree with this.

On 1/16/06, Edgar Toernig <[hidden email]> wrote:
> PS: I don't like the name osex.  I'd simply put them into os.


Reply | Threaded
Open this post in threaded view
|

Re: Problems with lposix on win32

Chris Marrin
In reply to this post by Edgar Toernig
Edgar Toernig wrote:
Chris Marrin wrote:

[...] so here is the current proposed list:

errno()		return the last error string,number


errno is pointless.  Unlike C, Lua has multiple return values.
And I would suggest to use strings like "ENOTEMPTY" instead of
errno numbers.  The numbers are totally useless.

It's no more pointless than in C, where you can compare it against constants for useful results. But I agree that Lua does not have switch() or enums making it difficult to use integer return values. I will cobble together an initial (small) set of string return values. I am already returning multiple values (string,number) because that is what lposix did. The string is the strerror() string. Perhaps a better approach would be to return a string value for the error number along with a numeric version and add a separate strerror() function. This would allow lookup of error codes that do not come from these functions. If errno is a value for which I don't have a symbolic constant, I would return nil for the string and still return the number. That gives us fully general functionality.

How does that sound?



mkdir(path)		create a directory at the passed path


mkdir(path [, mode])

Sounds good. Done. But mode will have no effect on win32 because mkdir() on that platform does not take a mode param and I don't think win32 obeys the umask when creating directories. Anyone have info to the contrary?


Ciao, ET.


PS: I don't like the name osex.  I'd simply put them into os.

I don't think that's possible using the 5.1 package model. I suppose it is possible to make:

    require "osex"

add things to the 'os' table, assuming it exists. But that seems, at least, poor practice.

But I did just notice that this reads like "o-sex", so maybe "exos" would be better? :-)

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
        ,.`           ,b`    ,`                            , 1$'
     ,|`             mP    ,`                              :$$'     ,mm
   ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
  m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
 b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
 ```"```'"    `"`         `""`        ""`                          ,P`

123