Proposal: Virtualization of Filesystem Operations for Lua 5.2.

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Georg Lehner
Hello!

I propose to include a virtualization layer for filesystem operations in
Lua 5.2.

Problems with the current approach to filesystem operations are:

- incomplete functionality (e.g. directory operations)
- their behaviour depends on the platform where Lua is built
- in some environments (deeply embedded..) the current functionality
might be
   meaningless (and must be stripped out when porting)

Advantages of virtualization, besides releaving the problems, are (see
also [1])

  - improved portability both for the Lua interpreter and for Lua scripts
  - extensibility, generalization and reusability:  Write portable Lua
scripts
    for anything what can be made to look like a filesystem.

Tcl/Tk features a successful filesystem virtualization layer[2].

A simplified proof of concept implementation is available at [1] with
about 400 lines
of Lua code.  I claim that an implementation of the virtualization layer
will be
slightly less code then actually contained in the Lua interpreter
(without including
a "real" filesystem driver).


I propose the following features (in more or less erratic order):

Extension of existing functions

- open(path,"d+") creates a directory
- open(path,"d"[,attributes]) returns an iterator over the directory path,
   returning filenames of existing files matching the attributes table
- remove(path) also removes (empty) directories


Additional functionality

- stat(path,[attributes]) sets/gets file attributes of path

- cd([path]) sets/gets the working directory of the current function
- mount(volume, path[,options..]) attaches the volume (see later) onto
   the mount point path
- bind(from, to [options..]) creates links between directory trees
- unmount(volume[, path]) deattaches volumes or links from the mount point

See the paper "Lexical File Names in Plan 9 or Getting Dot-Dot Right" from
Rob Pike [3] for the proposed mount/bind/unmount semantic

Implementation specifics

- each function has its own filesystem, which is by default inherited from
   its creator, but can be restricted (for sandboxes)
- '/' is the path separator, each filesystem driver provides translation to
   its specific path representation

- filesystem drivers implement only low level access to filesystem objects:
   read/write strings from/to files, create/delete files and directories,
   query directories and file attributes.
- the virtualization layer implements higher level features like lines(),
   formatted read and write.
- Lua scripts and pure Lua libraries implement high level applications like
   file copying or removing.

- A lua state has to be set up by its creator (e.g. the standalone Lua
   interpreter) to mount a specific filesystem driver.  This is however an
   optional step
- The current functionality of the io and os library can be maintained in
   the Lua interpreter and expanded as a filesystem driver which is mounted
   by default.  This maintains backwards compatiblity for existing Lua
script
   without to much hassle and overhead.

- Volume instance specific identifiers like "C:", "D:",
   "http://www.magma.com.ni/sw/lua/luafs" are set up as the "root" of a
   volume by the file system driver.


Permissions and Authorization

- Authorization can be done when a volume is instantiated, i.e. not Lua
   interpreter specific but filesystem driver specific
- Lua only needs to operate on already accessible filesystem objects,
permission
   manipulation can be filessystem driver specific


Terms

- filesystem object: any object which can be represented by a directed graph
   with named edges and where each node can hold a set of named objects
with
   a byte stream content and optional attributes
- filesystem driver: Lua library which operates on a specific class of
   filesystem objects
- volume: filesystem driver instance which operates on a filesystem object.


Filesystem class examples

- Lua tables
- Windows Registry
- Websites
- IMAP Mailaccounts
- Floppies, USB-Sticks, lokal and remote Windows drives
- zipfiles, tarballs, any other archives
- source repositories, git, svn, ...
- C-String arrays/data structures for deeply embedded Lua


The case of the Lua module search path

- with the proposed union mounts, search paths for Lua modules
   might become obsoleted by binding all locations where required
   modules live to a standard path in the virtual filesystems namespace

Best Regards,

     Jorge-León


[1] http://sw.magma.com.ni/LuaFs
[2] http://wiki.tcl.tk/2138
[3] http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Luiz Henrique de Figueiredo
> - open(path,"d+") creates a directory
> - open(path,"d"[,attributes]) returns an iterator over the directory path,
>   returning filenames of existing files matching the attributes table

Interesting but perhaps it is not a good idea to overload open when mkdir
exists and also readdir etc.

> - remove(path) also removes (empty) directories

os.remove already does that, at least under POSIX. I don't know about Windows.

Perhaps your proposal can be first fleshed out as a working library
so that we evaluate it...

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Louis Mamakos
In reply to this post by Georg Lehner

On Nov 2, 2011, at 3:53 PM, Georg Lehner wrote:

> Hello!
>
> I propose to include a virtualization layer for filesystem operations in Lua 5.2.
>
> Problems with the current approach to filesystem operations are:
>
> - incomplete functionality (e.g. directory operations)
> - their behaviour depends on the platform where Lua is built
> - in some environments (deeply embedded..) the current functionality might be
>  meaningless (and must be stripped out when porting)
>
> Advantages of virtualization, besides releaving the problems, are (see also [1])
>
> - improved portability both for the Lua interpreter and for Lua scripts
> - extensibility, generalization and reusability:  Write portable Lua scripts
>   for anything what can be made to look like a filesystem.
>
> Tcl/Tk features a successful filesystem virtualization layer[2].

[…]

One of the useful bits of the Tcl I/O interfaces is exposing a set of
operations common across different I/O objects and operations, including
communication interfaces (e.g., serial ports, network connections, etc.)  One
aspect of that was also having a generalized interface for asynchronous
notifications (in Tcl, the "fileevent" mechanism which would enable in a
generalized way asynchronous I/O and a common event loop mechanism within
Tcl that other extensions could hook into.

It would be really great if these aspects might be considered as part of the
file system virtualization that you propose.  I've often wondered if simply
porting the Tcl I/O channel architecture to Lua might be an interesting
experiment.

Louis Mamakos


Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Georg Lehner
In reply to this post by Luiz Henrique de Figueiredo
Hello!

On 11/02/11 21:02, Luiz Henrique de Figueiredo wrote:
>> - open(path,"d+") creates a directory
>> - open(path,"d"[,attributes]) returns an iterator over the directory path,
>>    returning filenames of existing files matching the attributes table
>>      
> Interesting but perhaps it is not a good idea to overload open when mkdir
> exists and also readdir etc.
>    
mkdir and readdir seem not to exist in the core Lua libraries - at least
I did not find them.

I tried not to add new functions, to keep the interface small - however
I have no strong opinion
with respect to splitting file and directory manipulation into different
functiongroups or
maintain them merged together.

I like the io.lines(path) functionality for iterating over files, so I
guessed iterating over
directories should operate in a similar way, hence open(path,"d")
returns an iterator.  It could
also be named readdir(path) or anything similar of course.

Note:  My proposal, and the Proof-Of-Concept module are named "fs" and
containd all functions for
file and directory manipulation:  fs.open(), fs.close(), fs.flush(),
fs.read(), fs.write(), ...

io.write() or os.remove() would not necessarily go away,  they would
rather be wrapped eventually
inside fs.write(), fs.remove().


>> - remove(path) also removes (empty) directories
>>      
> os.remove already does that, at least under POSIX. I don't know about Windows.
>
> Perhaps your proposal can be first fleshed out as a working library
> so that we evaluate it...
>
>    
Thats why I offer a Proof-Of-Concept module for the higher level
(fs.lua) and the lower level
functions (fsio.lua), see:

http://sw.magma.com.ni/LuaFs

Writing a module which reads/writes/iterates over a Lua table should be
trivial too and could
illustriate some points, e.g. persistence of tables simply by a
recursive file copy operation.
Somebody interested?


Regards,

     Jorge-León


Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Patrick Rapin
It is probably already too late to introduce a small feature in the
upcoming Lua 5.2, so such a big feature will have to wait until at
least Lua 5.3 ...
This apart, I like very much that proposal. If such a file system
visualization existed in 5.1, I could have used it to load modules
from a ZIP file, instead of patching Lua source code.
And I could have written a driver for our printer firmware pseudo file
system, instead of removing the "io" library and writing a custom
binding.
So yes, it is definitely a good idea. I just fear that it could
generate a huge amount of code when everything is put together.

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Petite Abeille

On Nov 2, 2011, at 11:11 PM, Patrick Rapin wrote:

> This apart, I like very much that proposal. If such a file system
> visualization existed in 5.1, I could have used it to load modules
> from a ZIP file, instead of patching Lua source code.
> And I could have written a driver for our printer firmware pseudo file
> system, instead of removing the "io" library and writing a custom
> binding.

FWIW, I like Georg Lehner proposal as well :)

That said, one could easily "virtualize/overload" the existing io module to suit its need.

For example:

local io = require( 'io' )
local ioopen = io.open

io.open = function( filename, mode, ... )
  -- do stuff depending on mode

  -- or default back to the standard implementation
  return ioopen( filename, mode )
end


Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Rena
On Wed, Nov 2, 2011 at 16:29, Petite Abeille <[hidden email]> wrote:

>
> On Nov 2, 2011, at 11:11 PM, Patrick Rapin wrote:
>
>> This apart, I like very much that proposal. If such a file system
>> visualization existed in 5.1, I could have used it to load modules
>> from a ZIP file, instead of patching Lua source code.
>> And I could have written a driver for our printer firmware pseudo file
>> system, instead of removing the "io" library and writing a custom
>> binding.
>
> FWIW, I like Georg Lehner proposal as well :)
>
> That said, one could easily "virtualize/overload" the existing io module to suit its need.
>
> For example:
>
> local io = require( 'io' )
> local ioopen = io.open
>
> io.open = function( filename, mode, ... )
>  -- do stuff depending on mode
>
>  -- or default back to the standard implementation
>  return ioopen( filename, mode )
> end
>
>
>

One of my favourite features of Lua is you can replace anything. You
could add support for reading zip files transparently by replacing
io.open and friends with your own wrappers, for example. Talk about
"drivers" that seem to be meant to define how functions behave seems a
bit redundant when you can change their behaviour already by replacing
them.

Also, I'd definitely want "chdir" rather than "cd". Readability is good.

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Miles Bader-2
HyperHacker <[hidden email]> writes:
> One of my favourite features of Lua is you can replace anything. You
> could add support for reading zip files transparently by replacing
> io.open and friends with your own wrappers, for example. Talk about
> "drivers" that seem to be meant to define how functions behave seems
> a bit redundant when you can change their behaviour already by
> replacing them.

One has to be a bit careful how one goes about it though, since it's
not uncommon to cache module functions in local variables... so any
wrapping needs to be done in code that is guaranteed to run before
anything uses the wrapped functions...

> Also, I'd definitely want "chdir" rather than "cd". Readability is good.

Agreed!

-miles


--
The car has become... an article of dress without which we feel uncertain,
unclad, and incomplete.  [Marshall McLuhan, Understanding Media, 1964]

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Sean Conner
In reply to this post by Patrick Rapin
It was thus said that the Great Patrick Rapin once stated:
> It is probably already too late to introduce a small feature in the
> upcoming Lua 5.2, so such a big feature will have to wait until at
> least Lua 5.3 ...
> This apart, I like very much that proposal. If such a file system
> visualization existed in 5.1, I could have used it to load modules
> from a ZIP file, instead of patching Lua source code.

  You need to modify Lua sources to do that?  Just add an entry to
package.loaders to scan a ZIP file for modules.  

        package.loaders

        A table used by require to control how to load modules.

        Each entry in this table is a searcher function. When looking for a
        module, require calls each of these searchers in ascending order,
        with the module name (the argument given to require) as its sole
        parameter. The function can return another function (the module
        loader) or a string explaining why it did not find that module (or
        nil if it has nothing to say).

  No direct modification of Lua required.

  -spc (Unless there's something I'm missing ... )


Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Patrick Rapin
>  You need to modify Lua sources to do that?  Just add an entry to
> package.loaders to scan a ZIP file for modules.

You are right. The fact is that I was not aware of the package.loaders
feature at the time I needed the special loading.

Anyway, let's take another example.
I currently would like to pass UTF-8 encoded path names on Windows.
That concerns at least loadfile, dofile, require, os.execute,
os.rename, os.remove, os.tmpname, io.open, io.input, io.output,
io.lines and io.popen.
What would be the best solution on current Lua ? Is it to rewrite all
those functions?
Probably could it be much easier if there is a well designed common
virtualization layer.

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Daniel Silverstone
On Thu, Nov 03, 2011 at 09:26:59AM +0100, Patrick Rapin wrote:
> Probably could it be much easier if there is a well designed common
> virtualization layer.

To play devil's advocate a little here -- why can this 'virtualisation layer'
not be LuaFilesystem ?

That way a large amount of the work is already done and used by plenty of code.

D.

--
Daniel Silverstone                         http://www.digital-scurf.org/
PGP mail accepted and encouraged.            Key Id: 3CCE BABE 206C 3B69

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

steve donovan
On Thu, Nov 3, 2011 at 10:43 AM, Daniel Silverstone
<[hidden email]> wrote:
> To play devil's advocate a little here -- why can this 'virtualisation layer'
> not be LuaFilesystem ?

Well, that's it - require 'lfs' is a very common way to start writing
a program ;)

There's lots of code using it, and plenty of scope for extending
LuaFileSystem so that (e.g) it can provide common abstractions for
similar file system concepts that are implemented differently, like
symbolic links.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Patrick Rapin
>> To play devil's advocate a little here -- why can this 'virtualisation layer'
>> not be LuaFilesystem ?
>
> Well, that's it - require 'lfs' is a very common way to start writing
> a program ;)

True, there is a common goal between LuaFileSystem and the proposed
virtualization system.

But all the 12 functions I have previously listed, like dofile,
io.open or os.execute, are not in LuaFileSystem but in the standard
library distribution. So you are talking about a more generalized
version of lfs, not the current 1.5 version.
Also, even in lfs 1.5, supporting UTF-8 for Windows filenames like in
my example would require a lot of modifications.

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Jerome Vuarand
2011/11/3 Patrick Rapin <[hidden email]>:

>>> To play devil's advocate a little here -- why can this 'virtualisation layer'
>>> not be LuaFilesystem ?
>>
>> Well, that's it - require 'lfs' is a very common way to start writing
>> a program ;)
>
> True, there is a common goal between LuaFileSystem and the proposed
> virtualization system.
>
> But all the 12 functions I have previously listed, like dofile,
> io.open or os.execute, are not in LuaFileSystem but in the standard
> library distribution. So you are talking about a more generalized
> version of lfs, not the current 1.5 version.
> Also, even in lfs 1.5, supporting UTF-8 for Windows filenames like in
> my example would require a lot of modifications.

It didn't seem like a lot when I did it. In my Windows builds, I patch
Lua [1] and LFS [2] to use UTF-8 encoding for all filenames.

[1] https://bitbucket.org/doub/electronmeat/src/tip/srcweb/lua-5.1.4/lua-wstring.patch
[2] https://bitbucket.org/doub/electronmeat/src/tip/srcweb/luafilesystem-1.2.1/win32-utf-8.patch

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Timm S. Mueller
In reply to this post by Georg Lehner
On Wed, 02 Nov 2011 20:53:24 +0100
Georg Lehner <[hidden email]> wrote:

Hello Georg,
 
> I propose to include a virtualization layer for filesystem operations in
> Lua 5.2.

> [..]

not to criticize your proposal, just a few general thoughts, as I was
once concerned with the same issue.

I would come out in opposition to any half-baked solution becoming part
of the Lua core. Everybody is free to implement an additional
filesystem library of her own, everybody is free to modify the core,
and everybody is free to link Lua against a different C library, which
may implement nifty features.

Modifications to the Lua core should not impair its simplicity,
transparency, and to some extent, its size and performance. The sooner
you call out to a library, the better. Sticking to what is offered
through stdio.h is one of the greater achievements of Lua. Any
complication can be justified only if it offers a comprehensive
solution to real problems.

I have once written a vfs in the shape of a filesystem and device
driver architecture for some kind of embedded operating system. It
supported mounting, multiple roots, loops, asynchronous I/O, locking,
automounting etc. Lua was adapted to this and part of the distribution.
The result was fully transparent, in that require, dofile, module
loading, and even stdin and stdout went through the vfs layer, and
modules could be loaded from mountable loop devices such as zip files.
Stdio was shielded completely, not just a little here and there. [1]

But while this is what I would call a comprehensible solution to a real
problem, I would recommend against it in the general case:

1. The vfs was too large and complex.

2. The vfs layer was written in C and had no knowledge of Lua, hence
new drivers could only be written in C. From a Lua perspective, I would
find this unacceptable.

3. The vfs introduced concepts which may look unfamiliar to people
programming applications on Linux or Windows (directory handles, file
handles and filesystem locks all based on the same 'IO packet', for
example, which could be used for asynchronous I/O directly).

4. The vfs introduced a distinct filesystem notation of its own, which
may look even more unfamiliar - it did not resemble either Windows or
Unix. I suspect that this would be hardly tolerable for the general
purpose.

On the other hand, I would just love to have some 'hooks' in the Lua
core to plug in a filesystem layer - but not just one, ANY filesystem
layer. For the chicken-egg-condition that it imposes, it may appear as
if a vfs should be linked in statically and become part of the Lua
core. But I would suggest to think again and to come up with just the
hooks that allow for dynamic reprogramming of a Lua filesystem's
semantics, i.e. a fully require'able vfs that, once it is loaded, is
also retargeted to by the 'io' library. (New functions, OTOH, should
probably go into a library table of its own.)

It would be very gratifying if someone had a go at it and tried to come
up with such a comprehensible vfs design in mind (no need to actually
implement it right at the outset), and just the (hopefully few,
surgical) hooks needed in the Lua core. Then I'm all for it.

- Timm

[1] http://hg.neoscientists.org/teklib/file/418c081f184c/doc/text/io.txt

--
Timm S. Mueller <[hidden email]>
Schulze & Mueller GbR, Jungstr. 2, 10247 Berlin,
Gesellschafter: Franciska Schulze, Timm S. Mueller,
Tel. +49 30 85610000, http://www.schulze-mueller.de/

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Sean Conner
In reply to this post by Patrick Rapin
It was thus said that the Great Patrick Rapin once stated:

> >  You need to modify Lua sources to do that?  Just add an entry to
> > package.loaders to scan a ZIP file for modules.
>
> You are right. The fact is that I was not aware of the package.loaders
> feature at the time I needed the special loading.
>
> Anyway, let's take another example.
> I currently would like to pass UTF-8 encoded path names on Windows.
> That concerns at least loadfile, dofile, require, os.execute,
> os.rename, os.remove, os.tmpname, io.open, io.input, io.output,
> io.lines and io.popen.
> What would be the best solution on current Lua ? Is it to rewrite all
> those functions?

  Lua is based upon what ANSI C (1989) provides.  And C89 does not provide
anything but a way to open a file with a given name.  What that name is, how
it is formatted, is outside the scope of C89, and thus, outside the scope of
Lua.  

  With that said, what does fopen() under Windows do when given a UTF-8
encoded filename?  The UTF-8 encoding was designed to work well (given
sufficient values of "well") with ASCII (well, such that str*() functions
won't break any more with a UTF-8 encoding than with an ASCII encoding), and
usually fopen() ends up calling the underlying operating system functions to
"open a file" and simply passes along a blob of data terminated with a zero
byte as a "name".  

> Probably could it be much easier if there is a well designed common
> virtualization layer.

  This isn 't a new problem; it's been around for years.  For example,
here's the Common Lisp approach:

        http://www.gigamonkeys.com/book/files-and-file-io.html

because it had [1] to handle filenames like:

System          File Name
==================================================
TOPS-20         <LISPIO>FORMAT.FASL.13
TOPS-10         FORMAT.FAS[1,4]
ITS             LISPIO;FORMAT FASL
MULTICS         >udd>LispIO>format.fasl
TENEX           <LISPIO>FORMAT.FASL;13
VAX/VMS         [LISPIO]FORMAT.FAS;13
UNIX            /usr/lispio/format.fasl
==================================================

(from http://www.ida.liu.se/imported/cltl/clm/node203.html)

  -spc (And that's not even the tip of the iceberg ... )  

[1] Today, you have Unix and Windows.  And they're *mostly* compatible
        [2].

[2] No one seems to realize that Windows can accept '/' as part of a
        path.  In fact, the following two file names point to the same file:

        C:/windows/windows32/bin
        C:\windows\windows32\bin

        It was only CMD.COM that enforced the use of '\' as a path
        separator, because MS-DOS inherited behavior from CP-M (which in
        turn inherited the behavior from RT-11) of using '/' to designate
        command switches (which under Unix is the '-' character).  The
        kernel of Windows (and MS-DOS, going back to 2.0) never cared which
        was used; you could use either one.

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Sean Conner
In reply to this post by Timm S. Mueller
It was thus said that the Great Timm S. Mueller once stated:
>
> I have once written a vfs in the shape of a filesystem and device
> driver architecture for some kind of embedded operating system. It
> supported mounting, multiple roots, loops, asynchronous I/O, locking,
> automounting etc. Lua was adapted to this and part of the distribution.
> The result was fully transparent, in that require, dofile, module
> loading, and even stdin and stdout went through the vfs layer, and
> modules could be loaded from mountable loop devices such as zip files.
> Stdio was shielded completely, not just a little here and there. [1]

  At one point, I had written my own replacement for C's stdio.h mechanism
since I found it limiting [2].  The first implementation was buggy and hard
to use.  The second implmentation was less buggy and a bit easier to use.
The third implementation was better still, but then I found out about the
various extentions that the GNU C library provides, thought hard about where
my code was going to run (the particular program had run exclusively on
Linux) and being realistic about things, decided to scrap all that code and
go with the GNU extentions (since it provided everything I felt missing from
C's stdio.h).  

  Sure, it's less portible now, but it's also less code I have to maintain.
Six of one, half-dozen the other.

  -spc (Likes that Lua is small ... )

[1] Not my footnote.

[2] The biggest limitation---you can't treat an arbitrary region of
        memory as a FILE *.  

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Luiz Henrique de Figueiredo
> then I found out about the various extentions that the GNU C library provides

For the lazy among us, could you please summarize those extensions or
point to a document? Thanks.

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Tony Finch
In reply to this post by Sean Conner
Sean Conner <[hidden email]> wrote:
>
> The biggest limitation---you can't treat an arbitrary region of
> memory as a FILE *.

On BSD (including Mac OS) there is funopen() which is roughly equivalent
to glibc's fopencookie(). Both allow you to plug in your own IO functions
behind stdio's buffering.

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
Trafalgar: Southwesterly veering northwesterly 6 to gale 8, perhaps severe
gale 9 later in south. Very rough in southeast, otherwise high or very high.
Rain or squally, thundery showers. Moderate or poor.

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

Sean Conner
In reply to this post by Luiz Henrique de Figueiredo
It was thus said that the Great Luiz Henrique de Figueiredo once stated:
> > then I found out about the various extentions that the GNU C library provides
>
> For the lazy among us, could you please summarize those extensions or
> point to a document? Thanks.

  I found out about them by going through stdio.h under Linux, and doing a
man on the functions found therein.  For instance:

NAME
       fmemopen, open_memstream, open_wmemstream -  open memory as stream

SYNOPSIS
       #define _GNU_SOURCE
       #include <stdio.h>

       FILE *fmemopen(void *buf, size_t size, const char *mode);

       FILE *open_memstream(char **ptr, size_t *sizeloc);

       #define _GNU_SOURCE
       #include <wchar.h>

       FILE *open_wmemstream(wchar_t **ptr, size_t *sizeloc);

DESCRIPTION
       The  fmemopen()  function opens a stream that permits the access speci‐
       fied by mode.  The stream allows I/O to be performed on the  string or
       memory  buffer  pointed  to  by buf.  This buffer must be at least size
       bytes long.

        ...

NAME
       fopencookie - opening a custom stream

SYNOPSIS
       #define _GNU_SOURCE
       #include <stdio.h>

       FILE *fopencookie(void *cookie, const char *mode,
                         cookie_io_functions_t io_funcs);

DESCRIPTION
       The  fopencookie()  function  allows  the programmer to create a custom
       implementation for a standard  I/O  stream.   This  implementation can
       store the stream's data at a location of its own choosing; for example,
       fopencookie() is used to implement fmemopen(3), which provides a stream
       interface to data that is stored in a buffer in memory.

        ...

  There's more at
http://www.gnu.org/software/libc/manual/html_mono/libc.html (warning:  huge
file) but it does describe not only the standard functions, but other
functions than the GNU C library provides (including the above).

  -spc (And that covers the library calls---the GNU C compiler has any
        number of extentions as well ... )

12