[ANN] Lua 5.2.0 (beta-rc2) now available

classic Classic list List threaded Threaded
70 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Roberto Ierusalimschy
> On Wed, Jun 22, 2011 at 12:21 AM, Luiz Henrique de Figueiredo
> <[hidden email]> wrote:
> > All feedback welcome. Thanks.
>
> Just an observation about the folly of depending on undocumented internals.
>
> Previously people could hijack the existing file objects by packing
> their own FILE* in the LUA_FILEHANDLE userdata, which was just FILE**.
>  Now that userdata looks like this:
>
> typedef struct LStream {
>    FILE *f;  /* stream */
>    lua_CFunction closef;  /* to close stream (NULL for closed streams) */
>  } LStream;
>
> (LuaJIT uses something similar)
>
> I mention this because I think there's a need for a standard way for
> extensions to create Lua file objects, or even on level of a
> 'io.fdopen' function that would create a file object from an existing
> file-like handle.  One particular use case would be a Windows
> extension wishing to provide a UTF-8 aware io.open, or to redefine
> io.popen so it does not depend on the particular broken implementation
> of the Windows CRT.

The idea of this change is to allow that. Probably LStream should be
exported with a small documentation about how to use it; more or
less something like this:

   - create a userdata with a structure that is an "extension" of LStream.
   - set its metatable to "FILE*" (luaL_setmetatable)
   - set field 'f'
   - set field 'closef'

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Roberto Ierusalimschy
In reply to this post by David Manura
> On Tue, Jun 21, 2011 at 6:21 PM, Luiz Henrique de Figueiredo
> <[hidden email]> wrote:
> > Lua 5.2.0 (beta-rc2) is now available at
>
> Do the new setlocale calls in ldo.c change thread safety assumptions
> Lua?  http://stackoverflow.com/questions/4057319/is-setlocale-thread-safe-function

It seems so :(

Suggestions?

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Mike Pall-28
In reply to this post by Roberto Ierusalimschy
Roberto Ierusalimschy wrote:
> Probably LStream should be exported [...]

It's never a good idea to export an internal structure. Especially
not, if it's allocated externally. You cannot add an element or
increase the struct size without breaking the ABI. You already
made that mistake with lua_Debug, please don't make it again.

Either add an API function to create a proper file object or
disallow injecting such objects.

--Mike

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

steve donovan
On Wed, Jun 22, 2011 at 3:20 PM, Mike Pall <[hidden email]> wrote:
> not, if it's allocated externally. You cannot add an element or
> increase the struct size without breaking the ABI.

To be fair, Lua does not make guarantees about the ABI with new versions.

But it is true that an API function would isolate the programmer from
the actual mechanism; LuaJIT could provide it as well without having
to expose internal details.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Roberto Ierusalimschy
In reply to this post by Mike Pall-28
> It's never a good idea to export an internal structure. Especially
> not, if it's allocated externally. You cannot add an element or
> increase the struct size without breaking the ABI.

Usually new Lua versions break the ABI in several other ways, so
that does not seem a big problem.


> You already made that mistake with lua_Debug, please don't make it
> again.

What was (is) the problem with lua_Debug?

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Gé Weijers
In reply to this post by Lorenzo Donati-2
On Wed, 2011-06-22 at 06:57 +0200, Lorenzo Donati wrote:
> I've already argued on the other thread (5.2 beta-rc1) that I'd prefer a
> limit "per-block". There I motivated my preference with an use case for
> error management.
>

I agree. There are two possible restrictions that would work:

- your suggestion: allow one definition of a label per block. Algol
scope rules apply within a function:

        ::label::
        do
                goto label -- should jump FORWARD

        ::label::
        end

- disallow a definition of a label if that label is already visible in
the scope. The above example would be illegal, but the next one would be
ok:

        for i in something() do
                ...
                goto continue
                ...
                ::continue:: This is a little tricky in a one-pass compiler:
        end
        for i in something() do
                ...
                goto continue
                ...
        ::continue::
        end

Advantage: you can decide on the fly whether a jump is forward or
backward, and you disallow one label hiding another. This is a little
more restrictive, but your 'continue' example works.




Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Roberto Ierusalimschy
In reply to this post by steve donovan
> On Wed, Jun 22, 2011 at 3:20 PM, Mike Pall <[hidden email]> wrote:
> > not, if it's allocated externally. You cannot add an element or
> > increase the struct size without breaking the ABI.
>
> To be fair, Lua does not make guarantees about the ABI with new versions.
>
> But it is true that an API function would isolate the programmer from
> the actual mechanism; LuaJIT could provide it as well without having
> to expose internal details.

The advantage of exporting the structure is that the user may add her
own extra fields, if needed. (Are there real cases for this need?)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Mike Pall-28
In reply to this post by Roberto Ierusalimschy
Roberto Ierusalimschy wrote:
> > It's never a good idea to export an internal structure. Especially
> > not, if it's allocated externally. You cannot add an element or
> > increase the struct size without breaking the ABI.
>
> Usually new Lua versions break the ABI in several other ways, so
> that does not seem a big problem.

So when you want to change the internal implementation, you may
need to break the ABI. Or you can't change the implementation.
Either case is bad.

ABI breakage, especially if it can be avoided, is not a virtue.

> > You already made that mistake with lua_Debug, please don't make it
> > again.
>
> What was (is) the problem with lua_Debug?

The struct is allocated outside the core VM. So it has a fixed
size and there's only one element for use as a context by the
core. There's no provision for extensibility (like padding).
That's seriously limiting the implementation choices for the core.

[You can look into the sources of LuaJIT what strange kind of
workarounds were needed to fit the context into 'int i_ci'.]

--Mike

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Roberto Ierusalimschy
In reply to this post by Roberto Ierusalimschy
> > On Wed, Jun 22, 2011 at 3:20 PM, Mike Pall <[hidden email]> wrote:
> > > not, if it's allocated externally. You cannot add an element or
> > > increase the struct size without breaking the ABI.
> >
> > To be fair, Lua does not make guarantees about the ABI with new versions.
> >
> > But it is true that an API function would isolate the programmer from
> > the actual mechanism; LuaJIT could provide it as well without having
> > to expose internal details.
>
> The advantage of exporting the structure is that the user may add her
> own extra fields, if needed. (Are there real cases for this need?)

There seems to be another problem with the function approach (assuming
an opaque structure). A subtle detail when creating files in Lua is
that the creation of the userdata may fail (memory error). So, if the
user opens a file and then calls this new function, Lua may raise an
error and the open file is lost forever. The natural order is to first
create the userdata and then open the file. (Once the file is stored in
the userdata, the garbage collector will ensure it will be eventually
closed.) With an opaque structure, that would demand two functions
(one to create the userdata and another to store the FILE*).

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Peter Cawley
On Wed, Jun 22, 2011 at 3:10 PM, Roberto Ierusalimschy
<[hidden email]> wrote:
> There seems to be another problem with the function approach (assuming
> an opaque structure). A subtle detail when creating files in Lua is
> that the creation of the userdata may fail (memory error). So, if the
> user opens a file and then calls this new function, Lua may raise an
> error and the open file is lost forever.

Then make this new function return NULL rather than throwing an error.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Roberto Ierusalimschy
> On Wed, Jun 22, 2011 at 3:10 PM, Roberto Ierusalimschy
> <[hidden email]> wrote:
> > There seems to be another problem with the function approach (assuming
> > an opaque structure). A subtle detail when creating files in Lua is
> > that the creation of the userdata may fail (memory error). So, if the
> > user opens a file and then calls this new function, Lua may raise an
> > error and the open file is lost forever.
>
> Then make this new function return NULL rather than throwing an error.

For that it would need to enclose the "lua_newuserdata" into a
lua_pcall. It is doable, but the "export structure" approach
seems simpler (does not need lua_pcall) and more flexible (user
may add other fields).

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

James Graves-2
In reply to this post by David Kastrup
David Kastrup <[hidden email]> wrote:

> Alexandre Rion <[hidden email]> writes:
>
>> I also think that labels should be per block.
>
> +1
>
> Local variables are per block.  This makes it possible to have
> self-contained syntactical entities with minimal outside interaction.
> function-local labels break this concept, and it is not clear to me how
> this would affect label/goto outside of functions.

I think the one-label-per-function restriction is fine.

If your function is soooo big that you're losing track of what labels
you've been using, then you've got other problems.  Having unique
labels in each function makes it easier for me to read and understand
your code.

I would like to encourage all developers to:

A) Use 'goto' sparingly, of course.  Error handling and a 'continue'
replacement are acceptable to me.  However, if a little
re-organization would allow you to use some other control structure,
that is preferred.

B) Put as much thought into naming labels as you would naming local
variables.  Please use descriptive names, such as ::retry_read::
instead of just ::retry:: and ::skip_blank_line:: instead of
::continue::, to draw from some recent examples.

James Graves

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Lorenzo Donati-2
In reply to this post by David Kastrup
On 22/06/2011 8.50, David Kastrup wrote:

> Lorenzo Donati<[hidden email]>  writes:
>
>> I was just thinking of it right now. I'm not a fan of continue, but I
>> find it really useful sometimes.
>>
>> Now that the goto machinery is in place, wouldn't a "continue" keyword
>> be useful and easily introduced? If I remember well, Lua authors
>> weren't contrary to it in principle, but because of the
>> incompatibilities with "until".
>>
>> Now this seems to work:
>
>>     ::continue::  -- hidden label if continue were introduced
>> until word ==  nil
>
> I mentioned this before: the purpose of "continue" is a short-circuit of
> a loop before its regular completion.  A while-loop _starts_ with the
> check of the loop condition as a prerequisite to its regular execution.
> A repeat-loop _finishes_ with the check of the loop condition as a
> _consequence_ of its regular execution.
>
> For that reason, you want "continue"-like behavior to resume a
> repeat-until loop _at_ _the_ _top_ of the loop body, not at the point
> where the condition is checked, since the condition is a _product_ and
> not a prerequisite of the regular execution, and continue short-circuits
> the regular execution.
>
> If you claim possibility for confusion: show me real existing code
> (rather than theoretic speculation) using "continue" inside of "do
> .. while".
>
The last sentence confused me. Did you mean "inside of repeat..until" or
"inside while..do..end" ? "do .. while" doesn't exist in Lua. Or are you
hinting at another language usage?

Anyway I never claimed any possibility for confusion. Please reread my
post (maybe I wasn't clear about it).

As I said, I think I read in an old message by Roberto or Luiz (in an
old of those endless threads about the beauty of continue) that Lua team
wasn't in principle contrary to the introduction of continue as a
counterpart of break, but the problem was that it was too tricky to get
it right with the current (5.1.4) semantics of repeat-until because of
the possibility of bypassing the scope of a local declared after the
continue (*IIRC* - please feel free to correct me if I'm wrong).

So the example above wasn't intended as a smart or cool way to show how
continue could/should be used, but a way to show that, with the proposed
5.2 goto, it could be feasible to introduce continue just as syntactic
sugar for something semantically well defined (in 5.2 beta rc2).

As I stated above, I'm not a fan of continue, in the sense that I don't
think it would cure all the pain in the world :-), nor that I need it at
all costs because it is at the heart of my programming style.

Nonetheless I find it useful sometimes (discount the example above,
please), since in some cases it increases readability (the last time I
wished I had continue at hand, IIRC, was with and endless loop in which
I wanted to take decisions step by step, skipping the loop or breaking
out of it according to some path in the decision process. And, yes, I
know in 99% of cases you can do without continue and be happy anyway. :-)


-- Lorenzo

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Ignacio Burgueño
In reply to this post by Roberto Ierusalimschy
On Wed, Jun 22, 2011 at 10:09 AM, Roberto Ierusalimschy <[hidden email]> wrote:
> On Tue, Jun 21, 2011 at 6:21 PM, Luiz Henrique de Figueiredo
> <[hidden email]> wrote:
> > Lua 5.2.0 (beta-rc2) is now available at
>
> Do the new setlocale calls in ldo.c change thread safety assumptions
> Lua?  http://stackoverflow.com/questions/4057319/is-setlocale-thread-safe-function

It seems so :(

Suggestions?


I think there's no easy way. On posix systems you can use newlocale / uselocale / freelocale to change it per-thread [1]
On Windows you have SetThreadLocale [2]. So, changing locale in a thread safe way is not portable. Some #ifdefs will be needed.


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

dcharno
In reply to this post by James Graves-2




On Jun 22, 2011, at 10:50 AM, James Graves <[hidden email]> wrote:

> David Kastrup <[hidden email]> wrote:
>> Alexandre Rion <[hidden email]> writes:
>>
>>> I also think that labels should be per block.
>>
>> +1
>>
>> Local variables are per block.  This makes it possible to have
>> self-contained syntactical entities with minimal outside interaction.
>> function-local labels break this concept, and it is not clear to me how
>> this would affect label/goto outside of functions.
>
> I think the one-label-per-function restriction is fine.
>
> If your function is soooo big that you're losing track of what labels
> you've been using, then you've got other problems.  Having unique
> labels in each function makes it easier for me to read and understand
> your code.
>
> I would like to encourage all developers to:
>
> A) Use 'goto' sparingly, of course.  Error handling and a 'continue'
> replacement are acceptable to me.  However, if a little
> re-organization would allow you to use some other control structure,
> that is preferred.
>
> B) Put as much thought into naming labels as you would naming local
> variables.  Please use descriptive names, such as ::retry_read::
> instead of just ::retry:: and ::skip_blank_line:: instead of
> ::continue::, to draw from some recent examples.
>
> James Graves
>

Your suggestions might be a good idea, but the language shouldn't enforce them; thats what coding standards and code reviews are for.  There is a good comment about this in PIL, something to the effect "Lua does not set policy.  Lua provides powerful mechanisms for developers to implement the policies that best suit them."
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

David Kastrup
In reply to this post by Lorenzo Donati-2
Lorenzo Donati <[hidden email]> writes:

> On 22/06/2011 8.50, David Kastrup wrote:
>> Lorenzo Donati<[hidden email]>  writes:
>>
>>> I was just thinking of it right now. I'm not a fan of continue, but I
>>> find it really useful sometimes.
>>>
>>> Now that the goto machinery is in place, wouldn't a "continue" keyword
>>> be useful and easily introduced? If I remember well, Lua authors
>>> weren't contrary to it in principle, but because of the
>>> incompatibilities with "until".
>>>
>>> Now this seems to work:
>>
>>>     ::continue::  -- hidden label if continue were introduced
>>> until word ==  nil
>>
>> I mentioned this before: the purpose of "continue" is a short-circuit of
>> a loop before its regular completion.  A while-loop _starts_ with the
>> check of the loop condition as a prerequisite to its regular execution.
>> A repeat-loop _finishes_ with the check of the loop condition as a
>> _consequence_ of its regular execution.
>>
>> For that reason, you want "continue"-like behavior to resume a
>> repeat-until loop _at_ _the_ _top_ of the loop body, not at the point
>> where the condition is checked, since the condition is a _product_ and
>> not a prerequisite of the regular execution, and continue short-circuits
>> the regular execution.
>>
>> If you claim possibility for confusion: show me real existing code
>> (rather than theoretic speculation) using "continue" inside of "do
>> .. while".
>>
> The last sentence confused me. Did you mean "inside of repeat..until"
> or "inside while..do..end" ? "do .. while" doesn't exist in Lua. Or
> are you hinting at another language usage?

It would be rather hard to show real existing code using "continue" in
Lua, wouldn't it?

> As I said, I think I read in an old message by Roberto or Luiz (in an
> old of those endless threads about the beauty of continue) that Lua
> team wasn't in principle contrary to the introduction of continue as a
> counterpart of break, but the problem was that it was too tricky to
> get it right with the current (5.1.4) semantics of repeat-until
> because of the possibility of bypassing the scope of a local declared
> after the continue (*IIRC* - please feel free to correct me if I'm
> wrong).

That's because they decided to think about continue in the C way, which
is a rather useless way.

--
David Kastrup


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Roberto Ierusalimschy
In reply to this post by Lorenzo Donati-2
> As I said, I think I read in an old message by Roberto or Luiz (in
> an old of those endless threads about the beauty of continue) that
> Lua team wasn't in principle contrary to the introduction of
> continue as a counterpart of break, but the problem was that it was
> too tricky to get it right with the current (5.1.4) semantics of
> repeat-until because of the possibility of bypassing the scope of a
> local declared after the continue (*IIRC* - please feel free to
> correct me if I'm wrong).

You are :) The problem with repeat-until was only a detail. The main
problem with continue was (and still is) that, for us, it is just one of
many different forms of "domesticated goto" and we did not see why to
include this one without including the others. (We could argue the
same about 'break', BTW.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Alexandre Erwin Ittner
In reply to this post by steve donovan
2011/6/22 steve donovan <[hidden email]>:

> Previously people could hijack the existing file objects by packing
> their own FILE* in the LUA_FILEHANDLE userdata, which was just FILE**.
> (...)

Documenting this feature or creating an API for it will also ease the
binding of functions which received FILE pointers. Sometimes the most
natural solution is to pass a standard Lua file object in the Lua
version of that function -- not so easy to do without such a trick
with undocumented features. I got some of these and they will bite me
in the upgrade to Lua 5.2.


--
Alexandre Erwin Ittner - [hidden email]
OpenPGP pubkey 0x0041A1FB @ http://pgp.mit.edu

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Roberto Ierusalimschy
In reply to this post by Gé Weijers
> - disallow a definition of a label if that label is already visible in
> the scope. The above example would be illegal, but the next one would be
> ok:
>
> [...]

Beware that a label is visible before its declaration. So, there is
no sensible difference between these two examples:

  -- example 1
  ::label::   -- 1
  do
    goto label
    ::label::   -- 2
  end

  -- example 2
  do
    ::label::   -- 1
    goto label
  end
  ::label::   -- 2

So, your second option should also disallow example 2. The first
label is not visible when the second is declared, but the second
label is visible when the first one is declared!  (Nevertheless,
this seems a nice compromise between one-per-function and one-per-
block.)
 
-- Roberto


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.2.0 (beta-rc2) now available

Lorenzo Donati-2
In reply to this post by Roberto Ierusalimschy
On 22/06/2011 19.02, Roberto Ierusalimschy wrote:

>> As I said, I think I read in an old message by Roberto or Luiz (in
>> an old of those endless threads about the beauty of continue) that
>> Lua team wasn't in principle contrary to the introduction of
>> continue as a counterpart of break, but the problem was that it was
>> too tricky to get it right with the current (5.1.4) semantics of
>> repeat-until because of the possibility of bypassing the scope of a
>> local declared after the continue (*IIRC* - please feel free to
>> correct me if I'm wrong).
>
> You are :) The problem with repeat-until was only a detail. The main
> problem with continue was (and still is) that, for us, it is just one of
> many different forms of "domesticated goto" and we did not see why to
> include this one without including the others. (We could argue the
> same about 'break', BTW.)

Thanks for the clarification! I really got It wrong! Sorry for the
noise, then. :-)

BTW, what about the limit per-block as opposed to the current limit
per-function?

Don't you think that the "per-function" approach limits the clean usage
of goto in common foreseeable use cases?

It seems to me that with current approach goto will either be exclusive
domain of code generators or be a source of ugly code. I had the
impression that, although you don't want goto to be abused, you
encourage its use in a clean way for those common cases.

Am I wrong here too? :-)

>
> -- Roberto
>
>
>

Cheers!
-- Lorenzo

1234