Small suggestions for luaL_Buffers & their documentation

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Small suggestions for luaL_Buffers & their documentation

frank@kastenholz.org
The manual does not say how to release the space occupied by a luaL_Buiffer.  Presumably luaL_pushresult() will reclaim resources.  The manual does not say how to handle errors.  Looking through mailing lists/etc, it looks like simply returning from the function in which the buffer was created will reclaim resources … but the manual does not say so..

Second, is there an “approved”  way to determine the current size of a liuaL_Buffer.  I see that there is a field, n, in the structure that is the :”number of characters in the buffer”. Is this field guaranteed to always be there? (If so, can it be documented?). If not, can a function be added to retrieve the current size?  I have a use are where I need to ensure that the string does not grow “too big” - so I’d like to be able to find the current length in a manner that is formally part of the API.  (Alternatively, a way to limit the buffer’s maximum size would be ok - except then all the functions that add values to the buffer need to return a status… this seems to break the existing api/etc.

thanks
frank kastenholz


-
Reply | Threaded
Open this post in threaded view
|

Re: Small suggestions for luaL_Buffers & their documentation

Roberto Ierusalimschy
> The manual does not say how to release the space occupied by a
> luaL_Buiffer.  Presumably luaL_pushresult() will reclaim resources.
> The manual does not say how to handle errors.  Looking through mailing
> lists/etc, it looks like simply returning from the function in which
> the buffer was created will reclaim resources … but the manual does
> not say so..

The manual also does not say how to release the space occupied by a
table or by a string or by a coroutine. This is true for everything in
Lua. Like everything else, buffers are subject to garbage collection and
their space will be eventually released.


> Second, is there an “approved” way to determine the current size
> of a liuaL_Buffer.  I see that there is a field, n, in the structure
> that is the :”number of characters in the buffer”. Is this field
> guaranteed to always be there? (If so, can it be documented?). If not,
> can a function be added to retrieve the current size?  I have a use
> are where I need to ensure that the string does not grow “too big”
> - so I’d like to be able to find the current length in a manner
> that is formally part of the API.  (Alternatively, a way to limit the
> buffer’s maximum size would be ok - except then all the functions
> that add values to the buffer need to return a status… this seems to
> break the existing api/etc.

Again, that does not match the way Lua is tipically used. There is
no "approved" way to determine the current size of a table (or a
coroutine) nor a way to limit its size.

Note that, unlike tables, buffers are implemented outside Lua, in the
auxiliary library. (In Lua 5.4, the entire implementation takes less
than 200 lines of code.) If you have those specific needs, you might
consider writing your own buffers, so that you can tune it as you like.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Small suggestions for luaL_Buffers & their documentation

Viacheslav Usov
On Thu, Jan 28, 2021 at 2:33 PM Roberto Ierusalimschy
<[hidden email]> wrote:

> Note that, unlike tables, buffers are implemented outside Lua, in the
> auxiliary library. (In Lua 5.4, the entire implementation takes less
> than 200 lines of code.) If you have those specific needs, you might
> consider writing your own buffers, so that you can tune it as you like.

I would recommend taking this advice very seriously. The buffers are a
dangerous facility that even experienced users of Lua misunderstand.
The consequences of the misunderstanding can be pretty masty.

In my opinion, the buffers should be deprecated as a public API and
reserved strictly for Lua itself.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Small suggestions for luaL_Buffers & their documentation

Gé Weijers
On Thu, Jan 28, 2021 at 12:46 PM Viacheslav Usov <[hidden email]> wrote:
>
>
> In my opinion, the buffers should be deprecated as a public API and
> reserved strictly for Lua itself.

The buffers work fine if you follow the rules. While you're filling
the contents of the buffer it may use extra stack slots, so you should
refer to stack entries created before the buffer object is initialized
with absolute indices only, and only do stack balanced operation
sequences between buffer API calls.

The reason the buffer API is a bit tricky to use is that it only uses
an extra stack slot if your string grows beyond a certain size,
shorter strings are stored inside the luaL_Buffer object.
An easier API could always use a single stack slot, but that would
incur garbage collector overhead even for tiny strings.

--

Reply | Threaded
Open this post in threaded view
|

Re: Small suggestions for luaL_Buffers & their documentation

Viacheslav Usov
On Fri, Jan 29, 2021 at 12:08 AM Gé Weijers <[hidden email]> wrote:

> The buffers work fine if you follow the rules.

The problem with the rules is that they are lost even on experienced
users of the Lua's C API (compare [2] and [3]). Some of them can be
adamant that the rules (with all their implications) are wrong even
after the rules were cited to them, with a detailed explanation why
[1].

Ultimately, as Roberto confirms, the buffers rely on no internal magic
and something similar can be implemented externally.

Cheers,
V.

[1] http://lua-users.org/lists/lua-l/2017-11/msg00005.html

[2] http://lua-users.org/lists/lua-l/2017-11/msg00013.html

[3] http://lua-users.org/lists/lua-l/2020-12/msg00168.html