Proposal: Make creating large arrays faster and "allow" nils in arrays

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Proposal: Make creating large arrays faster and "allow" nils in arrays

Mason Bogue
We are all generally aware of the issues with the length operator on
tables containing nil. There was an idea to fix this in 5.4 by
creating a special t[x] = undef syntax, which was considered too ugly.
Here I propose an alternative:

#t = 5 -- sets the length of the array portion of t to 5

Essentially we allow the length operator to be used in an lvalue (we
can already invoke functions in lvalues, so this is not
unprecedented). If I understand the implementation correctly, the
length will not be altered unless the table is reindexed, which only
happens if we make a lot of insertions to the hash-map. If this is
still too volatile, there could be a flag similar to `isrealasize(t)`
that doesn't allow the size to shrink if it was set this way. If
necessary for semantic consistency, doing this could also set t[5] =
false when otherwise t[5] = nil.

This also speeds up the creation of large arrays. Previously you had to do this:

t = {}
for i = 1, 10000 do t[i] = false end

During that loop the array would be reallocated several times. With
length assignment (`#t = 10000`) you get *one* reindex. Much nicer.
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Sean Conner
It was thus said that the Great Mason Bogue once stated:
> We are all generally aware of the issues with the length operator on
> tables containing nil. There was an idea to fix this in 5.4 by
> creating a special t[x] = undef syntax, which was considered too ugly.
> Here I propose an alternative:
>
> #t = 5 -- sets the length of the array portion of t to 5

  What happens if the __len metamethod is overridden?

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Sam Pagenkopf
local function setlength (t, len)
  local mt = getmetatable(t) or {}
  setmetatable(t,mt)
  mt.__len = function () return len end
end

^ Sean, are you referring to this type of solution? I am sure it would cover some bases.

The other solutions I can think of are:
1. Using pairs to find the largest numerical index of a table (slow to get length, but allows nil within the table)
2. Overriding __newindex to keep track of a max value. This enables 't[1000] = true' to cause '#t = 1000'. Could get hairy with removing values.


On Thu, Jul 16, 2020 at 3:40 PM Sean Conner <[hidden email]> wrote:
It was thus said that the Great Mason Bogue once stated:
> We are all generally aware of the issues with the length operator on
> tables containing nil. There was an idea to fix this in 5.4 by
> creating a special t[x] = undef syntax, which was considered too ugly.
> Here I propose an alternative:
>
> #t = 5 -- sets the length of the array portion of t to 5

  What happens if the __len metamethod is overridden?

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Sean Conner
It was thus said that the Great Sam Pagenkopf once stated:
> local function setlength (t, len)
>   local mt = getmetatable(t) or {}
>   setmetatable(t,mt)
>   mt.__len = function () return len end
> end
>
> ^ Sean, are you referring to this type of solution? I am sure it would
> cover some bases.

  No, I'm talking about existing code where #t becomes a left-hand side
value.  Code written to the current standard will only expect one parameter
to the __len metamethod, and swapping it out with that could break existing
code.

  Also, userdata can add behavior to the __len metamethod---what happens in
those cases?  You just can't assume that # only applies to tables.

  What should happen in the following cases?

        s = "some string"
        t = { "some table" }
        u = make_new_userdata()

        print(#s)
        print(#t)
        print(#u)

        #s = 10 assert(#s == 10)
        #t = 10 assert(#t == 10)
        #u = 10 assert(#u == 10)

        #s = 0 -- does this free the string?  Turn it into ""?
        #t = 0 -- does this mark all elements are garbage?
        #u = 0

        t = { 1 , 2 , 3 , 4 , 5 }
        #t = 10 -- elements 6 through 10 are now nil, right?
        table.remove(t,1)   -- #t is now what?  What are the contents?
        table.remove(t)     -- what about now?
        table.insert(t,'x') -- and so on ...
        table.sort(t)    -- this could be problematic as well

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Mason Bogue
In reply to this post by Sean Conner
> What happens if the __len metamethod is overridden?

Good question! The best idea I have is to treat this like __index and add a __setlen metamethod. Length assignment on any non-table value (userdata eg) would then be a type error unless there is a __setlen metamethod defined for that value. Of course the __setlen metamethod then calls for a rawsetlen() function which only works on tables.

You can't currently implement this behavior in pure Lua — you need to change the internal representation of the table to get the speedup. What I wanted was to set the length of the array segment. If there is a __len metamethod, the programmer must ensure their code makes sense, but it's true in that case `#t = 5` does not guarantee the truth of `#t == 5`. 

It seems like this should mean that if the table length is set downwards, all entries between the old (raw) len and new len are deleted. (Hence we may also want `rawgetlen()`, although that is already a question raised by __len). 

On Thursday, July 16, 2020, Sean Conner <[hidden email]> wrote:
It was thus said that the Great Mason Bogue once stated:
> We are all generally aware of the issues with the length operator on
> tables containing nil. There was an idea to fix this in 5.4 by
> creating a special t[x] = undef syntax, which was considered too ugly.
> Here I propose an alternative:
>
> #t = 5 -- sets the length of the array portion of t to 5

  What happens if the __len metamethod is overridden?

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Robert Burke
In reply to this post by Mason Bogue
I thought the undef proposal[1] was nice, but I don't think it
addresses the issue fully. I would like, ideally, to be able to use a
json or msgpack library without having to learn how the json or
msgpack library tries to decide which tables are maps and which tables
are arrays. So I would like a data type for arrays, which know their
own size and can contain nils, and a separate data type for maps,
which probably also know their own size but maybe cannot contain nils.

Roberto commented[2] that this could be bad because it slows down
table access by adding a check for the type of the table. I don't
think this needs to be a net slowdown though, since the code for
looking for integer keys in the array-part and hash-part of tables
could be removed.

[1]: http://lua-users.org/lists/lua-l/2018-03/msg00155.html
[2]: http://lua-users.org/lists/lua-l/2018-03/msg00434.html
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Lorenzo Donati-3
In reply to this post by Mason Bogue
On 16/07/2020 21:07, Mason Bogue wrote:

> We are all generally aware of the issues with the length operator on
> tables containing nil. There was an idea to fix this in 5.4 by
> creating a special t[x] = undef syntax, which was considered too ugly.
> Here I propose an alternative:
>
> #t = 5 -- sets the length of the array portion of t to 5
>
> Essentially we allow the length operator to be used in an lvalue (we
> can already invoke functions in lvalues, so this is not
> unprecedented). If I understand the implementation correctly, the
> length will not be altered unless the table is reindexed, which only
> happens if we make a lot of insertions to the hash-map. If this is
> still too volatile, there could be a flag similar to `isrealasize(t)`
> that doesn't allow the size to shrink if it was set this way. If
> necessary for semantic consistency, doing this could also set t[5] =
> false when otherwise t[5] = nil.
>
> This also speeds up the creation of large arrays. Previously you had to do this:
>
> t = {}
> for i = 1, 10000 do t[i] = false end
>
> During that loop the array would be reallocated several times. With
> length assignment (`#t = 10000`) you get *one* reindex. Much nicer.
>
Really, I can't understand why the simpler, easiest solution, has not
been implemented yet: simply add a new function in table library for
those efficiency aware programs where preallocating space for a table is
paramount.

The function is already there in C API (lua_createtable), so it can't
possibly be considered "adding bloat" since it would be just a few bytes
more to interface add the corresponding table library function.

All the other proposals are really "hacks" and complicate matters adding
conceptual overhead to the language and its syntax for a corner case.
This corner case is, IMO, important enough to grant for an added
`table.create` function, but it is not important enough to mess with the
syntax (how many times would a user need to write #T=maxlen ?). Not to
speak of the interactions with other mechanisms and syntax, as pointed
out by other in this thread.

Your proposal, moreover, will allocate array slots /after/ the table has
been created, so it is not optimal as it could be.

Please, understand I'm not "ranting" against you. I know there is really
the need sometimes to allocate tables efficiently (I've been there and
I'd like to be able to do it in pure Lua). But the real (elegant,
no-bloat, easy, readable) solution is what I told you and "screams" to
be implemented.

Why Lua team didn't do it yet (while struggling with yet another GC
implementation), I can't really understand. IIRC years ago they replied
it would lead to "bad" programming practice, since it leaks an
implementation detail (array part vs. hash part). But really I don't
agree on that. That leak is already there in all the "sequence vs,
non-sequence" thing and in the whole table library.

Moreover, if one wants to be abstract and non-leaky, an hypothetical
`table.create(narr, nhash)` can be defined at Lua level just as
"hinting" the Lua engine to create a table efficiently for a sequence of
`narr` elements and that the user means to fill with `nhash` keys. This
doesn't mention "array parts" and "hash parts" whatsoever and use
concept well defined at Lua level. So it is non-leaky. The
implementation would be free to implement table.create by mapping it to
lua_createtable (straightforward implementation) or to whatever future
mechanism would be needed for efficient allocation, would the underlying
implementation of table change.

So we can have our cake and eat it too, in this case. Easy-peasy.

Another, more elegant long-term solution (maybe), would be to introduce
a way to specify this kind of hint in any table ctor.

Just off the top of my head:

{:1000: ... --[[rest of the ctor, as usual]]} -- preallocate 1000 array
slots


{:1000,2000:...} -- preallocate 1000 array and 2000 hash slots


{:nil,2000:...} -- preallocate 2000 hash slots

I said "preallocate", so this /is/ implementation details leaking. If
you care, substitute it with "hinting" as I said above.

The syntax could be defined by stating that the engine would do its best
to optimize whatever allocation is needed to satisfy those parameters
(without mentioning hash/array parts).


That (or similar) syntax could be easily extended to other "hinting" or
table properties that could, in the future, lead to other optimizations
or useful features.

e.g.:

{:"const": ....} -- table that cannot be changed
{:"noresize":...} -- table whose number of entries cannot change

Heck, maybe, some could be used to automatically give a table a
metatable with suitable defined metamethods (just brainstorming here),
as in:

{:"weak":...}
{:"ephemeron":...}


Cheers!

-- Lorenzo






Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Philippe Verdy-2
I tend to disagree. The "sequence" meaning does not imply that ipairs() enumerates only the array part. It had to enumerate both parts to find keys that arte integers in the seuqnce range 1 to N (until it finds  a nil).
So the parts are implementation details.

As I said, nothing prohibits another change on the implementation of tables (I will soon experiment on integrating a variant of my old code using b-trees (for un unrelated project) and soee how it behaves in Lua. There will NOT be any separation between an "array" part and an "hash part" as there will be a single part for all, the b-tree. Still it will support ipairs() as a selective enumerator for sequences while still supporting pairs() for the uselective full enumerator. And it will be efficient on both cases for fast enumeration of sequences, and efficient in terms of memory allocation and data locality.

If you want tuning a specific implementation a separate API should be used for this specific implementation where you could pass or query an array of parameters.

As well the "length" (as returned by #t) should not be settable. the effective length should be a queriable property only for the sequence part of the table independantly of its implementation.

Le lun. 20 juil. 2020 à 11:40, Lorenzo Donati <[hidden email]> a écrit :
On 16/07/2020 21:07, Mason Bogue wrote:
> We are all generally aware of the issues with the length operator on
> tables containing nil. There was an idea to fix this in 5.4 by
> creating a special t[x] = undef syntax, which was considered too ugly.
> Here I propose an alternative:
>
> #t = 5 -- sets the length of the array portion of t to 5
>
> Essentially we allow the length operator to be used in an lvalue (we
> can already invoke functions in lvalues, so this is not
> unprecedented). If I understand the implementation correctly, the
> length will not be altered unless the table is reindexed, which only
> happens if we make a lot of insertions to the hash-map. If this is
> still too volatile, there could be a flag similar to `isrealasize(t)`
> that doesn't allow the size to shrink if it was set this way. If
> necessary for semantic consistency, doing this could also set t[5] =
> false when otherwise t[5] = nil.
>
> This also speeds up the creation of large arrays. Previously you had to do this:
>
> t = {}
> for i = 1, 10000 do t[i] = false end
>
> During that loop the array would be reallocated several times. With
> length assignment (`#t = 10000`) you get *one* reindex. Much nicer.
>
Really, I can't understand why the simpler, easiest solution, has not
been implemented yet: simply add a new function in table library for
those efficiency aware programs where preallocating space for a table is
paramount.

The function is already there in C API (lua_createtable), so it can't
possibly be considered "adding bloat" since it would be just a few bytes
more to interface add the corresponding table library function.

All the other proposals are really "hacks" and complicate matters adding
conceptual overhead to the language and its syntax for a corner case.
This corner case is, IMO, important enough to grant for an added
`table.create` function, but it is not important enough to mess with the
syntax (how many times would a user need to write #T=maxlen ?). Not to
speak of the interactions with other mechanisms and syntax, as pointed
out by other in this thread.

Your proposal, moreover, will allocate array slots /after/ the table has
been created, so it is not optimal as it could be.

Please, understand I'm not "ranting" against you. I know there is really
the need sometimes to allocate tables efficiently (I've been there and
I'd like to be able to do it in pure Lua). But the real (elegant,
no-bloat, easy, readable) solution is what I told you and "screams" to
be implemented.

Why Lua team didn't do it yet (while struggling with yet another GC
implementation), I can't really understand. IIRC years ago they replied
it would lead to "bad" programming practice, since it leaks an
implementation detail (array part vs. hash part). But really I don't
agree on that. That leak is already there in all the "sequence vs,
non-sequence" thing and in the whole table library.

Moreover, if one wants to be abstract and non-leaky, an hypothetical
`table.create(narr, nhash)` can be defined at Lua level just as
"hinting" the Lua engine to create a table efficiently for a sequence of
`narr` elements and that the user means to fill with `nhash` keys. This
doesn't mention "array parts" and "hash parts" whatsoever and use
concept well defined at Lua level. So it is non-leaky. The
implementation would be free to implement table.create by mapping it to
lua_createtable (straightforward implementation) or to whatever future
mechanism would be needed for efficient allocation, would the underlying
implementation of table change.

So we can have our cake and eat it too, in this case. Easy-peasy.

Another, more elegant long-term solution (maybe), would be to introduce
a way to specify this kind of hint in any table ctor.

Just off the top of my head:

{:1000: ... --[[rest of the ctor, as usual]]}   -- preallocate 1000 array
slots


{:1000,2000:...} -- preallocate 1000 array and 2000 hash slots


{:nil,2000:...} -- preallocate 2000 hash slots

I said "preallocate", so this /is/ implementation details leaking. If
you care, substitute it with "hinting" as I said above.

The syntax could be defined by stating that the engine would do its best
to optimize whatever allocation is needed to satisfy those parameters
(without mentioning hash/array parts).


That (or similar) syntax could be easily extended to other "hinting" or
table properties that could, in the future, lead to other optimizations
or useful features.

e.g.:

{:"const": ....} -- table that cannot be changed
{:"noresize":...} -- table whose number of entries cannot change

Heck, maybe, some could be used to automatically give a table a
metatable with suitable defined metamethods (just brainstorming here),
as in:

{:"weak":...}
{:"ephemeron":...}


Cheers!

-- Lorenzo






Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Sean Conner
It was thus said that the Great Philippe Verdy once stated:
> I tend to disagree. The "sequence" meaning does not imply that ipairs()
> enumerates only the array part.

  You never bothered to even check the source code, did you?  Because if you
did, you would see that the implementation returns an iterator that starts
with the number 1, and incrments the index on each pass until lua_geti()
returns LUA_TNIL.

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Philippe Verdy-2
And where does it contradict what I said ? I just said that ipairs() does not enumerate only the array part, because of course integer keys in the sequence can also be in the hashed part (given the fact that the arry part is only populated by constructors, and all the rest is populated by assignments, the array part is not evolving, there's no strategy to move items from the hash part to the array part. So items of the sequences may be alternatively in each part.

All this shows that the "optimization" make in tables (using an array part) was half-done, never really finalized.

As well the order of evaluation becomes completely random. This old optimization is unsafe, in fact Lua could even live without this array part and have used only the hashed part. This would have not done a major difference. But at least it would have simplified the specification and allowed to develop a correct behavior and semantics of table constructors, with a well defined order of evaluations of expression (for keys and values) and the assignment of keys in the table.

The current implementation of tables is simply unsafe. And even the hashed part is incorrectly optimized. That's very strange given that tables are a fundamental feature of Lua: there are tables almost everywhere (except arrays for upvalues, the stack of calls and the array of local variables), accessing/indexing tables is extremely frequent (including for lookup of metamethods in metatables).


Le sam. 25 juil. 2020 à 03:14, Sean Conner <[hidden email]> a écrit :
It was thus said that the Great Philippe Verdy once stated:
> I tend to disagree. The "sequence" meaning does not imply that ipairs()
> enumerates only the array part.

  You never bothered to even check the source code, did you?  Because if you
did, you would see that the implementation returns an iterator that starts
with the number 1, and incrments the index on each pass until lua_geti()
returns LUA_TNIL.

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Jasper Klein
Op zaterdag 25 juli 2020 08:29:47 CEST schreef Philippe Verdy:
> and all the rest is populated by
> assignments, the array part is not evolving, there's no strategy to move
> items from the hash part to the array part.

A while back I did some investigation about forcing the size for the array part and the hash part of a table.
In Lua 5.3.5, when the table grows, it may trigger a rehash.
When rehashing, an integer key may be moved to the array part if the sequence is contiguous.
This happens in ltable.c in the functions 'luaH_newkey', 'rehash', 'luaH_resize' and 'reinsert'.

-- Jasper

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Philippe Verdy-2
I agree with this approach, as long as you keep the paradigm of the split between an array part and an hashed part. However I'm not even convinced that this is the best way to handle tables, the benefit it gives for a supposed optimization of speed or access time or memory allocation/GC is quite doubtful. And it has brought various bugs or instabilities or incompatible behaviors.

A safe implementation should use invariants, but Lua does not fix them reliably. I'm now convinced that it is a problem to create a good ecosystem of reusable libraries. And the obvious symptom of this is that most projects we eloped in Lua are stocked to a si gle version and refuse to upgrade. In fine, o'er versions are left unmaintained, security holes are found but not fixed in versions that application supports, or this effectively cause many forks to be developed or maintained separately. This is bad for the long term evolutions of Lua. The only safe way is to remove the unsoefied behavior of tables. 

Le sam. 25 juil. 2020 à 13:28, Jasper Klein <[hidden email]> a écrit :
Op zaterdag 25 juli 2020 08:29:47 CEST schreef Philippe Verdy:
> and all the rest is populated by
> assignments, the array part is not evolving, there's no strategy to move
> items from the hash part to the array part.

A while back I did some investigation about forcing the size for the array part and the hash part of a table.
In Lua 5.3.5, when the table grows, it may trigger a rehash.
When rehashing, an integer key may be moved to the array part if the sequence is contiguous.
This happens in ltable.c in the functions 'luaH_newkey', 'rehash', 'luaH_resize' and 'reinsert'.

-- Jasper

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Egor Skriptunoff-2
In reply to this post by Lorenzo Donati-3
On Mon, Jul 20, 2020 at 12:40 PM Lorenzo Donati wrote:
{:1000,2000:...} -- preallocate 1000 array and 2000 hash slots
{:nil,2000:...} -- preallocate 2000 hash slots
{:"noresize":...} -- table whose number of entries cannot change
{:"const": ....} -- table that cannot be changed
{:"weak":...}
{:"ephemeron":...}

We already have a syntax for optional attributes, so why invent a new one?
Just allow a table constructor to have optional attribute:

{...} <size 1000,2000>
{...} <size 0,2000>
{...} <size 0,2000 noshrink>
{...} <const>
{...} <weak>
{...} <ephemeron>

This doesn't break existing code: all these expressions were syntax errors in previous Lua versions.
The parser should look ahead three lexemes to decide whether it is an attribute or not.
Attribute has highest priority:
func{...} <weak>
is equivalent to
func({...} <weak>)
 
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Egor Skriptunoff-2
The parser should look ahead three lexemes to decide whether it is an attribute or not.

Actually 4 lexemes are needed, example:
x = {...} < weak noshrink >
x = {...} < weak noshrink ()

But 3 lexemes are enough if each attribute has its own angle brackets:
{...} <weak> <noshrink> <size 1000,2000>

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Robert Burke
In reply to this post by Egor Skriptunoff-2
On Sun, Sep 6, 2020 at 4:21 PM Egor Skriptunoff
<[hidden email]> wrote:

> We already have a syntax for optional attributes, so why invent a new one?
> Just allow a table constructor to have optional attribute:
>
> {...} <size 1000,2000>
> {...} <size 0,2000>
> {...} <size 0,2000 noshrink>
> {...} <const>
> {...} <weak>
> {...} <ephemeron>
>

x = {} <weak>
hello()

is a single statement in Lua, so you may not have a great time with that one.
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Egor Skriptunoff-2
On Sun, Sep 6, 2020 at 12:51 PM Robert Burke wrote:
x = {} <weak>
hello()

is a single statement in Lua

Indeed!

Chained comparisons
a < b > c
are syntactically correct in Lua, but useless in real code
(due to comparison always returns boolean value)

I believe nobody uses chained comparisons,
so no old code would be broken

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

Viacheslav Usov
In reply to this post by Egor Skriptunoff-2
On Sun, Sep 6, 2020 at 10:10 AM Egor Skriptunoff
<[hidden email]> wrote:

> {...} <weak> <noshrink> <size 1000,2000>

I went to the bottom of the thread, and the original proposal, which
is still reflected in the subject line, was to allow nils as table
values, while making creating large arrays faster was a nicety we
would just get along the way.

Now it seems the nils as table values are not really under discussion,
and the discussion is about a syntax to specify array/table sizes at
the table creation time.

Has anyone ever bothered to check how much speedup one might hope to
get with this?

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

DarkWiiPlayer
In reply to this post by Mason Bogue
This was actually my first thought when I read the proposal: There
should be a `__setlen` metamethod to allow resizing of custom types.
Probably also a `rawsetlen` method that can be called within the
metamethod of a table.

On 17/07/2020 17:53, Mason Bogue wrote:

> > What happens if the __len metamethod is overridden?
>
> Good question! The best idea I have is to treat this like __index and
> add a __setlen metamethod. Length assignment on any non-table value
> (userdata eg) would then be a type error unless there is a __setlen
> metamethod defined for that value. Of course the __setlen metamethod
> then calls for a rawsetlen() function which only works on tables.
>
> You can't currently implement this behavior in pure Lua — you need to
> change the internal representation of the table to get the speedup.
> What I wanted was to set the length of the array segment. If there is
> a __len metamethod, the programmer must ensure their code makes sense,
> but it's true in that case `#t = 5` does not guarantee the truth of
> `#t == 5`. 
>
> It seems like this should mean that if the table length is set
> downwards, all entries between the old (raw) len and new len are
> deleted. (Hence we may also want `rawgetlen()`, although that is
> already a question raised by __len). 
>
> On Thursday, July 16, 2020, Sean Conner <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     It was thus said that the Great Mason Bogue once stated:
>     > We are all generally aware of the issues with the length operator on
>     > tables containing nil. There was an idea to fix this in 5.4 by
>     > creating a special t[x] = undef syntax, which was considered too
>     ugly.
>     > Here I propose an alternative:
>     >
>     > #t = 5 -- sets the length of the array portion of t to 5
>
>       What happens if the __len metamethod is overridden?
>
>       -spc
>


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Make creating large arrays faster and "allow" nils in arrays

DarkWiiPlayer
In reply to this post by Lorenzo Donati-3
On 20/07/2020 11:39, Lorenzo Donati wrote:

> Really, I can't understand why the simpler, easiest solution, has not
> been implemented yet: simply add a new function in table library for
> those efficiency aware programs where preallocating space for a table
> is paramount.
>
> The function is already there in C API (lua_createtable), so it can't
> possibly be considered "adding bloat" since it would be just a few
> bytes more to interface add the corresponding table library function.
>
> All the other proposals are really "hacks" and complicate matters
> adding conceptual overhead to the language and its syntax for a corner
> case. This corner case is, IMO, important enough to grant for an added
> `table.create` function, but it is not important enough to mess with
> the syntax (how many times would a user need to write #T=maxlen ?).
> Not to speak of the interactions with other mechanisms and syntax, as
> pointed out by other in this thread.
You mean like this?
https://github.com/DarkWiiPlayer/table_new/blob/master/table/new.c

(That's a rock, by the way, in case you want to use it)



signature.asc (849 bytes) Download Attachment