New to-be-closed variable list implementation question

[Sergey Zakharchenko]

Hello,

I (also) have a question related to this commit:
https://github.com/lua/lua/commit/4e47f81188d37e29027158b76271d02a781242e2
. Aside from performance reasons, is there anything stopping us from
storing the 'to-be-closed flag' in e.g. an unused pointer or tt_ bit
inside the stack value and keeping the old StackValue definition?
Keeping a count of such variables and the index of the last of them in
the state could address performance issues in some use common cases...

Best regards,

--
DoubleF

[Roberto Ierusalimschy]

> I (also) have a question related to this commit:
> https://github.com/lua/lua/commit/4e47f81188d37e29027158b76271d02a781242e2
> . Aside from performance reasons, is there anything stopping us from
> storing the 'to-be-closed flag' in e.g. an unused pointer or tt_ bit
> inside the stack value and keeping the old StackValue definition?
> Keeping a count of such variables and the index of the last of them in
> the state could address performance issues in some use common cases...

Aside from performance reasons, everything in programming could be quite
different :-) But I have not understood your suggestion.  Once we close
the last of them, how to find the next one?

-- Roberto

[Xavier Wang]

Roberto Ierusalimschy <[hidden email]>于2021年3月6日 周六00:33写道：

> I (also) have a question related to this commit:
> https://github.com/lua/lua/commit/4e47f81188d37e29027158b76271d02a781242e2
> . Aside from performance reasons, is there anything stopping us from
> storing the 'to-be-closed flag' in e.g. an unused pointer or tt_ bit
> inside the stack value and keeping the old StackValue definition?
> Keeping a count of such variables and the index of the last of them in
> the state could address performance issues in some use common cases...

Aside from performance reasons, everything in programming could be quite
different :-) But I have not understood your suggestion.  Once we close
the last of them, how to find the next one?

-- Roberto

Just close slots in order from stack top to base？

--

[Sergey Zakharchenko]

Roberto,

Just to be clear: as Xavier guessed above, I was implying a linear search in the stack (having a total count of to-be-closed vars would help avoid searching part of it). 

The reason for this question is that I'm the unlucky one for whom the extra fields added to stack value structure could matter... so there might be a speed/space tradeoff. I'm not suggesting that you implement that, just asking if it makes sense to you.

[Roberto Ierusalimschy]

> The reason for this question is that I'm the unlucky one for whom the extra
> fields added to stack value structure could matter...

Just for curiosity, how is this architecture (alignment, sizes, etc.)?

-- Roberto

[Sergey Zakharchenko]

Hello Roberto,

Roberto Ierusalimschy <[hidden email]>:
>
> > The reason for this question is that I'm the unlucky one for whom the extra
> > fields added to stack value structure could matter...
>
> Just for curiosity, how is this architecture (alignment, sizes, etc.)?

NaN packing:). Like
lua-users.org/files/wiki_insecure/power_patches/5.1/pack_value.patch
but for 5.4. TValues are thus 8 bytes (architecture is 32-bit MIPS).

Best regards,

--
DoubleF

[Roberto Ierusalimschy]

> Hello Roberto,
>
> Roberto Ierusalimschy <[hidden email]>:
> >
> > > The reason for this question is that I'm the unlucky one for whom the extra
> > > fields added to stack value structure could matter...
> >
> > Just for curiosity, how is this architecture (alignment, sizes, etc.)?
>
> NaN packing:). Like
> lua-users.org/files/wiki_insecure/power_patches/5.1/pack_value.patch
> but for 5.4. TValues are thus 8 bytes (architecture is 32-bit MIPS).

Are you using 32-bit integers? You still have one extra byte free,
is that right?

-- Roberto

[Sergey Zakharchenko]

Roberto,

Roberto Ierusalimschy <[hidden email]>:
> Are you using 32-bit integers?

Correct; LUA_INT_TYPE = LUA_INT_INT, LUA_FLOAT_TYPE = LUA_FLOAT_DOUBLE.

> You still have one extra byte free, is that right?

Not exactly, considering that the full range of double precision
values is supported. Non-floating-point values do seem to have a byte
that's unused.

Best regards,

--
DoubleF

[Roberto Ierusalimschy]

> > You still have one extra byte free, is that right?
>
> Not exactly, considering that the full range of double precision
> values is supported. Non-floating-point values do seem to have a byte
> that's unused.

Sure, silly question of mine.

-- Roberto

[Varanda]

Hello All,

I would like to share a project that I am starting: The LUA EOS

It provides an event driven non-preemptive LUA tasks for embedded systems.

Yes, nothing new as Coroutines are already there. But the idea is to put together a framework to make easier the integration between app layer  and the native RTOS implementation.

It is the same successful idea that has been used in many systems where Java provides a secure way for end users to write applications without compromising the native implementation. Similarly,  LUA replaces Java for the same means.

LUA EOS intents to provide a ready to use framework with a LUA scheduler on top of FreeRTOS. BTW... there is a small OS abstraction layer and it can actually run on top of other OS's. Its simulator based on Qt is an example.

The following video shows the concept:

https://rumble.com/veh9ed-lua-eos-for-embedded-systems.html

In Portuguese:

https://rumble.com/veh9lv-lua-eos-systema-operacional-para-embarcados.html

I am still figuring out its license model but I would like to make it free for non-commercial use (licensed otherwise). I am trying to get sponsorship from Semiconductor suppliers to port the framework to their targets. I hope that works out in getting resources to the project.

Anyone seriously considering to join this project please let me know. We can share the cake if the project gears up.

Regards,

Marcelo Varanda

[Roberto Ierusalimschy]

> Just to be clear: as Xavier guessed above, I was implying a linear search
> in the stack (having a total count of to-be-closed vars would help avoid
> searching part of it).
>
> The reason for this question is that I'm the unlucky one for whom the extra
> fields added to stack value structure could matter... so there might be a
> speed/space tradeoff. I'm not suggesting that you implement that, just
> asking if it makes sense to you.

If you are using the NaN-trick representation, I imagine that even that
one bit that you need must be stored inside the value itself (cleverly
assuming that float values cannot be marked to-be-closed). But then you
will have the problem Dibyendu talked about, that any copy of stack
values would need to clear that bit, which in turn demands a test,
because floats don't have that bit. That probably will be somewhat
expensive for any code, even if it does not use tbc variables.

Usually, stacks in Lua comprise a very small fraction of a program's
memory footprint. Even doubling their sizes should have a small
overall impact.

-- Roberto

[Sergey Zakharchenko]

Roberto,

Roberto Ierusalimschy <[hidden email]>:
> any copy of stack
> values would need to clear that bit, which in turn demands a test,
> because floats don't have that bit. That probably will be somewhat
> expensive for any code, even if it does not use tbc variables.

Thanks for your detailed analysis and related changes!

Best regards,

--
DoubleF