[Proposal-ish?] Lua should bind the Lua C API as a Lua module

classic Classic list List threaded Threaded
75 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

[Proposal-ish?] Lua should bind the Lua C API as a Lua module

Soni "They/Them" L.
Reading material:
https://en.wikipedia.org/wiki/Operating-system-level_virtualization
https://en.wikipedia.org/wiki/Protection_ring

Some game developers would significantly benefit from a slightly
stripped-down[1] Lua C API as a built-in Lua module, as it would give
the ability to create perfectly-isolated sub-states from Lua.

Because you'd be able to register (host) Lua functions as (guest) C
functions, they'd appear as opaque C functions to the sub-states, which
means the debug.* API could be trivially (and safely!) made available to
them. For example:

GUEST | call os.time()
HOST | call guestostime()
HOST | <operations to implement the guest's os.time()>
HOST | return to guest
GUEST | get result created by host
GUEST | call debug.* functions against os.time, but get no useful
results other than it being a C function.

It is technically possible to implement this in pure-Lua as it currently
stands, at a hefty amount of overhead, both in terms of code/maintenance
overhead, and CPU time. (it basically involves reimplementing the
debug.* API by checking if the given function is a valid target for the
debug API, and carefully wrapping host functions so they look like C
functions to the guest, and so on.)

[1] - We don't want to expose luaL_newstate, we want lua_newstate to use
the current state's allocator and stuff, and things like that.

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Sean Conner
It was thus said that the Great Soni They/Them L. once stated:

>
> Some game developers would significantly benefit from a slightly
> stripped-down[1] Lua C API as a built-in Lua module, as it would give
> the ability to create perfectly-isolated sub-states from Lua.
>
> It is technically possible to implement this in pure-Lua as it currently
> stands, at a hefty amount of overhead, both in terms of code/maintenance
> overhead, and CPU time. (it basically involves reimplementing the
> debug.* API by checking if the given function is a valid target for the
> debug API, and carefully wrapping host functions so they look like C
> functions to the guest, and so on.)
>
> [1] - We don't want to expose luaL_newstate, we want lua_newstate to use
> the current state's allocator and stuff, and things like that.

  As I said to you in June of 2015 [2]:

>   Then wrap Lua up into a module:

  And in September of 2015 [3]:

>   I think I may have mentioned this before, but wrap Lua in Lua.

  After two years of no one biting on this, I think the onus is on you to
provide this instead of asking other people to do the work.

  -spc (Think of all the accolades you'll get from the game developers ... )

[2] http://lua-users.org/lists/lua-l/2015-06/msg00573.html

[3] http://lua-users.org/lists/lua-l/2015-09/msg00341.html

Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Soni "They/Them" L.


On 2017-11-28 03:06 AM, Sean Conner wrote:

> It was thus said that the Great Soni They/Them L. once stated:
>> Some game developers would significantly benefit from a slightly
>> stripped-down[1] Lua C API as a built-in Lua module, as it would give
>> the ability to create perfectly-isolated sub-states from Lua.
>>
>> It is technically possible to implement this in pure-Lua as it currently
>> stands, at a hefty amount of overhead, both in terms of code/maintenance
>> overhead, and CPU time. (it basically involves reimplementing the
>> debug.* API by checking if the given function is a valid target for the
>> debug API, and carefully wrapping host functions so they look like C
>> functions to the guest, and so on.)
>>
>> [1] - We don't want to expose luaL_newstate, we want lua_newstate to use
>> the current state's allocator and stuff, and things like that.
>    As I said to you in June of 2015 [2]:
>
>>    Then wrap Lua up into a module:
>    And in September of 2015 [3]:
>
>>    I think I may have mentioned this before, but wrap Lua in Lua.
>    After two years of no one biting on this, I think the onus is on you to
> provide this instead of asking other people to do the work.
>
>    -spc (Think of all the accolades you'll get from the game developers ... )
>
> [2] http://lua-users.org/lists/lua-l/2015-06/msg00573.html
>
> [3] http://lua-users.org/lists/lua-l/2015-09/msg00341.html
>

I can't make it part of the standard libraries. So I have to ask for it.

https://github.com/keplerproject/rings

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Paige DePol
Soni They/Them L. <[hidden email]> wrote:

> On 2017-11-28 03:06 AM, Sean Conner wrote:
>>   After two years of no one biting on this, I think the onus is on you to
>> provide this instead of asking other people to do the work.
>>
>>   -spc (Think of all the accolades you'll get from the game developers ... )
>>
>> [2] http://lua-users.org/lists/lua-l/2015-06/msg00573.html
>>
>> [3] http://lua-users.org/lists/lua-l/2015-09/msg00341.html
>
> I can't make it part of the standard libraries. So I have to ask for it.
>
> https://github.com/keplerproject/rings

I am confused, why can't you make it part of the standard library?

Download the Lua source and hack away, then add your new functionality
to the game-oriented Lua fork you are working on!

The great thing about Lua, and other open source software, is that you
have the ability to modify the code to do whatever your heart desires!

Good luck with your project! :)

~Paige



Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Soni "They/Them" L.


On 2017-11-28 09:43 AM, Paige DePol wrote:

> Soni They/Them L. <[hidden email]> wrote:
>
>> On 2017-11-28 03:06 AM, Sean Conner wrote:
>>>    After two years of no one biting on this, I think the onus is on you to
>>> provide this instead of asking other people to do the work.
>>>
>>>    -spc (Think of all the accolades you'll get from the game developers ... )
>>>
>>> [2] http://lua-users.org/lists/lua-l/2015-06/msg00573.html
>>>
>>> [3] http://lua-users.org/lists/lua-l/2015-09/msg00341.html
>> I can't make it part of the standard libraries. So I have to ask for it.
>>
>> https://github.com/keplerproject/rings
> I am confused, why can't you make it part of the standard library?
>
> Download the Lua source and hack away, then add your new functionality
> to the game-oriented Lua fork you are working on!

But that isn't Lua. That isn't the Lua standard libraries, as defined by
the reference manual.

That's just a Lua fork.

> The great thing about Lua, and other open source software, is that you
> have the ability to modify the code to do whatever your heart desires!
>
> Good luck with your project! :)
>
> ~Paige
>
>
>

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Paige DePol
Soni They/Them L. <[hidden email]> wrote:

> On 2017-11-28 09:43 AM, Paige DePol wrote:
>> Soni They/Them L. <[hidden email]> wrote:
>>
>>> I can't make it part of the standard libraries. So I have to ask for it.
>>>
>>> https://github.com/keplerproject/rings
>> I am confused, why can't you make it part of the standard library?
>>
>> Download the Lua source and hack away, then add your new functionality
>> to the game-oriented Lua fork you are working on!
>
> But that isn't Lua. That isn't the Lua standard libraries, as defined by
> the reference manual.
>
> That's just a Lua fork.

Right... I thought you were working on a Lua fork, Cratera, right?

So why wouldn't you add your desired functionality to your fork? At least
do the initial work to show the benefits of your implementation and why
it should be something worth consideration of adoption upstream.

If this is something *you* desire to have in Lua then I think *you* will
need to do some of the groundwork towards implementation, no?

~Paige


Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Luiz Henrique de Figueiredo
In reply to this post by Soni "They/Them" L.
> But that isn't Lua. That isn't the Lua standard libraries, as
> defined by the reference manual.
>
> That's just a Lua fork.

No, it's not a Lua fork.  
The FAQ at https://www.lua.org/faq.html#1.7 says:

        1.7 - What do I call software derived from Lua?
       
        [...] If the syntax and the semantics of the language (that
        is, the parser and the virtual machine) remain the same,
        then the language is still Lua. If you simply add new
        libraries, or even replace the standard Lua libraries with
        your own libraries, the language is still the same, and you
        don't need to (and probably shouldn't) give it a completely
        different name.

Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Paige DePol
Luiz Henrique de Figueiredo <[hidden email]> wrote:

>> But that isn't Lua. That isn't the Lua standard libraries, as
>> defined by the reference manual.
>>
>> That's just a Lua fork.
>
> No, it's not a Lua fork.  
> The FAQ at https://www.lua.org/faq.html#1.7 says:
>
> 1.7 - What do I call software derived from Lua?
>
> [...] If the syntax and the semantics of the language (that
> is, the parser and the virtual machine) remain the same,
> then the language is still Lua. If you simply add new
> libraries, or even replace the standard Lua libraries with
> your own libraries, the language is still the same, and you
> don't need to (and probably shouldn't) give it a completely
> different name.

Wikipedia defines a fork as follows:

"In software engineering, a project fork happens when developers take a copy
of source code from one software package and start independent development
on it, creating a distinct and separate piece of software."

My custom variant of Lua makes some very significant changes to the original
language so I would definitely call it a fork. However, in your view if Soni
made a copy of Lua and did not change the language itself, but rather only
the libraries or other non parser/VM changes that does not count as a fork?
Even if the new libraries offer functionality not present in vanilla Lua?

I suppose this could just be a semtantic difference where your determination
of "fork" is when the language itself is changed, where I thought a fork was
when development occurred by anyone who wasn't the original developers.

~Paige


Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Luiz Henrique de Figueiredo
> in your view if Soni made a copy of Lua and did not change the
> language itself, but rather only the libraries or other non parser/VM
> changes that does not count as a fork? Even if the new libraries offer
> functionality not present in vanilla Lua?

No, in both cases, it's not a fork. That's what the FAQ says.

Reply | Threaded
Open this post in threaded view
|

What counts as a fork? (was: Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module)

Soni "They/Them" L.


On 2017-11-28 10:52 AM, Luiz Henrique de Figueiredo wrote:
>> in your view if Soni made a copy of Lua and did not change the
>> language itself, but rather only the libraries or other non parser/VM
>> changes that does not count as a fork? Even if the new libraries offer
>> functionality not present in vanilla Lua?
> No, in both cases, it's not a fork. That's what the FAQ says.
>

If someone changes an opcode in the VM to bring old behaviour back (like
lua 5.3.4's OP_SELF only accepting strings), is that a fork?

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: What counts as a fork? (was: Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module)

Luiz Henrique de Figueiredo
> If someone changes an opcode in the VM to bring old behaviour back
> (like lua 5.3.4's OP_SELF only accepting strings), is that a fork?

I'm not sure what you mean but if external behaviour is the same then
it's not a fork. If syntax or semantics change, it's a fork.

Reply | Threaded
Open this post in threaded view
|

What counts as a fork?

Paige DePol
In reply to this post by Luiz Henrique de Figueiredo
Luiz Henrique de Figueiredo <[hidden email]> wrote:

>> in your view if Soni made a copy of Lua and did not change the
>> language itself, but rather only the libraries or other non parser/VM
>> changes that does not count as a fork? Even if the new libraries offer
>> functionality not present in vanilla Lua?
>
> No, in both cases, it's not a fork. That's what the FAQ says.

Actually, the FAQ itself doesn't mention the word "fork" anywhere. It just
discusses what to call a derivative language, which is a different issue.

For example, GitHub[1] defines a fork as simply as a copy of a repository:

"A fork is a copy of a repository. Forking a repository allows you to freely
experiment with changes without affecting the original project."

Another definition of fork, from the Linux Information Project[2]:

"A project fork, also called a fork or forking, is the creation of a new
branch of a software development project that is independent of the existing
project."

There are other definition out there as well, I mentioned the Wikipedia one
in a previous post. All of which define a fork as an independently developed
copy of the original project.

If someone copies Lua and makes any changes then by definition I would have
to say they've created a fork. The FAQ covers what name that fork can use.

Please don't think I am trying to argue with you... I was just surprised at
your definition of "fork" being valid only if certain parts of the code were
changed, which is not how I've understood the term to be used. However, if
that is the definition team Lua wishes to use then you might want to add it
specifically to the FAQ as it does not match the usual definition.

For clarification the terms "soft fork" and "hard fork" could be adopted
from the BitCoin community. A "hard fork" could be used for changes to the
semantics of the language, whereas a "soft fork" would be your definition.

~Paige

[1] https://help.github.com/articles/fork-a-repo/
[2] http://www.linfo.org/project_fork.html


Reply | Threaded
Open this post in threaded view
|

Re: What counts as a fork?

Soni "They/Them" L.
In reply to this post by Luiz Henrique de Figueiredo


On 2017-11-28 11:12 AM, Luiz Henrique de Figueiredo wrote:
>> If someone changes an opcode in the VM to bring old behaviour back
>> (like lua 5.3.4's OP_SELF only accepting strings), is that a fork?
> I'm not sure what you mean but if external behaviour is the same then
> it's not a fork. If syntax or semantics change, it's a fork.
>

OP_SELF changed between 5.3.(?) and 5.3.4. Originally it accepted any
object as key, but since at least 5.3.4 it only accepts strings. Would
changing this back, e.g. because you need this because some of your
.luac files contain hand-written bytecode, be considered a fork?

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: What counts as a fork?

Luiz Henrique de Figueiredo
In reply to this post by Paige DePol
> Actually, the FAQ itself doesn't mention the word "fork" anywhere. It just
> discusses what to call a derivative language, which is a different issue.

Oh, I'm using "fork" to mean "incompatible derivative". So we're probably
takling about different things. Sorry for the noise.

Reply | Threaded
Open this post in threaded view
|

Re: What counts as a fork?

Paige DePol
Luiz Henrique de Figueiredo <[hidden email]> wrote:

>> Actually, the FAQ itself doesn't mention the word "fork" anywhere. It just
>> discusses what to call a derivative language, which is a different issue.
>
> Oh, I'm using "fork" to mean "incompatible derivative". So we're probably
> takling about different things. Sorry for the noise.

Ah, I thought that might have been the case! No worries about the "noise",
at least now we're on the same page! :)

The more I think about it the more I like the idea of using "soft fork" and
"hard fork" to disambiguate between compatible and incompatible variants of
Lua, especially when people are discussing Lua variants.

Does this seem like a good idea to anyone else? For example, Soni could say
they are creating Cratera, a soft fork, and we would know that it is Lua
compatible. While I could say I am creating Lunia, a hard fork, and we
would know I am creating a semantically incompatible variant of Lua.

I am currently creating a website to host my power patch collection and one
of the things I will be adding is information about the subject of Lua
forks. Specifically, compatible (soft) and incompatible (hard) forks of Lua
and when the name of the language should be changed.

In each patch I will also place a warning about "hard forks" requiring a
name change, along with a URL to an online tool that will generate a patch
file dynamically. This tool will ask for the new language name, author info,
versioning information, even a new binary signature, it will then create a
patch file to modify their hard fork to create a new Lua language variant.

I hope to have the site up by years end with 10 of my patches to start with.
There are more, but they're not ready to go live yet as I need to update
them to 5.3.4 from 5.3w2 amongst other tweaks. I will post when it is live!

~Paige

PS: I also have a GitHub page, but I wanted a stand-alone site to host the
    patches, the renaming tool, and other miscellaneous files related to Lua.


Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Russell Haley
In reply to this post by Paige DePol
Sorry for the top post. 

No offense Paige, but I will lean towards a standardized term, used by a university professor on a university project website over a rather airey social media definition of a term that grew from a git slang. I have many forks on my github site. I do not have any software derivatives though. 

Russ

Sent from my BlackBerry 10 smartphone on the Virgin Mobile network.
  Original Message  
From: Paige DePol
Sent: Tuesday, November 28, 2017 4:48 AM
To: Lua mailing list
Reply To: Lua mailing list
Subject: Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Luiz Henrique de Figueiredo <[hidden email]> wrote:

>> But that isn't Lua. That isn't the Lua standard libraries, as
>> defined by the reference manual.
>>
>> That's just a Lua fork.
>
> No, it's not a Lua fork.
> The FAQ at https://www.lua.org/faq.html#1.7 says:
>
> 1.7 - What do I call software derived from Lua?
>
> [...] If the syntax and the semantics of the language (that
> is, the parser and the virtual machine) remain the same,
> then the language is still Lua. If you simply add new
> libraries, or even replace the standard Lua libraries with
> your own libraries, the language is still the same, and you
> don't need to (and probably shouldn't) give it a completely
> different name.

Wikipedia defines a fork as follows:

"In software engineering, a project fork happens when developers take a copy
of source code from one software package and start independent development
on it, creating a distinct and separate piece of software."

My custom variant of Lua makes some very significant changes to the original
language so I would definitely call it a fork. However, in your view if Soni
made a copy of Lua and did not change the language itself, but rather only
the libraries or other non parser/VM changes that does not count as a fork?
Even if the new libraries offer functionality not present in vanilla Lua?

I suppose this could just be a semtantic difference where your determination
of "fork" is when the language itself is changed, where I thought a fork was
when development occurred by anyone who wasn't the original developers.

~Paige



Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Paige DePol
Russell Haley <[hidden email]> wrote:

> Sorry for the top post.
>
> No offense Paige, but I will lean towards a standardized term, used by a
> university professor on a university project website over a rather airey
> social media definition of a term that grew from a git slang. I have many
> forks on my github site. I do not have any software derivatives though.

You might want to read the rest of the posts that have occurred since
the one you quoted in your reply! ;)

The term "fork" has been around a lot longer than Git by the way, and in a
later post Luiz stated that he meant "incompatible derivative" specifically
when he used the term "fork", which is not really the standard meaning.

If you have forked repositories on GitHub, and then made modifications to
that forked code, is that not then a derivative of the original code?

Finally, university professors are human and can make mistakes, they are
not infallible by any means.

~Paige


Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Russell Haley
On Tue, Nov 28, 2017 at 9:01 AM, Paige DePol <[hidden email]> wrote:

> Russell Haley <[hidden email]> wrote:
>
>> Sorry for the top post.
>>
>> No offense Paige, but I will lean towards a standardized term, used by a
>> university professor on a university project website over a rather airey
>> social media definition of a term that grew from a git slang. I have many
>> forks on my github site. I do not have any software derivatives though.
>
> You might want to read the rest of the posts that have occurred since
> the one you quoted in your reply! ;)
>
> The term "fork" has been around a lot longer than Git by the way, and in a
> later post Luiz stated that he meant "incompatible derivative" specifically
> when he used the term "fork", which is not really the standard meaning.
>
> If you have forked repositories on GitHub, and then made modifications to
> that forked code, is that not then a derivative of the original code?
>
> Finally, university professors are human and can make mistakes, they are
> not infallible by any means.
>
> ~Paige
>
>
The mistake is to introduce the term fork where a standard definition
has existed for a very long time.  Mr. Henrique de Figueiredo mistake
was to apologize where he was correct. Soni has used a slang term -
not a definition - for software derivative. Mr.  Henrique de
Figueiredo pointed to the standardized definition for software
derivatives when referencing Lua.

Russ

Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Russell Haley
In reply to this post by Soni "They/Them" L.
On Tue, Nov 28, 2017 at 3:48 AM, Soni "They/Them" L. <[hidden email]> wrote:

>
>
> On 2017-11-28 09:43 AM, Paige DePol wrote:
>>
>> Soni They/Them L. <[hidden email]> wrote:
>>
>>> On 2017-11-28 03:06 AM, Sean Conner wrote:
>>>>
>>>>    After two years of no one biting on this, I think the onus is on you
>>>> to
>>>> provide this instead of asking other people to do the work.
>>>>
>>>>    -spc (Think of all the accolades you'll get from the game developers
>>>> ... )
>>>>
>>>> [2]     http://lua-users.org/lists/lua-l/2015-06/msg00573.html
>>>>
>>>> [3]     http://lua-users.org/lists/lua-l/2015-09/msg00341.html
>>>
>>> I can't make it part of the standard libraries. So I have to ask for it.
>>>
>>> https://github.com/keplerproject/rings
>>
>> I am confused, why can't you make it part of the standard library?
>>
>> Download the Lua source and hack away, then add your new functionality
>> to the game-oriented Lua fork you are working on!
>
>
> But that isn't Lua. That isn't the Lua standard libraries, as defined by the
> reference manual.
>
> That's just a Lua fork.
Are there many games that run 5.3 Lua? My rather dim understanding is
that most are pegged to 5.1 and don't change much. I'd LOVE to know
know more about game programming but I have trouble following what
you're trying to achieve as you give little context.

>> The great thing about Lua, and other open source software, is that you
>> have the ability to modify the code to do whatever your heart desires!
>>
>> Good luck with your project! :)
>>
>> ~Paige
>>
>>
>>
>
> --
> Disclaimer: these emails may be made public at any given time, with or
> without reason. If you don't agree with this, DO NOT REPLY.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [Proposal-ish?] Lua should bind the Lua C API as a Lua module

Etiene Dalcol
In reply to this post by Russell Haley
No offense Russ, but mo mistakes were made except for your insistence
on pointing mistakes, since the Lua FAQ does not make a definition
for the term "fork". Especially when, Dr. de Figueiredo, or LHF, and Paige
are discussing it in a quite relaxed way on a different thread and
establishing a common understanding of fork before moving on with
the conversation.  :)

On 28 November 2017 at 17:06, Russell Haley <[hidden email]> wrote:
On Tue, Nov 28, 2017 at 9:01 AM, Paige DePol <[hidden email]> wrote:
> Russell Haley <[hidden email]> wrote:
>
>> Sorry for the top post.
>>
>> No offense Paige, but I will lean towards a standardized term, used by a
>> university professor on a university project website over a rather airey
>> social media definition of a term that grew from a git slang. I have many
>> forks on my github site. I do not have any software derivatives though.
>
> You might want to read the rest of the posts that have occurred since
> the one you quoted in your reply! ;)
>
> The term "fork" has been around a lot longer than Git by the way, and in a
> later post Luiz stated that he meant "incompatible derivative" specifically
> when he used the term "fork", which is not really the standard meaning.
>
> If you have forked repositories on GitHub, and then made modifications to
> that forked code, is that not then a derivative of the original code?
>
> Finally, university professors are human and can make mistakes, they are
> not infallible by any means.
>
> ~Paige
>
>
The mistake is to introduce the term fork where a standard definition
has existed for a very long time.  Mr. Henrique de Figueiredo mistake
was to apologize where he was correct. Soni has used a slang term -
not a definition - for software derivative. Mr.  Henrique de
Figueiredo pointed to the standardized definition for software
derivatives when referencing Lua.

Russ




--
Etiene Dalcol

Software Engineer @ Red Badger 
Lua Space http://lua.space
LuaConf http://luaconf.com

1234