new "empty" value/type in Lua?

classic Classic list List threaded Threaded
102 messages Options
123456
Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Dirk Laurie-2
2013/6/28 Dirk Laurie <[hidden email]>:

> 2013/6/28 Tim Hill <[hidden email]>:
>
>> But I stand by my point; it's an oddity of the language that I cannot
>> use nil to mark an array entry as empty, and my suggestion was to fix
>> that in a clean simple manner. Nothing more, nothing less.
>
> It's a feature of the language, call it an oddity if you prefer, that
> unused array items quite likely do not occupy any actual storage. The
> Lua `nil` fits in very nicely with that. It is not intended to mark an
> array entry as empty. The ways to do so are domain-specific because the
> necessity to so is domain-specific.
>
> I ask again: should `empty` be a valid table key?

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Andrew Starks
In reply to this post by Enrico Colombini
On Fri, Jun 28, 2013 at 7:01 AM, Enrico Colombini <[hidden email]> wrote:

> On 28/06/2013 13.03, Tim Hill wrote:
>>
>> There are plenty of cases where being able to "mark" an array entry
>> as done/dead/empty etc is useful as part of an algorithm. In many
>> cases you can do that using a sentinel value that is outside the
>> normal range of expected array types and/or values, but it's all very
>> domain specific (can I use an empty string? -1? false? auxiliary
>> state?). I'm basically arguing that "empty" allows you to do this in
>> a nice, clean, self-documenting, portable manner. I don't really see
>> anything heretical in that.
>
>
> empty = {}
>
> --
>   Enrico
>

I think that Tim's point is not that there is not a workable solution.
It is that there are many workable solutions and since there is no
clear one way to solve the problem, it is likely to cause a mess
when/if there are libraries solving it in different ways.

Do I have that premise correctly stated?

Personally, I have never run into this issue, but I don't claim to be
that experienced, either. Furthermore, if it is an issue, it must be
miles down the list from "Hey, why isn't there a standard way to do
OOP?"

Some guy said it best, "Lua is not Perl." It's also probably why there
are relatively few (by JS or Ruby or Python standards) standard
libraries. I like to think of Lua as scripting starter kit for my
applications. The language and libraries are whatever I want them to
be. So,as suggested, you're kind of a light userdata value away from
exactly what you want, IF you scope your question to, "My version of
Lua should elegantly and efficiently solve this use case..."

If you long for a wider scope, then you're unlikely to be satisfied.
As demonstrated by the previous posts, there usually isn't a whole lot
of support behind adding new "things" to the language, especially when
they're "policy" things.

That is, there are many mechanisms in the language to accomplish
exactly what you want. You want a policy, which is not really what Lua
has traditionally been about.

-Andrew

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Tim Hill

On Jun 28, 2013, at 7:30 AM, Andrew Starks <[hidden email]> wrote:

> I think that Tim's point is not that there is not a workable solution.
> It is that there are many workable solutions and since there is no
> clear one way to solve the problem, it is likely to cause a mess
> when/if there are libraries solving it in different ways.
>
> Do I have that premise correctly stated?

Exactly. Everyone posts "look .. here's how you can do it" which is missing the point. I *know* how to do it, as I've repeatedly stated, in a dozen different ways (my preference is like yours, to use a light userdata wrapping a C NULL pointer btw).

My argument was:
(a) I see this as a useful generic language feature, that while lightweight is also light to implement and understand.
(b) It cleans up one of the side-cases of Lua, the odd behavior of Lua arrays when faced with nil values.

> If you long for a wider scope, then you're unlikely to be satisfied.
> As demonstrated by the previous posts, there usually isn't a whole lot
> of support behind adding new "things" to the language, especially when
> they're "policy" things.

Agreed, and 99 times out of 100 I'm on the side of "avoid feature bloat at all costs". But frankly some of the arguments used here seem zealous to say the least, and boil down to "Lua is perfect, any odd behaviors are by design and should stay odd". I really hope Lua isn't becoming a religion, rigidly set in stone because the "chosen ones" defend the holy language against the evil of change.

I *really* like Lua; it is, to my mind, the cleanest scripting language out there, by a WIDE margin. But i's not perfect; I was merely suggesting one area that is less elegant than other areas and a possible way to improve it.

> That is, there are many mechanisms in the language to accomplish
> exactly what you want. You want a policy, which is not really what Lua
> has traditionally been about.

Well it's a language level feature, which doesn't necessarily imply "policy", just a defined behavior that can be used in a number of different circumstances. The "spirit" of Lua, to my mind, is one of providing as much functionality and as little policy in as compact a language as possible, with as few surprises and enough sugar to make it familiar to those coming from other environments.

Regarding the "I see very little use for it" argument. I've now been involved in three MAJOR projects using Lua in a significant or key role (we are talking a LOT of Lua code and a LOT of C libraries here). In each case we have hit against real issues with interfacing between C and Lua where the lack of an "empty" feature has caused us to create workarounds of various types. And of course we've worked around them. But it left me thinking hard about how much simpler this would be if I had an "empty" value in the language and a few extra APIs like "lua_isempty()".

However, as you state, there is essentially zero chance of this happening, Thanks for your input, good discussion. As I stated as the start, I was expecting to be attacked, I think it's a pity no-one engaged beyond "hey look .. I know a workaround!" and "how dare you suggest we meddle with perfection!".

It would be amusing to speculate how people would react if Lua had always had "empty" and I posted a suggestion that it should be removed as being unnecessary, since there were obviously so many workarounds. :)

--Tim


Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Tim Hill
In reply to this post by Philipp Janda

On Jun 28, 2013, at 5:18 AM, Philipp Janda <[hidden email]> wrote:

Why do you care if it's an "array"? No matter what, you can access the values for the integer keys from 1 to n. If nil is not allowed as a value, Lua gives you a fast way to calculate that n. If nil may be in there somewhere, somebody better tell you where this table is supposed to end because that information is not obvious!

Yes, it's NOT obvious. It could be MADE obvious by using "empty" instead of nil, in which case the "fast way to calculate n" would be MORE useful, which seems to me a good thing, doesn't it?

Are you saying it's a bad idea to make arrays more useable?
Is it better to have 20 different workarounds to a common problem than one simple, clear, language supported fix?

--Tim

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Roberto Ierusalimschy
In reply to this post by Tim Hill
> My argument was:
> (a) I see this as a useful generic language feature, that while lightweight is also light to implement and understand.
> (b) It cleans up one of the side-cases of Lua, the odd behavior of Lua arrays when faced with nil values.

Maybe I am missing something obvious, but Lua already has an "empty"
value. It is called "false". Of course, it is not a perfect solution,
because now your array cannot have "false" values, as they denote
holes. So, ok, we create a new "empty" value to denote holes. But,,,
wait! As Dirk asked repeteadly without an answer, what if we want to
store an "empty" value into an array? After all, "empty" is a valid value
in the language, is it not? Functions can return "empty" (e.g., rawget),
and therefore I may want to store it in an array (e.g., all the results
of a function, or the results of an array of functions).

The solution seems obvious again: we create a new value, "undefined". But,
wait!...

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Petite Abeille
In reply to this post by Tim Hill

On Jun 28, 2013, at 11:17 AM, Tim Hill <[hidden email]> wrote:

> No-one has answered my original question: How can I write a CLEAN C library that emits a SQL rowset that might include NULL values? Without "empty", I cannot see a clean way to do this.

Again, as spelled out by Rena among others, what's so fundamentally wrong with mapping null to nil?!? I'm personally doing it every day, in and out, and don't feel like I'm missing anything at all. But perhaps it's just me…



Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Jorge Visca
In reply to this post by Roberto Ierusalimschy
On 28/06/13 18:24, Roberto Ierusalimschy wrote:
> Maybe I am missing something obvious, but Lua already has an "empty"
> value. It is called "false".

Now maybe I am missing something obvious :)
But empty is not a noun, is an adjective. Something can be empty. To be
capable of being empty, it must be capable of holding something in the
first place. Thus a table. This leads to

empty = function(t) return next(t)==nil end

If I want store emptiness, I should wrap all my stuff in tables

t[x] = nil --nothing
t[y] = {'stuff'} --something not emptiness
t[z] = {} --something: emptiness

I consider this argumentation completely irrefutable. Hope nobody
restarts it using the term "void" :-\

Jorge

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Tim Hill
In reply to this post by Roberto Ierusalimschy
Of course the recursive nature of such a model is obvious (I was aware of it when I suggested empty), but I stand by my argument: Since Lua arrays cannot be sparse, you must either (a) structure the data in a form that is not an array or (b) have a sentinel value that acts as a placeholder for "empty" elements. Since Lua has no language-defined placeholder, it is necessary to choose an existing Lua value to act in that capacity, be it false, an empty table, userdata, or the string "this-is-not-an-element".

Such a choice is domain-specific, which is fine in a closed application model, where you are free to create the necessary conventions (and hopefully enforce them). But when developing libraries for more general use, plucking a convention out of the air is asking for trouble. Everyone will choose a different one, and interoperability issues turn up. My feeling was that a sanctioned placeholder would help alleviate this situation. After all, why does SQL have NULL values? You could argue it doesn't need them for the same reason Lua doesn't need empty, but it would force each table schema to define it's own "placeholder" in a domain-specific manner, which I don't think the database community would like much.

And yes, of course you can store empty into an array (I believe I did answer this). As I stated, "empty" to my mind is simply a regular Lua value that is not equal to any other value except itself. So it can be used as a value, a function argument, a return value, a table key etc etc. Doesn't seem too complex a concept to me, but maybe I'm missing something obvious...

--Tim


On Jun 28, 2013, at 2:24 PM, Roberto Ierusalimschy <[hidden email]> wrote:

>> My argument was:
>> (a) I see this as a useful generic language feature, that while lightweight is also light to implement and understand.
>> (b) It cleans up one of the side-cases of Lua, the odd behavior of Lua arrays when faced with nil values.
>
> Maybe I am missing something obvious, but Lua already has an "empty"
> value. It is called "false". Of course, it is not a perfect solution,
> because now your array cannot have "false" values, as they denote
> holes. So, ok, we create a new "empty" value to denote holes. But,,,
> wait! As Dirk asked repeteadly without an answer, what if we want to
> store an "empty" value into an array? After all, "empty" is a valid value
> in the language, is it not? Functions can return "empty" (e.g., rawget),
> and therefore I may want to store it in an array (e.g., all the results
> of a function, or the results of an array of functions).
>
> The solution seems obvious again: we create a new value, "undefined". But,
> wait!...
>
> -- Roberto
>


Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Tim Hill
In reply to this post by Petite Abeille
You can do this of course, but only at the expense of not using arrays to hold rows. I guess more or less what I'm saying is this:

-- The cleanest, most obvious way to map a SQL row into Lua is as a Lua array.
-- Most databases have numbers, strings, boolean (maybe), NULL and BLOB (maybe)
-- Again, there is an obvious clean mapping of each database type to a Lua type (BLOBs aside)
-- Except if SQL NULL maps to nil, you cannot put the NULL (=nil) value into the array

So you either have to map NULL to something other than nil, or choose a different representation in Lua, which means moving away from the obvious mapping to a non-obvious one to work-around the nil-array problem. At this point, in my mind, you are swimming upstream *against* the language, rather than having it help you (which, in most other ways, Lua excels at).

--Tim



On Jun 28, 2013, at 2:30 PM, Petite Abeille <[hidden email]> wrote:

>
> On Jun 28, 2013, at 11:17 AM, Tim Hill <[hidden email]> wrote:
>
>> No-one has answered my original question: How can I write a CLEAN C library that emits a SQL rowset that might include NULL values? Without "empty", I cannot see a clean way to do this.
>
> Again, as spelled out by Rena among others, what's so fundamentally wrong with mapping null to nil?!? I'm personally doing it every day, in and out, and don't feel like I'm missing anything at all. But perhaps it's just me…
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Rena
In reply to this post by Tim Hill

On 2013-06-28 6:47 PM, "Tim Hill" <[hidden email]> wrote:
>
> And yes, of course you can store empty into an array (I believe I did answer this). As I stated, "empty" to my mind is simply a regular Lua value that is not equal to any other value except itself. So it can be used as a value, a function argument, a return value, a table key etc etc. Doesn't seem too complex a concept to me, but maybe I'm missing something obvious...
>
> --Tim

That sounds like you're describing nil. What difference is there between empty and nil except that one can be stored in a table? That seems a lot more confusing than nil itself.
It can help to rethink how tables work. It's not true that a table can't contain nil - it contains nil in every key that doesn't have something else in it. The problem is that leaves no way to iterate over keys that contain nil, since you'd never know when to stop - there are an infinite number of them in every table.

Like I said before, I do think there's benefit to having a single standard "empty" value across the entire Lua state, so that each module doesn't have to have their own - I think this was the main point you intended to make. I just don't see why it needs to be a new "special" type, and not just a global variable created by the standard libraries, just like string and math, pointing to an empty table or userdata.

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Javier Guerra Giraldez
In reply to this post by Tim Hill
On Fri, Jun 28, 2013 at 5:56 PM, Tim Hill <[hidden email]> wrote:
> You can do this of course, but only at the expense of not using arrays to hold rows.


given that SQL query results have a defined number of columns, there's
no problem having nil'ed entries.

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Tim Hill
In reply to this post by Rena

On Jun 28, 2013, at 3:59 PM, Rena <[hidden email]> wrote:

That sounds like you're describing nil. What difference is there between empty and nil except that one can be stored in a table? That seems a lot more confusing than nil itself.
It can help to rethink how tables work. It's not true that a table can't contain nil - it contains nil in every key that doesn't have something else in it. The problem is that leaves no way to iterate over keys that contain nil, since you'd never know when to stop - there are an infinite number of them in every table.

Wel yes, and really the point here is that nil is somewhat overloaded; it means "nothing" sometimes (such as in return values of functions) and other times means "delete" (as in table key/value pairs). And yes, I get (and really like) the concept of all tables having an infinite number of entries, most of which are nil. "empty", to my mind, is just a value stored as a placeholder so that an array stays an array when it has "holes" .. perhaps what's messing people around is my use of the name "empty" .. perhaps "hole" might be better (? "black hole").

Like I said before, I do think there's benefit to having a single standard "empty" value across the entire Lua state, so that each module doesn't have to have their own - I think this was the main point you intended to make. I just don't see why it needs to be a new "special" type, and not just a global variable created by the standard libraries, just like string and math, pointing to an empty table or userdata.


And that would be perfectly fine too .. it's not really an issue of it having to be baked into the language, it's just a standard that everyone can assume is available. My current work-around is pretty much as you describe; it's a light userdata holding a C (void*)0 pointer. It has all the attributes I need, and it works just fine. But without someone saying "if you need to create an "holding place" within an array, use "xxx", everyone will come up with different ways to do this, and we have chaos. After all, look at the all the different suggestions in this thread; all valid but all different. I'm reminded of the old Adventure game: "You are in a twisty maze of little passages, all different". :)

--Tim


Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Javier Guerra Giraldez
On Fri, Jun 28, 2013 at 6:09 PM, Tim Hill <[hidden email]> wrote:
> My current work-around is pretty much as you describe; it's a light userdata
> holding a C (void*)0 pointer. It has all the attributes I need, and it works
> just fine. But without someone saying "if you need to create an "holding
> place" within an array, use "xxx", everyone will come up with different ways
> to do this, and we have chaos.


on one side, i think Roberto's position is the only one sensible from
a language-theoretical point of view for a language that strives for
minimalism.

on the other side, a theoretically pure language could be too
cumbersome to use, so one have to balance with practical approaches.
(of course going too far in that direction you find Python, which is
so practical, but so 'dirty'... but that's OT)

so, I'm curious about the real, specific objections to current
practices.  One point that you mention is the diversity of options,
which reflects the lack of a single "right" solution.  But as I see
it, there are different choices because there are different
situations.  From the top of my head, i recognize these:

A: if there are fixed length arrays, i just use nil.

B: if I create a list, set or table of values and need a placeholder,
just use false, unless it's a list of booleans, of course.

C: if i store values that come from other libraries (or modules
developed independently), i want to use a value that no other library
could use, so i use a unique empty table as the 'nothing' value.  in
that case, it must be a private value, handled only by the container
code.  even if i have different container modules (say, a heap and an
ordered dict), each one has it's own 'nothing'.  in most cases, i
avoid returning the 'nothing' value.  presence/absence is signaled
'offline' or replaced with nils if appropriate.

in my opinion, these are not workarounds for the lack of a 'nothing'
value.  these are different uses, so they need different solutions.
having a single extra value would be the same in A, a little more
'semantic' in the B case and less adequate on the C case.

as you can see, i don't agree with the idea that "this is just policy,
create a global 'nothing' and use everywhere",  but my own Lua
experience is mostly from lots and lots (and lots) of small and medium
scripts, very few large and complex systems, so i'm honestly curious
how having an extra language-defined 'empty' value could help in your
experience.

Are you thinking of a different case besides those three? or do you
disagree they're so different?


--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Dirk Laurie-2
In reply to this post by Tim Hill
2013/6/28 Tim Hill <[hidden email]>:

> I'm going to get shot for suggesting this… (hides under bed)
>
> I've been writing a lot of Lua code recently, some of it quite generic in nature. One thing I frequently come up against is the need to handle data polymorphically in tables. And an effect of this is to hit against the overloaded use of "nil" in Lua.
>
> It seems to me that "nil" is overloaded; it means both a noun: "no value" and a verb: "delete this table entry". Now, most of the time, the delete is hidden since reading a non-existent table entry returns nil. So having an explicitly stored nil versus having no value stored at all is equivalent (all this I love, btw … 100 points to Lua).
>
> But it all rather breaks down with arrays (sequences). I cannot have sparse arrays, which in Lua terms means I cannot have "nil" in an array element. So, when writing polymorphic code, how do I handle cases (which, trust me, i DO have to handle, with "no value" in an array element? There are, of course, a whole bunch of work-arounds: Use a sentinel value; use an empty table (assuming empty tables aren't value data, and ignoring that each one is unique); use a magic userdata (assuming I have access to C code to make one); etc etc.
>
> But, let's face it; they ARE all workarounds. I think there IS a case to be made for being able to store a value in a Lua array that is uniquely NOT any other Lua value. Essentially, I think that value is "empty"; a valid Lua value that is simply not the same as any other Lua value (including nil).
>
> Basically, empty would act like nil in most regards, except:
> -- Storing empty in a table element actually stores the value, rather than deletes it.
> -- When used in a boolean context, the empty value evaluates to true (unlike nil).
> -- Has a type() of "empty"
>
> Other arguments in favor of this:
> -- Judging by the mail list chatter, a lot of Lua beginners struggle with the problem of nil in sequences.
>
> -- Better impedance matching to SQL NULL values; since empty can bridge to a SQL NULL when a Lua array is used to hold a SQL row.
>
> Thoughts anyone?
> --Tim
>
>

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Dirk Laurie-2
In reply to this post by Tim Hill
2013/6/28 Tim Hill <[hidden email]>:
> I'm going to get shot for suggesting this… (hides under bed)

You were not. But your repeated persistence in re-trotting out the
same argument at length, despite finding no support, just might
push some of us into oiling the old equalizer.

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Tim Hill
In reply to this post by Javier Guerra Giraldez
No, you make very good points, and of course these are very workable solutions … ALL the suggestions are that. My problem stems from good software design. The system I am designing is inherently very complex (has to be .. long story), and in any complex project you strive to manage that complexity, otherwise you are doomed. As we all know, one way to manage complexity is well defined module boundaries, and clearly delineated areas that do or do not have knowledge of other parts of the system (information hiding).

One aspect of this is the need for a generic way to loft SQL results into Lua in an efficient (re: fast) manner where the underlying schema is NOT known to the transport code; it's sole job is one of packaging. In this model it's crazy to suggest (as some have stated) "you know your schema, so you know the structure of the results", because the lofting code does NOT know the schema .. nor SHOULD it, architecturally. In addition, this lofting has to happen on (OS) threads outside of Lua, and so CANNOT be linked (directly) to things like well-known magic tables. So I have to be able to (a) recognize NULLs, (b) loft them, (c) present them to Lua in a simple to consume manner.

When I design any interface or API, I strive to make it as "user friendly" as I can .. in fact one reason I like Lua is that the libraries seem to have that same philosophy .. tight, minimal, but powerful. Now, my experience of developers leads me to think that a naive developer, having just got to grips with Lua, would immediately think .."hmmm, database row … sounds like a Lua array", and I think he'd be right .. it's heterogenous, ordered (like a SQL row), and clean. And in a system where the schema (or perhaps even the SELECT statement being processed) is not known to the intermediate layers, a Lua array also carries type information for each element (via Lua type()) and the column count (via #). It's clean, easy, simple.

But it doesn't work, as Lua doesn't have enough "width" to its type system to handle the basic set of SQL types, and careful analysis says the problem here is NULL … the obvious mapping here is NULL -> nil, but then we hit on the issue where Lua arrays with "holes" (nil elements) are not arrays any more (any yes, I really, really, REALLY get why that is, trust me). So we need to invent a type that CAN tunnel in from an external non-Lua source and be efficiently surfaced. Hence my choice of a (void*)0 light userdata. Works perfectly.

But when I thought carefully, I felt that this had other applications in other areas of Lua. It allowed algorithms that had a need to mark an array item as "dealt with" or "processed" in a way that would not break the "array-ness" of the table that using nil does. Hence my suggestion. It wasn't a "how do I work around this?", it was a "if we think about this problem, isn't this something that might warrant further discussion?". Apparently, the answer is a resounding "shut up", which to be honest I think is a bit sad.

I've been working on complex software architectures for decades, and I've seen a lot of places where the lack of a "blessed" way to do something created a truly horrific mess. Perhaps i'm sensitized to this, but I still feel the Lua arrays are one of the more "odd" behaviors of the language. One moment its an array, the next its not. So when someone learning Lua says (a good question, btw), "ok, I get i'm not supposed to put nil in an array because they cannot be sparse, but what DO i put in?" .. and you say "it depends", I think you are punting a bit. Roberto suggested false, which sounds like he feels no-one ever needs to store booleans in an array .. a position I find odd.

--Tim

On Jun 28, 2013, at 8:01 PM, Javier Guerra Giraldez <[hidden email]> wrote:

as you can see, i don't agree with the idea that "this is just policy,
create a global 'nothing' and use everywhere",  but my own Lua
experience is mostly from lots and lots (and lots) of small and medium
scripts, very few large and complex systems, so i'm honestly curious
how having an extra language-defined 'empty' value could help in your
experience.

Are you thinking of a different case besides those three? or do you
disagree they're so different?

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Tim Hill
In reply to this post by Dirk Laurie-2
As you wish. I consider the matter closed. Sad.

On Jun 28, 2013, at 9:42 PM, Dirk Laurie <[hidden email]> wrote:

> 2013/6/28 Tim Hill <[hidden email]>:
>> I'm going to get shot for suggesting this… (hides under bed)
>
> You were not. But your repeated persistence in re-trotting out the
> same argument at length, despite finding no support, just might
> push some of us into oiling the old equalizer.
>


Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Javier Guerra Giraldez
In reply to this post by Tim Hill
On Fri, Jun 28, 2013 at 11:51 PM, Tim Hill <[hidden email]> wrote:
> I still feel the Lua arrays are one of the more "odd" behaviors of the
> language. One moment its an array, the next its not. So when someone
> learning Lua says (a good question, btw), "ok, I get i'm not supposed to put
> nil in an array because they cannot be sparse, but what DO i put in?" .. and
> you say "it depends"


a different approach is not to use sequences, nor expect any table to
be one.  if you need an array-like structure that can hold nils as any
value, then just create a specific structure: a few short functions
that store values in a table while keeping score of the 'length'.  i
think penlight has one of these.

sometimes people here advocate here the idea of not using 'naked'
tables, but rather use them to build the exact structures you need.  a
vector structure is an easy one and can be just as efficient as native
arrays on other languages.

again, this could be just a new "it depends" answer... but that's how
engineering works, no? by knowing many useful idioms and using them to
avoid reinventing whenever possible.


--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Jerome Vuarand
In reply to this post by Tim Hill
2013/6/28 Tim Hill <[hidden email]>:
> Regarding the "I see very little use for it" argument.

You apparently have a use for it. Since it's been shown it's easy to implement
that 'empty' value on top of existing Lua, and in a way that is, from Lua
code, indistinguishable from the actual integration of the concept in the VM,
did you actually implement it?

> I've now been involved in three MAJOR projects using Lua in a significant or
> key role (we are talking a LOT of Lua code and a LOT of C libraries here).
> In each case we have hit against real issues with interfacing between C and
> Lua where the lack of an "empty" feature has caused us to create workarounds
> of various types. And of course we've worked around them. But it left me
> thinking hard about how much simpler this would be if I had an "empty" value
> in the language and a few extra APIs like "lua_isempty()".

So, in these three MAJOR projects, did you extend Lua to have that "empty"
value? Given the relatively small size of Lua compared to any project that can
be qualified as MAJOR in capitals, there was no reason not to.

Did you implement the "empty" value in the same way for the three projects?
Did the concept (and eventually the implementation) apply equally well to the
three projects?

What are the few extra APIs that you had to define?

Did you initially abuse the concept to later realize it was useful in some
cases, but not that much in others?

> However, as you state, there is essentially zero chance of this happening,

I may be overlooking something, but that whole modification of Lua is much
less than 1000 lines of code, which is basically nothing (and I feel I'm being
quite generous with such an estimate). It's been said and repeated that there
is no technical obstacle to its implementation. So did you make it happen? If
not, why not?

When basic logic doesn't convince, hard numbers can help. A detailed analysis
of a use case and a conclusion in the style of "my implementation of 'empty'
shaved X% of my code base, improved performance by Y%, and fixed Z bugs" will
certainly gain you some supporters.

> "how dare you suggest we meddle with perfection!".
>
> It would be amusing to speculate how people would react if Lua had always
> had "empty" and I posted a suggestion that it should be removed as being
> unnecessary, since there were obviously so many workarounds. :)

I for one would have quoted Saint-Exupéry to support such a case :-)

Reply | Threaded
Open this post in threaded view
|

Re: new "empty" value/type in Lua?

Philipp Janda
In reply to this post by Tim Hill
Am 28.06.2013 20:01 schröbte Tim Hill:
> It would be amusing to speculate how people would react if Lua had
> always had "empty" and I posted a suggestion that it should be removed
> as being unnecessary, since there were obviously so many workarounds. :)

One example of a recently removed "policy" feature is the module
function. Obviously removing an existing feature produces much more
heat, because people have come accustomed to it, and there is code using
this feature which will break. Fortunately the Lua authors never had to
remove a complete type yet.

>
> --Tim
>

Philipp



123456