Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Jay Carlson

(This is aimed at the "community edition" thread too.)

On Sat, Feb 11, 2012 at 6:45 PM, Patrick Rapin <[hidden email]> wrote:
>> I hoped that the 5.2 -E interpreter option would be backported to 5.1.5. Any
>> chance of getting that?
>
> Most certainly none.

Lately it seems like many people are quite certain about things they have no control of. I think "very unlikely, based on policy and precedent" is about as far as anybody could say, except for two people.

Anyway, there's a significant Lua fork with a bunch of users; it does not use version numbers like 5.1.5. In addition the Debian fork has made changes like this in the past.

On Unix systems, a wrapper script

#!/bin/sh
# in case you wanted to put them back later or something
old_LUA_INIT="$LUA_INIT"
old_LUA_PATH="$LUA_PATH"
old_LUA_CPATH="$LUA_CPATH"
export old_LUA_INIT old_LUA_PATH old_LUA_CPATH
unset LUA_PATH LUA_INIT LUA_CPATH
exec lua "$@"

will *almost* have the same effect, and the cost is near zero.

Oh, you're on Windows.

> If you want -E on 5.1, you can backport it yourself for your usage.

Well, if you think it would significantly help on Windows you may want to have a word with Antonio Scuri, who's listed as the owner of record of the LuaBinaries project for Win32. Or talk to the Lua For Windows committers. I think a patch with some tests would help.

> But asking for this official change is utopia...

I think the reason there isn't even a common basic library (at this point "standard library" is a poisonous term) is there is not sufficient leadership interested in it. What I mean by leadership is:

"I'll listen to this person/these people because I trust them to make good decisions, and I know many other people do too. And they have working code."

It feels like learned helplessness to just say "upstream isn't going to do it, so it's hopeless." I think repeated statements *from* upstream approve of branches (as long as it's not marked to confuse people.)

Official isn't the last word. A long time ago Linus's kernels were official and standard and everybody shipped them, but that hasn't been true for ages; every significant Linux distro ships their own kernels. In my opinion it's been good for mainline. Lack of any forking (for various reasons) can be near fatal, oddly; see egcs, Emacs 19, glibc--although those were arguably just the very visible symptoms, not the problem itself.

Jay

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Dirk Laurie-2
I think the reason there isn't even a common basic library (at this point "standard library" is a poisonous term) is there is not sufficient leadership interested in it. What I mean by leadership is:

"I'll listen to this person/these people because I trust them to make good decisions, and I know many other people do too. And they have working code."

There are at least two such projects already: stdlib and penlight, both maintained by well-respected members of the community.  Maybe we should start debating the relative merits of the two very different approaches taken, as a basis for growing into the kind of community-accepted library that poeple clamour for,
Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Jon Akhtar
Perhaps if we could get an endorsement on the result from upstream,
and either inclusion in the release (as an external source based
library), or at least a link off of the main page. Otherwise - it is
sort of pointless to bless one library vs another.

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Alex Queiroz
Hallo,

On Mon, Feb 13, 2012 at 2:00 PM, Jon Akhtar <[hidden email]> wrote:
> Perhaps if we could get an endorsement on the result from upstream,
> and either inclusion in the release (as an external source based
> library), or at least a link off of the main page. Otherwise - it is
> sort of pointless to bless one library vs another.
>

I should start counting how many times this common library discussions
come full circle and end up nowhere. Maybe this is the 500th one. I'll
just put a counter on "bless" and "blessed".

I really have no idea what's with people and blessing.

--
-alex
http://www.artisancoder.com/

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

steve donovan
On Mon, Feb 13, 2012 at 3:07 PM, Alex Queiroz <[hidden email]> wrote:
> I really have no idea what's with people and blessing.

It's a emotional habit. If people could think with the equivalence
'Torvalds:Linux Kernel :: PUC Rio:Lua" then they will see that it's
like insisting on Linus recommending a window manager.  Or (more
directly relevant) asking the ISO committee to suggest an XML library
for C.   Upstream may have opinions, but these are not 'canonical'
opinions (another word of religious origin)

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

KHMan
In reply to this post by Alex Queiroz
On 2/13/2012 9:07 PM, Alex Queiroz wrote:

> On Mon, Feb 13, 2012 at 2:00 PM, Jon Akhtar wrote:
>> Perhaps if we could get an endorsement on the result from upstream,
>> and either inclusion in the release (as an external source based
>> library), or at least a link off of the main page. Otherwise - it is
>> sort of pointless to bless one library vs another.
>
> I should start counting how many times this common library discussions
> come full circle and end up nowhere. Maybe this is the 500th one. I'll
> just put a counter on "bless" and "blessed".
>
> I really have no idea what's with people and blessing.

Yeah, but I suppose we should trundle out some old arguments for
the benefit of the new folks... :-) :-) :-)

What a Lua+everything thingy needs is stable leadership,
management and manpower. Not to mention absorbing small projects
under one umbrella. Until that is created and we have a thingy
shipping for a few quarters, there is no use to ask for blessing.
But a long-term thingy is harder to start and run than it looks
(for a small community)... it's not like this is the first time
people are discussing it, as Alex says.

--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Axel Kittenberger
So this thread is complaining once again, instead of doing or
suggesting anything concrete?

At the end it boils down to single persons standing up for it. Like
Mike Pall doing LuaJIT; also without any "blessing", so can we just
ban that word for god? I'll bless you all!

stdlib and penlight are laudable efforts, they just don't yet cut it.
Nor have the focus to be a proper overall meta-system.

My personal best candidate for a new umbrella is Luvit. Like node.js
turned out to be the common platform for tons of javascript stuff. It
just misses documentation :-) As far I can say Tim started with
offering a good module system, similar to CommonJS.

- Axel

On Mon, Feb 13, 2012 at 3:21 PM, KHMan <[hidden email]> wrote:

> On 2/13/2012 9:07 PM, Alex Queiroz wrote:
>
>> On Mon, Feb 13, 2012 at 2:00 PM, Jon Akhtar wrote:
>>>
>>> Perhaps if we could get an endorsement on the result from upstream,
>>> and either inclusion in the release (as an external source based
>>> library), or at least a link off of the main page. Otherwise - it is
>>> sort of pointless to bless one library vs another.
>>
>>
>> I should start counting how many times this common library discussions
>> come full circle and end up nowhere. Maybe this is the 500th one. I'll
>> just put a counter on "bless" and "blessed".
>>
>> I really have no idea what's with people and blessing.
>
>
> Yeah, but I suppose we should trundle out some old arguments for the benefit
> of the new folks... :-) :-) :-)
>
> What a Lua+everything thingy needs is stable leadership, management and
> manpower. Not to mention absorbing small projects under one umbrella. Until
> that is created and we have a thingy shipping for a few quarters, there is
> no use to ask for blessing. But a long-term thingy is harder to start and
> run than it looks (for a small community)... it's not like this is the first
> time people are discussing it, as Alex says.
>
> --
> Cheers,
> Kein-Hong Man (esq.)
> Kuala Lumpur, Malaysia
>

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

KHMan
On 2/13/2012 10:31 PM, Axel Kittenberger wrote:
> So this thread is complaining once again, instead of doing or
> suggesting anything concrete?

Chill. I'm not saying that it's good or bad. Yeah, we all want a
ready-made all-in distro we can utterly rely on, but it hasn't
happened. There have been attempts to get organized, like Lua
Cheia etc, but it's hard to sustain that in a resource-limited
environment. But if it sounds like pouring cold water to you,
don't get riled up too easily okay?

Now, if there is a group of fellas with enough spare time to
sustain such an effort for a few years, then maybe. Getting
organized is important -- libraries need to survive their main
developer getting abducted by aliens and so on.

> At the end it boils down to single persons standing up for it. Like
> Mike Pall doing LuaJIT; also without any "blessing", so can we just
> ban that word for god? I'll bless you all!
> [snip snip]

> On Mon, Feb 13, 2012 at 3:21 PM, KHMan wrote:
>> On 2/13/2012 9:07 PM, Alex Queiroz wrote:
>>> On Mon, Feb 13, 2012 at 2:00 PM, Jon Akhtar wrote:
>>>>[snip snip snip]

--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Jon Akhtar
In reply to this post by Axel Kittenberger
On Mon, Feb 13, 2012 at 9:31 AM, Axel Kittenberger <[hidden email]> wrote:
> My personal best candidate for a new umbrella is Luvit. Like node.js
> turned out to be the common platform for tons of javascript stuff.

Are you suggesting we have a standard runtime environment? That seems unlikely.

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Jay Carlson
In reply to this post by KHMan
On Feb 13, 2012, at 9:21 AM, KHMan wrote:

> On 2/13/2012 9:07 PM, Alex Queiroz wrote:
>> On Mon, Feb 13, 2012 at 2:00 PM, Jon Akhtar wrote:
>>> Perhaps if we could get an endorsement on the result from upstream,
>>> and either inclusion in the release (as an external source based
>>> library), or at least a link off of the main page. Otherwise - it is
>>> sort of pointless to bless one library vs another.
>>
>> I should start counting how many times this common library discussions
>> come full circle and end up nowhere. Maybe this is the 500th one. I'll
>> just put a counter on "bless" and "blessed".
>>
>> I really have no idea what's with people and blessing.
>
> Yeah, but I suppose we should trundle out some old arguments for the benefit of the new folks... :-) :-) :-)
>
> What a Lua+everything thingy needs is stable leadership, management and manpower.

We're far better off than in the past. luarocks can handle the "everything" part and let people pick and choose; nobody has to decide if lposix is in or out. Going to the store[0], gonna buy some batteries. What's left that's cross-cutting?

I want to cut features ruthlessly until we mostly agree we all use something from time to time, and it looks and acts mostly the same--or there's an obvious winner for correctness/efficiency/simplicity. NO. NEW. FEATURES. If it's not present in luarocks at least three times in mostly compatible form it's not mature enough. If I look through the WoW base libraries I should see it over and over again. Pave the cowpaths.

I'm looking at you, object systems. Lua already has a basic object system and it's not in liblualib, it's in *core*: setting __index to a table. That solves, what, ~70% of problems?

It's nice to have a deep pretty printer (or serializer) around, but I will settle for function table.print(t) for k,v in pairs(t) do print(k,v) end. I think it would be cool to have table.copy(t) actually be the merge [[ function table.copymerge(t,override) local nt = {} for k,v in ipairs(t) do nt[k] = (override and override[k]) or t[k] end return nt end ]] but I'll drop it like a hot rock if nobody else writes that.[1]

The object system code I most want is a "proxy everything" empty table, where all __index and __newindex get handed off to another hidden table. I don't know how much more complicated I want to get, because no new features means NO. NEW. DESIGNS. too. On the positive side, memoize and its weak-table friend probably go here.[2]

It's lots of fun to build new scaffolding in Lua, and people should keep doing that. You can't stop me anyway. Penlight (for example) should continue to be the place where Steve puts all his common code (and other people grab it), and maybe eventually more of it becomes something people commonly use so often it would save time to just put it one place.

It's possible I'm overestimating the amount of commonality we have...well, at least it makes the maintenance easier. Or tells us something interesting about the language.

If luarocks really wants to help, add an implicit dependency on a base package to everything; every rock then can count on that basic library being around. Jam it into LUA_INIT. I dunno. Ideally I want something to be no-require, just like the rest of _G. That's a significant threshold for likelihood of use, and constrains what goes in (must not change existing code behavior, for starters). If there's some question about whether it should be in a full-fledged module, maybe it should go there instead.

Jay

[0]: normal guy normal walk. http://www.youtube.com/watch?v=iRZ2Sh5-XuM

[1] yes yes, hoist the override == nil check out of the loop body if we're going to do it once the right way. and because of how multiple-returns work and how table.copy is often used for safety, copy = copymerge would be bad for the typical case of copy(f(...)) anyway.

[2]: We are hopefully past the worst of this, but there used to be complex dances needed to use some features like weak tables correctly, and that'd be my biggest concession to exceptions to "must be in ubiquitous": when YOU'RE DOING IT WRONG. os.execute on an array of args maybe.
Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Axel Kittenberger
> I want to cut features ruthlessly until we mostly agree we all use something from time to time, and it looks and acts mostly the same

That would be vanilla Lua as you get it. I don't get it what you are
aiming at, or just don't see through it through the badly written
text.

Reply | Threaded
Open this post in threaded view
|

paving cowpaths with libraries (was Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available))

Jay Carlson

On Feb 13, 2012, at 1:45 PM, Axel Kittenberger wrote:

>> I want to cut features ruthlessly until we mostly agree we all use something from time to time, and it looks and acts mostly the same
>
> That would be vanilla Lua as you get it. I don't get it what you are
> aiming at, or just don't see through it through the badly written
> text.
>

I'm sorry if I was unclear. I either need more coffee or less coffee.

My thesis is that there is an ad-hoc basic library everybody already uses. It's not solely what's in _G (and when *was* the last time you called string.reverse?)

I think the reason that some previous standard library efforts lost momentum was that they did too much new stuff, or attempted to do things that there was not general agreement on. I don't want to see another object system, or even multiple object systems. I think they reduce the likelihood any library as a whole is widely adopted--even though you don't have to use them, they are irritants. This goes double for the use of an object system to implement a library.

The jargon "paving the cowpaths" describes my goal; these are things I think almost everybody does from time to time and usually in the same way. (I have a slight bias for inclusion of things not as commonly used but prone to error on casual reimplementation.)

Let me give some examples. Maybe it's the whole list, or even a superset. Feel free to say, "I never use that" or "I never use that and I can't see why anybody would". Or say "using a library function for that would be just as verbose as inlining it." Don't feel free to say,"that's trivial". The more trivial something is, the more likely we agree on how it works. I am so tired of writing/pasting these trivial functions, and looking for what they're called in other people's code.

Tables
======

Shallow-copy a table. I'd use a shallow merge-copy too.

If you are interactively working with tables, you have some mechanism to inspect their contents. I don't mean overriding tostring(), I mean "just print what's in the table to the console".  I'll offer for k,v in pairs(t) do print(k,v) end

Given an array, produce a "set". local s={}; for _,v in ipairs(a) do s[v]=true done end return s.

The opposite. Return the keys of a table as an array.

tmap(). tfilter(). The key,value version for tables. Not as widely used as:

imap(), ifilter(). The same thing, except with array-flavor, like ipairs.

The in-place variant of those. No, you can't change the key in tmap_s().[1]

I use a variant of tmap() and tfilter() which accumulates results in an array, usually for table.concat(). I don't know if anybody else does.

I don't use/write these much and hesitate to bring them up: ireduce() and izip(). If you're going to have ireduce, you probably want sum(a,b) and friends.

In verbose-lambda languages, it seems like the world is ending up with list comprehensions etc instead, since map/filter/reduce are bandages over how verbose the in-line imperative looping versions are.

Functions
=========

Given a function f, return a function memoizing f(v). A variant: keep the memoized value in a weak table.

Given a function f and a value v, return function (...) return f(v, ...) end. (Calling it currying makes it seem mysterious; it's called "bind" in luvit, for instance.)

Do it or die: return a function wrapping another function with an assert.

Files
=====

Test whether a file exists. (Mostly important for avoiding overwriting files.)

Read the entire contents of a named file into a string.

Iterate over all the lines of a file which match a pattern. Use captures as the loop variables.

Split a file path into {root, directories, filename}; split a filename.extension.

Objects
=======

In general, please no. A function wrapping the "make an {__index=parent}, set metatable" boilerplate is about as far as I'm willing to go.

I'd like to have "empty proxy with private storage" but this is not so much a basic library thing as "what do you miss from Lua 4.0".

Strings
=======

String interpolation. I despise it, but I find a long-running " ab '"..x.." cd"..y construct worse; you'll notice I forgot to close the quote begun after "ab" which would have been slightly more obvious in " ab'$x cd$y" . I don't care if it's $abc or ${abc} syntax or both. Just stop the quotation/concatenation pain.

Things I do but feel guilty about: xml_escape(), xml_attr_escape(). I don't use the inverse; if I'm consuming XML I for sure need a real parser.

I wish I had some kind of way of shortening the "read a 4k buffer, run some code on all the lines/frames/balanced-expressions, keep the unmatched tail part for the next 4k buffer." That's tedious and error-prone.

Probably a few more. I'm not touching getopt()....

Jay

[1]: BTW, anybody have a good naming convention for mutators, like Scheme's "set!" exclamation point?





Reply | Threaded
Open this post in threaded view
|

Re: paving cowpaths with libraries (was Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available))

Sean Conner
It was thus said that the Great Jay Carlson once stated:

>
> Let me give some examples. Maybe it's the whole list, or even a superset.
> Feel free to say, "I never use that" or "I never use that and I can't see
> why anybody would". Or say "using a library function for that would be
> just as verbose as inlining it." Don't feel free to say,"that's trivial".
> The more trivial something is, the more likely we agree on how it works. I
> am so tired of writing/pasting these trivial functions, and looking for
> what they're called in other people's code.
>
> Tables
> ======
>
> Shallow-copy a table. I'd use a shallow merge-copy too.

  I'm iffy on these.  I've found table copies rare, and the two times (in
the same project) I wanted a merge operation, I just wrote my own inline,
since the two tables were quite different.  In fact, here's the code:

  -- --------------------------------------
  -- merge system limits with _DEFAULT
  -- --------------------------------------
  for resource,value in pairs(service.limits) do
    proc.limits.soft[resource] = value
    proc.limits.hard[resource] = value
  end

  for resource,value in pairs(_DEFAULT.limits) do
    if not service.limits[resource] then
      proc.limits.soft[resource] = value
      proc.limits.hard[resource] = value
    end
  end
 
  -- --------------------------------------
  -- now merge the environment
  -- --------------------------------------

  local environment = {}

  for name,value in pairs(_DEFAULT.env) do
    environment[name] = value
  end

  for name,value in pairs(service.env) do
    environment[name] = value
  end

  The first bit, with the resource limits, deals with a proxy table that
wraps calls to setrlimit() within the __newindex method.  Here, we set the
values first, then any not specified specifically get a default value (using
setrlimit() has limits---you can lower, but it's hard to raise).

  The second bit is easier in that the environment table is just a
name/value pairs of strings.  Here, overwriting is fine.  

> If you are interactively working with tables, you have some mechanism to
> inspect their contents. I don't mean overriding tostring(), I mean "just
> print what's in the table to the console".  I'll offer for k,v in pairs(t)
> do print(k,v) end

  I did that at first.  I then wrote my own "pretty print a table" function
that I find easier to read, but it's a bit more complicated than just the
for loop.  

> Given an array, produce a "set". local s={}; for _,v in ipairs(a) do
> s[v]=true done end return s.

  I don't see the reason for this.  The only two values that evaluate to
false are false and nil.  

> The opposite. Return the keys of a table as an array.

  I can see the benefit of this.  I had to do this on a project to report
key/value pairs in a consistent manner (sorted by key).  

> tmap(). tfilter(). The key,value version for tables. Not as widely used as:
>
> imap(), ifilter(). The same thing, except with array-flavor, like ipairs.
>
> The in-place variant of those. No, you can't change the key in tmap_s().[1]

  I haven't really done anything that required these functions, but I can
seen a use for some people.

> I use a variant of tmap() and tfilter() which accumulates results in an
> array, usually for table.concat(). I don't know if anybody else does.

  I don't recall doing anything like this.

> I don't use/write these much and hesitate to bring them up: ireduce() and
> izip(). If you're going to have ireduce, you probably want sum(a,b) and
> friends.

  Again, I don't recall doing anything like this.

> In verbose-lambda languages, it seems like the world is ending up with
> list comprehensions etc instead, since map/filter/reduce are bandages over
> how verbose the in-line imperative looping versions are.
>
> Functions
> =========
>
> Given a function f, return a function memoizing f(v). A variant: keep the
> memoized value in a weak table.

  I've done memoizing, but on a table (using __index/__newindex) and not on
a function.

> Given a function f and a value v, return function (...) return f(v, ...)
> end. (Calling it currying makes it seem mysterious; it's called "bind" in
> luvit, for instance.)

  Eh.  While I have used the Y-combinator [1] I haven't wanted a function to
do this.  

> Do it or die: return a function wrapping another function with an assert.

  Eh.  I never did enough Perl programming to want this (or do this).

> Files
> =====
>
> Test whether a file exists. (Mostly important for avoiding overwriting
> files.)

  Useful.  Even better if it took advantage of any system specific calls (it
can be done using fopen(), but Unix has access()).

> Read the entire contents of a named file into a string.

  I'd rather have a method of indicating to Lua that what I'm giving it is a
"constant" string and thus, it doesn't need to be interred.  I mean, I has
always bugged me that

        lua_pushliteral(L,"constant")

  will cause Lua to copy the string "constant".  It doesn't need to be
copied at all.  I have a few projects that I would benefit from "constant"
strings (I already have the strings in memory, as part of a mmap()'ed file;
there's no need for Lua to copy N megabytes).

  Now, Lua does technically have a way of reading a named file into a
string:

        s = io.open("/tmp/foo","r").read("*a")

but if Lua had a way of specifying constant strings, I could then mmap() a
file into memory, pass the pointer to Lua as a constant string and avoid a
whole mess of IO (and memory allocation, and ... ).

  So, yes (it would be nice) but no (because of the current way Lua works).

> Iterate over all the lines of a file which match a pattern. Use captures
> as the loop variables.

  Hmm, so both f:lines() and s:gmatch() in one ... I could see that as being
somewhat useful in certain contexts.

> Split a file path into {root, directories, filename}; split a
> filename.extension.

  Ah yes, I thought this came up a few times, and it has:

        http://lua-users.org/lists/lua-l/2009-12/msg00641.html
        http://lua-users.org/lists/lua-l/2011-02/msg01592.html
        http://lua-users.org/lists/lua-l/2011-11/msg00125.html

  Now, given the above, what's a use case senario for this?

> Objects
> =======
>
> In general, please no. A function wrapping the "make an {__index=parent},
> set metatable" boilerplate is about as far as I'm willing to go.
>
> I'd like to have "empty proxy with private storage" but this is not so
> much a basic library thing as "what do you miss from Lua 4.0".

  I don't really have an opinion on this, so my default position is "keep
the status quo".

> Strings
> =======
>
> String interpolation. I despise it, but I find a long-running " ab
> '"..x.." cd"..y construct worse; you'll notice I forgot to close the quote
> begun after "ab" which would have been slightly more obvious in " ab'$x
> cd$y" . I don't care if it's $abc or ${abc} syntax or both. Just stop the
> quotation/concatenation pain.

  I would vote this down---how do you escape the string interpolation
characters?  Do strings in '' not undergo interpolation?  Or was it ""?  Do
strings in [[ ]] undergo interpolation?  All strings?  Sometimes?  

  Sorry, if I want Perl or bash, I know where to find it.

> Things I do but feel guilty about: xml_escape(), xml_attr_escape(). I
> don't use the inverse; if I'm consuming XML I for sure need a real parser.
>
> I wish I had some kind of way of shortening the "read a 4k buffer, run
> some code on all the lines/frames/balanced-expressions, keep the unmatched
> tail part for the next 4k buffer." That's tedious and error-prone.

  I'm not sure I understand this.  Lua (and C) provide different levels of
buffering (by character, by line, by size).

> Probably a few more. I'm not touching getopt()....

  I'm still not satisfied with any of the getopt() implementations,
including my own ...

> [1]: BTW, anybody have a good naming convention for mutators, like
> Scheme's "set!" exclamation point?

  Nope.  

  -spc (I'm recently finding myself hating abstration just for abstraction's
        sake)

[1] A way to write recursive anonymous functions

Reply | Threaded
Open this post in threaded view
|

Re: paving cowpaths with libraries (was Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available))

Miles Bader-2
Sean Conner <[hidden email]> writes:
>> Given an array, produce a "set". local s={}; for _,v in ipairs(a) do
>> s[v]=true done end return s.
>
>   I don't see the reason for this.  The only two values that evaluate to
> false are false and nil.  

It's actually pretty useful, because it allows a nice syntax for
writing fast table-based predicates.

E.g.:

   -- Return a table containing every key in KEYS as a key, with value true.
   --
   local function set (keys)
      local s = {}
      for i,v in ipairs (keys) do
         s[v] = true
      end
      return s
   end

   local keyword_set = set {'if', 'while', 'do', 'end'}

   function some_wacky_parsing_function (...)
      ...
      if keyword_set[token] then
        -- do some keywordy thing
      else
        -- do some other kinda thing
      end
      ...
   end

You could of course just write out:

   local keyword_set = {['if'] = true, ['while'] = true, ...}
   
but it can be kind of nice to use the more concise syntax.

Like most of the mentioned functions it's very simple, so I just
always include a local copy wherever I use it... but maybe some of
them are so widely used they're worth making standard, I dunno...

-miles

--
Bride, n. A woman with a fine prospect of happiness behind her.

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Ross Bencina
In reply to this post by Axel Kittenberger


On 14/02/2012 5:45 AM, Axel Kittenberger wrote:
> I don't get it what you are
> aiming at, or just don't see through it through the badly written
> text.

I'm going to attempt an answer, from my perspective:

Use case #1: application development language

Currently I use Lua embedded in my app for internal projects. I use it
to implement customised application logic for specific projects. My
impression is that this is how it's used for many projects -- as an
internally embeded language, much like any other application development
language. The only developers that see (or care about) the Lua code are
the other project developers.

In this scenario, having a common base library would serve the following
aims:

- It would make it easier for new developers to join in, because they
would be familiar with the base library.

- It would make starting a new project like this much easier because I
wouldn't have to make basic design decisions like "how should objects
work" or "where can I find a set of classes for basic stuff that I expect".

Maybe this "common base" is only 100 LoC, I don't know, but it would be
much better starting point for new developers than "every time I embed
lua I have to make all these design decisions about my language
environment, and write a bunch of container classes to give the expected
semantics".


Use case #2: user extension language

This is the more imporant use-case from my point of view. If I embed Lua
as an user extension language (which I would like to consider doing), I
would be providing my users with something called "Lua" that they can
use to extend my app. However, there is a bunch of basic stuff that they
will need (object support, basic data structures, etc) that I will need
to provide them, because Lua doesn't. In this sense, what I'm providing
is something called "Lua" but it could easily be incompatible with every
other "Lua" the user has ever seen -- depending on how I implement
certain basic runtime features.

In this scenario, a common base library would serve the end users, and
the app developer:

- As an embedder I don't have to make basic decisions like how an object
is implemented. I just do it "the common base way".

- The end users would get familiar facilities that they have used in
other "Lua" environments. Rather than confusion: "it works like this in
this app, but in this other "Lua" app it works like that".

Sure, there could be design decisions I decide to break, but right now I
am forced to research and decide on everything myself.


Don't get me wrong, I'm not criticising core-Lua at all. I'm a big fan
of the current design approach. However, without a common base library
it is confusing and unsettling to embed -- there is too much
infrastructure stuff I have to do myself -- when I really want to be
getting on with writing application code.


I agree with Jay that forming a minimal common base out of existing
best-practice would be the ideal. I'm no Lua expert, so I'm not the one
to write it, but I would certainly deploy it.

Note here I'm not talking about runtime facilities (like luvit) or even
about domain-specific infrastructure libraries (like XML libraries,
JSON, logging) .. just basic stuff needed to organise and express medium
size software systems: basic data abstractions, an object system,
whatever other tools are commonly needed.

For use case #1 above (developing) it probably doesn't matter, people
can do what they like, but for use case #2 (user extension) I really
don't want to wear the "object system and language library designer" hat
and be held accountable by my users of how such basic facilities work --
I'm more than happy to do it the "standard base way".

Does that make sense?

Ross.














Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

steve donovan
In reply to this post by Jay Carlson
On Mon, Feb 13, 2012 at 8:22 PM, Jay Carlson <[hidden email]> wrote:
> We're far better off than in the past. luarocks can handle the "everything" part and let people pick and choose;

I mostly agree, although there's a fair amount of work to make LR
really sing on Windows.  And it annoys Linux people because it does
not recognize the native package manager - there's a _lot_ of good
basic Lua functionality in the Debian repositories. If a package needs
lfs then LR will pull in lfs and not see liblua5.1-filesystem0 as
satisfying that need. (This is _not_ an easy problem to solve, because
of the versioning issues, especially hard to frame as a cross-platform
solution.)

But LR isn't an obvious way to deliver stuff to an embedder.

> I want to cut features ruthlessly until we mostly agree we all use something from time to time, and it looks and acts mostly the same--or there's an obvious winner for correctness/efficiency/simplicity.

That's an admirable approach, much admired by Roberto and lhf.  In
practice, it's very hard to get us to agree on _anything_. I remember
a discussion about the signature for a map() function; no agreement
was possible on _one_ such function because there's actually a family
of such functions,  table -> table, iterator -> iterator, etc.

> If luarocks really wants to help, add an implicit dependency on a base package to everything;

Ah, but it's a no-brainer for a rock author to add a base dependency;
the question is what will that be?  Call it 'baselib' and get ruthless
with it, a hundred well-curated utility functions, maybe 2KLoc.  It's
understood that it will be a disappointment to many, that's cool.  (I
could make something like that by tomorrow by taking a sharp knife to
Penlight, but that's the easy part.)

A promising approach is to define luarocks meta-packages  - e.g. we
can call one 'lua-batteries' and it will be a package that simply
pulls in a set of commonly-useful stuff packages.  That's a reasonable
way to do a cross-platform Lua distribution.

The intention remains to use LuaDist as the sane cross-platform
package builder and LuaRocks as the distribution channel.  However, we
keep getting distracted ;)

While it's ramble-week on lua-l, I'd like to return to the 'static
linking' meme.  You can give a short but powerful Python script to
your chums because they have the platform. Making that into an
executable for the world is a painful exercise.  Now with something
like Squish, you can write a short but powerful Lua script, 'compile'
it into a self-contained script and glue that script to a Lua
executable using srlua.  The result is going to be in the region of
200K, (depending on what Lua binary extensions need to be statically
linked in) and that's practically small change by modern standards.
Combine this strategy with LR and you _do_ have a way for an embedder
to use existing batteries without (for instance) pulling in the whole
10Kloc of penlight.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Sean Conner
In reply to this post by Ross Bencina
It was thus said that the Great Ross Bencina once stated:
>
> I'm going to attempt an answer, from my perspective:
>
> Use case #1: application development language

  [ snip ]

> Use case #2: user extension language

  [ snip ]

> For use case #1 above (developing) it probably doesn't matter, people
> can do what they like, but for use case #2 (user extension) I really
> don't want to wear the "object system and language library designer" hat
> and be held accountable by my users of how such basic facilities work --
> I'm more than happy to do it the "standard base way".
>
> Does that make sense?

  Not really.  I don't see any real difference between #1 and #2, so I'm not
even sure what you are asking.  Also, I'm not a big proponent of object
oriented programming, and I find what Lua provides just fine for my own
needs (I stay as far away from C++ as possible), so I don't quite understand
this whole obsession over objects.  From what I can see, Lua support of
"objects" is fine.

  At work, I use Lua to test our software.  And for one platform (Solaris)
I've had to create what I call "The Kitchen Sink Lua"---it's basically a
custom Lua standalone interpreter that has all the modules we need to run
(both in C and Lua) built into a single executable.  And yes, stuff that
would normally go into /usr/local/lib/lua/5.1 or /usr/local/share/lua/5.1
have been compiled statically into this "Kitchen Sink Lua" (primarily
because it's difficult to get the required shared objects/Lua files
installed on the deployment systems).

  So for my use case, it's important to be able to statically compile
modules into the program.  It's less important to have a "base library" or
"base object system" because what Lua provides is enough (or rather, I have
to supply quite a bit, but it's all custom coding anyway).  

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

steve donovan
On Tue, Feb 14, 2012 at 10:39 AM, Sean Conner <[hidden email]> wrote:
> ...  Also, I'm not a big proponent of object
> oriented programming, and I find what Lua provides just fine for my own
> needs (I stay as far away from C++ as possible), so I don't quite understand
> this whole obsession over objects.

Well, twenty years of OOP has ingrained a way of thinking about
problems and enshrined it as the Way.  It's still a fine way to
organize solutions to some common problems.  It would obviously be bad
if the language enforced it, but (fortunately) that's not going to
happen.  [0]

Newcomers would be more comfortable if there was a straightforward
pre-packaged way of making classes; the trouble comes when they see
something like LOOP and run screaming to the comforts of their
favourite OOP scripting language.  Experienced developers have another
problem, which is that they cannot share more than the assurance that
duck:quack() works.[1]  (We can't even assume that getmetatable(duck)
represents a Duck 'class' and that's a feature, not a bug [2])

So (like Jay) says: keep it really simple.  Encourage newcomers to use
a little library if they feel the urge to make class hierarchies, even
if it just contains the mt.__index = mt incantation.

steve d.

[0] This powerful urge results in derivative languages which provide
the much-needed class sugar: CoffieScript for JavaScript, and now
MoonScript for Lua.
[1] Some people enjoy saying duck.quack(), and Lua can satisfy them,
at the cost of confusing everyone else and creating an unbounded
number of closures.
[2] Similar cognitive dissonance happens with people new to Go, which
basically does compile-time duck typing.

Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Ross Bencina
In reply to this post by Sean Conner
On 14/02/2012 7:39 PM, Sean Conner wrote:
> It was thus said that the Great Ross Bencina once stated:

Should I laugh or cry?

>> >
>> >  I'm going to attempt an answer, from my perspective:
>> >
>> >  Use case #1: application development language
>    [ snip ]
>
>> >  Use case #2: user extension language
>    [ snip ]
>
>> >  For use case #1 above (developing) it probably doesn't matter, people
>> >  can do what they like, but for use case #2 (user extension) I really
>> >  don't want to wear the "object system and language library designer" hat
>> >  and be held accountable by my users of how such basic facilities work --
>> >  I'm more than happy to do it the "standard base way".
>> >
>> >  Does that make sense?
>    Not really.  I don't see any real difference between #1 and #2, so I'm not
> even sure what you are asking.

I would put your use in case #1: you control the development and runtime
environment, the whole code base, *and* the developers, *and* the users.
You have captive users too -- you can treat them any way you like.

Case #2 is like, well I can't think of a Lua example, but let's say,
it's like "Visual Basic for Applications". You give the user an App, and
it has a scripting environment where they can write extensions. The user
will write code. In this case you're providing a programming environment
to the user -- either it's plain Lua or it comes with some "batteries."
Even though I don't expect "batteries" with Lua, end-users sure will --
and it really shouldn't be a nightmare on the scale of this thread for
an embedder to provide "the standard batteries".

All I'm saying is that for cases where batteries are included, they
should be the same batteries (or at least compatible batteries), and it
should be easy to find and package them. Otherwise App A embedding "Lua"
could use it in a completely different and incompatible way to App B.
That's fine if there's a good reason for it, but it's a big headache if
it's just becausefunction names or core idioms aren't standardised. Or
the embedder (like me) wasn't a Lua guru, and didn't really know what
the right batteries looked like.


 > Also, I'm not a big proponent of object
 > oriented programming, and I find what Lua provides just fine
 > for my own needs (I stay as far away from C++ as possible),
 > so I don't quite understand this whole obsession over objects.
 > From what I can see, Lua support of "objects" is fine.

Fair enough. The same would apply if we were discussing functional
programming. There are a standard set of functions in that style too
(e.g. map() )


 > So for my use case, it's important to be able to statically
 > compile modules into the program.  It's less important to
 > have a "base library" or "base object system" because what
 > Lua provides is enough (or rather, I have to supply quite a bit,
 > but it's all custom coding anyway).

Do you think every line of what you supply needs to be custom? Are you
really using absolutely no standard utility functions? no reusable
abstractions? generic functions? If not, then colour me surprised. I had
to write a bunch of abstractions just to get started. Sure, they were
light weight and simple, but they were still necessary to make the code
readable.

My assertion is that if there is no "base library" then every Lua
embedder has to concoct their own base library. Which is fine for custom
jobs, but not so easy to manage when the whole thing is then published
as an open programming system for other end-user-developers to use.
Every such publisher becomes their own Cardinal (to use the previous
blessing analogy) -- I have enough jobs already, I don't want to be a
Lua language Cardinal, I just want to publish something with a "standard
base" that's usable.. whether the standard base is object-oriented or
functional is less important that it is usable by end users without me
or end users writing a bunch of expected "standard" infrastructure.

Ross.







Reply | Threaded
Open this post in threaded view
|

Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available)

Axel Kittenberger
In reply to this post by steve donovan
> It would obviously be bad if the language enforced it

I find when reading texts, that adverbs like "obviously", "naturally"
etc. stand for "I cannot or do not want to reason this".

I think its fine when a language does it, and I think its fine when a
language doesn't it. Languages are different. Enjoy!

12