Lua library bank? (Was: Ruby philosophy vs Lua philosophy

classic Classic list List threaded Threaded
63 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Dirk Laurie-2
Somewhere the thread changed course from not just Ruby vs Lua
to that reliable old performer, Lua's lack of "batteries".

I don't think anybody seriously means "like Python."  Python is
way past "batteries included". It's reached the stage of "any old
junk repackaged". I've just asked Ubuntu's package manager how many
packages have "python" and "binding" in their names.

Python, 342.

Ruby, 101.

Perl, 81.

Lua, 13. None of which I have actually installed.

I can think of several reasons.

1. Lua is not as pervasive as the others.
2. It's quite easy in Lua (and trivial in LuaJIT) to write bindings for
    just the few routines from a large C library that you actually need.
3. Lua fans don't like taking code someone else wrote on trust. Core
    Lua is OK, it goes through several rc versions, we all had a chance
    to put it through its paces. But not just anything.

Do we really need another library bank? We have luarocks.
It's a pons asinorum. Packaging something as a rock is just tricky
enough to eliminate the nine-day wonders and let through only
those authors with a little staying power.

Personally, I find that of the dozens of Lua 5.1 non-rocks packages
that I installed and tried, only four (lfs, lpeg and two others) made it
to my 5.2 library. And I'm pretty sure lfs and lpeg are 5.2 rocks too
by now.

What we do need is something like Amazon's user review
system. I need say a json reader, the LuaWiki lists several and
it's really hard to figure out which to try first. It's not going be the
same for everyone. But with a user review system I can see
which product gets a good rating from people with my sort of
setup.

Isn't it just a question of installing the right open-source
package on say lua-users.org and waiting for us Lua users
to provide the input?

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Kevin Martin
I use Lua purely as an embedded language in our optimisation engine, I've built quite a few custom libraries where perhaps one out there already would have been better.

One of the reasons for this has been the perceived work (admittedly it might be easier than it looks) integrating the libraries into our build system. I would like to see something similar to the SQLite amalgamation for Lua libraries, so that I could just download one file (c or lua) and just add it to package.preload.

This would make me much more likely to use and get involved with the libraries than more/different package management.

Thanks,
Kev

Sent from my iPhone

On 28 Feb 2013, at 10:39, Dirk Laurie <[hidden email]> wrote:

> Somewhere the thread changed course from not just Ruby vs Lua
> to that reliable old performer, Lua's lack of "batteries".
>
> I don't think anybody seriously means "like Python."  Python is
> way past "batteries included". It's reached the stage of "any old
> junk repackaged". I've just asked Ubuntu's package manager how many
> packages have "python" and "binding" in their names.
>
> Python, 342.
>
> Ruby, 101.
>
> Perl, 81.
>
> Lua, 13. None of which I have actually installed.
>
> I can think of several reasons.
>
> 1. Lua is not as pervasive as the others.
> 2. It's quite easy in Lua (and trivial in LuaJIT) to write bindings for
>    just the few routines from a large C library that you actually need.
> 3. Lua fans don't like taking code someone else wrote on trust. Core
>    Lua is OK, it goes through several rc versions, we all had a chance
>    to put it through its paces. But not just anything.
>
> Do we really need another library bank? We have luarocks.
> It's a pons asinorum. Packaging something as a rock is just tricky
> enough to eliminate the nine-day wonders and let through only
> those authors with a little staying power.
>
> Personally, I find that of the dozens of Lua 5.1 non-rocks packages
> that I installed and tried, only four (lfs, lpeg and two others) made it
> to my 5.2 library. And I'm pretty sure lfs and lpeg are 5.2 rocks too
> by now.
>
> What we do need is something like Amazon's user review
> system. I need say a json reader, the LuaWiki lists several and
> it's really hard to figure out which to try first. It's not going be the
> same for everyone. But with a user review system I can see
> which product gets a good rating from people with my sort of
> setup.
>
> Isn't it just a question of installing the right open-source
> package on say lua-users.org and waiting for us Lua users
> to provide the input?
>

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

steve donovan
In reply to this post by Dirk Laurie-2
On Thu, Feb 28, 2013 at 12:39 PM, Dirk Laurie <[hidden email]> wrote:
> Do we really need another library bank? We have luarocks.
> It's a pons asinorum. Packaging something as a rock is just tricky
> enough to eliminate the nine-day wonders and let through only
> those authors with a little staying power.

Well, I can honestly say that I am an ass when it comes to rockspecs,
so I write little utilities to write rockspecs, and forget how to use
them.

> What we do need is something like Amazon's user review
> system. I need say a json reader, the LuaWiki lists several
> ... But with a user review system I can see
> which product gets a good rating from people with my sort of
> setup.

Ah, you mean _anonymous voting_ ;)  We do have a gentleman's agreement
not to criticise each others modules...

Your last sentence used the magic word 'just' ;)  We had big ideas
about the LuaForge replacement; the Lua Snippets site was a
contribution to that debate, and Sputnik can certainly do that kind of
nifty trick.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

steve donovan
In reply to this post by Kevin Martin
On Thu, Feb 28, 2013 at 2:13 PM, Kevin Martin <[hidden email]> wrote:
> One of the reasons for this has been the perceived work (admittedly it might be easier than it looks) integrating the libraries into our build system. I would like to see something similar to the SQLite amalgamation for Lua libraries, so that I could just download one file (c or lua) and just add it to package.preload.

Well, we have Mathew Wild's Squish and Jay Carlson's soar for making
Lua amalgamations. In LuaBuild I combined soar with lhf's srlua with a
mini 5.2 distribution to create a build system which could package a
Lua program with both its Lua and C dependencies into one executable.
What LB does _not_ do is provide a way of generating similar shared
library amalgamations. Maybe that's the logical next step.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Luiz Henrique de Figueiredo
In reply to this post by Kevin Martin
> I would like to see something similar to the SQLite amalgamation for Lua libraries, so that I could just download one file (c or lua) and just add it to package.preload.

See these threads:
        http://lua-users.org/lists/lua-l/2012-01/msg00601.html
        http://lua-users.org/lists/lua-l/2011-12/msg00249.html

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Luiz Henrique de Figueiredo
> > I would like to see something similar to the SQLite amalgamation for Lua libraries

Oops, I missed "libraries" here. The links I've posted are about an
amalgamation for the Lua engine. Sorry for the noise.

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Javier Guerra Giraldez
In reply to this post by Dirk Laurie-2
On Thu, Feb 28, 2013 at 5:39 AM, Dirk Laurie <[hidden email]> wrote:
> What we do need is something like Amazon's user review
> system. I need say a json reader, the LuaWiki lists several and
> it's really hard to figure out which to try first. It's not going be the
> same for everyone. But with a user review system I can see
> which product gets a good rating from people with my sort of
> setup.


that reminds me of djangopackages.org, a great way to choose add-on
modules for Django projects (a Pythono web app framework).  the
authors created opencomparison.org, which offers "free subdomains to
open-source languages, web frameworks, and other projects."   it's
still somewhat Python centric, and seems to favor pypi-installable
packages, so it might not work out of the box for Lua, but there are
some good ideas to mull over:

besides the clasification, what i use most is the repository activity
graph, which pulls data from github and bitbucket.  it makes it easy
to weed out abandonware.  now, it has already been noted that Lua
mature modules have very little changes without detriment to quality.

another useful stat is the number of projects using it..... not sure
how it would translate to Lua

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Andrew Starks
On Thu, Feb 28, 2013 at 7:46 AM, Javier Guerra Giraldez
<[hidden email]> wrote:

> On Thu, Feb 28, 2013 at 5:39 AM, Dirk Laurie <[hidden email]> wrote:
>> What we do need is something like Amazon's user review
>> system. I need say a json reader, the LuaWiki lists several and
>> it's really hard to figure out which to try first. It's not going be the
>> same for everyone. But with a user review system I can see
>> which product gets a good rating from people with my sort of
>> setup.
>
>
> that reminds me of djangopackages.org, a great way to choose add-on
> modules for Django projects (a Pythono web app framework).  the
> authors created opencomparison.org, which offers "free subdomains to
> open-source languages, web frameworks, and other projects."   it's
> still somewhat Python centric, and seems to favor pypi-installable
> packages, so it might not work out of the box for Lua, but there are
> some good ideas to mull over:
>
> besides the clasification, what i use most is the repository activity
> graph, which pulls data from github and bitbucket.  it makes it easy
> to weed out abandonware.  now, it has already been noted that Lua
> mature modules have very little changes without detriment to quality.
>
> another useful stat is the number of projects using it..... not sure
> how it would translate to Lua
>
> --
> Javier
>

Some kind of community editorial system is really important. It's good
for both the package writer and the community. Feedback, is always
good, especially in a community that supports good manners.

I'd also advocate for some standards, where they would provide
exponential benefit. As a starting point, I might suggest:

A brief description
Works with Lua 5.2
Works on Windows, Linux and Mac (what Windows and Linux means would
need to be defined by smarter people than me)
LuaDoc (LDoc) documentation for all consumed interfaces.
One quick tutorial that brings the package user from installation to first use.
[test coverage] (this one might be more difficult to automate)

Maybe the most critical aspect is the "just" part that Steve
mentioned. How can this be done, as close to "right now" as possible,
without swallowing too much?

You know what I'm a huge fan of? Drupal. Getting a Drupal site going
would be relatively simple and one can always migrate later. I might
be ignorant of a Lua alternative that would include modules for
voting, etc.

-Andrew

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Andrew Starks
Sorry for the noise, but another project that is interesting to look
at is Homebrew [1]

It doesn't cover all of the ground here (voting, etc), but it works
really well, it's simple and it solves a lot of hard problems. I don't
want to imply anything against LuaRocks, either.

Also, resurrecting an existing Lua web site (like LuaForge) sounds
awesome to me, if that's possible. If not, Can I suggest:

[2] Room 100


-Andrew

[1] http://mxcl.github.com/homebrew/
[2] http://kottke.org/05/02/loo-etymology

On Thu, Feb 28, 2013 at 9:29 AM, Andrew Starks <[hidden email]> wrote:

> On Thu, Feb 28, 2013 at 7:46 AM, Javier Guerra Giraldez
> <[hidden email]> wrote:
>> On Thu, Feb 28, 2013 at 5:39 AM, Dirk Laurie <[hidden email]> wrote:
>>> What we do need is something like Amazon's user review
>>> system. I need say a json reader, the LuaWiki lists several and
>>> it's really hard to figure out which to try first. It's not going be the
>>> same for everyone. But with a user review system I can see
>>> which product gets a good rating from people with my sort of
>>> setup.
>>
>>
>> that reminds me of djangopackages.org, a great way to choose add-on
>> modules for Django projects (a Pythono web app framework).  the
>> authors created opencomparison.org, which offers "free subdomains to
>> open-source languages, web frameworks, and other projects."   it's
>> still somewhat Python centric, and seems to favor pypi-installable
>> packages, so it might not work out of the box for Lua, but there are
>> some good ideas to mull over:
>>
>> besides the clasification, what i use most is the repository activity
>> graph, which pulls data from github and bitbucket.  it makes it easy
>> to weed out abandonware.  now, it has already been noted that Lua
>> mature modules have very little changes without detriment to quality.
>>
>> another useful stat is the number of projects using it..... not sure
>> how it would translate to Lua
>>
>> --
>> Javier
>>
>
> Some kind of community editorial system is really important. It's good
> for both the package writer and the community. Feedback, is always
> good, especially in a community that supports good manners.
>
> I'd also advocate for some standards, where they would provide
> exponential benefit. As a starting point, I might suggest:
>
> A brief description
> Works with Lua 5.2
> Works on Windows, Linux and Mac (what Windows and Linux means would
> need to be defined by smarter people than me)
> LuaDoc (LDoc) documentation for all consumed interfaces.
> One quick tutorial that brings the package user from installation to first use.
> [test coverage] (this one might be more difficult to automate)
>
> Maybe the most critical aspect is the "just" part that Steve
> mentioned. How can this be done, as close to "right now" as possible,
> without swallowing too much?
>
> You know what I'm a huge fan of? Drupal. Getting a Drupal site going
> would be relatively simple and one can always migrate later. I might
> be ignorant of a Lua alternative that would include modules for
> voting, etc.
>
> -Andrew

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Geoff Leyland
In reply to this post by Kevin Martin
On 1/03/2013, at 1:13 AM, Kevin Martin <[hidden email]> wrote:

> I use Lua purely as an embedded language in our optimisation engine, I've built quite a few custom libraries where perhaps one out there already would have been better.

What's your optimisation engine?
Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

William Ahern
In reply to this post by Dirk Laurie-2
On Thu, Feb 28, 2013 at 12:39:05PM +0200, Dirk Laurie wrote:

> Somewhere the thread changed course from not just Ruby vs Lua
> to that reliable old performer, Lua's lack of "batteries".
>
> I don't think anybody seriously means "like Python."  Python is
> way past "batteries included". It's reached the stage of "any old
> junk repackaged". I've just asked Ubuntu's package manager how many
> packages have "python" and "binding" in their names.
>
> Python, 342.
>
> Ruby, 101.
>
> Perl, 81.
>
> Lua, 13. None of which I have actually installed.

FWIW, I've started a project "Crow"*, to support a run-time module fetching
system. Basically, you seed your local configuration file with URLs to
package databases, elliptic curve public keys (which can be very short), and
other parameters, and modules which aren't found locally are downloaded
automatically and placed into a local cache at require time.

It'll also support automatic run-time updating, so long-running server
instances can be automatically patched for bug or security fixes--at least
for pure-Lua modules.

This is one reason why I've added so many bindings to OpenSSL in my cqueues
library. cqueues will handle all the asynchronous I/O happening behind a
simple "require" call, and the OpenSSL bindings support signature
verification, including validating chains of trust.

This way, one doesn't have to worry about installing a package locally,
including making decisions like "is it really worth it?" You can just use
the package in code immediately; if you don't like a module and stop using
it, it just quietly disappears.

Because of the way cqueues handles events, the Crow system can be readily
embedded into existing applications, too, without much (or any) adaptation
to the aynchronous network I/O.

It's also not centralized. This is why public-key crypto is built-in at the
core. And I'm striving to make the tools to generate keys, and package and
sign code as trivial to use as possible. I imagine people (like myself) who
prefer to host their projects themselves can just publish a URL and a short
70-character public key for people to copy into their local config file
(with as many constraining parameters as the user wishes, of course). Or
they can federate with centralized repositories using trust chaining
techniques--e.g. luarocks.org signing the repository information for Bob
Smith, restricted to loading bob.smith.* modules.


* I recently registered murder.io. crow.io was already taken. I hope people
are familiar with terms of venery and don't think I'm just a really dark
person ;)


Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Petite Abeille
In reply to this post by Dirk Laurie-2

On Feb 28, 2013, at 11:39 AM, Dirk Laurie <[hidden email]> wrote:

> I don't think anybody seriously means "like Python."  Python is
> way past "batteries included". It's reached the stage of "any old
> junk repackaged".

Ah! Indeed.

Case in point… say one implements an IMAP server [sic]… nothing fancy, just the core IMAP4rev1 specification… which is about 10 year old this year… anyhow… for testing purpose,  one rounds up the usual suspects… the UW c‑client library as a baseline … and the latest in Java, Perl, Python, and Ruby… they all sport some sort of IMAP client module "out-of-the-box"... battery included and all… great… piece of cake.

But, nay, all these packages fail miserably at the most basic interoperability level… none of them implement the spec properly… none… epic fail.

Only the UW c‑client library complies to the spec with flying color… honorable mention for the Perl module as well… which was not as dysfunctional as the other assorted clowns…

In fairness, the IMAP spec is a bit of a pathological case… but still… if all those "batteries included" are radioactive junk blowing up in your face at the slightest ignition… just don't bother.




Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

William Ahern
On Thu, Feb 28, 2013 at 09:04:05PM +0100, Petite Abeille wrote:

>
> On Feb 28, 2013, at 11:39 AM, Dirk Laurie <[hidden email]> wrote:
>
> > I don't think anybody seriously means "like Python."  Python is
> > way past "batteries included". It's reached the stage of "any old
> > junk repackaged".
>
> Ah! Indeed.
>
> Case in point… say one implements an IMAP server [sic]… nothing fancy,
> just the core IMAP4rev1 specification… which is about 10 year old this
> year… anyhow… for testing purpose, one rounds up the usual suspects… the
> UW c‑client library as a baseline … and the latest in Java, Perl, Python,
> and Ruby… they all sport some sort of IMAP client module
> "out-of-the-box"... battery included and all… great… piece of cake.
>
> But, nay, all these packages fail miserably at the most basic
> interoperability level… none of them implement the spec properly… none…
> epic fail.
>
> Only the UW c‑client library complies to the spec with flying color…
> honorable mention for the Perl module as well… which was not as
> dysfunctional as the other assorted clowns…

This is where CPAN's size and age helps, I think. Most code is crap, I think
we can all agree, for a multitude of reasons. But there are just more gems
in CPAN than in, say, RubyGems.


Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Tom N Harris
In reply to this post by Dirk Laurie-2
On Thursday, February 28, 2013 05:39:05 AM Dirk Laurie wrote:
> I've just asked Ubuntu's package manager how many
> packages have "python" and "binding" in their names.
>
> Python, 342.
> Ruby, 101.
> Perl, 81.
> Lua, 13. None of which I have actually installed.
>

Not wanting to take those numbers at face value, I queried Debian sid.

lib*-perl: 2961
python-*: 1751
ruby-*: 438
python3-*: 215
lua-*: 90
node-*: 85

python bind: 573
ruby bind: 174
perl bind: 139
lua bind: 58
node bind: 25

Make of it what you will.

--
tom <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Dirk Laurie-2
In reply to this post by William Ahern
2013/2/28 William Ahern <[hidden email]>:

> This is where CPAN's size and age helps, I think. Most code is
> crap,

Aha! A new acronym!

Comprehensive Ruby And Python.

… those CRAP libraries

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

oliver-2
In reply to this post by Dirk Laurie-2
How about OSQA (www.osqa.net). There is a bitnami stack for it: http://bitnami.org/stack/osqa. The tagging would make it easy to filter for specific lua package: tag "lfs" would find all posts related to lfs pros/cons; or for "task" questions (what is best package for xyz): tag "filesystem" would find all posts that answer questions about lfs and other filesystem-related items.


On Thu, Feb 28, 2013 at 5:39 AM, Dirk Laurie <[hidden email]> wrote:
Somewhere the thread changed course from not just Ruby vs Lua
to that reliable old performer, Lua's lack of "batteries".

I don't think anybody seriously means "like Python."  Python is
way past "batteries included". It's reached the stage of "any old
junk repackaged". I've just asked Ubuntu's package manager how many
packages have "python" and "binding" in their names.

Python, 342.

Ruby, 101.

Perl, 81.

Lua, 13. None of which I have actually installed.

I can think of several reasons.

1. Lua is not as pervasive as the others.
2. It's quite easy in Lua (and trivial in LuaJIT) to write bindings for
    just the few routines from a large C library that you actually need.
3. Lua fans don't like taking code someone else wrote on trust. Core
    Lua is OK, it goes through several rc versions, we all had a chance
    to put it through its paces. But not just anything.

Do we really need another library bank? We have luarocks.
It's a pons asinorum. Packaging something as a rock is just tricky
enough to eliminate the nine-day wonders and let through only
those authors with a little staying power.

Personally, I find that of the dozens of Lua 5.1 non-rocks packages
that I installed and tried, only four (lfs, lpeg and two others) made it
to my 5.2 library. And I'm pretty sure lfs and lpeg are 5.2 rocks too
by now.

What we do need is something like Amazon's user review
system. I need say a json reader, the LuaWiki lists several and
it's really hard to figure out which to try first. It's not going be the
same for everyone. But with a user review system I can see
which product gets a good rating from people with my sort of
setup.

Isn't it just a question of installing the right open-source
package on say lua-users.org and waiting for us Lua users
to provide the input?


Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Pierre-Yves Gérardy
In reply to this post by Dirk Laurie-2
On Thu, Feb 28, 2013 at 11:39 AM, Dirk Laurie <[hidden email]> wrote:
>
> Somewhere the thread changed course from not just Ruby vs Lua
> to that reliable old performer, Lua's lack of "batteries".
> [...]
> Do we really need another library bank? We have luarocks.
> It's a pons asinorum. Packaging something as a rock is just tricky
> enough to eliminate the nine-day wonders and let through only
> those authors with a little staying power.


Component.js [0] and the Julia Pkg system [1] both use git as backend
(and GitHub as default host).
Julia packages must be registered in a central METADATA.jl repository,
but Componenent is completely decentralized.

    $ component install owner/repo@tag_or_branch

... installs a given component from GitHub. You can also provide a
full URL for packages hosted elsewhere. Component has a central search
index, but it is optional.

If a new package system were to be designed, it would be great to use
them as inspiration. In both cases, creating/uploading a package is trivial
(if you're a bit familiar with git).

> What we do need is something like Amazon's user review
> system. I need say a json reader, the LuaWiki lists several and
> it's really hard to figure out which to try first. It's not going be the
> same for everyone. But with a user review system I can see
> which product gets a good rating from people with my sort of
> setup.

Hosting all packages on GitHub does not give you a star rating system,
but you can see how popular a given package is, how reactive the
author is to bug reports, etc..

-- Pierre-Yves

[0] https://github.com/component/component
[1] https://github.com/JuliaLang/METADATA.jl

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Andrew Starks
On Fri, Mar 8, 2013 at 3:36 PM, Pierre-Yves Gérardy <[hidden email]> wrote:

>
> Hosting all packages on GitHub does not give you a star rating system,
> but you can see how popular a given package is, how reactive the
> author is to bug reports, etc..
>
> -- Pierre-Yves

I like the idea of using GitHub as the base.

Also, there isn't much wrong with luarocks that couldn't be fixed or
extended away. For me, I have trouble with it in Windows and I can't
seem to make it believe me when I ask it to use other directories. The
second problem is probably me, and the first might be better fixed
with a another package.searcher.

Wants/desires/I-haven't-figured-it-out-yet are:

- Version/Updating modules.
- Keeping module repositories and the installed builds of them
straight, so my patches are more easily managed.
- Modules that need non-lua libraries that I don't have installed and
don't really want to install in my /usr/local/ tree.
- Modules that need hard-to-build libraries (like ZMQ on Windows).
- Ability to change environments for using, developing, testing, and deployment.

Some ideas...

I think Lua's method for storing .o/.so/DLLs and lua scripts makes
sense for the core language. It feels like an approach for package
management that is aiming for simplicity might be better off
considering ways to make cross-platform deployment "not suck." Perhaps
the addition of a package searcher and an aping of TeX/Context/LaTeX
TEXMF trees? Instead of separating  cpath and path directories into
different roots (or at least roots at different levels), use a single
tree, such as this one that i just thought up right now:
moddir/repos      --(optional, probably only in the system tree) a git
repo that only has module URLs...
                            --I guess the way that luarocks does it
now works fine. No need to reinvent that...
moddir/git            --the raw repos for all installed modules.
"moddir/git/mymodule", etc
moddir/lua           --instead of /usr/local/share/lua/5.2
moddir/lib            --instead of /usr/local/lib
                            --perhaps the makefile could automate
putting not-found, dependent, non-lua libraries here too, such as
"curl" or "expat"?
moddir/resources  --for other non-exec binaries that a module or
distribution might need. Would help with more application-y
distribution, like Love
moddir/doc      --perhaps typing luadoc mymodulename looks in all of
the moddir trees for that module's documentation and loads a viewer?
                        -- Again, stolen from TeX: texdoc
moddir/include --perhaps for header files that are used for FFI or C
libraries that are being built with lua?
moddir/bin        --perhaps for lua-centric commands, like build
utilities? Would help with Love?

Standard array of 'moddir's with fall-through rules apply. The hope is
that it works well on Windows and *nix. On *nix machines, the
system-level moddir goes in '/usr/local/share/lua/5.2/luamod', Windows
in '/Windows/System32/lua/5.2/luamod'. Then, also, the user would have
one installed at "~/lua/5.2/luamod" and "$HOMEPROFILE/Documents" or
whatever it is in Windows. :)

I'm positive that I'm violating 75% of everyone's sensibilities.
Precedent has been set for this approach, by TeX, which is a battle
hardened world. Unlike with Lua, I can make a wicked complicated,
extremely dependency-laden LaTeX project compile on Windows, Mac and
Linux, all from very different distributions (MacTeX, MikTeX and
TexLive). They even work well between WinEdt and Sublime and our CI
server.

To help solve versioning issues, swapping environments and keeping
repos and module directories straight:

- keep the whole repo in tact in 'moddir/git'
- use git fetch as the caching mechanism for detecting updates.
- build in the repo's build directory (moddir/git/mymodule/build).
    -- The module author can choose to .gitignore in there or not.
    -- If not, they can have subdirectories for platform binaries in
"bin" and in "libs". This would be good for Windows.
    -- If repo bloat is a problem, use branches to solve it. Have a
"with-binaries" branch that includes the binary files.
    -- The repo's build directory has the same tree structure as a
regular moddir.
- Maybe most important: symlink all of the files in the build
directory (moddir/git/mymodule/build/lua, etc) to that moddir tree's
root versions (moddir/lua, moddir/lib, etc)

The hope is that...

- when that module is updated and rebuilt, the symlinks are updated.
- if the user wants to patch the module, they can branch it and the
symlinks still work.
  -- if we enforce the policy that they must branch, luarocks can
still maintain updates from the author, without breaking local
changes.
  -- If luarocks detects local commits on master, then it can complain
and offer a "reset --hard" alias.
- luarocks can use branching to provide facilities for named "module
environments" that can be envoked for testing, development, general
use, etc.

Environment switching could be done the normal way (like a script that
holds the environment names and sates), or it could be done with git
subtree merging. With subtree merging, the whole 'moddir/git' is a git
repo and "git subtree add ..." is used to add each module. Each module
becomes a branch, with only it inside of it. They would be named
something like "mymodule_module". New environment branches can be made
from master. In those branches, modules could be updated/deleted/etc.
These branches might be named "debug_myproject_env", etc.

Shipped binaries might be organized like so (stealing from TeX):

moddir/git/build/lib/common --default.
moddir/git/build/OSX/10.6 --perhaps if you're 10.7, it would use this?
moddir/git/build/OSX/10.5
moddir/git/build/OSX --default for mac
moddir/git/build/Windows --etc...

only relevant binaries are symlinked into 'moddir/lib'

I've gone on long enough. Sorry. You are all saints.

Andrew Starks
Tightrope Media Systems

"Your stupid." -favorite bumper sticker

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

steve donovan
In reply to this post by Pierre-Yves Gérardy
On Fri, Mar 8, 2013 at 11:36 PM, Pierre-Yves Gérardy <[hidden email]> wrote:
> Hosting all packages on GitHub does not give you a star rating system,
> but you can see how popular a given package is, how reactive the
> author is to bug reports, etc..

But git (and specifically github) isn't everyone's way to doing
things. It forces people to use a particular way of doing things.
Granted, Michal Kottman's lua-git means that the _user_ doesn't need
Git, but still.

Making something like this work automagically depends on everyone
having a standard repo layout as well.  And how does one specify
dependencies?

I can't see the advantage over LuaRocks, really, and that system
already exists.  I will grant you that there is  a learning curve
(Dirk's 'pons asinorum') involved in packaging modules.  There is
already a self-service LR publishing option (rocks.moonscript.org)
where you can put up rockspecs and even rocks.  It would be rather
cool if it came with a simple wizard for generating rockspecs.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

Dirk Laurie-2
In reply to this post by Andrew Starks
2013/3/9 Andrew Starks <[hidden email]>:

> Perhaps the addition of a package searcher and an aping of
> TeX/Context/LaTeX TEXMF trees?

Why another package searcher? Can't we simply use kpse? Remember, Lua
is part of TeX now. There are over 600 Lua source files in texlive2012.

I've made a little experiment: created a "lua" subdirectory in
my $HOME/texmf/tex, put some modules in there, ran texhash.
`kpsewhich` finds the .lua files but not the .so files. So there is
clearly already something special about the .lua extension, even
though there is no specialized file type 'lua'.

1234