Is "scripting" truly Lua's future?

classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Is "scripting" truly Lua's future?

Matt Campbell-4
The following quote from the recently published Computerworld interview with Roberto caught my attention:

Question:  Where do you envisage Lua's future lying?

Answer: Scripting. It is a pity that the term "scripting language" is becoming a synonym for "dynamic language". A scripting language, as its name implies, is a language that is mainly used for scripting. The origins of the name are the shell languages that have been used to script other programs. Tcl enlarged it for scripting a program, but later people started applying the term for languages like Perl or Python, which are not scripting languages (in that original meaning) at all. They are dynamic languages. For real scripting, Lua is becoming a dominant language.

[end of quote]

I agree with Roberto about the abuse of the term "scripting language"; that is why in my talk at the Lua Workshop, I only used the term "dynamic language". However, I was shocked at the implication, perhaps a misunderstanding on my part, that one should not use Lua as the primary implementation language for an application as one would use Perl or Python, but instead for scripting an application written primarily in C or the like. Though Roberto's answer made me think for a moment that my use of Lua is somehow wrong, My experience has been that given enough libraries, Lua is quite good as a primary implementation language. Perhaps Roberto is saying that Perl, Python, and the like are better in that role, so the best use of Lua is for scripting, the purpose for which Lua was originally developed. However, Lua's minimal core and relatively painless C API set Lua apart in my mind. By combining this small core with an appropriate collection of libraries, many of which are themselves small, one can use a powerful dynamic language without a bloated runtime environment. This is especially useful for developing desktop and "smart client" applications on Windows, where even the .NET Framework isn't yet ubiquitous, let alone Python or Ruby. Still, it's more difficult to use Lua in this way than it is with the more mainstream dynamic languages, especially if one has to write one's own bindings (as I have). I hope that this will change as more libraries and tools (e.g. something like py2exe) become available. The recent interest in Lua for Windows indicates to me that there is growing interest in using Lua for more than just scripting.

Matt

Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

Roberto Ierusalimschy
> Though Roberto's answer made me think for a moment that  
> my use of Lua is somehow wrong, My experience has been that given enough  
> libraries, Lua is quite good as a primary implementation language.  
> Perhaps Roberto is saying that Perl, Python, and the like are better in  
> that role, so the best use of Lua is for scripting, the purpose for  
> which Lua was originally developed.  However, Lua's minimal core and  
> relatively painless C API set Lua apart in my mind. [...]

I agree with you, but I think it is difficult to sell this idea of
"small is beautiful" to a larger audience. So, you may qualify that
answer as "For mainstream relevance, the future of Lua is scripting."

(BTW, your own emphasis on a "painless C API" also seems to point to a
"scripting" approach, does it not? ;)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

Javier Guerra Giraldez
On Fri, Sep 12, 2008 at 11:25 AM, Roberto Ierusalimschy
<[hidden email]> wrote:
> (BTW, your own emphasis on a "painless C API" also seems to point to a
> "scripting" approach, does it not? ;)

i value the "painless C API" more for knowing that any (sane) C
library is just a few hours away from a Lua extension.


-- 
Javier

Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

Paul Moore-3
In reply to this post by Matt Campbell-4
2008/9/12 Matt Campbell <[hidden email]>:
> The following quote from the recently published Computerworld interview with
> Roberto caught my attention:
>
> Question:  Where do you envisage Lua's future lying?
>
> Answer:  Scripting.
[...]
> My experience has been that given enough libraries, Lua is
> quite good as a primary implementation language.

I use Lua for scripting extensively - generally, if I see a program
with any sort of plugin interface, my first thought is "can I write a
Lua plugin for this?" :-) (And usually I can, the Lua C API really is
a wonderful thing!)

For use standalone, however, I find that I don't look at Lua. The
basic reason for this is embodied in your "given enough libraries"
statement, combined with the ridiculously painful C runtime mess on
Windows.

For scripting (embedding) use, I contend that really the only sensible
option is to use msvcrt (assuming you're not statically linking Lua
and disabling all extension use, when using the same CRT as the main
application is clearly the sensible option). The reason for this is
that msvcrt works when it is used by a DLL (and hence is not
guaranteed to be set up by the application startup code). For msvcr90
(definitely, and I suspect older versions as well) the CRT gets most
upset if it isn't initialised by the main application startup.

But the LuaBinaries standard is msvcr80 - and so, using anything else
requires me to build my own binaries (which for something complex,
like wxLua or Alien, may well be effectively impossible). So for
standalone use, I need to use LuaBinaries, and then I cannot build my
own extensions (I only have VC 2008 Express, which is msvcr90, and
mingw, which is msvcrt - although maybe mingw can be persuaded to
build with msvcr80, if I can get past the manifest rigmarole).

So for scripting (embedding) and standalone, I need to maintain 2
completely separate Lua installations. That's not a disaster, but it
doesn't make for a simple environment. So I stick with Lua for
scripting, and don't use it standalone.

I'd love to see the CRT mess sorted out sensibly, but I don't really
hold out much hope in the short term. Until that happens, though, I
doubt I'll use Lua much for standalone use. (But I'll probably carry
on whingeing about CRT issues on the list :-))

Paul.

Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

Veli-Pekka Tätilä
In reply to this post by Matt Campbell-4
Matt Campbell wrote:
The following quote from the recently published Computerworld interview
with Roberto caught my attention:

Question:  Where do you envisage Lua's future lying?
Answer:  Scripting. <snip> scripting language, as its
name implies, is a language that is mainly used for scripting. The
origins of the name are the shell languages that have been used to
script other programs. Tcl enlarged it for scripting a program, but
later people started applying the term for languages like Perl or
Python, which are not scripting languages (in that original meaning) at
all. <snip>
[end of quote]

My experience has been that given enough libraries, Lua is quite good as a
primary implementation language.
Perhaps Roberto is saying that Perl,
Python, and the like are better in that role, so the best use of Lua is
for scripting, the purpose for which Lua was originally developed.
Not sure what kind of answers you were after, but here's a scripter's view whose mostlyh been coding for the fun of it:

I'm looking at this coming from Perl, and have only done some apps on my own that are a couple of kloc of code at most. As to why Perl, it let's me prototype ideas so much quicker than the lower level languages, and has libraries for stuff I find important. Briefly libraries are the thing in Perl: COM, off-line and on-line MIDI handling, GUi automation, building accessible WIn32::GUIs, text processing, and so on. So first and for most: CPAN, the PPM command-line. and a keep it simple yet convenient coding culture that's markedly different from your average Java package -- you don't have to read 10 pages of docs and understand heavy OOP abstractions to get started. As to what I've done, I think my biggest Perl project is gotta be a Win32::GUI based, fully keyboard usable MIDI drum machine for the blind and other keyboard users, since I got fed up with not being able to use any of the soft drum machines out there:

http://vtatila.kapsi.fi/beats_me.html

As for Perl features helping you write large programs that would be absent from Lua, I don't think there is anything major. Both can easily do OOP and name spaces that I tend to use. I like Perl's POD documentation format, however, as well as good conventions of where the packages and docs go automagically. It is much quicker for me to look up Perl docs compared to any other language due to perldoc and the neat dir structure.

Another thing I find helpful is that Perl's strictness is compile-time so my variable typos. and I do these a lot coding with speech, are caught much earlier apart from method calls. Would be great to have such support for Lua, as a special build, add-on Lintlike program or some such thing. Other than that I don't complain. I tend to view Lua as a kind of minimalist, orthogonal Perl with the Unixisms ripped out and a change in philosophy. most of the core mechanisms allowing you to do the Perlisms if you want to are there, and even more. Cloning Ruby's arrays and related mixins would be a fun coding exercise in Lua OOP, <grin>.

It might even be that because Lua givse you less syntax, you write programs that are cleaner for new programmers to understand. I miss autovivification, hash slices, map and grep, push, regexp lookaround etc... But this is just minor quibbling and redundant convenience. While code not using it might be a bit more wordy, it does not require you to understand those concepts, and might be easier to maintain. I find that because perl has all these mechanisms at the language rather than library level, I tend to use them where it makes sense, often optimizing for brevity or elegance as opposed to coding for maintainability. Lua allows you to stay maintainable as it has fewer temptations. Still, I am, maybe as a result of Perl, a big fan of programmer convenience still as in Ruby. I haven't done Python so cannot comment on it, though.

--
With kind regards Veli-Pekka Tätilä
Accessibility, Apps and Coding plus Synths and Music:
http://vtatila.kapsi.fi


Reply | Threaded
Open this post in threaded view
|

typo checks considered mandatory (was Re: Is "scripting" truly Lua's future?)

Chip Salzenberg
On Fri, Sep 12, 2008 at 07:56:29PM +0300, Veli-Pekka T?til? wrote:
> Another thing I find helpful is that Perl's strictness is compile-time so 
> my variable typos.

This, this, a hundred times this.

Humans make mistakes; pretending otherwise is just ignoring reality.  I
don't want a computer to second- guess my design decisions; but when it
comes to spelling, I *need* help.

I invented a strict.lua but it only works at runtime, not compile time.
That's not really good enough.
-- 
Chip Salzenberg <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: typo checks considered mandatory (was Re: Is "scripting" truly Lua's future?)

Alexander Gladysh
> Humans make mistakes; pretending otherwise is just ignoring reality.  I
> don't want a computer to second- guess my design decisions; but when it
> comes to spelling, I *need* help.
>
> I invented a strict.lua but it only works at runtime, not compile time.
> That's not really good enough.

Have you tried Metalint? (I advise to get some recent version from Git.)

    http://lua-users.org/lists/lua-l/2008-04/msg00082.html

See also this page:

    http://lua-users.org/wiki/DetectingUndefinedVariables

HTH,
Alexander.

Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

Matt Campbell-4
In reply to this post by Roberto Ierusalimschy
The responses so far have been interesting. The question that came to my mind when I saw Roberto's answer to the "future of Lua" question in Computerworld is this: If I'm designing a new application, should I write the application core in C and then use Lua for scripting on top of that, or should I write the application in Lua, using C extension modules only where needed? Roberto's answer on the future of Lua, as well as his answer to the question about sysadmin tools, seemed to indicate that the former is correct. But if that's so, where do I draw the line between C and Lua? Coming from Python, I'm quite used to developing primarily in a dynamic language. This approach is dominant in the Python community; for example, see the rant "Extending Vs. Embedding" by a well-known Python programmer:

http://twistedmatrix.com/users/glyph/rant/extendit.html

So I naturally bring this approach with me to Lua. Roberto's statements in Computerworld made me think it was wrong to use Lua like this, but now I understand that he was only saying that the most natural and popular use of Lua is for embedded scripting.

Matt

Reply | Threaded
Open this post in threaded view
|

Re: typo checks considered mandatory (was Re: Is "scripting" trulyLua's future?)

Veli-Pekka Tätilä
In reply to this post by Alexander Gladysh
Alexander Gladysh wrote:
> Humans make mistakes; pretending otherwise is just ignoring reality.  I
> don't want a computer to second- guess my design decisions; but when it
> comes to spelling, I *need* help.
> I invented a strict.lua but it only works at runtime, not compile time. > <snip>
Have you tried Metalint? (I advise to get some recent version from Git.)

http://lua-users.org/lists/lua-l/2008-04/msg00082.html

See also this page:

http://lua-users.org/wiki/DetectingUndefinedVariables

Thanks for the help. I read the relevant section in the excellent Lua users Wiki and found that Luac can do the hardest part of the job. They gave a unix example but something like the following works on Windows:

luac -l -p test.lua |findstr /r /i "[gs]etglobal"

It produces output like:

       2       [1]     SETGLOBAL       0 -1    ; foo
       6       [3]     GETGLOBAL       0 -4    ; prit
etc...

The idea being parse, don't compile code, list disassembled lines and filter only those that involve sets or gets to global variables, if I got it right. The neat thing is that I don't need to download any extras to use this method. Now I only need a whitelist of allowed globals including built-ins and other modules (via reflection), and maybe a small Lua script around that for proper filtering. All that should be easily doable. So thanks for indirectly pointing me to one simple solution that just might be good enough for my needs.

--
With kind regards Veli-Pekka Tätilä
Accessibility, Apps and Coding plus Synths and Music:
http://vtatila.kapsi.fi


Reply | Threaded
Open this post in threaded view
|

Re: typo checks considered mandatory (was Re: Is "scripting" trulyLua's future?)

Luiz Henrique de Figueiredo
> Now I only need a whitelist of allowed globals including built-ins and
> other modules (via reflection), and maybe a small Lua script around
> that for proper filtering.

See http://lua-users.org/lists/lua-l/2006-05/msg00306.html

Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

David Given
In reply to this post by Matt Campbell-4
Matt Campbell wrote:
> If I'm designing a new application, should I
> write the application core in C and then use Lua for scripting on top of
> that, or should I write the application in Lua, using C extension
> modules only where needed?

Well, I did the latter for WordGrinder, partly as an experiment to see
how well it worked.

<plug> http://wordgrinder.sourceforge.net </plug>

The short answer: it works awesomely well.

What I have is a complete if simple word processor, written from
scratch, in 5600 lines of Lua and 1400 lines of C (not including the Lua
and LFS engines, of course). The C contains low-level screen drawing
routines, UTF-8 read/write routines, various bits of contorted setup,
and a very few specially optimised routines for doing stuff with words.
Everything else is in Lua.

Lua's talent at storing complex data structures intuitively allowed me
to put together a data model for storing documents that's fast and
reasonably efficient: a document is an array of paragraphs, a paragraph
is an array of words, and each word is a table. Because arrays are
tables, that allows me to attach methods and cached state to every
object. For example, line wrapping is done by storing, with each
paragraph, an array of lines, where each line is itself an array of the
words in the line. This then gets regenerated on demand. Since drawing
occurs at the cursor position and works up and down until the screen
bounds are reached, this means that the bulk of the document won't get
wrapped until you're actually looking at it --- which means it's fast.

There's nothing there that's particularly hard. I could do it all in C++
with the STL, for example. But it would have taken me a lot longer than
a month to do, which is how long the first working version of
WordGrinder took.

It's also very easy to modify. For the most recent version, 0.3.1, I
added a Table Of Contents feature: a dialogue that produces a
hierarchical list of headers and allows you to jump to one. This took
about an hour to write and is 97 lines of code, including comments and
whitespace:

http://wordgrinder.svn.sourceforge.net/viewvc/wordgrinder/wordgrinder/src/lua/addons/goto.lua?revision=121&view=markup

It's also *efficient*. New with 0.3.1 is a Windows version. Because
Windows machines tend not to come with package managers, I had to embed
the Lua and LFS code into the executable. Since I was doing this, I also
decided to byte-compile the Lua source code and embed that, as well, to
produce a single, stand-alone executable. This is 266kB uncompressed
EXE. That's the WordGrinder Lua code, the WordGrinder specialist C
bindings, the Lua engine itself, the LFS bindings, and all the overhead
needed for a Windows app, which is not insubstantial (including the icon
--- all 45kB of it!). I think that's quite impressive.

However, doing things in Lua also has its disadvantages. Like all
languages, Lua has its quirks --- details below. However, the major
architectural issue is that Lua, being a dynamic language which does
most things at run-time, allows you to compile in typos. 0.3.1 is a
bugfix release of 0.3 which I released when someone pointed out that if
you did File->New the application crashed. Highly embarrassing. While
fixing that I found another similar crash. I hope there aren't any more
--- but it's very hard to know.

Also, writing code in Lua, once you have it straight in your head, is
*so* fast it requires iron discipline to keep the code sensible, well
organised, and well commented. It's absolutely vital to maintain coding
standards, naming conventions, file organisation, etc. It's very
tempting to let your program fill up with quick hacks, which means it
quickly turns into an unmaintainable spaghetti mess full of special
cases. At least it's very quick to redraft large passages when you
decide to change an algorithm or data structure; this lends itself well
to evolutionary design, which is how I tend to code.

...

That concludes the 'my experiences writing a moderately complex app in
Lua' essay, so now on to some of the specific rants:

- Lua doesn't have a preprocessor. My app was made up of multiple source
files, and most of the time I wanted the same common header at the top
of each source file (importing C functions, mostly). Many's the time I
wished for #include.

- the module structure is a pain when writing single applications ---
because once you've done module(), the standard library is no longer
available. You have to import everything with 'local print = print'
statements. And if you forget one you get no warnings or error messages
until it crashes at run time, either. Again, #include would have helped
with this. Eventually I gave up on modules.

- 1-based string offsets... do, actually, cause problems. I think I've
talked about this before; the problem is string positions and string
deltas. With 0-based offsets, they're the same. With 1-based offsets,
they're not, which means you need *different code* depending on whether
you're referring to something by position or by delta. This genuinely
did cause me a lot of grief until I got everything straightened out.

However, I should point out that I would *still* rather cope with Lua's
quirks than, say, Symbian's C++ dialect, which is what I'm currently
struggling with in my day job...

-- 
ââââ ïïïïïïïïïïïïïï âââââ http://www.cowlark.com âââââ
â
â "All power corrupts, but we need electricity." --- Diana Wynne Jones,
â _Archer's Goon_

Attachment: signature.asc
Description: OpenPGP digital signature

Reply | Threaded
Open this post in threaded view
|

Re: Cons of 5.1 Style Modules (Was: Is "scripting" truly Lua's future?)

Veli-Pekka Tätilä
David Given <[hidden email]> wrote:
<heavy snippage>
- the module structure is a pain when writing single applications ---
because once you've done module(), the standard library is no longer
available. You have to import everything with 'local print = print'
statements. And if you forget one you get no warnings or error messages
until it crashes at run time, either. Again, #include would have helped
with this. Eventually I gave up on modules.
Errm, maybe I'm missing something here, but couldn't you just say:

module( ..., package.seeall )

The way I've understood it this will:
* name and create your global module table (aka. class) after the physical file basename * make sure that "global" assignments incl. function definitions go in your module table * fetch stuff from _G if it is not found in your module, when looking for "globals"

The only disadvantage I've found is that I can no longer use the colon syntax. I would say:

function class:setX(x)

But now, it becomes:

function setX(self, x)

The self does feel quite cozy as a Perler, but would be even neater to omit that in the param list. The new module syntax makes the old syntactic sugar less sweet. But that's just cosmetic quibbling, <grin>.

Hth.

--
With kind regards Veli-Pekka TÃtilÃ
Accessibility, Apps and Coding plus Synths and Music:
http://vtatila.kapsi.fi


Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

Mark Meijer-2
In reply to this post by David Given
That passage from Roberto's interview stuck with me too :) That's not
to say I was shocked or anything, obviously Lua is extremely well
suited for embedded use (the intended meaning here being, embedded in
a host app). It's true that right now, there are some issues when
trying to use Lua as an extensible language (as in embedding vs.
extending), i.e. writing apps primarily in Lua with support of
extension libraries. Which is why this currently is not the
"mainstream relevance" of Lua, as Roberto well put it a couple of
posts ago.

No "standard" set of libraries, can be difficult to get libraries to
work on some platforms, no preprocessor, lacking compile-time
validation, etc. But all of these problems can be, and are being,
solved. And I don't mean stopgap measures. I mean, if this continues,
then I think Lua (or rather, some "complete" Lua distribution) can be
at least as valid a choice for application programming as any. Imho,
the idea that Lua, in principle, somehow isn't suitable for this, does
not do justice. I don't think that's what Roberto meant to say, but I
just feel his statement about the future of Lua might be true only in
part. I'm sure it will be scripting, but I don't see why it can't also
be as a primary implementation language. It might not "dominate the
market" as such, but I think it can stand on its own two feet well
enough.

I don't see it as a "wrong way" (in Matt's words) of using Lua, when
using Lua as an application programming language. For my part, this is
exactly at the heart of the recent discussions about LuaRocks and Lua
for Windows, and likely a key motivation behind most "Lua
distributions" (for today's purpose, this phrase includes LuaRocks :P
). It is precisely the appeal of using Lua, with a complete set of
libraries, as the primary language to write apps in. I hope this
*will* have mainstream relevance in the future of Lua (albeit *not* at
the cost of mainstream embedded use, unlike python), and I think it is
well within the realm of possibility. I see efforts such as LuaRocks,
LfW and others, as great steps in that direction, regardless of the
different constraints and approaches taken.

Also, is the Kepler project not a case in point? Unless I'm sorely
mistaken, the Kepler project is meant to provide a Lua-based platform
for web application development. Are web applications not
applications? I know it's often called "server side scripting", and
technically Lua is in this case embedded in the webserver or plugin.
But on the other hand, the webserver itself may very well be
implemented in Lua (or an extension library), and also, I wouldn't
make the mistake of calling a web application a script.

When I think about what application development boils down to more and
more these days, it is, in an oversimplified nutshell, the glueing
together of functionality which resides in libraries. This is nothing
new of course, but the ratio of library code vs. glue code is getting
ever more extreme. Fifteen years ago, I wrote my own cheesy 3D-library
in C and assembly, down to the function that rendered gouraud-shaded
triangles on my screen in the all-popular mode-X. Who in his right
mind does this today?

When you're writing in "pure" C++ today, think about what remains of
your application (not talking about system programming here) when you
take out all the libraries, let's say all the code not written by you
(or your team). Glue and business logic. How is it that we're still
using C++ for that? I'm no longer a professional developer (by
choice), but it's still one of my hobbies, and I am desparately trying
to switch entirely to Lua for writing my hobby apps (in case nobody
noticed yet ;). Call me weird, but it makes a lot of sense to me.

Is not the entire .NET effort based on this very notion (and I quote
one sentence from the very first paragraph of
http://en.wikipedia.org/wiki/.NET_Framework ) - "It includes a large
library of pre-coded solutions to common programming problems and a
virtual machine that manages the execution of programs written
specifically for the framework." That sums it up nicely. Ok, .NET or
say Java have *a lot* of advanced features for what is termed
enterprise applications, and are often used for that. So where do
these advanced features come from? Even if they call it "runtime"
(java runtime, common language runtime, C runtime, runtime this
runtime that)... are runtime libraries not libraries? This is not to
say we should be building a framework of such scope for Lua. But it's
certainly possible.

Many Lua modules consist of bindings to existing libraries. Libraries
that are otherwise finding use in applications written in a "regular"
application programming language (which, if the language runs on a VM,
including aforementioned .NET and Java, requires bindings of their
own). How is that use of libraries different from use of the same
libraries in Lua? Libraries for networking, database access, web
access, serialisation, multimedia, user interfaces, hardware
abstraction, containers/data structures/algorithms/templates (yes STL
too), maths, compression, encryption, authentication, text
manipulation, image manipulation, DSP. For goodness sake even complete
text editors and web browsers are being provided as library
components. And let's not forget 3D engines, AI and pathfinding,
animation, physics, input devices, 3D audio, etc. A great many of
these are already available to Lua in some form or another. In other
words, what makes Lua such a poor choice to write your glue and
business logic in?

In short, in this regard, I see no difference between Lua and any
other language typically used for application programming. Lua, the
core language, should keep doing what it's doing. But on top of that
(as a separate effort), I'm already seeing more and more convergence
towards creating distributions of Lua with a complete set of libraries
that can be easily deployed, and used as a development platform in its
own right. And I don't see why not :)

I'm not a python programmer, I've never written one line of python
code in my life. I'm not taking a stand as to whether Lua should be
used for embedding or extending (yes I've read the python article
about that quite some time ago). I just think Lua can be excellent for
both. And it took me this long and incoherent post to explain (sorry
it's late and I'm rambling)...

Cheers
Mark


2008/9/12 David Given <[hidden email]>:
> Matt Campbell wrote:
>> If I'm designing a new application, should I
>> write the application core in C and then use Lua for scripting on top of
>> that, or should I write the application in Lua, using C extension
>> modules only where needed?
>
> Well, I did the latter for WordGrinder, partly as an experiment to see
> how well it worked.
>
> <plug> http://wordgrinder.sourceforge.net </plug>
>
> The short answer: it works awesomely well.
>
> What I have is a complete if simple word processor, written from
> scratch, in 5600 lines of Lua and 1400 lines of C (not including the Lua
> and LFS engines, of course). The C contains low-level screen drawing
> routines, UTF-8 read/write routines, various bits of contorted setup,
> and a very few specially optimised routines for doing stuff with words.
> Everything else is in Lua.
>
> Lua's talent at storing complex data structures intuitively allowed me
> to put together a data model for storing documents that's fast and
> reasonably efficient: a document is an array of paragraphs, a paragraph
> is an array of words, and each word is a table. Because arrays are
> tables, that allows me to attach methods and cached state to every
> object. For example, line wrapping is done by storing, with each
> paragraph, an array of lines, where each line is itself an array of the
> words in the line. This then gets regenerated on demand. Since drawing
> occurs at the cursor position and works up and down until the screen
> bounds are reached, this means that the bulk of the document won't get
> wrapped until you're actually looking at it --- which means it's fast.
>
> There's nothing there that's particularly hard. I could do it all in C++
> with the STL, for example. But it would have taken me a lot longer than
> a month to do, which is how long the first working version of
> WordGrinder took.
>
> It's also very easy to modify. For the most recent version, 0.3.1, I
> added a Table Of Contents feature: a dialogue that produces a
> hierarchical list of headers and allows you to jump to one. This took
> about an hour to write and is 97 lines of code, including comments and
> whitespace:
>
> http://wordgrinder.svn.sourceforge.net/viewvc/wordgrinder/wordgrinder/src/lua/addons/goto.lua?revision=121&view=markup
>
> It's also *efficient*. New with 0.3.1 is a Windows version. Because
> Windows machines tend not to come with package managers, I had to embed
> the Lua and LFS code into the executable. Since I was doing this, I also
> decided to byte-compile the Lua source code and embed that, as well, to
> produce a single, stand-alone executable. This is 266kB uncompressed
> EXE. That's the WordGrinder Lua code, the WordGrinder specialist C
> bindings, the Lua engine itself, the LFS bindings, and all the overhead
> needed for a Windows app, which is not insubstantial (including the icon
> --- all 45kB of it!). I think that's quite impressive.
>
> However, doing things in Lua also has its disadvantages. Like all
> languages, Lua has its quirks --- details below. However, the major
> architectural issue is that Lua, being a dynamic language which does
> most things at run-time, allows you to compile in typos. 0.3.1 is a
> bugfix release of 0.3 which I released when someone pointed out that if
> you did File->New the application crashed. Highly embarrassing. While
> fixing that I found another similar crash. I hope there aren't any more
> --- but it's very hard to know.
>
> Also, writing code in Lua, once you have it straight in your head, is
> *so* fast it requires iron discipline to keep the code sensible, well
> organised, and well commented. It's absolutely vital to maintain coding
> standards, naming conventions, file organisation, etc. It's very
> tempting to let your program fill up with quick hacks, which means it
> quickly turns into an unmaintainable spaghetti mess full of special
> cases. At least it's very quick to redraft large passages when you
> decide to change an algorithm or data structure; this lends itself well
> to evolutionary design, which is how I tend to code.
>
> ...
>
> That concludes the 'my experiences writing a moderately complex app in
> Lua' essay, so now on to some of the specific rants:
>
> - Lua doesn't have a preprocessor. My app was made up of multiple source
> files, and most of the time I wanted the same common header at the top
> of each source file (importing C functions, mostly). Many's the time I
> wished for #include.
>
> - the module structure is a pain when writing single applications ---
> because once you've done module(), the standard library is no longer
> available. You have to import everything with 'local print = print'
> statements. And if you forget one you get no warnings or error messages
> until it crashes at run time, either. Again, #include would have helped
> with this. Eventually I gave up on modules.
>
> - 1-based string offsets... do, actually, cause problems. I think I've
> talked about this before; the problem is string positions and string
> deltas. With 0-based offsets, they're the same. With 1-based offsets,
> they're not, which means you need *different code* depending on whether
> you're referring to something by position or by delta. This genuinely
> did cause me a lot of grief until I got everything straightened out.
>
> However, I should point out that I would *still* rather cope with Lua's
> quirks than, say, Symbian's C++ dialect, which is what I'm currently
> struggling with in my day job...
>
> --
> ┌─── dg@cowlark.com ───── http://www.cowlark.com ─────
> │
> │ "All power corrupts, but we need electricity." --- Diana Wynne Jones,
> │ _Archer's Goon_
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Cons of 5.1 Style Modules (Was: Is "scripting" truly Lua's future?)

Mark Meijer-2
In reply to this post by Veli-Pekka Tätilä
2008/9/13 Veli-Pekka Tätilä <[hidden email]>:
> [...]
> module( ..., package.seeall )
>
> The way I've understood it this will:
> * name and create your global module table (aka. class) after the physical
> file basename
> * make sure that "global" assignments incl. function  definitions go in your
> module table
> * fetch stuff from _G if it is not found in your module, when looking for
> "globals"
>
> The only disadvantage I've found is that I can no longer use the colon
> syntax. I would say:
>
> function class:setX(x)
>
> But now, it becomes:
>
> function setX(self, x)
>
> The self does feel quite cozy as a Perler, but would be even neater to omit
> that in the param list. The new module syntax makes the old syntactic sugar
> less sweet. But that's just cosmetic quibbling, <grin>.

This is based on the assumption that a module equals a class. I don't
think modules were necessarily meant to be used in that way.
Personally, I tend to view the module name as something like a
namespace. The namespace may contain just functions or entire classes,
depending on the design style for that particular project. But a
namespace containing functions (i.e. a module exposing functions) does
not necessarily equal a class, in my view. Hope I'm making sense.


Reply | Threaded
Open this post in threaded view
|

Re: Cons of 5.1 Style Modules (Was: Is "scripting" truly Lua'sfuture?)

Veli-Pekka Tätilä
Mark Meijer wrote:
2008/9/13 Veli-Pekka Tätilä <[hidden email]>:
> [...]
> module( ..., package.seeall )
>
> The way I've understood it this will:
> * name and create your global module table (aka. class) after the
> physical file basename
> * make sure that "global" assignments incl. function  definitions go in
> your module table
> * fetch stuff from _G if it is not found in your module, when looking
> for "globals"
>
> The only disadvantage I've found is that I can no longer use the colon
> syntax. I would say:
>
> function class:setX(x)
>
> But now, it becomes:
>
> function setX(self, x)
<snipping myself>
This is based on the assumption that a module equals a class. I don't
think modules were necessarily meant to be used in that way.
Personally, I tend to view the module name as something like a
namespace. The namespace may contain just functions or entire classes,
depending on the design style for that particular project. But a
namespace containing functions (i.e. a module exposing functions) does
not necessarily equal a class, in my view. Hope I'm making sense.
You are, I mostly agree. Take the math library for example, it is just a bunch of related functions grouped under a common name so as not to clash with other names. Or the IO library where, again, we have a collection of functions for stdin and stdout that just happen to also work as methods if you create a file handle to call the functions against. Similar syntax sugar is there for strings due to the metatable concept. It reminds me of Perl packages trying to support both OOP and procedural use.

Whether a module with maybe some data fields and a bunch of functions is a class is a matter of terminology. IF the language forces me to call it such syntactically, e.g. Java, I call it one. If not, say for Lua and Perl, I tend to call the conventional OOP stuff classes and the rest with the native name of such a construct i.e. modules in Lua. Somewhat inconsistant, I know, but calling everything a module tends to confuse people who don't know Lua 5.1 modules.

--
With kind regards Veli-Pekka Tätilä
Accessibility, Apps and Coding plus Synths and Music:
http://vtatila.kapsi.fi


Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

steve donovan
In reply to this post by Matt Campbell-4
'Scripting' is a great word, but carries prejudice; something 'lightweight', suitable for the quick jobs, not for the big workhorses which keep our civilization from falling apart. Or, something that one of these workhorses would add, as an afterthought, as 'end user customization'.

A very influential person in this debate was Osterhout (the Tcl man) who saw the need for a 'glue language' to tie C modules together. And this worked very well, despite the many weakness of tcl/tk (hint: Bash on steriods does not scale.) As it's been pointed out, applications are mostly glue to bind libraries together these days. I personally resisted this 'two languages' thing for a long while, I felt that a flexible language like C++ could stretch to both roles, especially with a good interpreter for the glue bits. The interpreter kept me busy for a few years and resulted in a severe allergy to C++;) But it is in fact useful that the two languages _look_ different, so that the correct mental framework is loaded, which is a point alluded to in the Lightroom presentation. Lua-tcc strikes me as a step in the wrong direction, because you want to keep modules and glue as separate entities.

Lua is a natural successor to Tcl, but never had a GUI app framework like tk. So anybody wanting to do a Lua application has to make choices: lua-gtk,LuaInterface,LuaJava,wxLua,IUP, etc, and basically have to do a 'due dilligence' investigation to see if a framework is mature enough, or currently maintained, etc. (For example, I am really impressed with LuaJava, but it _is_ ownerless at the moment; lua-gtk is going through intense refactoring, etc) This is particularly important if it's going to be used 'officially'! This isn't a call for standardization, however, I personally like choice, but understand that choice is confusing, especially if there are a few potential duds in the box of chocolates.

Another issue is support for 'programming in the large', which these days is mostly seen as tool support but still feeds into language design. The kind of happy hackery that works for 100 line scripts seems to lead to unhappiness in 100K applications (e.g. check out 'Dreaming in Code' about Chandler and Python); dynamic languages do not typically have explicit type annotations, so the tools have to work a lot harder. Eclipse with Java totally impresses me, but Eclipse with other languages was underwhelming (haven't tried the Lua plugin yet, however). We have the dynamic language paradox, usually expressed as 'conversion stories': former hard-line statically-typed X programmer discovers that they are enormously more productive in dynamic language Y for a weekend project, but miss the tool support with big projects. Maybe one needs explicit type annotations for the public interface of modules (defined here more loosely as a unit of code aggregation or programmer responsibility) but the internal module implementation can be more free wheeling? Are there means to make dynamic programming productivity scale well? More intelligent spell-checking? Source browsers that can hop around like mountain goats in million-line projects?

(My apologies for the mini-essay but it seems to be essay week on the Lua List ;))

steve d.





Reply | Threaded
Open this post in threaded view
|

Re: typo checks considered mandatory (was Re: Is "scripting" trulyLua's future?)

Fabien-3
In reply to this post by Veli-Pekka Tätilä
On Fri, Sep 12, 2008 at 9:47 PM, Veli-Pekka Tätilä <[hidden email]> wrote:
Now I only need a whitelist of allowed globals including built-ins and other modules (via reflection), and maybe a small Lua script around that for proper filtering. All that should be easily doable. So thanks for indirectly pointing me to one simple solution that just might be good enough for my needs.

This will catch typos such as taable.insert(), because the problem is in the global var. However, it won't catch table.iinsert(), because the global var 'table' indeed exists, and luac can't tell which field names are legal in it. For that you need a deeper analysis, metalint-style.

-- Fabien.
 

Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

Asko Kauppi
In reply to this post by steve donovan

Seems Roberto's comment on the future of Lua spawned some interesting, and needed discussion.

I share many of Steve's experiences, about the problems scaling "scripting" to actual application size projects (>1000 lines of Lua).

Here's what to do:

- start using type descriptions as an addon to "typeless" Lua. LuaSub and MetaLua already provide this.

- invent a "lua-lint" tool, which would use the same type descriptions to check as much of validity of the code, statically, as possible.

In my understanding, the lint-like tools exist in specific companies internal usage, but none is there out in the open. To me, this (and the GUI) seems to be the missing piece to "future Lua". Other pieces are syntax modding (already mentioned), LuaJIT 2.0, LuaRocks.

What's interesting here is that none of these need any changes from the core language whatsoever. Meaning, from Roberto's viewpoint the future may well be "scripting" but for some other of us, the future may be "hardwired scripting". The futures don't exclude each other.

-asko

p.s. One thing I'd like from Lua itself, is inclusion of a tolua-type interface in the language pack itself.


On Sat, 13 Sep 2008 11:22:00 +0200
 "Steve Donovan" <[hidden email]> wrote:
'Scripting' is a great word, but carries prejudice; something 'lightweight', suitable for the quick jobs, not for the big workhorses which keep our civilization from falling apart. Or, something that one of these workhorses would add, as an afterthought, as 'end user customization'.

A very influential person in this debate was Osterhout (the Tcl man) who saw the need for a 'glue language' to tie C modules together. And this worked very well, despite the many weakness of tcl/tk (hint: Bash on steriods does not scale.) As it's been pointed out, applications are mostly glue to bind libraries together these days. I personally resisted this 'two languages' thing for a long while, I felt that a flexible language like C++ could stretch to both roles, especially with a good interpreter for the glue bits. The interpreter kept me busy for a few years and resulted in a severe allergy to C++;) But it is in fact useful that the two languages _look_ different, so that the correct mental framework is loaded, which is a point alluded to in the Lightroom presentation. Lua-tcc strikes me as a step in the wrong direction, because you want to keep modules and glue as separate entities.

Lua is a natural successor to Tcl, but never had a GUI app framework like tk. So anybody wanting to do a Lua application has to make choices: lua-gtk,LuaInterface,LuaJava,wxLua,IUP, etc, and basically have to do a 'due dilligence' investigation to see if a framework is mature enough, or currently maintained, etc. (For example, I am really impressed with LuaJava, but it _is_ ownerless at the moment; lua-gtk is going through intense refactoring, etc) This is particularly important if it's going to be used 'officially'! This isn't a call for standardization, however, I personally like choice, but understand that choice is confusing, especially if there are a few potential duds in the box of chocolates.

Another issue is support for 'programming in the large', which these days is mostly seen as tool support but still feeds into language design. The kind of happy hackery that works for 100 line scripts seems to lead to unhappiness in 100K applications (e.g. check out 'Dreaming in Code' about Chandler and Python); dynamic languages do not typically have explicit type annotations, so the tools have to work a lot harder. Eclipse with Java totally impresses me, but Eclipse with other languages was underwhelming (haven't tried the Lua plugin yet, however). We have the dynamic language paradox, usually expressed as 'conversion stories': former hard-line statically-typed X programmer discovers that they are enormously more productive in dynamic language Y for a weekend project, but miss the tool support with big projects. Maybe one needs explicit type annotations for the public interface of modules (defined here more loosely as a unit of code aggregation or programmer responsibility) but the internal module implementation can be more free wheeling? Are there means to make dynamic programming productivity scale well? More intelligent spell-checking? Source browsers that can hop around like mountain goats in million-line projects?

(My apologies for the mini-essay but it seems to be essay week on the Lua List ;))

steve d.






Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

Matt Campbell-4
I agree with much of what has been said here, but:

Asko wrote:
p.s. One thing I'd like from Lua itself, is inclusion of a tolua-type interface in the language pack itself.

IMO, even this is too much for the standard Lua package. The means of generating Lua bindings for libraries vary significantly depending on the libraries. Factors include the library's target language (C or C++), API style (e.g. OO or not), and conventions with regard to memory allocation and object ownership. I believe it's best for the Lua team to defer to third-party packages for automated generation of Lua bindings, as they have done thus far.

Matt

Reply | Threaded
Open this post in threaded view
|

Re: Is "scripting" truly Lua's future?

Petite Abeille-2-2
In reply to this post by Matt Campbell-4

On Sep 12, 2008, at 9:37 PM, Matt Campbell wrote:

If I'm designing a new application, should I write the application core in C and then use Lua for scripting on top of that, or should I write the application in Lua, using C extension modules only where needed?

For what it's worth, as a hobby, I use Lua to write applications, only using extension modules when there are no other alternatives.

After all, one can get quite a bit of mileage out of just bare bone Lua :)

As a concrete example of an arguably full fledge application, Nanoki, a wiki server, is fully implemented in Lua, using only a handful of low level C modules:

http://svr225.stepx.com:3388/a

Just my 2¢.

Cheers,

--
PA.
http://alt.textdrive.com/nanoki/




123