Portability of Lua

classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Portability of Lua

Paul Baker
Hi,

I have been using Lua embedded within a C application for some time,
but am now considering using it for a project which requires
standalone programming. Previously I would probably have used Python
for this, but a major goal of this project is maximum portability.

I have investigated different languages which I could use, and have found:

- On some niche platforms, C may be the only language immediately
available. However, since the Lua interpreter is written in pure C, it
should also be possible to use Lua on all of these platforms - that
is, Lua seems to be as portable as C.

- On more popular platforms with plenty of languages to choose from, a
Lua interpreter may be the most readily available
interpreter/compiler. I could even distribute the interpreter
alongside my code.

These facts combine to mean that, quite possibly, Lua is the most
portable language in existence. Do people agree with this statement?
If not, which other languages approach Lua in terms of portability?

Thanks

Paul

Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

steve donovan
On Tue, Apr 30, 2013 at 1:01 PM, Paul Baker <[hidden email]> wrote:
These facts combine to mean that, quite possibly, Lua is the most
portable language in existence. Do people agree with this statement?
If not, which other languages approach Lua in terms of portability?

Well, the majority opinion would be naturally a big yes here, especially when you factor in the lean and mean aspect.

The gotcha is that you will need to select a set of batteries to complement the small set of built-in libraries (this comes directly out of C89 as an 'abstract platform').  E.g, LuaFileSystem. For POSIX platforms luaposix is a big win (you won't need LuaFileSystem and it gives you all the usual yummies) and it has a very configurable build.  It's possible to build a single executable which carries the extra batteries.

Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Laurent FAILLIE
In reply to this post by Paul Baker
Hi,

I'm working on open sources that have to be available on as many as
possibly POSIX platform (m$-windows is out of the scope, I will not wast
my time with "that").

* Lua itself is widely available in package managers : from AmigaOS up
to major Linux/BSD distribution
* LuaRocks as well ... sometime with very ancients version
* Rocks themselves are only POORLY supported on OSes' package managers.
Major ones as Socket are there but again with deprecated version, but
generally recent or confident ones not.

Despite of that, I choose Lua instead of Python or others, because it
has a very reduced footprint. I said to my "customers" to install rocks
using LuaRocks. The drawback is generally they have to take care about
libraries breakage when doing system upgrade.
Gentoo, the distribution I'm using, is taking care automatically of this
aspect but I'm not sure all distrib are smart enough ...

Bye

Laurent

Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Rob Kendrick-2
In reply to this post by Paul Baker
On Tue, Apr 30, 2013 at 11:01:56PM +1200, Paul Baker wrote:
> These facts combine to mean that, quite possibly, Lua is the most
> portable language in existence. Do people agree with this statement?
> If not, which other languages approach Lua in terms of portability?

It can't be more portable than the language it's implemented in.
Additionally, there are places you can use C where using Lua would be
very difficult, such as memory-starved microcontrollers or anything
requiring real-time.

Also, I'm not sure how well Lua reacts when char is not 8 bits.

B.

Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Michael Richter
I'd be hard-pressed to think of a scripting language more portable than Rexx in terms of having code Just Work across platforms.  That said Lua is the better language and it's on most popular platforms and easily ported to less popular ones.  You will get different behaviour across platforms, however.


On 30 April 2013 20:16, Rob Kendrick <[hidden email]> wrote:
On Tue, Apr 30, 2013 at 11:01:56PM +1200, Paul Baker wrote:
> These facts combine to mean that, quite possibly, Lua is the most
> portable language in existence. Do people agree with this statement?
> If not, which other languages approach Lua in terms of portability?

It can't be more portable than the language it's implemented in.
Additionally, there are places you can use C where using Lua would be
very difficult, such as memory-starved microcontrollers or anything
requiring real-time.

Also, I'm not sure how well Lua reacts when char is not 8 bits.

B.




--
"Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot."
--Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra.
Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Thomas Jericke
In reply to this post by Rob Kendrick-2
> It can't be more portable than the language it's implemented in.
True, but it would be possible to write Lua interpreters in other
languages to overcome this limitation.
> Additionally, there are places you can use C where using Lua would be
> very difficult, such as memory-starved microcontrollers or anything
> requiring real-time.
Actually we use Lua in a real-time environment. In general you need to
control the GC, which is possible in the new versions (>=5.1).
The only critical thing is, that you must be able to run at least
one/any GC step in your RT cycle. Also you must be able to run the GC
for long enough in general, otherwise your memory wont last long ;-)

It may not work on real time systems with very limited CPU and RAM but
it is possible to intergrate Lua in RT systems.

Of course it would help if a GC step was more deterministic in the run
time it needs. But I am quite happy as it is at the moment.
--
Thomas

Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Roberto Ierusalimschy
In reply to this post by Rob Kendrick-2
> It can't be more portable than the language it's implemented in.

Of course, that depends on your meaning of portability. C has way too
many non-portable features, so programs written in C usually are
non-portable. You can write very portable programs in C, but you must be
*very* careful. If we follow the standard, a lot of things we take
for granted in C are undefined and should not be used in portable programs
(e.g., any use of pointer subtraction and ptrdiff_t).

So, a language implemented in C, but making judicious use of its features,
can be more portable than C, in the meaning that the "chance" of an
"arbitrary" program in that language work in different platforms is
higher than the chance for an arbitrary program in C.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

William Ahern
On Tue, Apr 30, 2013 at 10:01:24AM -0300, Roberto Ierusalimschy wrote:
> > It can't be more portable than the language it's implemented in.
>
> Of course, that depends on your meaning of portability. C has way too
> many non-portable features, so programs written in C usually are
> non-portable. You can write very portable programs in C, but you must be
> *very* careful. If we follow the standard, a lot of things we take
> for granted in C are undefined and should not be used in portable programs
> (e.g., any use of pointer subtraction and ptrdiff_t).

Technically if you're not following the standard you're not writing in C ;)

I know lots of stuff is unspecified or implementation-defined, but if you're
following the standard then you're also checking to see if such behavior is
acceptably portable, or otherwise selecting your algorithm so it works
portably.

It's not like Lua solves all these issues, either. Floating point behavior
is just as dependant on implementation-defined behavior as similar C code
would be. I just ran into it last week when exploring why infinity was
changing between INT_MIN and INT_MAX depending on whether it was converted
at compile-time or runtime.


Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Jay Carlson
In reply to this post by Roberto Ierusalimschy
On Apr 30, 2013, at 9:01 AM, Roberto Ierusalimschy wrote:

>> It can't be more portable than the language it's implemented in.
>
> Of course, that depends on your meaning of portability. C has way too
> many non-portable features, so programs written in C usually are
> non-portable. You can write very portable programs in C, but you must be
> *very* careful. If we follow the standard, a lot of things we take
> for granted in C are undefined and should not be used in portable programs

Very well said. I wish this were printed in the preface to every text on the C language. The conforming ANSI C compiler on the 36-bit, word-addressed Sperry SX1100 is not the weirdest you can get.

Microprocessor development has been shaped by "make C fast" too, so rather than everything converging on a PDP-11/VAX/68000/i386-flat model, processor and OS designs are shaped by what's allowable in the standard and what won't break *too* much non-conforming code. (I think the SVR4 ELF dynamic linker is too deferential, but for some reason saying "Windows DLLs are healthy, like good vigorous exercise" is unpopular.)

> So, a language implemented in C, but making judicious use of its features,
> can be more portable than C, in the meaning that the "chance" of an
> "arbitrary" program in that language work in different platforms is
> higher than the chance for an arbitrary program in C.

This is parallel to how languages which compile to C as an intermediate language can be faster than C.

Jay
Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Paul E. Merrell, J.D.
In reply to this post by Rob Kendrick-2
On Tue, Apr 30, 2013 at 5:16 AM, Rob Kendrick <[hidden email]> wrote:

> Also, I'm not sure how well Lua reacts when char is not 8 bits.

This can be problematic absent a library for handling such characters.
For UTF-8 chars, I've yet to hit a problem using the utf-8.lua library
with either Lua 5.1.x or 5.2.x.
<http://www.curse.com/addons/wow/utf8>. Written in pure Lua, it
provides UTF-8 aware substitutes for string.len, string.sub,
string.reverse, string.upper, and string.lower. There are other
solutions. See generally, <http://lua-users.org/wiki/LuaUnicode>.

Paul

Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Coda Highland
On Tue, Apr 30, 2013 at 3:24 PM, marbux <[hidden email]> wrote:

> On Tue, Apr 30, 2013 at 5:16 AM, Rob Kendrick <[hidden email]> wrote:
>
>> Also, I'm not sure how well Lua reacts when char is not 8 bits.
>
> This can be problematic absent a library for handling such characters.
> For UTF-8 chars, I've yet to hit a problem using the utf-8.lua library
> with either Lua 5.1.x or 5.2.x.
> <http://www.curse.com/addons/wow/utf8>. Written in pure Lua, it
> provides UTF-8 aware substitutes for string.len, string.sub,
> string.reverse, string.upper, and string.lower. There are other
> solutions. See generally, <http://lua-users.org/wiki/LuaUnicode>.
>
> Paul
>

I think you missed what was being said. UTF-8 still uses an 8-bit code
unit, which can be stored in an array of type "char". But not all
architectures historically have had "char" be 8 bits long -- 7-bit
chars and 9-bit chars historically weren't rare (9-bit chars make a
lot of sense on a system with 36-bit words), and even on some modern
architectures (TI's C54x DSPs for example) "char" is 16 bits long.

/s/ Adam

Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

William Ahern
In reply to this post by Paul E. Merrell, J.D.
On Tue, Apr 30, 2013 at 03:24:30PM -0700, marbux wrote:

> On Tue, Apr 30, 2013 at 5:16 AM, Rob Kendrick <[hidden email]> wrote:
>
> > Also, I'm not sure how well Lua reacts when char is not 8 bits.
>
> This can be problematic absent a library for handling such characters.
> For UTF-8 chars, I've yet to hit a problem using the utf-8.lua library
> with either Lua 5.1.x or 5.2.x.
> <http://www.curse.com/addons/wow/utf8>. Written in pure Lua, it
> provides UTF-8 aware substitutes for string.len, string.sub,
> string.reverse, string.upper, and string.lower. There are other
> solutions. See generally, <http://lua-users.org/wiki/LuaUnicode>.
>

I believe they're referring to DSP systems where CHAR_BIT is very often 16.
This usually doesn't effect string encodings so much as invalidate width
calculations, unsigned overflow, and a host of other stuff that most C
programmers are only vaguely familiar with.

You know it's a lost cause when you see code with `sizeof (char)', although
at least the developer gets props for trying.


Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Tim Hill
In reply to this post by William Ahern

But the point is that the C "standard" leaves some things so open that it's very difficult to write code without making some guesses. Width of an integer? Behavior of C runtime functions? All have very subtle hidden assumptions. For example, when you call "malloc()" you assume that it won't take 5 seconds to allocate memory, but the standard is silent on such issues as API overhead. Writing in standard C is a REQUIREMENT for portability, but not a GUARANTEE.

--Tim


On Apr 30, 2013, at 11:41 AM, William Ahern <[hidden email]> wrote:

> On Tue, Apr 30, 2013 at 10:01:24AM -0300, Roberto Ierusalimschy wrote:
>>> It can't be more portable than the language it's implemented in.
>>
>> Of course, that depends on your meaning of portability. C has way too
>> many non-portable features, so programs written in C usually are
>> non-portable. You can write very portable programs in C, but you must be
>> *very* careful. If we follow the standard, a lot of things we take
>> for granted in C are undefined and should not be used in portable programs
>> (e.g., any use of pointer subtraction and ptrdiff_t).
>
> Technically if you're not following the standard you're not writing in C ;)
>


Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Rena
On Tue, Apr 30, 2013 at 9:51 PM, Tim Hill <[hidden email]> wrote:

But the point is that the C "standard" leaves some things so open that it's very difficult to write code without making some guesses. Width of an integer? Behavior of C runtime functions? All have very subtle hidden assumptions. For example, when you call "malloc()" you assume that it won't take 5 seconds to allocate memory, but the standard is silent on such issues as API overhead. Writing in standard C is a REQUIREMENT for portability, but not a GUARANTEE.

Would such a guarantee even be possible? Even the fastest systems might take a while to allocate memory if there's none left, they're busy paging things out already, etc. An idea I've had in the past was that if an allocation ever fails, the process could be paused until enough memory is available instead of letting it die; in that case malloc() might block for any amount of time.


--
Sent from my Game Boy.
Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

William Ahern
In reply to this post by Tim Hill
On Tue, Apr 30, 2013 at 06:51:23PM -0700, Tim Hill wrote:
>
> But the point is that the C "standard" leaves some things so open that
> it's very difficult to write code without making some guesses. Width of an
> integer?

If you could provide concrete examples, then it would be easier to discuss.

The byte width of any integer type is simply sizeof (type), and that's
sufficient for memory mangement. The arrangement of bits is almost always
inconsequential because one should always be operating on values without
concern for representation. unsigned char is guaranteed to never have any
padding bits, making it trivial to, e.g., load and store encoded byte
sequences from memory or a file. C is no worse than Java or C# here, and in
practice much easier to use. The marginal return on over-specifying behavior
the way Java does is almost nil.

(Also, exact-width unsigned types like uint32_t never have any padding bits
[and thus no trap representations], and C11 even provides alignof, so
masochists who really want to conflate value and represtation can actually
do so within the confines of the standard.)

> Behavior of C runtime functions? All have very subtle hidden assumptions.
> For example, when you call "malloc()" you assume that it won't take 5
> seconds to allocate memory, but the standard is silent on such issues as
> API overhead. Writing in standard C is a REQUIREMENT for portability, but
> not a GUARANTEE.

That's an odd argument to make. No general purpose language makes that
guarantee. Runtime limits are nearly impossible to standardize, although C
makes a passable attempt. Anyhow, on systems with pageable virtual memory,
even a block of memory already allocated can stall the whole execution
context. In any event C, just like Java or C#, is defined in terms of an
abstract "virtual" machine.

If you write your code using nothing but well-defined C, it's guaranteed to
compile with any C implementation. In many cases you can even use
unspecified and implementation-defined behavior in a portable manner.

Lua is a testament to the practicality of writing portable code. It does
make a handful of assumptions, but it chooses to do so. And while Lua code
is very clean and very strict C, it's not that exceptional. If you hang out
in the right forums (i.e. not the stackexchange, cargo-cult groups where the
notion of portability is already dismissed out-of-hand ;) you can see it
done on a regular basis.

Nobody said it's easy to write compliant C code, but if you don't have at
least one copy of a C standard on your desktop that you reference regularly,
you're not even trying. And compared to other languages with a written
specification, C is actually rather easy to wrap your head around.


Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

William Ahern
In reply to this post by Rena
On Tue, Apr 30, 2013 at 09:56:26PM -0400, Rena wrote:

> On Tue, Apr 30, 2013 at 9:51 PM, Tim Hill <[hidden email]> wrote:
>
> >
> > But the point is that the C "standard" leaves some things so open that
> > it's very difficult to write code without making some guesses. Width of an
> > integer? Behavior of C runtime functions? All have very subtle hidden
> > assumptions. For example, when you call "malloc()" you assume that it won't
> > take 5 seconds to allocate memory, but the standard is silent on such
> > issues as API overhead. Writing in standard C is a REQUIREMENT for
> > portability, but not a GUARANTEE.
> >
>
> Would such a guarantee even be possible? Even the fastest systems might
> take a while to allocate memory if there's none left, they're busy paging
> things out already, etc. An idea I've had in the past was that if an
> allocation ever fails, the process could be paused until enough memory is
> available instead of letting it die; in that case malloc() might block for
> any amount of time.
>

I believe that could easily result in a deadlock scenario.


Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Jay Carlson
On Apr 30, 2013, at 10:48 PM, William Ahern wrote:

> On Tue, Apr 30, 2013 at 09:56:26PM -0400, Rena wrote:
>> An idea I've had in the past was that if an
>> allocation ever fails, the process could be paused until enough memory is
>> available instead of letting it die; in that case malloc() might block for
>> any amount of time.

Welcome to OS X! Run out of memory then run out of disk space for swap on your startup disk and the system will start pausing apps. (After first telling apps to throw any ballast over the side--Terminal.app discards scrollback for instance.)

Not that it always works out so well. Something terrible seems to happen when Safari's content slave (the usual perpetrator in fact) gets paused before Safari.app.

> I believe that could easily result in a deadlock scenario.

Dead, deadlock, it's all just words. I prefer "dirt nap". That is not dead which can eternal lie etc etc

Just because the process never completes or even makes progress doesn't mean it ever violated the execution model. It's no worse than the Linux out-of-memory task killer--at least, given the OOM killer's historical habit of going after the X server with kill -9. Boy, that freed up a lot of memory! Why is my CRT making that funny noise?

Jay
Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Tim Hill
In reply to this post by William Ahern

int a, b, c;
a = 10000;
b = 5000;
c = a * b;

This will generate different results on different platforms, depending on the width of int. For those of us who started when CPUs were 16 bits, you could NOT use code like this without overflow. In fact, all those silly dialog boxes with impossible numbers ("The copy will complete in -150009999 seconds") are mostly traceable to invalid assumptions about things like this that are most certainly standard C and yet did not port well.

I don't think at the runtime argument is odd. You stated that writing to standard C would make code portable. I merely said that while standard C was a requirement, it was not the ONLY requirement.

Also, C is NOT defined in terms of a VM (real or virtual), unlike Java, which is why there is far more variability in C than Java. Java has its faults (legion), but in general Java code (and, to a lesser extent, C#) is more portable than C for this reason. Spend any time looking at the Lua header files to see what incredible lengths Roberto's team had to jump through to make Lua as portable as it is, they certainly did NOT just "write it in ANSI C".

--Tim

On Apr 30, 2013, at 7:47 PM, William Ahern <[hidden email]> wrote:

> On Tue, Apr 30, 2013 at 06:51:23PM -0700, Tim Hill wrote:
>>
>> But the point is that the C "standard" leaves some things so open that
>> it's very difficult to write code without making some guesses. Width of an
>> integer?
>
> If you could provide concrete examples, then it would be easier to discuss.
>
>
>> Behavior of C runtime functions? All have very subtle hidden assumptions.
>> For example, when you call "malloc()" you assume that it won't take 5
>> seconds to allocate memory, but the standard is silent on such issues as
>> API overhead. Writing in standard C is a REQUIREMENT for portability, but
>> not a GUARANTEE.
>
> That's an odd argument to make. No general purpose language makes that
> guarantee. Runtime limits are nearly impossible to standardize, although C
> makes a passable attempt. Anyhow, on systems with pageable virtual memory,
> even a block of memory already allocated can stall the whole execution
> context. In any event C, just like Java or C#, is defined in terms of an
> abstract "virtual" machine.


Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Paul E. Merrell, J.D.
In reply to this post by Coda Highland
On Tue, Apr 30, 2013 at 3:35 PM, Coda Highland <[hidden email]> wrote:

> I think you missed what was being said. UTF-8 still uses an 8-bit code
> unit, which can be stored in an array of type "char". But not all
> architectures historically have had "char" be 8 bits long -- 7-bit
> chars and 9-bit chars historically weren't rare (9-bit chars make a
> lot of sense on a system with 36-bit words), and even on some modern
> architectures (TI's C54x DSPs for example) "char" is 16 bits long.

I didn't miss it; I'm just ignorant so read what I do understand into
what I read. Thanks for the education and sorry for the noise.

Paul

Reply | Threaded
Open this post in threaded view
|

Re: Portability of Lua

Roberto Ierusalimschy
In reply to this post by William Ahern
> You know it's a lost cause when you see code with `sizeof (char)', although
> at least the developer gets props for trying.

I strongly desagree with this view. I frequently use 'sizeof(char)',
although I am very aware that this is 1 by definition. I think it helps
both to document that you are computing sizes of things with chars (e.g.,
strings) and to change char to some other type (e.g., wchar_t) if the
need comes.

It is a similar point of using '\0' instead of 0; it documents what you
are doing.

-- Roberto


12