[mildly OT] Some info about Python

classic Classic list List threaded Threaded
121 messages Options
1234 ... 7
Reply | Threaded
Open this post in threaded view
|

[mildly OT] Some info about Python

Lorenzo Donati-3
Hi all!

On a recent thread ("Dead Batteries" ) I argued that Lua lost terrain
over Python.

I won't bother anyone repeating what I already said, but I stumbled on
this article which may explain something:

https://www.techrepublic.com/article/python-is-eating-the-world-how-one-developers-side-project-became-the-hottest-programming-language-on-the-planet/

Moreover the recent StackOverflow Developer Survey Results show
Python as one of the most known language among programmers (even among
professionals), while Lua doesn't even make it on the list.

https://insights.stackoverflow.com/survey/2019#technology

https://insights.stackoverflow.com/survey/2019#technology-_-most-loved-dreaded-and-wanted-languages

https://insights.stackoverflow.com/survey/2019#technology-_-what-languages-are-associated-with-the-highest-salaries-worldwide

I thought it could be useful info to share.

I'm just sad because while still loving Lua, it just seems that there is
no point in investing further time in developing in Lua, unless one has
a very specific need.

I won't comment further, since I'm bored to death to beat dead horses.

Cheers!

-- Lorenzo

Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Stefan-2
Hello,

Am 15.01.2020 um 16:59 schrieb Lorenzo Donati:

> Moreover the recent StackOverflow Developer Survey Results show
> Python as one of the most known language among programmers (even among
> professionals), while Lua doesn't even make it on the list.
>
> https://insights.stackoverflow.com/survey/2019#technology
>
> https://insights.stackoverflow.com/survey/2019#technology-_-most-loved-dreaded-and-wanted-languages 
>
>
> https://insights.stackoverflow.com/survey/2019#technology-_-what-languages-are-associated-with-the-highest-salaries-worldwide 
>
>
> I thought it could be useful info to share.

Of course everybody loves Python because of powerful libraries
and easy scripting. It does not make any sense to start another
full-fledged Better Version Of Python That Is Actually Lua.

I like Python alot myself and have written many programs in it;
professionally too.

Python perfectly fills the general-purpose scripting niche, where
programs are usually text files only, written and maintained by a
single person and performance just needs to be good enough for
interactivity, i.e. when you can call "python my_script.py" and
don't have to wait multiple seconds for JVM/.Net startup.
Or to build a simple GUI without jitters (has anybody seen a Ruby
GUI in the wild? I haven't). A Python example would be the Linux
Mint update manger:
https://github.com/linuxmint/mintupdate

But anything beyond that and it becomes tedious. I once worked with
a >1MLOC wxPython project and it was an unbelievable heap of garbage.
A depressing experience - apparently 40 man-years were poured into it.

And Python also has glaring issues with the upcoming multicore
architectures that will soon be the new standard.
And Jython/IronPython are not the solution - they negate the main
benefits of CPython.

> I'm just sad because while still loving Lua, it just seems that there is
> no point in investing further time in developing in Lua, unless one has
> a very specific need.

Here is an example:
My current project is an HTTP and Websocket server for interactive web
sites and online games. It connects each request to one user script.
I really want to use Lua for this since it fits perfectly IMO:
  * Friendly for non-programmers - "like JavaScript without brackets"
  * Web sites are usually a big collection of small scripts and media
    files, not one monolithic program that does everything.
    "And I also want a chat and an image gallery in my webshop"
  * Extremely fast interpreter startup (>100K/second/core)
  * Low memory overhead even for many thousand interpreter states
  * Bytecode caching is very easy

Because Websockets are long-lived full-duplex communication channels,
they are very problematic with "big" scripting languages like PHP/Ruby/
Python/Perl since they use up a huge amount of memory per connection.
And browsers allow up to 200 connections, so even one malicious user
can almost DOS your server in that case.

The NodeJS solution is to only run one script that handles all
connections. This has good performance (memory leaks aside) for small
programs, but it doesn't scale without ugly hacks and makes user and
application seperation very hard - the "LAMP stack" is successful for
a reason.

But with Lua many states aren't a problem and the simpler, more secure
and scalable Apache model can be used! And it is beginner-friendly!

So, the only show stopper is the lack of a standard library.

I'm currently experimenting with a new C module system and my goal is:
  * Fast state creation and destruction even with many C functions
  * Only allow thread-safe C functions because the Lua states are run
    by multiple threads
  * Everything statically linked since using Lua's require/dlopen for
     every request doesn't make any sense (dlopen has a cache, too)

Currently I think the best solution is something like the luaL_requiref
that luaL_openlibs uses, but with a metatable to only load what is
actually used.

An open repository of robust and portable C functions that can be
chosen to taste and staticially linked with a given Lua version
would also be a better fit with Lua, IMO.
Dynamic linking assumes that there is "the" Lua with a stable ABI and
that is just not how Lua is actually used.

>
> I won't comment further, since I'm bored to death to beat dead horses.
>
> Cheers!
>
> -- Lorenzo
>

Sorry to hear that!

The Lua ecosystem is certainly in a sorry state right now, but let's
just hope that this can and will change in the future :)

Greetings

Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Andre Leiradella-2
On 15/01/20 21:18, Stefan wrote:
 * Fast state creation and destruction even with many C functions

I once toyed with Lua state rollback and reuse, using linear allocators. The idea was to have instant Lua states for use, and fast memory management, for short-lived scripts.

The process was like this:

1. Create a linear allocator
2. Create a Lua state using the linear allocator to manage memory (note that the linear allocator does not free memory)
3. Check and save the number of bytes used by the Lua state used after initialization, including all necessary modules (state_size)
4. Allocate state_size bytes from the allocator and save the pointer (inited_state), and memcpy the initialized Lua state to that new area
5. Check and save the number of bytes used in the linear allocator (rollback)
6. Use the Lua state to run a script
7. Memcpy inited_state over the state at the beginning of the linear allocator
8. Set the number of used bytes by the linear allocator to rollback (uncommit the unused pages)
9. Go to 6

Of course each linear allocator must have enough space reserved for your worst case, but that's not a problem in 64-bit OSes since the address space is huge.

Cheers,

Andre
Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Luiz Henrique de Figueiredo
> I once toyed with Lua state rollback and reuse, using linear allocators.

Any practical conclusions?

Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Andre Leiradella-2

On 15/01/20 22:02, Luiz Henrique de Figueiredo wrote:

> Any practical conclusions?

Unfortunately no. It worked just fine, but I didn't measure the
performance difference against recreating the state every time, against
using the system allocator, nor the memory usage impact of not freeing
memory until the script ends.

Cheers,

Andre



Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Stefan-2
In reply to this post by Andre Leiradella-2
Hey Andre,

Am 15.01.2020 um 22:56 schrieb Andre Leiradella:
> On 15/01/20 21:18, Stefan wrote:
>>  * Fast state creation and destruction even with many C functions
>
> I once toyed with Lua state rollback and reuse, using linear allocators.
> The idea was to have instant Lua states for use, and fast memory
> management, for short-lived scripts.
>

I too tried using Lua with an arena allocator; it speeds up script
execution by about 10-20% since Lua makes heavy use of the allocator.

> The process was like this:
>
> 1. Create a linear allocator
> 2. Create a Lua state using the linear allocator to manage memory (note
> that the linear allocator does not free memory)
> 3. Check and save the number of bytes used by the Lua state used after
> initialization, including all necessary modules (state_size)
> 4. Allocate state_size bytes from the allocator and save the pointer
> (inited_state), and memcpy the initialized Lua state to that new area
> 5. Check and save the number of bytes used in the linear allocator
> (rollback)
> 6. Use the Lua state to run a script
> 7. Memcpy inited_state over the state at the beginning of the linear
> allocator
> 8. Set the number of used bytes by the linear allocator to rollback
> (uncommit the unused pages)
> 9. Go to 6
>
> Of course each linear allocator must have enough space reserved for your
> worst case, but that's not a problem in 64-bit OSes since the address
> space is huge.
>
> Cheers,
>
> Andre

Well, that's exactly the problem with area allocators: What if your
data no longer fits into your area?
Abusing virtual memory is extremely hacky.

I haven't yet figured out what exactly takes so long in luaL_openlibs -
but it is probably the sum of unneccessary object creation/string
interning/hash table construction and collection of the whole shebang
afterwards.

Greetings

Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Andre Leiradella-2

On 15/01/20 22:17, Stefan wrote:
>
>
> Well, that's exactly the problem with area allocators: What if your
> data no longer fits into your area?
You can create huge linear allocators. Bonus: if a bogus script exhausts
the allocator, the script will end.
> Abusing virtual memory is extremely hacky.

Not at all.

Cheers,

Andre



Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Stefan-2


Am 15.01.2020 um 23:35 schrieb Andre Leiradella:

>
> On 15/01/20 22:17, Stefan wrote:
>>
>>
>> Well, that's exactly the problem with area allocators: What if your
>> data no longer fits into your area?
> You can create huge linear allocators. Bonus: if a bogus script exhausts
> the allocator, the script will end.
>> Abusing virtual memory is extremely hacky.
>
> Not at all.
>
> Cheers,
>
> Andre

Well, it certainly is an interesting idea. But it has many drawbacks:
  * It only works for one state at a time (if I get your approach right)
  * There will be a lot of dead memory (Lua uses realloc shrinking often)
  * It assumes you have enough virtual memory (only x64)
  * vm.overcommit_memory or equivalent must be enabled
  * Users will complain about what top/Taks manager shows them
  * Memory use has an hard upper limit now; or you need to check it for
every allocation

Also, the current glibc allocator is *damn* fast. I tried to outsmart it
for a very narrow use case[1] and glibc malloc was still not that much
slower - while being a general allocator and threadsafe.

But thanks for the input anyways :)

Greetings

[1] https://github.com/evelance/slaballoc64

Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Sean Conner
It was thus said that the Great Stefan once stated:

>
>
> Am 15.01.2020 um 23:35 schrieb Andre Leiradella:
> >
> >On 15/01/20 22:17, Stefan wrote:
> >>
> >>
> >>Well, that's exactly the problem with area allocators: What if your
> >>data no longer fits into your area?
> >You can create huge linear allocators. Bonus: if a bogus script exhausts
> >the allocator, the script will end.
> >>Abusing virtual memory is extremely hacky.
> >
> >Not at all.
> >
> >Cheers,
> >
> >Andre
>
> Well, it certainly is an interesting idea. But it has many drawbacks:
>  * It only works for one state at a time (if I get your approach right)

  I didn't get that impression at all.  lua_newstate() takes a allocation
function and a pointer to user data (it's luaL_newstate() that uses
malloc()/realloc()/free()).  It's simple enough to say, give 5M to each
state (or 50M or whatever limit you want).

>  * There will be a lot of dead memory (Lua uses realloc shrinking often)

  Andre did mention it was for short-lived scripts, and not for something
that runs for a long time (hours?  days?  For ever?)

>  * It assumes you have enough virtual memory (only x64)
>  * vm.overcommit_memory or equivalent must be enabled
>  * Users will complain about what top/Taks manager shows them
>  * Memory use has an hard upper limit now; or you need to check it for
> every allocation

  It sounds like a hard upper limit.  And there's *always* a hard upper
limit, even with VM.  It's just that in most cases, you'll never hit it.

  I can see this method working for the right work case.

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Coda Highland
In reply to this post by Stefan-2
On Wed, Jan 15, 2020 at 3:18 PM Stefan <[hidden email]> wrote:
But with Lua many states aren't a problem and the simpler, more secure
and scalable Apache model can be used! And it is beginner-friendly!

If by "Apache model" you mean plugins loaded into the webserver that spin up separate script processors for every view, that is not simpler, nor more secure, nor more scalable.

If you're referring to the CGI model where each view is handled by a separate heavyweight process, that is indeed very simple and very secure, but it is notorious for its poor scaling. Any model that involves initializing a new interpreter for every request is going to have poor scaling because you have to reinitialize the application state for each page view. The problem isn't an issue of HAVING many states; the problem is in STARTING many states.

You claim that the NodeJS model is bad for scalability without "ugly hacks" -- this is completely false. It's true that there is a security risk involved in the app server model because of the shared memory space, but the app server model is the best possible model for scalability. This is because it allows for state to persist across requests without needing to serialize it and deserialize it with every request, and you don't have to reinitialize the entire application every time. There's a reason that EVERY major high-scalability service uses the app server model, whether Java, C#, Python, Javascript, or whatever.

The "ugly hacks" you decry are what you have to do ANYWAY in order to scale past what can be handled by a single machine, regardless of what underlying model you're using.

Lua's lightweight states do work very nicely with the app server model. You can create a new state for every session, providing some level of isolation between users while maintaining persistent state. I've not done this with Lua before so I'm not familiar with any of the existing tools, but I know how I'd approach it if I were building such a thing from scratch.

/s/ Adam
Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Stefan-2
In reply to this post by Sean Conner
Hello,

>>   * It only works for one state at a time (if I get your approach right)
>
>    I didn't get that impression at all.  lua_newstate() takes a allocation
> function and a pointer to user data (it's luaL_newstate() that uses
> malloc()/realloc()/free()).  It's simple enough to say, give 5M to each
> state (or 50M or whatever limit you want).

Yes, but you give Lua pointers to *your buffer*. The already initialized
state that gets stored in the background only works with exactly this
buffer and you cannot copy it into another buffer.
Therefore, if you want to have two Lua states, you need to do the
initialization + backup trick twice.

>
>>   * There will be a lot of dead memory (Lua uses realloc shrinking often)
>
>    Andre did mention it was for short-lived scripts, and not for something
> that runs for a long time (hours?  days?  For ever?)

Yes it will work well with scripts that are short lived, don't use
much data and get destroyed and re-run a lot. This is just a very
narrow use-case.

>
>>   * It assumes you have enough virtual memory (only x64)
>>   * vm.overcommit_memory or equivalent must be enabled
>>   * Users will complain about what top/Taks manager shows them
>>   * Memory use has an hard upper limit now; or you need to check it for
>> every allocation
>
>    It sounds like a hard upper limit.  And there's *always* a hard upper
> limit, even with VM.  It's just that in most cases, you'll never hit it.
>
>    I can see this method working for the right work case.
>
>    -spc

With the default malloc, this upper limit is your main memory (+ swap
partition if you have that). But with an arena allocator the upper limit
would be  50MB since realloc/mremap may copy your area to another place,
which cannot be handled by the area allocator.

Greetings

Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Pierre Chapuis
In reply to this post by Coda Highland
On Thu, Jan 16, 2020, at 00:30, Coda Highland wrote:
If you're referring to the CGI model where each view is handled by a separate heavyweight process, that is indeed very simple and very secure, but it is notorious for its poor scaling. Any model that involves initializing a new interpreter for every request is going to have poor scaling because you have to reinitialize the application state for each page view. The problem isn't an issue of HAVING many states; the problem is in STARTING many states.

Scalability is relative, but for many applications the performance of Lua CGI is good enough, thanks to the low interpreter startup time. This characteristic is shared with Perl, but not Python or Ruby for instance.

I have written Lua CGI code that could easily serve hundreds of dynamic pages per second on a single machine. The main issue is not really starting processes, it is opening connections to the database if you use one. This can be worked around by using things such as SQL Relay (but I never needed it myself).

Of course, if you need to serve more dynamic pages than that, it's probably better to choose a different model. But for things like personal and small business websites, it still works very well.

-- 
Pierre Chapuis
Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Andre Leiradella-2
In reply to this post by Sean Conner

On 15/01/20 23:22, Sean Conner wrote:
Well, it certainly is an interesting idea. But it has many drawbacks:
  * It only works for one state at a time (if I get your approach right)
   I didn't get that impression at all.  lua_newstate() takes a allocation
function and a pointer to user data (it's luaL_newstate() that uses
malloc()/realloc()/free()).  It's simple enough to say, give 5M to each
state (or 50M or whatever limit you want).


Exactly. Create one linear allocator per Lua state.



  * There will be a lot of dead memory (Lua uses realloc shrinking often)
   Andre did mention it was for short-lived scripts, and not for something
that runs for a long time (hours?  days?  For ever?)


That was my use case. In my implementation, I didn't allocate new memory when the block was shrinking, in this case I just updated the block header with the new size.

I've also added dlmalloc [1] on top of the linear allocator, dlmalloc would request new memory blocks from the linear allocator whenever it couldn't satisfy an allocation request. You still have a hard limit on the maximum memory for each state, but with 64-bit OSes you can in theory have 4 billion+ 4 GiB linear allocators which should be enough for everyone (TM). Both the Lua and the dlmalloc states were reset with memcpy at the end of script execution, so it wasn't necessary to event close the state to free up memory.

Cheers,

Andre


[1] http://gee.cs.oswego.edu/dl/html/malloc.html

Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Stefan-2
Am 16.01.2020 um 13:03 schrieb Andre Leiradella:

>
> That was my use case. In my implementation, I didn't allocate new memory
> when the block was shrinking, in this case I just updated the block
> header with the new size.
>
> I've also added dlmalloc [1] on top of the linear allocator, dlmalloc
> would request new memory blocks from the linear allocator whenever it
> couldn't satisfy an allocation request. You still have a hard limit on
> the maximum memory for each state, but with 64-bit OSes you can in
> theory have 4 billion+ 4 GiB linear allocators which should be enough
> for everyone (TM). Both the Lua and the dlmalloc states were reset with
> memcpy at the end of script execution, so it wasn't necessary to event
> close the state to free up memory.

That sounds interesting! Do you want to share the sources?

Btw. the 4 billion allocators won't work in practice. The paging
hardware of current processors seems to only support 48 bits
instead of the full 64 bit, so in fact you are limited to
2 ^ 48 / ( 4 * 1024 ^ 3 ) = 65536 states with 4GiB each.
Although this is still a lot, it can be exhausted.

Greetings

Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Marc Balmer
In reply to this post by Lorenzo Donati-3


> Am 15.01.2020 um 16:59 schrieb Lorenzo Donati <[hidden email]>:
>
> Hi all!
>
> On a recent thread ("Dead Batteries" ) I argued that Lua lost terrain over Python.
>
> I won't bother anyone repeating what I already said, but I stumbled on this article which may explain something:
>
> https://www.techrepublic.com/article/python-is-eating-the-world-how-one-developers-side-project-became-the-hottest-programming-language-on-the-planet/
>
> Moreover the recent StackOverflow Developer Survey Results show
> Python as one of the most known language among programmers (even among professionals), while Lua doesn't even make it on the list.
>
> https://insights.stackoverflow.com/survey/2019#technology
>
> https://insights.stackoverflow.com/survey/2019#technology-_-most-loved-dreaded-and-wanted-languages
>
> https://insights.stackoverflow.com/survey/2019#technology-_-what-languages-are-associated-with-the-highest-salaries-worldwide
>
> I thought it could be useful info to share.
>
> I'm just sad because while still loving Lua, it just seems that there is no point in investing further time in developing in Lua, unless one has a very specific need.

I usually pick my tools to fit the work at hand, not by means of what is currently en vogue...

>
> I won't comment further, since I'm bored to death to beat dead horses.
>
> Cheers!
>
> -- Lorenzo
>


Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Andre Leiradella-2
In reply to this post by Stefan-2

On 16/01/20 13:30, Stefan wrote:

> Am 16.01.2020 um 13:03 schrieb Andre Leiradella:
>>
>> That was my use case. In my implementation, I didn't allocate new
>> memory when the block was shrinking, in this case I just updated the
>> block header with the new size.
>>
>> I've also added dlmalloc [1] on top of the linear allocator, dlmalloc
>> would request new memory blocks from the linear allocator whenever it
>> couldn't satisfy an allocation request. You still have a hard limit
>> on the maximum memory for each state, but with 64-bit OSes you can in
>> theory have 4 billion+ 4 GiB linear allocators which should be enough
>> for everyone (TM). Both the Lua and the dlmalloc states were reset
>> with memcpy at the end of script execution, so it wasn't necessary to
>> event close the state to free up memory.
>
> That sounds interesting! Do you want to share the sources?
Unfortunately I lost the source code during a PC migration. It wasn't
particularly difficult to implement, but I remember having to tweak
dlmalloc a bit to make it request more memory from its backing linear
allocator instead of using sbrk or mmap.
>
> Btw. the 4 billion allocators won't work in practice. The paging
> hardware of current processors seems to only support 48 bits
> instead of the full 64 bit, so in fact you are limited to
> 2 ^ 48 / ( 4 * 1024 ^ 3 ) = 65536 states with 4GiB each.
> Although this is still a lot, it can be exhausted.

Aren't those 48 bits the upper bits of the virtual address, allowing
2^48 pages of 64 KiB each? I think that's the case with AArch64.

>
> Greetings
>
Cheers,

Andre



Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Stefan-2
Am 17.01.2020 um 15:24 schrieb Andre Leiradella:>
 > Aren't those 48 bits the upper bits of the virtual address, allowing
 > 2^48 pages of 64 KiB each? I think that's the case with AArch64.
 >
No, it is the lower part[1]. Under Linux, the userspace part is
even smaller, "only" 128 TB[2], allowing 32768 states with 4GB.
But this should still be enough for anything... it really is an
interesting approach!
It could probably be used to build a simple and efficient snapshot +
rollback algorithm for a whole program.

[1] https://en.wikichip.org/wiki/arm/aarch64 "MMU"
[2] https://www.kernel.org/doc/Documentation/x86/x86_64/mm.txt

Reply | Threaded
Open this post in threaded view
|

was: Some info about Python: Back to batteries

Steve Litt
In reply to this post by Pierre Chapuis
On Thu, 16 Jan 2020 09:58:55 +0100
"Pierre Chapuis" <[hidden email]> wrote:


> I have written Lua CGI code that could easily serve hundreds of
> dynamic pages per second on a single machine.

Here it is. We're asked "why should we enable Lua as a general purpose
language?" Pierre used Lua's natural speed to make it a better CGI
language than PP&R. Imagine how many more of us would do this if Lua
had tested and curated packages for DOM, XML, and the like, so that you
never have to spend time evaluating competing softwares, and whatever
software you write is likely to have its dependencies available
anywhere the Lua Standard Library exists.

The existence of Python doesn't make Lua useless as a general purpose
language.
 
SteveT

Steve Litt
January 2020 featured book: Troubleshooting: Just the Facts
http://www.troubleshooters.com/tjust

Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Lorenzo Donati-3
In reply to this post by Marc Balmer
On 17/01/2020 09:03, Marc Balmer wrote:

>
>
>> Am 15.01.2020 um 16:59 schrieb Lorenzo Donati
>> <[hidden email]>:
>>
>> Hi all!
>>
>> On a recent thread ("Dead Batteries" ) I argued that Lua lost
>> terrain over Python.
>>
[ snip]

>
> I usually pick my tools to fit the work at hand, not by means of what
> is currently en vogue...
>

Exactly!

And what I reported seemed to indicate that a great majority in the
world find Python more apt at solving their problems than Lua.

Unless you think all those Python programmers/users are funny people
that choose a language using the criteria they would when choosing a
fancy new scarf.

Python is no longer a "new kid on the block", which may look cool to
someone. It has been shunned by many (me too) in the years for various
reasons. But now it has reached a critical mass of users that allowed it
to show all its capabilities and "self market" itself.

One could argue ad infinitum, but the hard data show that more and more
people find Python solves their problems "better" [1] than Lua. And Lua
is losing user base on a daily basis (at least in relative numbers).

[1] "better" meaning: more readily, more thoroughly, more rapidly, more
whatever. Heck! They might not even know Lua exists for what matters.

One of the strength of C that makes it so widespread and a good
investment is the huge codebase/userbase, despite being quite ugly and
dangerous. But when you learn it you know that in 10 years time your
competences won't go down the drain because some new hyped language will
substitute it (not even C++ has succeeded in one of its initial goal to
be the "next C").

In the same way, IMO, I think Python now is here to stay.

One may argue Python is a worse language than Lua (as I do, wrt. the
core language and its implementation). But as "fitting the work at hand"
Lua is becoming irrelevant.

Ask yourself: would you (a company) prefer a Python solution that is
bloated and slow, but still "fast enough", well supported (because
"everyone" know Python) and that is created in 10 days, or an equivalent
Lua solution that takes 2 month to be created and that cannot be
supported long term because you don't even know where to find an
alternative Lua dev in your area would the need arise? Not to mention
the level of support of libraries.

Again, Lua is better at some things, but that doesn't make it a "good"
general purpose PL (and being Turing complete doesn't count in practice)
if people don't /use/ it as a GP-PL.

And, btw, this huge user base means that Python use is growing even in
areas where Lua definitely shine: embedding. I find more and more
projects that use Python as embedded language. I can't say how difficult
is to embed Python, but I'm sure that if the demand grows, someone in
the Python community will find a way to make it simpler.

Then Lua would lose another edge.

I really think Lua Team should reconsider some of their positions on the
development model of Lua, otherwise they risk a slow descent into
irrelevance in the PL landscape in a couple of years (except maybe some
very limited niches).

Its an hard reality of open source projects, their success depend
heavily on user base.

Cheers!

-- Lorenzo





Reply | Threaded
Open this post in threaded view
|

Re: [mildly OT] Some info about Python

Steve Litt
On Sat, 18 Jan 2020 16:05:29 +0100
Lorenzo Donati <[hidden email]> wrote:

> On 17/01/2020 09:03, Marc Balmer wrote:
> >
> >  
> >> Am 15.01.2020 um 16:59 schrieb Lorenzo Donati
> >> <[hidden email]>:
> >>
> >> Hi all!
> >>
> >> On a recent thread ("Dead Batteries" ) I argued that Lua lost
> >> terrain over Python.
> >>  
> [ snip]
>
> >
> > I usually pick my tools to fit the work at hand, not by means of
> > what is currently en vogue...
> >  
>
> Exactly!
>
[snip]

> One could argue ad infinitum, but the hard data show that more and
> more people find Python solves their problems "better" [1] than Lua.
> And Lua is losing user base on a daily basis (at least in relative
> numbers).
>
> [1] "better" meaning: more readily, more thoroughly, more rapidly,
> more whatever. Heck! They might not even know Lua exists for what
> matters.

The preceding assertion, which I agree with in a limited context, is,
in my opinion, the most important in this email. I'd like to paraphrase
the preceding assertion as "If you start a project in Python, you're
more likely to finish it in Python as opposed to if you'd started it in
Lua and expect to finish it in Lua."

More on that later...


[snip several similar assertions and arguments]
 
> Again, Lua is better at some things, but that doesn't make it a
> "good" general purpose PL (and being Turing complete doesn't count in
> practice) if people don't /use/ it as a GP-PL.

[snip]

> I really think Lua Team should reconsider some of their positions on
> the development model of Lua, otherwise they risk a slow descent into
> irrelevance in the PL landscape in a couple of years (except maybe
> some very limited niches).

This thread was spawned by the "Dead Batteries", but this particular
post takes no position on "batteries" (a well curated standard Lua
library). This email could be taken one of two ways:

1) Python's more likely to solve your problem because and only because
   Python has "batteries" and Lua doesn't, so Lua batteries are
   necessary and sufficient to make Lua preferable for most programming
   than Python, so Lua should acquire batteries.

2) Python's more likely to solve your problem because and only because
   Python has "batteries" and Lua doesn't, so because Python's more
   likely to solve your problem, Lua should confine itself to embedded,
   where Python just can't go, and not bother with batteries.

In my opinion #2 is circular thinking non-logic.

By the way, it's not just Python. I've been studying Tcl for the last
ten days. Tcl has "batteries" galore, they're in one standard library
package that's separate from the Tcl package itself, and so far I've
found they all work perfectly except for the Huddle object
(serialization). Even though I just started Tcl, because of batteries,
I'd be more confident of starting a difficult project in Tcl than in
Lua.

IMHO in this day and age, you need a language that can easily and
reliably and without experimentation interact with XML, HTML(5), Xhtml,
databases, JSON, YAML, DOM, sockets, IPC, complex numbers, and probably
twenty other things I haven't thought of yet. And with small appliance
devices doing ever more mainstream things, some of these batteries will
probably become more important, in embedded programming, in the future.

So, in my opinion, Lua would greatly benefit from an official, blessed,
curation of libraries available both in a single package for when disk
space isn't at a premium, and a-la-carte for embedded.

SteveT

Steve Litt
January 2020 featured book: Troubleshooting: Just the Facts
http://www.troubleshooters.com/tjust

1234 ... 7