The Lua interpreter and large scripts

classic Classic list List threaded Threaded
47 messages Options
123
Reply | Threaded
Open this post in threaded view
|

The Lua interpreter and large scripts

Sean Conner

  I've noticed some strange behavior with the Lua interpreter [1].  I'm
trying to load two scripts, one is 1101 bytes in size (just dumps the
contents of a table) and the other one is 83414279 bytes in size (yes,
80M---generated by a C program and contains a *really* large table of around
80,000 items [2]).

[spc]lucy:/tmp/lua>ll
total 81548
-rw-------  1 spc spc 83414279 Nov 10 05:30 default.lua
-rw-r--r--  1 spc spc     1101 Nov 10 05:30 show.lua

  Okay.  

[spc]lucy:/tmp/lua>time lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> dofile("show.lua")
> dofile("default.lua")
> os.exit()

real    0m16.130s
user    0m6.311s
sys     0m0.365s
[spc]lucy:/tmp/lua>time lua -i default.lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> dofile("show.lua")
> os.exit()

real    0m12.354s # file is probably cached at this point
user    0m6.293s
sys     0m0.413s
[spc]lucy:/tmp/lua>time lua -i show.lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> dofile("default.lua")
> os.exit()

real    5m49.052s # okay ... what happened?
user    5m43.279s
sys     0m0.577s
[spc]lucy:/tmp/lua>

  I can work around this, but I'm very curious as to why it's happening.

  -spc (Who finds this very odd behavior ... )

[1] Linux 2.6, dual core Pentium-4 2.6GHz with a 1G of RAM

[2] Basically, an index of all the headers from over 80,000 messages
        from 2,300 files (of which the names, sizes and timestamps are
        saved as well in the table).

Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Daniel Stephens
Does show.lua call os.exit() ?

Sean Conner wrote:

>   I've noticed some strange behavior with the Lua interpreter [1].  I'm
> trying to load two scripts, one is 1101 bytes in size (just dumps the
> contents of a table) and the other one is 83414279 bytes in size (yes,
> 80M---generated by a C program and contains a *really* large table of around
> 80,000 items [2]).
>
> [spc]lucy:/tmp/lua>ll
> total 81548
> -rw-------  1 spc spc 83414279 Nov 10 05:30 default.lua
> -rw-r--r--  1 spc spc     1101 Nov 10 05:30 show.lua
>
>   Okay.  
>
> [spc]lucy:/tmp/lua>time lua
> Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
>  
>> dofile("show.lua")
>> dofile("default.lua")
>> os.exit()
>>    
>
> real    0m16.130s
> user    0m6.311s
> sys     0m0.365s
> [spc]lucy:/tmp/lua>time lua -i default.lua
> Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
>  
>> dofile("show.lua")
>> os.exit()
>>    
>
> real    0m12.354s # file is probably cached at this point
> user    0m6.293s
> sys     0m0.413s
> [spc]lucy:/tmp/lua>time lua -i show.lua
> Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
>  
>> dofile("default.lua")
>> os.exit()
>>    
>
> real    5m49.052s # okay ... what happened?
> user    5m43.279s
> sys     0m0.577s
> [spc]lucy:/tmp/lua>
>
>   I can work around this, but I'm very curious as to why it's happening.
>
>   -spc (Who finds this very odd behavior ... )
>
> [1] Linux 2.6, dual core Pentium-4 2.6GHz with a 1G of RAM
>
> [2] Basically, an index of all the headers from over 80,000 messages
> from 2,300 files (of which the names, sizes and timestamps are
> saved as well in the table).
>
>
>  

Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Sean Conner
It was thus said that the Great Daniel Stephens once stated:
> Does show.lua call os.exit() ?

  No it doesn't.  What you see is what I typed into the interpreter.
Besides, if show.lua called os.exit(), then the last example wouldn't have
taken nearly six minutes to run, since I load show.lua first.

  -spc (I can work around the six minute problem, I'm just curious as to
        what could cause it ... )



Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Daniel Stephens
According to your email you say:

show.lua -> default.lua    16 seconds
show.lua                         12 seconds
default.lua                       6 minutes

Which makes me think that in the FIRST example, default.lua isn't
running at all.

Daniel

Sean Conner wrote:

> It was thus said that the Great Daniel Stephens once stated:
>  
>> Does show.lua call os.exit() ?
>>    
>
>   No it doesn't.  What you see is what I typed into the interpreter.
> Besides, if show.lua called os.exit(), then the last example wouldn't have
> taken nearly six minutes to run, since I load show.lua first.
>
>   -spc (I can work around the six minute problem, I'm just curious as to
> what could cause it ... )
>
>
>
>
>  

Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Sean Conner
It was thus said that the Great Daniel Stephens once stated:
> According to your email you say:
>
> show.lua -> default.lua    16 seconds
> show.lua                         12 seconds
> default.lua                       6 minutes
>
> Which makes me think that in the FIRST example, default.lua isn't
> running at all.

  First example was running the Lua interpreter, and typing
        dofile("show.lua")
        dofile("default.lua")

  Second example was running the Lua interpreter with the "-i" option
loading up default.lua, then typing
        dofile("show.lua")

  Third example was running the Lua interpreter with the "-i" option loading
up show.lua, then typing
        dofile("default.lua")

  It was the third example that took six minutes.

  -spc

Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Mike Pall-9
Sean Conner wrote:
>   Third example was running the Lua interpreter with the "-i" option loading
> up show.lua, then typing
> dofile("default.lua")
>
>   It was the third example that took six minutes.

The "-i" option loads readline which as a side-effect initializes
the NLS locale for the current process from the environment
variables.

This may slow down all conversions of strings to numbers, numbers
to strings and string comparisons. Looks like it's 30x slower in
your case.

Try os.setlocale("C") before the dofile().

--Mike
Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Luiz Henrique de Figueiredo
> The "-i" option loads readline which as a side-effect initializes
> the NLS locale for the current process from the environment variables.

If available, readline is compiled into the interpreter. You don't need
to use -i to activate it.  Do you mean that when you do use interactive
input then a function from the readline library gets called and only then
the OS loads the readline library? And so readline is not loaded at all
when running the interpreter in batch?
--lhf
Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Sean Conner
In reply to this post by Mike Pall-9
It was thus said that the Great Mike Pall once stated:

> Sean Conner wrote:
> >   Third example was running the Lua interpreter with the "-i" option loading
> > up show.lua, then typing
> > dofile("default.lua")
> >
> >   It was the third example that took six minutes.
>
> The "-i" option loads readline which as a side-effect initializes
> the NLS locale for the current process from the environment
> variables.
>
> This may slow down all conversions of strings to numbers, numbers
> to strings and string comparisons. Looks like it's 30x slower in
> your case.
>
> Try os.setlocale("C") before the dofile().

  I did that.  Still took six minutes.  I even set the locale from the
command line and tried, and it took six minutes.

  I then did a quick look at the Lua interpeter (lua.c).  The Lua function
dofile() does:

        int n = lua_gettop(L);
        luaL_loadfile(L,name);
        lua_call(L,0,LUA_MULTRET);
        return lua_gettop(L) - n;

whereas the "-i" option appears to do:

        int base = lua_gettop(L) - narg
        luaL_loadfile(L,name);
        lua_pushcfunction(L, traceback);
        lua_insert(L, base);
        lua_pcall(L, narg, (clear ? 0 : LUA_MULTRET), base);
        lua_remove(L, base);
       
  I'm still not sure why loading a small file via the "-i" option before
loading the 80M script via dofile() takes so darned long.

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Mike Pall-9
In reply to this post by Luiz Henrique de Figueiredo
Luiz Henrique de Figueiredo wrote:
> > The "-i" option loads readline which as a side-effect initializes
> > the NLS locale for the current process from the environment variables.
>
> If available, readline is compiled into the interpreter. You don't need
> to use -i to activate it.  Do you mean that when you do use interactive
> input then a function from the readline library gets called and only then
> the OS loads the readline library? And so readline is not loaded at all
> when running the interpreter in batch?

I should have been more precise. The shared library is always
loaded (which is a waste without -i). The first time a readline
function is called (only with -i), it's initialized and it sets
the locale.

You can verify that by comparing the output of these two commands:
  strace -e open lua -v
  strace -e open lua -i </dev/null
If you have the environment variables LANG or LC_* set, the output
of the 2nd command shows it opens lots of locale related files.

--Mike
Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Sean Conner
It was thus said that the Great Mike Pall once stated:

> Luiz Henrique de Figueiredo wrote:
> > > The "-i" option loads readline which as a side-effect initializes
> > > the NLS locale for the current process from the environment variables.
> >
> > If available, readline is compiled into the interpreter. You don't need
> > to use -i to activate it.  Do you mean that when you do use interactive
> > input then a function from the readline library gets called and only then
> > the OS loads the readline library? And so readline is not loaded at all
> > when running the interpreter in batch?
>
> I should have been more precise. The shared library is always
> loaded (which is a waste without -i). The first time a readline
> function is called (only with -i), it's initialized and it sets
> the locale.

  The "-i" option is used to load a script and continue into interactive
mode.  If you just run "lua" you still end up in interactive mode.  

  When I run "strace -e open lua" I get the following:

open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/tls/libm.so.6", O_RDONLY)    = 3
open("/lib/libdl.so.2", O_RDONLY)       = 3
open("/usr/lib/libreadline.so.4", O_RDONLY) = 3
open("/usr/lib/libhistory.so.4", O_RDONLY) = 3
open("/usr/lib/libncurses.so.5", O_RDONLY) = 3
open("/lib/tls/libc.so.6", O_RDONLY)    = 3
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
open("/usr/share/terminfo/x/xterm", O_RDONLY) = 3
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
open("/etc/inputrc", O_RDONLY)          = 3
open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = 3
>

  That final ">" is the Lua prompt.  Now, let's load a script with "strace
-e open lua -i default.lua" ("default.lua" being the 80M script---this takes
only 15 seconds):

open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/tls/libm.so.6", O_RDONLY)    = 3
open("/lib/libdl.so.2", O_RDONLY)       = 3
open("/usr/lib/libreadline.so.4", O_RDONLY) = 3
open("/usr/lib/libhistory.so.4", O_RDONLY) = 3
open("/usr/lib/libncurses.so.5", O_RDONLY) = 3
open("/lib/tls/libc.so.6", O_RDONLY)    = 3
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
open("default.lua", O_RDONLY)           = 3
open("/usr/share/terminfo/x/xterm", O_RDONLY) = 3
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
open("/etc/inputrc", O_RDONLY)          = 3
open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = 3
>

  Hmm ... curious.  The script is loaded first, then the locale stuff.  What
has me really bugged is when I do this:

[spc]lucy:/tmp/lua>lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> dofile("show.lua") --NOTE!  I'm typing this
> dofile("default.lua") --this too!
>

it only takes less than 15 seconds.  Yet doing:

[spc]lucy:/tmp/lua>lua -i show.lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> dofile("default.lua") --NOTE!  I'm typing this
>

takes six minutes.  Other than the slight differences in how the script is
loaded via "-i" on the command line and dofile() from within the
interpreter, I don't why one way should take 15 seconds and the other way 6
minutes.

  So I decided to profile Lua under the various conditions.  First, "lua -i
show.lua / dofile('default.lua')":

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total          
 time   seconds   seconds    calls   s/call   s/call  name    
 97.71    314.31   314.31  2888775     0.00     0.00  luaS_newlstr
  0.34    315.40     1.09  8932847     0.00     0.00  luaH_getstr
  0.30    316.37     0.97  2154477     0.00     0.00  read_string
  0.27    317.23     0.85 10442038     0.00     0.00  llex
  0.13    317.63     0.41 67142148     0.00     0.00  save
  0.08    317.89     0.26  3264403     0.00     0.00  addk
  0.07    318.10     0.21  3891495     0.00     0.00  newkey
        [ snip ]

Doing any other combination ("lua -i default.lua / dofile('show.lua')" or
"lua / dofile('show.lua') dofile('default.lua')" or "lua /
dofile('default.lua') dofile('show.lua')" results in the following profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total          
 time   seconds   seconds    calls   s/call   s/call  name    
 17.75      1.04     1.04  2154477     0.00     0.00  read_string
  9.47      1.59     0.56 10442038     0.00     0.00  llex
  9.39      2.15     0.55  2888775     0.00     0.00  luaS_newlstr
  7.34      2.58     0.43  8932844     0.00     0.00  luaH_getstr
  7.00      2.98     0.41 67142145     0.00     0.00  save
  3.24      3.17     0.19        4     0.05     0.22  luaV_execute
  2.82      3.34     0.17   704940     0.00     0.00  setnodevector
  2.73      3.50     0.16  5422300     0.00     0.00  mainposition
  2.30      3.63     0.14  3891495     0.00     0.00  newkey
        [ snip ]

And yes, all three of those return *identical* profiling results.  And I can
verify that in all four cases, the entirety of the 80m script is loaded.  

  -spc (Curiouser and curiouser ... )

Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Sean Conner
In reply to this post by Mike Pall-9
It was thus said that the Great Mike Pall once stated:

> Luiz Henrique de Figueiredo wrote:
> > > The "-i" option loads readline which as a side-effect initializes
> > > the NLS locale for the current process from the environment variables.
> >
> > If available, readline is compiled into the interpreter. You don't need
> > to use -i to activate it.  Do you mean that when you do use interactive
> > input then a function from the readline library gets called and only then
> > the OS loads the readline library? And so readline is not loaded at all
> > when running the interpreter in batch?
>
> I should have been more precise. The shared library is always
> loaded (which is a waste without -i). The first time a readline
> function is called (only with -i), it's initialized and it sets
> the locale.
>
> You can verify that by comparing the output of these two commands:
>   strace -e open lua -v
>   strace -e open lua -i </dev/null
> If you have the environment variables LANG or LC_* set, the output
> of the 2nd command shows it opens lots of locale related files.

  I also did a generic build of Lua, to avoid the whole readline issue and
the results are the same as before:  "lua -i show.lua /
dofile('default.lua')" takes 6 minutes, other combinations take 15 seconds
or less.

  -spc (Just in case anyone asks)

Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Mike Pall-9
In reply to this post by Sean Conner
Sean Conner wrote:
>  97.71    314.31   314.31  2888775     0.00     0.00  luaS_newlstr
>   9.39      2.15     0.55  2888775     0.00     0.00  luaS_newlstr

Umm, can you get a line-by-line profile? If not, then maybe
compile lstring.c with -O2 -fno-inline and see what pops up.

One explanation for this difference would be excessive string hash
collisions. But I don't know how that could be triggered by the
interactive mode (well, it creates a few strings).

You may want to measure the length of the chain in luaS_newlstr
and print a message if it exceeds (say) 50. The chain length
should be somwhere between 1 and 3 on average.

You could also trace the calls to luaS_resize. The string table
usually grows and only occasionally shrinks. But there shouldn't
be continuous resizing.

--Mike
Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Sean Conner
It was thus said that the Great Mike Pall once stated:

> Sean Conner wrote:
> >  97.71    314.31   314.31  2888775     0.00     0.00  luaS_newlstr
> >   9.39      2.15     0.55  2888775     0.00     0.00  luaS_newlstr
>
> Umm, can you get a line-by-line profile? If not, then maybe
> compile lstring.c with -O2 -fno-inline and see what pops up.
>
> One explanation for this difference would be excessive string hash
> collisions. But I don't know how that could be triggered by the
> interactive mode (well, it creates a few strings).
>
> You may want to measure the length of the chain in luaS_newlstr
> and print a message if it exceeds (say) 50. The chain length
> should be somwhere between 1 and 3 on average.
>
> You could also trace the calls to luaS_resize. The string table
> usually grows and only occasionally shrinks. But there shouldn't
> be continuous resizing.

  Hmm ... interesting.  In the six minute run, luaS_resize is called 469488
times, but in the other three cases, it's only called 15 times.  And in
looking over the profiler output, there are a lot of similarities, but some
very curious differences as well.  

Six minute run:

-----------------------------------------------
                0.00    0.00       1/469488      f_luaopen [136]
                0.14    0.00  469487/469488      newlstr [53]
[62]     0.0    0.14    0.00  469488         luaS_resize [62]
                0.00    0.00      12/2101456     luaM_realloc_ [76]
-----------------------------------------------

15 second run:

-----------------------------------------------
                0.01    0.00       1/15          f_luaopen [107]
                0.08    0.00      14/15          newlstr [49]
[56]     1.5    0.09    0.00      15         luaS_resize [56]
                0.00    0.00      30/2101448     luaM_realloc_ [86]
-----------------------------------------------

  -spc (Perhaps some pathological case I'm hitting?)

Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Mike Pall-9
Sean Conner wrote:
> -----------------------------------------------
>                 0.00    0.00       1/469488      f_luaopen [136]
>                 0.14    0.00  469487/469488      newlstr [53]
> [62]     0.0    0.14    0.00  469488         luaS_resize [62]
>                 0.00    0.00      12/2101456     luaM_realloc_ [76]
> -----------------------------------------------

Yes, that's an excessive number of calls to luaS_resize. But it
still spends very little time in that function and it only causes
12 actual resizes. So I think it's hitting this check:

  if (G(L)->gcstate == GCSsweepstring)
    return;  /* cannot resize during GC traverse */

What this probably means is that the string table needs to grow,
but can't, because the GC is stuck in the sweepstring phase. It
tries very often (on every string allocation), but never succeeds.
This way the hash chains grow without bounds, which seriously
destroys the performance.

The #1 question is now: why is the GC stuck? Because usually every
string allocation is accompanied with a GC check. And once the
threshold is reached, the GC is driven forward and should quickly
get out of the sweepstring phase.

I think luaX_newstring() in llex.c must be the culprit. You can
try to add a GC check just before the 'return ts' at line 123:
  luaC_checkGC(L);

If this solves the problem, then you've found a bug in Lua.
Congratulations -- it doesn't happen that often. :-)

--Mike
Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Sean Conner
It was thus said that the Great Mike Pall once stated:

> The #1 question is now: why is the GC stuck? Because usually every
> string allocation is accompanied with a GC check. And once the
> threshold is reached, the GC is driven forward and should quickly
> get out of the sweepstring phase.
>
> I think luaX_newstring() in llex.c must be the culprit. You can
> try to add a GC check just before the 'return ts' at line 123:
>   luaC_checkGC(L);
>
> If this solves the problem, then you've found a bug in Lua.
> Congratulations -- it doesn't happen that often. :-)

  I tried that, and nope.  Still takes six minutes.  

  -spc (And I'd like to thank you for your help so far 8-)

Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

KHMan
Sean Conner wrote:

> It was thus said that the Great Mike Pall once stated:
>> The #1 question is now: why is the GC stuck? Because usually every
>> string allocation is accompanied with a GC check. And once the
>> threshold is reached, the GC is driven forward and should quickly
>> get out of the sweepstring phase.
>>
>> I think luaX_newstring() in llex.c must be the culprit. You can
>> try to add a GC check just before the 'return ts' at line 123:
>>   luaC_checkGC(L);
>>
>> If this solves the problem, then you've found a bug in Lua.
>> Congratulations -- it doesn't happen that often. :-)
>
>   I tried that, and nope.  Still takes six minutes.  
>
>   -spc (And I'd like to thank you for your help so far 8-)

Any chance you can post some equivalent script files for the list
to hack on? Turns out this thingy might benefit if more people
have their hands on something they can run and debug.

--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia

Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Sean Conner
It was thus said that the Great KHMan once stated:
>
> Any chance you can post some equivalent script files for the list
> to hack on? Turns out this thingy might benefit if more people
> have their hands on something they can run and debug.

  I think I can do that, but it probably won't be until Friday when I get a
chance to randomize the data in the huge file (it's the headers of a ton of
email since 1991).  

  -spc



Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Luiz Henrique de Figueiredo
>   I think I can do that, but it probably won't be until Friday when I get a
> chance to randomize the data in the huge file (it's the headers of a ton of
> email since 1991).  

Changing the data will probably change the problem if it's related to hashing.
Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Roberto Ierusalimschy
In reply to this post by Mike Pall-9
> I think luaX_newstring() in llex.c must be the culprit. You can
> try to add a GC check just before the 'return ts' at line 123:
>   luaC_checkGC(L);
>
> If this solves the problem, then you've found a bug in Lua.
> Congratulations -- it doesn't happen that often. :-)

I guess this is a bug in Lua even if it does not solve this particular
problem. The idea of not calling the collector during a parse is that it
creates very little garbage. But the possibility of the collector being
held at a particular bad state is real.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: The Lua interpreter and large scripts

Roberto Ierusalimschy
In reply to this post by Sean Conner
> > The #1 question is now: why is the GC stuck? Because usually every
> > string allocation is accompanied with a GC check. And once the
> > threshold is reached, the GC is driven forward and should quickly
> > get out of the sweepstring phase.
> >
> > I think luaX_newstring() in llex.c must be the culprit. You can
> > try to add a GC check just before the 'return ts' at line 123:
> >   luaC_checkGC(L);
> >
> > If this solves the problem, then you've found a bug in Lua.
> > Congratulations -- it doesn't happen that often. :-)
>
>   I tried that, and nope.  Still takes six minutes.  

Nevertheless it would be useful to check Mike's theory. A simple (and
somewhat dirty ;) way to do it would be to add the following line
at the beginning of luaS_resize (just after variable definitions):

  fprintf(stderr, "gcs: %d strgc: %d size: %d nuse: %d bytes: %d tsh: %d\n",
        G(L)->gcstate, G(L)->sweepstrgc, G(L)->strt.size, G(L)->strt.nuse,
        G(L)->totalbytes, G(L)->GCthreshold);

(That will generate 469488 lines of output. But any chunk after the
first thousand lines could give us some clue of what is going one. You
can abort the run after the first few thousand lines.)

-- Roberto
123