How can a module intentionally fail to load?

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

How can a module intentionally fail to load?

Sean Conner

  I'm working on two modules, both related to libtls [1].  The first one is
just a C wrapper around libtls (which is done for the most part), and I
thought it would be nice to support as many versions as possible.

  The second module is a Lua module (or rather, a collection of modules)
that handle TLS (or TCP) connections via coroutines---each connection is
wrapped up in a coroutine and there's a main event loop that schedules
everything.  This one too, is mostly done.

  Enough of the background, now the problem.  The Lua module will only work
with libtls 2.5.0 (the earliest version that supports user-controlled
sockets).  It's not enough that, say, org.conman.tls exists, but unless it
was compiled against libtls 2.5.0 (or higher), it won't work.  I'm looking
for something like:

        -----[ my-tls.lua / loaded as a module ]----

        local tls = require "org.conman.tls"

        if tls.LIBRESSL_VERSION < 0x2050000f then
          ??? there's no point in continuing with loading
          ??? this module, because libtls will not let us
          ??? control the socket.
        end

  How to I abort the module?  There's nothing I can find in the
documentation to deal with this, unless I'm missing something.

  -spc

[1] I started this with the latest version and got it working.  Now I'm
        working backwards, supporting earlier verions of the library.

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

nobody
On 2018-08-07 04:32, Sean Conner wrote:>

> -----[ my-tls.lua / loaded as a module ]----
>
> local tls = require "org.conman.tls"
>
> if tls.LIBRESSL_VERSION < 0x2050000f then
>  ??? there's no point in continuing with loading
>  ??? this module, because libtls will not let us
>  ??? control the socket.
> end
>
>    How to I abort the module?  There's nothing I can find in the
> documentation to deal with this, unless I'm missing something.

Does just throwing an error or assert()ing the test work?

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Sean Conner
It was thus said that the Great nobody once stated:

> On 2018-08-07 04:32, Sean Conner wrote:>
> > -----[ my-tls.lua / loaded as a module ]----
> >
> > local tls = require "org.conman.tls"
> >
> > if tls.LIBRESSL_VERSION < 0x2050000f then
> >  ??? there's no point in continuing with loading
> >  ??? this module, because libtls will not let us
> >  ??? control the socket.
> > end
> >
> >   How to I abort the module?  There's nothing I can find in the
> >documentation to deal with this, unless I'm missing something.
>
> Does just throwing an error or assert()ing the test work?

  It kind of works.  I get:

> tls = require "org.conman.nfl.tls"
/usr/local/share/lua/5.1/org/conman/nfl/tls.lua:41: too old a version of TLS
stack traceback:
        [C]: in function 'assert'
        /usr/local/share/lua/5.1/org/conman/nfl/tls.lua:41: in main chunk
        [C]: in function 'require'
        stdin:1: in main chunk
        [C]: ?
>

But ...

> print(package.loaded['org.conman.nfl.tls'])
userdata: 0x8064634
>

I think the userdata is the .so file (because each userdata in the TLS
module has a __tostring() method assigned to it), which is an unexpected
result.

I suppose it's good enough.

By the way, when it works,

> tls = require "org.conman.nfl.tls"
> print(package.loaded['org.conman.nfl.tls'])
table: 0x843fc18
>

  -spc

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Daurnimator
On 7 August 2018 at 14:19, Sean Conner <[hidden email]> wrote:
> But ...
>
>> print(package.loaded['org.conman.nfl.tls'])
> userdata: 0x8064634
>>
>
> I think the userdata is the .so file (because each userdata in the TLS
> module has a __tostring() method assigned to it), which is an unexpected
> result.

Nope. It's a hack by lua5.1 to detect require loops. It was removed in
5.2 (yet another reason to upgrade!)

$ lua5.1 -e 'package.preload.foo=function() error("foo") end;
pcall(require,"foo"); print(package.loaded.foo)'
userdata: 0x4216d4
$ lua5.2 -e 'package.preload.foo=function() error("foo") end;
pcall(require,"foo"); print(package.loaded.foo)'
nil

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Dirk Laurie-2
In reply to this post by Sean Conner
2018-08-07 4:32 GMT+02:00 Sean Conner <[hidden email]>:

>
>         -----[ my-tls.lua / loaded as a module ]----
>
>         local tls = require "org.conman.tls"
>
>         if tls.LIBRESSL_VERSION < 0x2050000f then
>           ??? there's no point in continuing with loading
>           ??? this module, because libtls will not let us
>           ??? control the socket.
>         end
>
>   How to I abort the module?  There's nothing I can find in the
> documentation to deal with this, unless I'm missing something.

The answer is so simple that maybe I don't understand the question.

Inside that 'if', write a message on io.stderr and return false.

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Daurnimator
On 7 August 2018 at 16:05, Dirk Laurie <[hidden email]> wrote:
> Inside that 'if', write a message on io.stderr and return false.

Please don't!
This will manifest as e.g. a user reporting an error about trying to
index 'false'.
When they try and run code such as:

tls = require "org.conman.nfl.tls"
function dothething()
   tls.handshake(mysocket)
end
-- at some point later in response to user input:
    dothething()

It's better to fail fast and keep the error message close to the cause!


But also, writing to stderr from a library is an antipattern: please
never do this.
I might e.g. just be checking to see if your module is available and do:
pcall(require, "yourmodule")
I don't want output appearing in the user's terminal when I do that!

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Dirk Laurie-2
2018-08-07 8:38 GMT+02:00 Daurnimator <[hidden email]>:

> On 7 August 2018 at 16:05, Dirk Laurie <[hidden email]> wrote:
>> Inside that 'if', write a message on io.stderr and return false.
>
> Please don't!
> This will manifest as e.g. a user reporting an error about trying to
> index 'false'.
> When they try and run code such as:
>
> tls = require "org.conman.nfl.tls"
> function dothething()
>    tls.handshake(mysocket)
> end
> -- at some point later in response to user input:
>     dothething()
>
> It's better to fail fast and keep the error message close to the cause!

I don't agree. Returning 'false' is a Lua idiom.

>
> But also, writing to stderr from a library is an antipattern: please
> never do this.
> I might e.g. just be checking to see if your module is available and do:
> pcall(require, "yourmodule")
> I don't want output appearing in the user's terminal when I do that!

I'll concede that this is a plausible reason for not writing to stderr,
although If I were that user, I would not minded the hint about some
feature that the program would have exploited if available.

But this second reason nullifies the first. The purpose of returning
'false' is precisely so that a pcall will catch the error.

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Frank Kastenholz-2
In reply to this post by Sean Conner

Simply return nil?

Have the module return a table, just like it would if it worked, but populate the table with dummy functions that all return/throw a “module not properly loaded” error?

Put a status variable in the modules table that indicates it didn’t load properly and the additional reason information?




> On Aug 6, 2018, at 10:32 PM, Sean Conner <[hidden email]> wrote:
>
>
>  I'm working on two modules, both related to libtls [1].  The first one is
> just a C wrapper around libtls (which is done for the most part), and I
> thought it would be nice to support as many versions as possible.
>
>  The second module is a Lua module (or rather, a collection of modules)
> that handle TLS (or TCP) connections via coroutines---each connection is
> wrapped up in a coroutine and there's a main event loop that schedules
> everything.  This one too, is mostly done.
>
>  Enough of the background, now the problem.  The Lua module will only work
> with libtls 2.5.0 (the earliest version that supports user-controlled
> sockets).  It's not enough that, say, org.conman.tls exists, but unless it
> was compiled against libtls 2.5.0 (or higher), it won't work.  I'm looking
> for something like:
>
>    -----[ my-tls.lua / loaded as a module ]----
>
>    local tls = require "org.conman.tls"
>
>    if tls.LIBRESSL_VERSION < 0x2050000f then
>      ??? there's no point in continuing with loading
>      ??? this module, because libtls will not let us
>      ??? control the socket.
>    end
>
>  How to I abort the module?  There's nothing I can find in the
> documentation to deal with this, unless I'm missing something.
>
>  -spc
>
> [1]    I started this with the latest version and got it working.  Now I'm
>    working backwards, supporting earlier verions of the library.
>


Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Gé Weijers
Why not throw an error? (e.g. luaL_error(L, "libtls version too old, version = 0x%x", version);)

A regular return will register the module in the module table, for every value possible it seems. The error handling will bypass that.



On Tue, Aug 7, 2018 at 3:28 AM Frank Kastenholz <[hidden email]> wrote:

Simply return nil?

Have the module return a table, just like it would if it worked, but populate the table with dummy functions that all return/throw a “module not properly loaded” error?

Put a status variable in the modules table that indicates it didn’t load properly and the additional reason information?




> On Aug 6, 2018, at 10:32 PM, Sean Conner <[hidden email]> wrote:
>
>
>  I'm working on two modules, both related to libtls [1].  The first one is
> just a C wrapper around libtls (which is done for the most part), and I
> thought it would be nice to support as many versions as possible.
>
>  The second module is a Lua module (or rather, a collection of modules)
> that handle TLS (or TCP) connections via coroutines---each connection is
> wrapped up in a coroutine and there's a main event loop that schedules
> everything.  This one too, is mostly done.
>
>  Enough of the background, now the problem.  The Lua module will only work
> with libtls 2.5.0 (the earliest version that supports user-controlled
> sockets).  It's not enough that, say, org.conman.tls exists, but unless it
> was compiled against libtls 2.5.0 (or higher), it won't work.  I'm looking
> for something like:
>
>    -----[ my-tls.lua / loaded as a module ]----
>
>    local tls = require "org.conman.tls"
>
>    if tls.LIBRESSL_VERSION < 0x2050000f then
>      ??? there's no point in continuing with loading
>      ??? this module, because libtls will not let us
>      ??? control the socket.
>    end
>
>  How to I abort the module?  There's nothing I can find in the
> documentation to deal with this, unless I'm missing something.
>
>  -spc
>
> [1]    I started this with the latest version and got it working.  Now I'm
>    working backwards, supporting earlier verions of the library.
>




--
--

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Soni "They/Them" L.
In reply to this post by Daurnimator


On 2018-08-07 01:26 AM, Daurnimator wrote:

> On 7 August 2018 at 14:19, Sean Conner <[hidden email]> wrote:
>> But ...
>>
>>> print(package.loaded['org.conman.nfl.tls'])
>> userdata: 0x8064634
>> I think the userdata is the .so file (because each userdata in the TLS
>> module has a __tostring() method assigned to it), which is an unexpected
>> result.
> Nope. It's a hack by lua5.1 to detect require loops. It was removed in
> 5.2 (yet another reason to upgrade!)
>
> $ lua5.1 -e 'package.preload.foo=function() error("foo") end;
> pcall(require,"foo"); print(package.loaded.foo)'
> userdata: 0x4216d4
> $ lua5.2 -e 'package.preload.foo=function() error("foo") end;
> pcall(require,"foo"); print(package.loaded.foo)'
> nil
>

you *could* do it like this:

$ lua5.1 -e 'package.preload.foo=function() package.loaded.foo=nil;
error("foo") end; pcall(require,"foo"); print(package.loaded.foo)'
nil

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Dirk Laurie-2
In reply to this post by Gé Weijers
Op Di., 7 Aug. 2018 om 23:10 het Gé Weijers <[hidden email]> geskryf:
>
> Why not throw an error? (e.g. luaL_error(L, "libtls version too old, version = 0x%x", version);)
>
> A regular return will register the module in the module table, for every value possible it seems. The error handling will bypass that.

You don't want to bypass that.

The return value 'false' is an idiom that does not prevent 'require'
from trying to reload a module, but at the same time allows your
program to test whether there has been a previous unsuccessful
attempt.

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Gé Weijers
On Wed, Aug 8, 2018 at 1:49 AM Dirk Laurie <[hidden email]> wrote:
Op Di., 7 Aug. 2018 om 23:10 het Gé Weijers <[hidden email]> geskryf:
>
> Why not throw an error? (e.g. luaL_error(L, "libtls version too old, version = 0x%x", version);)
>
> A regular return will register the module in the module table, for every value possible it seems. The error handling will bypass that.

You don't want to bypass that.

The return value 'false' is an idiom that does not prevent 'require'
from trying to reload a module, but at the same time allows your
program to test whether there has been a previous unsuccessful
attempt.

I don't see a whole lot of use for this, if a module fails to load retrying is not going to help much in almost all use cases, except where you're loading components on demand. In that case I would suggest writing a wrapper around 'require' to get the behavior needed.

One downside I see is that the reason for the error is not reported, 'false' does not tell you much, and 'require' returns only one value. If there's one thing I try to avoid is programs that fail silently.

Another downside of returning 'false' is that you have to test the return value of each 'require', which just clutters up modules, and in the case that 'require' can't find the module it'll throw an error anyway. This creates an interface that reports failures in two different ways, which makes the code handling the error more complicated.


--
--

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Dirk Laurie-2
Op Wo., 8 Aug. 2018 om 19:00 het Gé Weijers <[hidden email]> geskryf:
> On Wed, Aug 8, 2018 at 1:49 AM Dirk Laurie <[hidden email]> wrote:
>> The return value 'false' is an idiom that does not prevent 'require'
>> from trying to reload a module, but at the same time allows your
>> program to test whether there has been a previous unsuccessful
>> attempt.
>
> I don't see a whole lot of use for this, if a module fails to load retrying is not going to help much in almost all use cases, except where you're loading components on demand. In that case I would suggest writing a wrapper around 'require' to get the behavior needed.

An idiom is a way of using language. It is not necessary for an idiom
to have a whole lot of use. It is only necessary that one is
consistent in how the idiom is used.

But I think you (and Daurnimator) miss the utility of 'false', so I'll
continue this reply.

> One downside I see is that the reason for the error is not reported, 'false' does not tell you much, and 'require' returns only one value. If there's one thing I try to avoid is programs that fail silently.

I use 'false' to mean that the module encountered an unanticipated
reason for failing to do what was asked. It is quite usual in Lua not
to raise an error, but to return nil, e.g. io.open. But you can't do
that with 'require':  modules do not need to return anything. They
could do things to global variables. If you return nil or nothing,
'require' translates that to 'true': there is nothing wrong. 'false'
is just the opposite of 'true': something is wrong.

Although 'require' returns only one value, it often puts stuff into
'package.loaded', not in the global environment, for you to require as
needed. For example, Penlight loads pl.util, pl.import_into and
pl.compat. Only pl.util is globally visible as util. Such a package
could profitably use 'false' to indicate that a particular submodule
is not available.

> Another downside of returning 'false' is that you have to test the return value of each 'require', which just clutters up modules, and in the case that 'require' can't find the module it'll throw an error anyway. This creates an interface that reports failures in two different ways, which makes the code handling the error more complicated.

I find myself testing the return value more often than not, even duck-typing it.

But you don't always want your module to raise an error, sometimes you
want your program to have control over what happens, maybe execute a
fallback.

Anyway, If you have no use for this, don't add it to your idiolect. It
just seemed to me to be tailor-made for the OP's situation: to
intentionally dail to load.

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Sean Conner
It was thus said that the Great Dirk Laurie once stated:
> Op Wo., 8 Aug. 2018 om 19:00 het Gé Weijers <[hidden email]> geskryf:

> > Another downside of returning 'false' is that you have to test the
> > return value of each 'require', which just clutters up modules, and in
> > the case that 'require' can't find the module it'll throw an error
> > anyway. This creates an interface that reports failures in two different
> > ways, which makes the code handling the error more complicated.
>
> I find myself testing the return value more often than not, even
> duck-typing it.
>
> But you don't always want your module to raise an error, sometimes you
> want your program to have control over what happens, maybe execute a
> fallback.
>
> Anyway, If you have no use for this, don't add it to your idiolect. It
> just seemed to me to be tailor-made for the OP's situation: to
> intentionally dail to load.

  And to inform the user why the module failed to load (in my case, that
their version of TLS is lacking a feature required for use).

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Dirk Laurie-2
Op Do., 9 Aug. 2018 om 02:05 het Sean Conner <[hidden email]> geskryf:

>
> It was thus said that the Great Dirk Laurie once stated:
> > Op Wo., 8 Aug. 2018 om 19:00 het Gé Weijers <[hidden email]> geskryf:
>
> > > Another downside of returning 'false' is that you have to test the
> > > return value of each 'require', which just clutters up modules, and in
> > > the case that 'require' can't find the module it'll throw an error
> > > anyway. This creates an interface that reports failures in two different
> > > ways, which makes the code handling the error more complicated.
> >
> > I find myself testing the return value more often than not, even
> > duck-typing it.
> >
> > But you don't always want your module to raise an error, sometimes you
> > want your program to have control over what happens, maybe execute a
> > fallback.
> >
> > Anyway, If you have no use for this, don't add it to your idiolect. It
> > just seemed to me to be tailor-made for the OP's situation: to
> > intentionally dail to load.
>
>   And to inform the user why the module failed to load (in my case, that
> their version of TLS is lacking a feature required for use).

So your module could return an error object instead of a module table,
and the documentation could say it does that.

-----[ my-tls.lua / loaded as a module ]----
--[[
Usage:
   my_tls = require "my-tls"
   if my_tls.errormessage then
     error(my_tls.errormessage)
   end
]]


local tls = require "org.conman.tls"

        if tls.LIBRESSL_VERSION < 0x2050000f then
          return { errormessage = [[ ??? there's no point in
continuing with loading
          this module, because libtls will not let us
          ??? control the socket.]]
        end

Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Frank Kastenholz-2
In reply to this post by Gé Weijers


> On Aug 8, 2018, at 1:00 PM, Gé Weijers <[hidden email]> wrote:
>
> I don't see a whole lot of use for this, if a module fails to load retrying is not going to help much in almost all use cases, except where you're loading components on demand. In that case I would suggest writing a wrapper around 'require' to get the behavior needed.

In a previous life I wrote a system where the lua app could require a particular module and, if the require failed, request that the module  be loaded from a depot and then re-require it.

The download-and-install functionality was in a separate module - we did not modify the require code in the PUC-Rio lua source due to project requirements.

It could have been a wrapper encapsulating the “require/install/re-require” logic ... but that only would have made life a bit easier for our application developers —- we would have still needed to re-require


> Another downside of returning 'false' is that you have to test the return value of each 'require',

Isn’t it good programming practice to always check return values?

Frank



Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Tomás Guisasola-2
Hi

> I don't see a whole lot of use for this, if a module fails to load retrying is not going to help much in almost all use cases, except where you're loading components on demand. In that case I would suggest writing a wrapper around 'require' to get the behavior needed.

In a previous life I wrote a system where the lua app could require a particular module and, if the require failed, request that the module  be loaded from a depot and then re-require it. 
Although I share the opinion of  Gé Weijers, your approach is quite reasonable.  Since it is not standard, I would still throw an error when there is any failure in require.  This is what Lua demands from developers.

The download-and-install functionality was in a separate module - we did not modify the require code in the PUC-Rio lua source due to project requirements.
Yes, this is an extension :-)

> Another downside of returning 'false' is that you have to test the return value of each 'require',

Isn’t it good programming practice to always check return values?
In case of require, I won't check, except in extreme cases.  As someone already mentioned, there is no error message from require, so it is useless to check the return value.  I think require should throw an error, as Gé mentioned.

Regards,
Tomás
Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Gé Weijers
In reply to this post by Frank Kastenholz-2


On Thu, Aug 9, 2018 at 4:24 AM Frank Kastenholz <[hidden email]> wrote:
Isn’t it good programming practice to always check return values?


Depends on the interface contract. When memory allocation fails in C (e.g. using malloc) you end up with a null pointer, which you check for. In C++ the contract is different, the failure causes an exception (unless you call the non-throwing version of 'new', I know), so you should not test for null, because the contract guarantees that the pointer will never be null (your program will continue in an exception handler if allocation fails).

Lua has two standard conventions for reporting errors: error/lua_error ('throwing' an error), or returning false plus an error string. 'require' only uses and supports the first one, so that's the idiomatic way to report errors. If you construct additional ways to report errors you modify the interface contract of a standard Lua interface. You have to have a really, really, really good reason for doing that, especially if you publish the code, or other people have to maintain it.

It's better to add your own variant of 'require' if you need something non-standard.

--


Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Viacheslav Usov
In reply to this post by Frank Kastenholz-2
On Thu, Aug 9, 2018 at 1:24 PM Frank Kastenholz <[hidden email]> wrote:

> Isn’t it good programming practice to always check return values?

It is. It is also often one severely neglected in practice, even in environments much less forgiving than Lua.

Therefore, one is advised to practise making it impossible for an unsuspecting user to ignore an error silently.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: How can a module intentionally fail to load?

Gé Weijers

On Mon, Aug 13, 2018 at 3:41 AM Viacheslav Usov <[hidden email]> wrote:
On Thu, Aug 9, 2018 at 1:24 PM Frank Kastenholz <[hidden email]> wrote:

> Isn’t it good programming practice to always check return values?

It is. It is also often one severely neglected in practice, even in environments much less forgiving than Lua.

Therefore, one is advised to practise making it impossible for an unsuspecting user to ignore an error silently.


I would suggest reading "A Philosophy of Software Design" by John Ousterhout. One of the suggestions he makes for C programs is to replace 'malloc' with a 'ckalloc' that checks for a NULL return and exits the program with an error message if you run out of memory. There's typically not much you can do when a C program runs out of memory.
The idea here is that you try not to saturate your code with error checks, which makes it harder to read. The idea is NOT that you ignore any errors.

Checking the return value only makes sense if the return value can actually indicate an error. In languages like C and Go that's the convention, so you absolutely should check return values. In languages like C++, Lua and others that use some kind of exception handling checking the return value may be pointless, depending on the interface you're using. 'require' comes out of the box 'throwing' an error string using 'lua_error', it does not return when it finds an error, and it does not support the "false, <errorstring>" convention. Other interfaces in Lua work differently (io.open for instance).

The suggestion was make 'require' return some kind of encoded error, which would then have to be checked. That's a change of behavior from the way 'require' normally works, which is not a good thing. Adding unexpected behavior to a well-known interface is a likely source of bugs. 'require' can also only return one value only, so you will have to come up with a non-standard way to encode an error return. I work with large code bases, and this kind of cleverness comes back to haunt you most of the time.


--

12