how to prevent bugs without static type checking?

classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

RE: how to prevent bugs without static type checking?

Thijs Schreijer
> Behalf Of steve donovan
> Sent: woensdag 24 oktober 2012 12:15
> Subject: Re: how to prevent bugs without static type checking?
>
> On Wed, Oct 24, 2012 at 11:43 AM, Thijs Schreijer
<[hidden email]>
> wrote:
> > If the "typical" library by Rob Hoelz can be combined with my "typex"
> > library, then that could pretty much fill this gap.
>
> I'd like to harmoize Penlight with this, and fill in MT.__type.
> Standards are a good thing!
>
> However, is the idea that __type (or tostring(__type)) should always
resolve
> to a sensible name for a type?

Yes, fallback it the lua-type. That is for userdata's and tables. If there
is a __type or similar (whatever the convention) value or function (for
tables also in the table itself, not just the metatable), then use that
If not, then a table is just a table, but for userdata's, collect the
metatable name from the registry, and report that (if not found, a userdata
is also just a userdata)

That would be the most complete approach I think.

Just a definition would be required on what names to look for; __type, or
.class(), .superclass(), etc.
And some standard preprocessor symbols

Thijs

And a devil in the details of argument checking; should the MT chain be
traversed to see whether a subclass satisfies as a class higher up in the
hierarchy (plant -> flower -> tulip, does a tulip satisfy an argument
requiring a plant-type), maybe that requires an __istype() or similar
function as well.





Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

steve donovan
On Wed, Oct 24, 2012 at 12:50 PM, Thijs Schreijer
<[hidden email]> wrote:
> And a devil in the details of argument checking; should the MT chain be
> traversed to see whether a subclass satisfies as a class higher up in the
> hierarchy

Ah, but again there are different ways to peel the avocado.  An MT
chain is one way, but I tend to use 'fat metatable inheritance' where
the contents of the base class is copied into the new derived class,
with a _base field pointing to the original base class.  Faster method
dispatch for the price of larger metatables - a good trade-off if the
number of objects outnumbers the number of classes significantly.

> maybe that requires an __istype() or similar  function as well.

That would be best, and would depend on the OOP scheme being used.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Rob Hoelz-2
On 10/24/12 1:06 PM, steve donovan wrote:

> On Wed, Oct 24, 2012 at 12:50 PM, Thijs Schreijer
> <[hidden email]> wrote:
>> And a devil in the details of argument checking; should the MT chain be
>> traversed to see whether a subclass satisfies as a class higher up in the
>> hierarchy
> Ah, but again there are different ways to peel the avocado.  An MT
> chain is one way, but I tend to use 'fat metatable inheritance' where
> the contents of the base class is copied into the new derived class,
> with a _base field pointing to the original base class.  Faster method
> dispatch for the price of larger metatables - a good trade-off if the
> number of objects outnumbers the number of classes significantly.
>
>> maybe that requires an __istype() or similar  function as well.
> That would be best, and would depend on the OOP scheme being used.
>
> steve d.
>
Just thought I'd chime in since lua-typical was mentioned; typical
currently works using the following algorithm:

  - If __type is a valid metafield for the value and is a string, return
__type.
  - If __type is a valid metafield for the value and is a function,
return __type(value).
  - If io.type(value) returns a non-nil value, return io.type(value).
  - Return type(value).

There's no reason that I couldn't incorporate Thijs' typex metatable
lookup logic into this; I just decided to do the minimum for
the first release.

-Rob

Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Geoff Leyland
In reply to this post by steve donovan
On 24/10/2012, at 10:19 PM, steve donovan <[hidden email]> wrote:

> On Wed, Oct 24, 2012 at 10:29 AM, Geoff Leyland
> <[hidden email]> wrote:
>> foo(
>> a, -- string
>> b) -- integer
>
> Yeah, I support this pattern with the last comment _before_ the
> closing parens - should not be difficult to check for this case, which
> reads better anyhow.

Don't forget this:

function foo(a) -- this is a bit of an ambiguous way to document a

and

function foo(
 a, -- a really really
    -- long description
 b) -- and a short one

One thing that's needed is a convention to tell if the comment is intended as a type annotation (for now I'm using a colon):

function foo(
 a)  -- string: a description

Otherwise this:

function foo(
  a) -- I don't want to get this type-checked

gets a complaint that a is not of type I.  Type aliases could work?

function foo(
  a)  -- @?string|1..3

> Next release (soon!) will define the primitive types as tparam
> aliases, so @string, etc. (@int is interesting one, since you apply an
> additional custom constraint here, very useful)

Is it a good idea to prefix the aliases with an @?  It means you have to distinguish between @string (a type), @tparam (followed by a type) and @see (not a type).  All possible of course, but picking a different character would make it simpler.  I guess there's a fairly large body of code already using ldoc.

> What could work is if there's an argcheck option to run a preprocessed
> version, with explicit type asserts (they can all be on the same line
> as 'function' so no line number confusions)

That's an interesting and completely different approach!  I guess if we sort out what the comments are supposed to look like then we can have competing implementations,


Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Geoff Leyland
In reply to this post by Thijs Schreijer
On 24/10/2012, at 10:43 PM, Thijs Schreijer <[hidden email]> wrote:

> I think this could combine very well with this message;
> http://lua-users.org/lists/lua-l/2012-10/msg00122.html 
>
> If the "typical" library by Rob Hoelz can be combined with my "typex"
> library, then that could pretty much fill this gap.

I tried to get typical's functionality by looking for __type, but I forgot to call it if it's a function.  Your typex is harder, since you can't access the registry from Lua.  I could pcall(require) your version, and act appropriately if it's available.

> A preprocessor that can be run on the command line (similar to LuaCov usage)
> would be nice then to prevent extra manual steps in the process

It's not a preprocessor in the sense that Steve suggested, but you do just go "lua -largcheck file.lua" like you do with luacov and luatrace.
Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Geoff Leyland
In reply to this post by Thijs Schreijer
On 24/10/2012, at 11:50 PM, Thijs Schreijer <[hidden email]> wrote:

> And a devil in the details of argument checking; should the MT chain be
> traversed to see whether a subclass satisfies as a class higher up in the
> hierarchy (plant -> flower -> tulip, does a tulip satisfy an argument
> requiring a plant-type), maybe that requires an __istype() or similar
> function as well.


My take on this is [1]: give the object's metatable a __typeinfo field that's a table that has fields that are true if the object is of that type, so, for if value is a tulip:

getmetatable(value).__typeinfo -> { table=true, plant=true, flower=true, tulip=true }

and there's a typeinfo function so you can check with "if typeinfo(value).flower then...".

(Having said that, I'm not sure that long inheritance chains are a symptom of good practice)


[1] https://github.com/geoffleyland/rima/blob/master/lua/rima/lib/object.lua
Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Henning Diedrich
In reply to this post by Enrico Tassi

On Oct 23, 2012, at 2:28 PM, Enrico Tassi <[hidden email]> wrote:

>  the only true advantage I see of static typing is when you do
> true research, i.e. you have no clear idea about the right data
> structure (or the right approach) to solve your problem

A very rare situation, of course.

And static type checking really is a blessing when doing small experimental changes where you might be tempted to not update the code parts you are currently not interested in.

Henning
Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

eugeny gladkih
In reply to this post by William Ahern
On 23.10.2012 9:38, William Ahern wrote:

> That often makes any bug more likely to be caught during QA testing.

it's an utopia talking about 100% code coverage in the tests. nobody did
that, even Knuth

Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

eugeny gladkih
In reply to this post by Geoff Leyland
On 23.10.2012 21:58, Geoff Leyland wrote:

>>>> I suppose I could write a function "checkargs"
>>>
>>> iirc the Sierra Wireless folks did something like that.
>>
>> Indeed we have...
>
> If you're talking about run-time rather than static type-checking, then [1] shamelessly copies the sierra wireless solution, except that it runs as a debug hook, reads documentation comments rather than requiring a check function and is in Lua as opposed to C (nothing against C, just that a Lua-only solution doesn't require compilation).
>
>
>      --- prints the string s n times
>      -- @tparam number n how many times to print the string
>      -- @tparam string s the string to print
>      function printn(s, n)
>        for i = 1, n do print(s) end
>      end
>
>      printn(10, "hello")
>
>
>      $ lua -largcheck test/ldoc.lua
>      lua: test/ldoc.lua:8: bad argument #1 to 'printn' (string expected, got number '10')

what about

printn("hello",'10')

?

Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Thijs Schreijer
In reply to this post by 乙振斐
Fwiw; you can access the registry through the debug library

Thijs


Geoff Leyland <[hidden email]> wrote:

>On 24/10/2012, at 10:43 PM, Thijs Schreijer <[hidden email]> wrote:
>
>> I think this could combine very well with this message;
>> http://lua-users.org/lists/lua-l/2012-10/msg00122.html 
>>
>> If the "typical" library by Rob Hoelz can be combined with my "typex"
>> library, then that could pretty much fill this gap.
>
>I tried to get typical's functionality by looking for __type, but I forgot to call it if it's a function.  Your typex is harder, since you can't access the registry from Lua.  I could pcall(require) your version, and act appropriately if it's available.
>
>> A preprocessor that can be run on the command line (similar to LuaCov usage)
>> would be nice then to prevent extra manual steps in the process
>
>It's not a preprocessor in the sense that Steve suggested, but you do just go "lua -largcheck file.lua" like you do with luacov and luatrace.
Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Thijs Schreijer
In reply to this post by 乙振斐
well that is one approach, yet, to make it generally useful it should work on all the common approaches I think.

(sorry for top-posting, client doesn't support it)

Geoff Leyland <[hidden email]> wrote:

>On 24/10/2012, at 11:50 PM, Thijs Schreijer <[hidden email]> wrote:
>
>> And a devil in the details of argument checking; should the MT chain be
>> traversed to see whether a subclass satisfies as a class higher up in the
>> hierarchy (plant -> flower -> tulip, does a tulip satisfy an argument
>> requiring a plant-type), maybe that requires an __istype() or similar
>> function as well.
>
>
>My take on this is [1]: give the object's metatable a __typeinfo field that's a table that has fields that are true if the object is of that type, so, for if value is a tulip:
>
>getmetatable(value).__typeinfo -> { table=true, plant=true, flower=true, tulip=true }
>
>and there's a typeinfo function so you can check with "if typeinfo(value).flower then...".
>
>(Having said that, I'm not sure that long inheritance chains are a symptom of good practice)
>
>
>[1] https://github.com/geoffleyland/rima/blob/master/lua/rima/lib/object.lua
Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Geoff Leyland
On 25/10/2012, at 9:37 AM, Thijs Schreijer <[hidden email]> wrote:

> Geoff Leyland <[hidden email]> wrote:
>

>> give the object's metatable a __typeinfo field that's a table that has fields that are true if the object is of that type, so, for if value is a tulip:
>>
>> getmetatable(value).__typeinfo -> { table=true, plant=true, flower=true, tulip=true }
>>
>> and there's a typeinfo function so you can check with "if typeinfo(value).flower then...".
>
> well that is one approach, yet, to make it generally useful it should work on all the common approaches I think.

Sorry, to be clear: I use this method sometimes for objects with inheritance chains.

argcheck, on the other hand, tries any method I can think of to guess whether an object is a given type.  I haven't got to typex yet, and I haven't written a two-line "add_type_checker" yet, but I'm bumbling in that direction.


Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Geoff Leyland
In reply to this post by eugeny gladkih
On 25/10/2012, at 8:43 AM, eugeny gladkih <[hidden email]> wrote:

>>     --- prints the string s n times
>>     -- @tparam number n how many times to print the string
>>     -- @tparam string s the string to print
>>     function printn(s, n)
>>       for i = 1, n do print(s) end
>>     end
>>
>>     printn(10, "hello")
>
> what about
>
> printn("hello",'10')
>
> ?

As in, does the "number" constraint check whether tonumber (or automatic type coercion) can coerce the argument into a number?  Currently, no, it just does 'type(value) == constraint'.  It'd be easy to change, but I get the feeling automatic coercion is not always a popular feature.

But you can go:

function printn(
  s, -- string
  n) -- number|string

or, for that matter

function printn(
  s, -- string
  n) -- 1..640: 640 ought to be enough for anybody

and so on.


Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Geoff Leyland
In reply to this post by Thijs Schreijer
On 25/10/2012, at 9:05 AM, Thijs Schreijer <[hidden email]> wrote:

> Fwiw; you can access the registry through the debug library

D'oh, so you can.  Thanks.

Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Geoff Leyland
In reply to this post by Thijs Schreijer
On 25/10/2012, at 9:05 AM, Thijs Schreijer <[hidden email]> wrote:

> Fwiw; you can access the registry through the debug library

Thanks again.  argcheck now looks in the registry to try to find a metatable match.
Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Andrew Starks
In reply to this post by spir ☣


On Tue, Oct 23, 2012 at 3:29 AM, spir <[hidden email]> wrote:
I noticed the following: after programming for a long time only in dynamic languages, when I then switch to static typing, for a while I have about no type error (at compile-time, indeed); then they start to happen, and finally I have many. I think, maybe, after some time we start "programming-thinking" differently. Dynamic programmers may have a clearer awareness about types --that is, ultimately, the meaning of symbols-- and unconsciously pay more attention to their coherence in the model. Conversely, after some time, static programmers may start to rely on the machine to do this job; and it does it well, provided your typing is correct and clear. They become so-to-say lazy about that, not uselessly spending their "neuronic power" in what a machine can do (better).
Both are good, in my opinion.



I have noticed the same thing. As background, I'd classify myself as a hacker. In my software company, I don't program for my job.

Whenever I program in Lua, I find myself setting up a quick argument checker for the group of functions that cover a particular area of a program that I'm writing. At first, it usually just checks for what is required, applies defaults to what isn't there and gives awesome errors (written like a manual) when it gets non-sense.

I was speaking with a developer about an error they had found in a shipping product. It struck me that they didn't check for this particular lexical error on one of the interfaces for an object's method. It was really alarming, given that the input came from a serial port.

Now, notwithstanding the recklessness of that oversight, it got me thinking about the nature of dynamic languages and how their cultures differ from each other and from those of static languages.

Prior to Ruby taking off, pretty much every dynamic language that I've been around had people that didn't go to school to learn to program as their target audience. Ruby's mission was "programmer happiness" and was picked up by a crowd that at least professed to strive for quality code. Ruby's testing culture boarders on nauseating.

An analogy might be the difference between a raw performance car (say an older Porche 911), and a modern, high-tech version of the same (Skyline / GTR, with all of its tele-metrics and four wheel drive).

An average driver will be much faster in the GTR, using all of the help that the advanced technology provides (type checking, syntax highlighting).

A great driver will be much faster in the 911, because the car is lighter and doesn't interfere when it shouldn't. 

I think that there is a sense of false security that people _can_ gain from all of the bells and whistles that an IDE can provide and that type checking provide. They are great, no doubt, but maybe arguments should always be checked, both for type and for sanity? Maybe type checking is just a really primitive and inflexible way of doing something that might be done more clearly and more completely, when the program is being designed?

I've never missed type checking in lua.

On a related note, I really wish that nil was not false, although I know I'm in the minority on that one. :)

-Andrew Starks
Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

steve donovan
On Fri, Oct 26, 2012 at 5:36 AM, Andrew Starks <[hidden email]> wrote:
> target audience. Ruby's mission was "programmer happiness" and was picked up
> by a crowd that at least professed to strive for quality code. Ruby's
> testing culture boarders on nauseating.

Yes, testing isn't so happy-making, but part of life. But I'd say that
the first generation of dynamic programmers were smart hackers, e.g
Unix shell, Tcl and Perl.  In the hands of masters, these hotrods
could do marvelous things, but less careful people tend to make
messes. (Which fits with the car analogy)

As an example of different language culture, Ruby tolerates
duck-typing and so people get into messes which are hard to reason
about. Lua people are much more reluctant to do that, even though it's
trivial to redefine string.match say.

> I think that there is a sense of false security that people _can_ gain from
> all of the bells and whistles that an IDE can provide and that type checking
> provide.

Formal guarantees of type correctness aren't a good indication of
program correctness! [1] Still, there are rules of thumb about errors
in dynamic languages - force them to happen as early as possible, and
crash noisily - a type assertion is a good example. And these dynamic
type assertions can be as intelligent as we need them to be.

I've been doing Android programming in Lua, and feel that a dynamic
solution has to give a big productivity boost to offset the loss of
the intelligent IDE support one gets with Java.  In particular, a REPL
is a fantastic thing for learning a platform and experimenting without
having to rebuild, reinstall and relaunch.[2]

> On a related note, I really wish that nil was not false, although I know I'm
> in the minority on that one. :)

Well, nil came before false in Lua's evolution ;)

steve d.

[1]  (Haskell people tend to say otherwise but then they have an
incredibly powerful type system which gives non-masters headaches.
Certainly for me).
[2] which can involve a number of seconds, which is a long time in a
programmer's head.

Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

spir ☣
In reply to this post by Andrew Starks
On 26/10/2012 05:36, Andrew Starks wrote:
> On a related note, I really wish that nil was not false, although I know
> I'm in the minority on that one. :)

I wish there would at least 4 notions: undef, none, false, failure (as in "no
such file'config.lua'"). And none of them would be a value, except for none as
default param. (I know I'm in the minority on that one. :)

Denis

Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Javier Guerra Giraldez
In reply to this post by steve donovan
On Fri, Oct 26, 2012 at 1:23 AM, steve donovan
<[hidden email]> wrote:
> As an example of different language culture, Ruby tolerates
> duck-typing and so people get into messes which are hard to reason
> about. Lua people are much more reluctant to do that, even though it's
> trivial to redefine string.match say.

don't you mean monkey-patching?  zoology can be hard!  ;-)

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: how to prevent bugs without static type checking?

Miles Bader-2
In reply to this post by Philippe Lhoste
Philippe Lhoste <[hidden email]> writes:
> - Unless you document very well your data structure (I guess you do),
> and document well the type of your parameters, etc., it can be hard to
> use the API.

Yup.

I tend to think of types in variable/field/parameter declarations (in
statically-typed languages) as _documentation_, which helps me
understand the program when reading other people's code, my old code,
etc.

The consistency-checking by the compiler is nice too of course... :]

-miles

--
春風の 穏やかな波 草を弾く
春草の 穏やかな波 風を惹く

123