An operator syntax for bitfields using metatables

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

An operator syntax for bitfields using metatables

John Hind
I've been following the discussion on the bit32 library and share the desire
for an operator rather than a functional syntax. However I realised that 99%
of my use of this library is for cracking and assembling bitfields for
communications protocols and register mapping in embedded systems. It is
straightforward to add a syntax for this in a C library by providing a
metatable for the number type with 'index' and 'newindex' metamethods. Then:

b = n[3]        -- Extract bit 3 as a boolean.
n[23] = true    -- Set bit 23.
n[6] = not n[6] -- Toggle bit 6.

If you really need to bitwise and two numbers:

for i=0, 31 do nr[i] = n1[i] and n2[i] end

Nowhere near as efficient as:

nr = bit32.band(n1,n2)

But more expressive and intuitive especially when you need to manipulate a
few bits in relatively complex ways.

This concept can be extended to extract and replace numeric fields within a
larger field (like the bit32 functions with these names). This requires that
the index encode two five bit integers which could be done using bitfields
within a single number or by using a string index which gets parsed in the
metamethod:

bitrange = function(s,e) return ((e * 0xFF) + 1) + s end  -- Range testing
omitted for clarity

nr = n[bitrange(10,15)]  -- Mask out bits 10 through 15 and shift right 10.
nr = n[bitrange(15,10)]  -- Mask out bits 10 through 15, shift and reverse
the bit order.
nr = n[bitrange(12,12)]  -- nr = 1 if bit 12 is set, else 0.
b = n[bitrange(12)]      -- b = true only if bit 12 is set (as before).
n[bitrange(4,6)] = 2     -- replace bits 4 through 6 with 010.




Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

D Burgess-4
FWIW. I like the syntax.

On Sat, Jul 13, 2013 at 1:24 AM, John Hind <[hidden email]> wrote:

> I've been following the discussion on the bit32 library and share the desire
> for an operator rather than a functional syntax. However I realised that 99%
> of my use of this library is for cracking and assembling bitfields for
> communications protocols and register mapping in embedded systems. It is
> straightforward to add a syntax for this in a C library by providing a
> metatable for the number type with 'index' and 'newindex' metamethods. Then:
>
> b = n[3]        -- Extract bit 3 as a boolean.
> n[23] = true    -- Set bit 23.
> n[6] = not n[6] -- Toggle bit 6.
>
> If you really need to bitwise and two numbers:
>
> for i=0, 31 do nr[i] = n1[i] and n2[i] end
>
> Nowhere near as efficient as:
>
> nr = bit32.band(n1,n2)
>
> But more expressive and intuitive especially when you need to manipulate a
> few bits in relatively complex ways.
>
> This concept can be extended to extract and replace numeric fields within a
> larger field (like the bit32 functions with these names). This requires that
> the index encode two five bit integers which could be done using bitfields
> within a single number or by using a string index which gets parsed in the
> metamethod:
>
> bitrange = function(s,e) return ((e * 0xFF) + 1) + s end  -- Range testing
> omitted for clarity
>
> nr = n[bitrange(10,15)]  -- Mask out bits 10 through 15 and shift right 10.
> nr = n[bitrange(15,10)]  -- Mask out bits 10 through 15, shift and reverse
> the bit order.
> nr = n[bitrange(12,12)]  -- nr = 1 if bit 12 is set, else 0.
> b = n[bitrange(12)]      -- b = true only if bit 12 is set (as before).
> n[bitrange(4,6)] = 2     -- replace bits 4 through 6 with 010.
>
>
>
>



--
David Burgess

Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Dirk Laurie-2
2013/7/13 David Burgess <[hidden email]>:

> FWIW. I like the syntax.
>
> On Sat, Jul 13, 2013 at 1:24 AM, John Hind <[hidden email]> wrote:
>> I've been following the discussion on the bit32 library and share the desire
>> for an operator rather than a functional syntax. However I realised that 99%
>> of my use of this library is for cracking and assembling bitfields for
>> communications protocols and register mapping in embedded systems. It is
>> straightforward to add a syntax for this in a C library by providing a
>> metatable for the number type with 'index' and 'newindex' metamethods. Then:
>>
>> b = n[3]        -- Extract bit 3 as a boolean.
>> n[23] = true    -- Set bit 23.
>> n[6] = not n[6] -- Toggle bit 6.
>>

Up to here I like it too.

>> This concept can be extended to extract and replace numeric fields within a
>> larger field (like the bit32 functions with these names). This requires that
>> the index encode two five bit integers which could be done using bitfields
>> within a single number or by using a string index which gets parsed in the
>> metamethod:
>>
>> bitrange = function(s,e) return ((e * 0xFF) + 1) + s end  -- Range testing
>> omitted for clarity
>>
>> nr = n[bitrange(10,15)]  -- Mask out bits 10 through 15 and shift right 10.
>> nr = n[bitrange(15,10)]  -- Mask out bits 10 through 15, shift and reverse
>> the bit order.
>> nr = n[bitrange(12,12)]  -- nr = 1 if bit 12 is set, else 0.
>> b = n[bitrange(12)]      -- b = true only if bit 12 is set (as before).
>> n[bitrange(4,6)] = 2     -- replace bits 4 through 6 with 010.

Here I would prefer string-valued indexing.

nr = n["10:15"]
n["4:6"] = 2

String-valued assignment can be handled too.

n["4:6"] = "010"

The available metamethods for numbers have not been exhausted yet.
What about:

__len  -- number of 1's in n
__call -- iterator over the positions of the 1's in n

Reply | Threaded
Open this post in threaded view
|

RE: An operator syntax for bitfields using metatables

John Hind
In reply to this post by John Hind
> -----Original Message-----
> From: John Hind [mailto:[hidden email]]
> Sent: 12 July 2013 16:25


> It is straightforward to add a syntax for this in
> a C library by providing a metatable for the number type with 'index'
> and 'newindex' metamethods. Then:
>
> b = n[3]        -- Extract bit 3 as a boolean.
> n[23] = true    -- Set bit 23.
> n[6] = not n[6] -- Toggle bit 6.
>

Arghh! This does not work! (But I still think it should.)

(Phew! I feared David Burgess and/or my old nemesis Dick Laurie had spotted
my idiocy, but no!)

The problem is that Lua does not honour a return value from the 'newindex'
metamethod so the second and third lines in the above example do not
actually change n. This is fine for tables and userdata, but makes the
'newindex' metamethod pretty useless for any other type!

So unless we patch Lua so this does work as (I, and apparently the other
two) expected, or alternatively implement the bit selection syntax directly,
this idea is a non-starter. Doing it directly as a syntax patch would have
the advantage that we could do n[10,12] directly rather than by bitfield
encoding or string parsing, but I still think the operation of the
metamethod should be also changed to make it usable for all types (a
non-breaking change if nil return is interpreted as 'silently do nothing' as
at present).

Dick: if using a string key, it is neat to have something like "bit_4_to_12"
so dot syntax can be used.

A lesson learned: test your code before trying to sell it!



Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Jay Carlson
In reply to this post by John Hind
On Jul 13, 2013, at 7:28 AM, John Hind wrote:

>> From: John Hind [mailto:[hidden email]]
>
>> It is straightforward to add a syntax for this in
>> a C library by providing a metatable for the number type with 'index'
>> and 'newindex' metamethods. Then:
>>
>> b = n[3]        -- Extract bit 3 as a boolean.
>> n[23] = true    -- Set bit 23.
>> n[6] = not n[6] -- Toggle bit 6.
>>
>
> Arghh! This does not work! (But I still think it should.)
>
> The problem is that Lua does not honour a return value from the 'newindex'
> metamethod so the second and third lines in the above example do not
> actually change n. This is fine for tables and userdata, but makes the
> 'newindex' metamethod pretty useless for any other type!

More broadly, this is an issue for any immutable type. Consider tuples:

  tpl = Tuple(1, 2, 3)
  orig_tpl = tpl
  assert( tpl[2] == 2 )

  tpl[2] = 0   ==> error("that's not really an lvalue")

  s = "123"
  s[2]="0"   ==> error("that's not an lvalue either")

So for __newindex-like-behavior to have any meaning, the assignment had to be syntactic sugar for

  tpl = tuple__copy_replacing_index(tpl, 2, 0)
  s = string__copy_replacing_index(s, 2, "0")

and note that definitionally tpl~=orig_tpl.

I think this is a scary transformation to do conditionally at runtime. I suppose the answer is to turn *all* t[k]=v statements into sugar; the table implementation doesn't need to copy of course. I'm still scared of it though....

Jay
Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Robert Virding
In reply to this post by John Hind
While operators for bit-fields are very useful I think a better way is a syntax which allows you describe/build the whole bitstring in one go. We have this in Erlang and it makes building/pulling apart protocol packets very much easier and clearer. Some simple examples of the top of my head:

64-bit IEEE floating point:
<<Sign:1,Exponent:11,Mantissa:52>>

IP datagram:
<<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16,
      ID:16, Flgs:3, FragOff:13,
      TTL:8, Proto:8, HdrChkSum:16,
      SrcIP:32, DestIP:32, RestDgram/binary>>

The second one is a bit longer than 32 bits. :-) I think you get the idea. We can also use the same syntax for pulling apart the binary/bitstring. This really is powerful and simplifies a lot of things and I think that something similar would be useful for Lua if you seriously intend to work with bitfields.

Robert

----- Original Message -----

> From: "John Hind" <[hidden email]>
> To: [hidden email]
> Sent: Saturday, 13 July, 2013 1:28:35 PM
> Subject: RE: An operator syntax for bitfields using metatables
>
> > -----Original Message-----
> > From: John Hind [mailto:[hidden email]]
> > Sent: 12 July 2013 16:25
>
>
> > It is straightforward to add a syntax for this in
> > a C library by providing a metatable for the number type with
> > 'index'
> > and 'newindex' metamethods. Then:
> >
> > b = n[3]        -- Extract bit 3 as a boolean.
> > n[23] = true    -- Set bit 23.
> > n[6] = not n[6] -- Toggle bit 6.
> >
>
> Arghh! This does not work! (But I still think it should.)
>
> (Phew! I feared David Burgess and/or my old nemesis Dick Laurie had
> spotted
> my idiocy, but no!)
>
> The problem is that Lua does not honour a return value from the
> 'newindex'
> metamethod so the second and third lines in the above example do not
> actually change n. This is fine for tables and userdata, but makes
> the
> 'newindex' metamethod pretty useless for any other type!
>
> So unless we patch Lua so this does work as (I, and apparently the
> other
> two) expected, or alternatively implement the bit selection syntax
> directly,
> this idea is a non-starter. Doing it directly as a syntax patch would
> have
> the advantage that we could do n[10,12] directly rather than by
> bitfield
> encoding or string parsing, but I still think the operation of the
> metamethod should be also changed to make it usable for all types (a
> non-breaking change if nil return is interpreted as 'silently do
> nothing' as
> at present).
>
> Dick: if using a string key, it is neat to have something like
> "bit_4_to_12"
> so dot syntax can be used.
>
> A lesson learned: test your code before trying to sell it!
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Luiz Henrique de Figueiredo
> 64-bit IEEE floating point:
> <<Sign:1,Exponent:11,Mantissa:52>>
>
> IP datagram:
> <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16,
>       ID:16, Flgs:3, FragOff:13,
>       TTL:8, Proto:8, HdrChkSum:16,
>       SrcIP:32, DestIP:32, RestDgram/binary>>

Given these strings it easy to parse them into a Lua table containing the
names and starting and end points of the bitfields described there. That
table can then be used to set index and newindex metamethods for get and
put the bitfields using Lua syntax, or perhaps pack and unpack functions:

template = "<<Sign:1,Exponent:11,Mantissa:52>>"
name = "64-bit IEEE floating point"
b = bitfelds.parse(name,template)
s = io.read(file,b.size)
t = b:unpack(s)
print(t.Sign,t.Exponent,t.Mantissa)
t.Exponent=0
s = b:pack(t)

Reply | Threaded
Open this post in threaded view
|

RE: An operator syntax for bitfields using metatables

Thomas Buergel
In reply to this post by Robert Virding
Diverging from the original topic a bit...

Robert wrote:

> While operators for bit-fields are very useful I think a
> better way is a syntax which allows you describe/build
> the whole bitstring in one go. We have this in Erlang and
> it makes building/pulling apart protocol packets very
> much easier and clearer.

...

> This really is powerful and simplifies a lot of things
> and I think that something similar would be useful for Lua
> if you seriously intend to work with bitfields.

I agree - something like vstruct [1, 2] could do the job. I have used it for just that, packing/unpacking network protocols.

Tom

[1] https://github.com/ToxicFrog/vstruct
[2] http://lua-users.org/lists/lua-l/2013-01/msg00433.html

Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Miles Bader-2
Thomas Buergel <[hidden email]> writes:
> I agree - something like vstruct [1, 2] could do the job. I have used
> it for just that, packing/unpacking network protocols.

The last time I looked, vstruct seemed fairly slow and made a lot of
garbage...  Roberto's struct or Luiz's lpack might be better bets...

-miles

--
Marriage, n. The state or condition of a community consisting of a master, a
mistress and two slaves, making in all, two.

Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Todd Coram
In reply to this post by Luiz Henrique de Figueiredo
On Mon, Jul 15, 2013, at 09:58 PM, Luiz Henrique de Figueiredo wrote:

> > 64-bit IEEE floating point:
> > <<Sign:1,Exponent:11,Mantissa:52>>
> >
> > IP datagram:
> > <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16,
> >       ID:16, Flgs:3, FragOff:13,
> >       TTL:8, Proto:8, HdrChkSum:16,
> >       SrcIP:32, DestIP:32, RestDgram/binary>>
>
> Given these strings it easy to parse them into a Lua table containing the
> names and starting and end points of the bitfields described there. That
> table can then be used to set index and newindex metamethods for get and
> put the bitfields using Lua syntax, or perhaps pack and unpack functions:
>
> template = "<<Sign:1,Exponent:11,Mantissa:52>>"
> name = "64-bit IEEE floating point"
> b = bitfelds.parse(name,template)
> s = io.read(file,b.size)
> t = b:unpack(s)
> print(t.Sign,t.Exponent,t.Mantissa)
> t.Exponent=0
> s = b:pack(t)
>

Erlang's bit syntax is a joy to work with and I miss it every time I use
another programming language to decode protocols.
In particular, I like the inline binding to variables as opposed to the
regex/scanf like approach taken by other languages.

An efficient analogue (to Erlang's bit syntax) in Lua would be
tremendously useful.

Lua tables would work great for something like this.  :)

/todd


Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Sean Conner
It was thus said that the Great Todd Coram once stated:

> On Mon, Jul 15, 2013, at 09:58 PM, Luiz Henrique de Figueiredo wrote:
> > > 64-bit IEEE floating point:
> > > <<Sign:1,Exponent:11,Mantissa:52>>
> > >
> > > IP datagram:
> > > <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16,
> > >       ID:16, Flgs:3, FragOff:13,
> > >       TTL:8, Proto:8, HdrChkSum:16,
> > >       SrcIP:32, DestIP:32, RestDgram/binary>>
> >
> > Given these strings it easy to parse them into a Lua table containing the
> > names and starting and end points of the bitfields described there. That
> > table can then be used to set index and newindex metamethods for get and
> > put the bitfields using Lua syntax, or perhaps pack and unpack functions:
> >
> > template = "<<Sign:1,Exponent:11,Mantissa:52>>"
> > name = "64-bit IEEE floating point"
> > b = bitfelds.parse(name,template)
> > s = io.read(file,b.size)
> > t = b:unpack(s)
> > print(t.Sign,t.Exponent,t.Mantissa)
> > t.Exponent=0
> > s = b:pack(t)
> >
>
> Erlang's bit syntax is a joy to work with and I miss it every time I use
> another programming language to decode protocols.
> In particular, I like the inline binding to variables as opposed to the
> regex/scanf like approach taken by other languages.
>
> An efficient analogue (to Erlang's bit syntax) in Lua would be
> tremendously useful.
>
> Lua tables would work great for something like this.  :)

  A good test for this would be DNS packets.  The header portion isn't that
bad:

        id:16,
        query:1, -- boolean, 0=query, 1=response
        opcode:4, -- type of query
        aa:1, -- boolean, authoritative answer
        tc:1, -- boolean, truncation
        rd:1, -- boolean, recursion desired
        ra:1, -- boolean, recursion available
        z:1, -- not used, must be 0
        ad:1, -- boolean, authentic bit
        cd:1, -- boolean, checking disabled
        rcode:4, -- return code
        qcount:16, -- query count
        ancount:16, -- answer count
        nscount:16, -- nameserver count
        adcount:16 -- additional count

But the record types are variable length:

        name:??, -- key for record, example: www.conman.org
        type:16, -- type of record
        class:16, -- class of record
        ttl:16, -- Time-To-Live
        rdlength:16, -- length of rdata
        rdata:?? -- rest of data

And the name portion isn't easy to skip---you *have* to parse it to find the
end (check RFC-1035 for the gory details).  

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Todd Coram
On Tue, Jul 16, 2013, at 11:50 AM, Sean Conner wrote:

> It was thus said that the Great Todd Coram once stated:
>>
> > Erlang's bit syntax is a joy to work with and I miss it every time I use
> > another programming language to decode protocols.
> > In particular, I like the inline binding to variables as opposed to the
> > regex/scanf like approach taken by other languages.
> >
> > An efficient analogue (to Erlang's bit syntax) in Lua would be
> > tremendously useful.
> >
> > Lua tables would work great for something like this.  :)
>
>   A good test for this would be DNS packets.  The header portion isn't
>   that
> bad:
>
> id:16,
> query:1, -- boolean, 0=query, 1=response
> opcode:4, -- type of query
> aa:1, -- boolean, authoritative answer
> tc:1, -- boolean, truncation
> rd:1, -- boolean, recursion desired
> ra:1, -- boolean, recursion available
> z:1, -- not used, must be 0
> ad:1, -- boolean, authentic bit
> cd:1, -- boolean, checking disabled
> rcode:4, -- return code
> qcount:16, -- query count
> ancount:16, -- answer count
> nscount:16, -- nameserver count
> adcount:16 -- additional count
>
> But the record types are variable length:
>
> name:??, -- key for record, example: www.conman.org
> type:16, -- type of record
> class:16, -- class of record
> ttl:16, -- Time-To-Live
> rdlength:16, -- length of rdata
> rdata:?? -- rest of data
>
> And the name portion isn't easy to skip---you *have* to parse it to find
> the
> end (check RFC-1035 for the gory details).  
>
>   -spc
>
>

I don't see the problem. You do have to do some parsing, but the header
is indeed a no-brainer. In Erlang i'd use the bit syntax parser for the
header, write a  parser for "name", then grab the rest of the record
with bit syntax (rdata can be captured by an addition bit string by
using the captured rdlength).

I am not suggesting that Lua add a "half baked" implementation of Erlang
bit syntax or that the syntax magically solves all parsing problems.
For sufficiently complex protocols  you are going to do some hand
parsing.

I just think that the declarative portion provided by Erlang bit syntax
is very nice and is worthy of study when considering add bit parser
support to Lua.

/todd

Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Jerome Vuarand
In reply to this post by Robert Virding
2013/7/16 Robert Virding <[hidden email]>:

> While operators for bit-fields are very useful I think a better way is a syntax which allows you describe/build the whole bitstring in one go. We have this in Erlang and it makes building/pulling apart protocol packets very much easier and clearer. Some simple examples of the top of my head:
>
> 64-bit IEEE floating point:
> <<Sign:1,Exponent:11,Mantissa:52>>
>
> IP datagram:
> <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16,
>       ID:16, Flgs:3, FragOff:13,
>       TTL:8, Proto:8, HdrChkSum:16,
>       SrcIP:32, DestIP:32, RestDgram/binary>>
>
> The second one is a bit longer than 32 bits. :-) I think you get the idea. We can also use the same syntax for pulling apart the binary/bitstring. This really is powerful and simplifies a lot of things and I think that something similar would be useful for Lua if you seriously intend to work with bitfields.

There are already plenty of libraries that simplify parsing such
binary structures, some using the Lua flexible syntax as a kind of DSL
to describe them. I don't see how another set of syntactic constructs
could make it any easier or clearer. Can you elaborate on that claim?

Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Robert Virding
> From: "Jerome Vuarand" <[hidden email]>
>
> 2013/7/16 Robert Virding <[hidden email]>:
> > While operators for bit-fields are very useful I think a better way
> > is a syntax which allows you describe/build the whole bitstring in
> > one go. We have this in Erlang and it makes building/pulling apart
> > protocol packets very much easier and clearer. Some simple
> > examples of the top of my head:
> >
> > 64-bit IEEE floating point:
> > <<Sign:1,Exponent:11,Mantissa:52>>
> >
> > IP datagram:
> > <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16,
> >       ID:16, Flgs:3, FragOff:13,
> >       TTL:8, Proto:8, HdrChkSum:16,
> >       SrcIP:32, DestIP:32, RestDgram/binary>>
> >
> > The second one is a bit longer than 32 bits. :-) I think you get
> > the idea. We can also use the same syntax for pulling apart the
> > binary/bitstring. This really is powerful and simplifies a lot of
> > things and I think that something similar would be useful for Lua
> > if you seriously intend to work with bitfields.
>
> There are already plenty of libraries that simplify parsing such
> binary structures, some using the Lua flexible syntax as a kind of
> DSL
> to describe them. I don't see how another set of syntactic constructs
> could make it any easier or clearer. Can you elaborate on that claim?

- I make no claims to be a Lua expert, so I am not surprised to hear that there are libraries that support this type of thing.

- My comment was on the suggestion to add operator syntax for bitfields. I have done enough bitfiddling in both C and Erlang to appreciate Erlang's bitsyntax for building/pulling apart bitstrings.

- If it is something that is to be done often and speed is critical then you need some for of "compiler" support. You can either add it to the language like in Erlang or maybe have it so that your library can compile a bit pattern into some descriptive data structure or code for processing the bitstring. Erlang can do something similar to this for regular expressions.

- To be clear I wasn't suggesting using the Erlang syntax, it is to Erlangy and wouldn't fit into Lua. Although it does have a similar feel as Lua syntax for constructing tables and you could probably add some special syntax which builds a suitable table.

Robert

Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Ico Doornekamp
In reply to this post by Jerome Vuarand
* On Wed Jul 17 01:20:42 +0200 2013, Jerome Vuarand wrote:
 
> 2013/7/16 Robert Virding <[hidden email]>:

> There are already plenty of libraries that simplify parsing such
> binary structures, some using the Lua flexible syntax as a kind of DSL
> to describe them.

Interesting. I'm only aware of Luiz Henrique de Figueiredo's lpack, but
as far as I know this supports regular data width types only.

Can you give pointers to any of the other libraries you are referring to?

--
:wq
^X^Cy^K^X^C^C^C^C

Reply | Threaded
Open this post in threaded view
|

Re: An operator syntax for bitfields using metatables

Rena
In reply to this post by John Hind

On 2013-07-12 11:26 AM, "John Hind" <[hidden email]> wrote:
>
> I've been following the discussion on the bit32 library and share the desire
> for an operator rather than a functional syntax. However I realised that 99%
> of my use of this library is for cracking and assembling bitfields for
> communications protocols and register mapping in embedded systems. It is
> straightforward to add a syntax for this in a C library by providing a
> metatable for the number type with 'index' and 'newindex' metamethods. Then:
>
> b = n[3]        -- Extract bit 3 as a boolean.
> n[23] = true    -- Set bit 23.
> n[6] = not n[6] -- Toggle bit 6.
>

This, I like. (Though I wonder if n.bit[3] would be even better.) It's a shame this can't be made to work with Lua as it's currently implemented, but maybe it could be possible in a future version.

Beyond that, the bit ranges and fields syntax starts to look pretty hairy.

I also do like the idea of a simple way to describe bitfield structures, but I don't think it needs to be built into the language; a module should be enough. (Also, "bitfield structure" sounds a lot like "struct", and now we're moving beyond the topic of bit twiddling...)