[ANN] lua-buffet

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] lua-buffet

Dmitry Meyer
Hi list,

I would like to present one of my recent projects.

This is not actually a new piece of software, but it has never been
announced here (I only tried /r/lua, where, as I understand it, only a
really small fraction of Lua developers is present), so better late than
never.

---

Summary:

Socket-like buffer objects suitable for data stream processing, unit
testing, and so on.

Links:

* Homepage: https://github.com/un-def/lua-buffet
* Examples: https://github.com/un-def/lua-buffet/tree/master/examples/
* Luarocks: https://luarocks.org/modules/undef/lua-buffet
* OPM: https://opm.openresty.org/package/un-def/lua-buffet/

Description:

A buffet is an object that has the same interface as socket objects in
the popular Lua libraries LuaSocket and Lua Nginx Module but doesn't do
any real network communication. Instead the network stack the buffet
receives data from an arbitrary source. The data source can be a string,
a table of strings or an iterator function producing strings.

The buffet works in a streaming fashion. That is, the buffet doesn't try
to read and store internally the whole source data at once but reads as
little as possible and only when necessary. That means that the buffet
can be efficiently used as a proxy for sources of unlimited amounts of
data such as real sockets or file I/O readers.

Another possible use is unit testing where the buffet can be used as a
mock object instead of the real socket object.

Fun fact: the word "buffet" is a portmanteau of "buffer" and "socket".

Status:

As of version 0.2.0, all of ngx.socket.tcp methods are
implemented/mocked. LuaSocket TCP interface is not implemented but
ngx.socket.tcp implementation should be compatible. UDP interfaces are
not implemented. All missing interfaces can be easily added if there is
a need.

License:

The MIT License.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] lua-buffet

Pierre Chapuis
On Fri, Nov 20, 2020, at 16:01, Dmitry Meyer wrote:

> A buffet is an object that has the same interface as socket objects in
> the popular Lua libraries LuaSocket and Lua Nginx Module but doesn't do
> any real network communication. Instead the network stack the buffet
> receives data from an arbitrary source. The data source can be a string,
> a table of strings or an iterator function producing strings.
>
> The buffet works in a streaming fashion. That is, the buffet doesn't try
> to read and store internally the whole source data at once but reads as
> little as possible and only when necessary. That means that the buffet
> can be efficiently used as a proxy for sources of unlimited amounts of
> data such as real sockets or file I/O readers.
>
> Another possible use is unit testing where the buffet can be used as a
> mock object instead of the real socket object.

Hello Dmitry,

this is interesting. Some of the goals kind of remind me with ltn12.
It would be interesting if you supported ltn12 source and sinks.

I can see certainly see myself using this for mocks someday.

--
Pierre Chapuis
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] lua-buffet

Dmitry Meyer
On 20/11/2020 19:31, Pierre Chapuis wrote:

> It would be interesting if you supported ltn12 source and sinks.


I didn't do anything special to support ltn12, but at first glance it "just works":

--- Code:

local ltn12 = require('ltn12')
local mime = require('mime')
local buffet = require('buffet')
local buffet_resty = require('buffet.resty')

local src = ltn12.source.chain(
    ltn12.source.cat(
        ltn12.source.string('Lorem ipsum dolor sit amet, consectetur '),
        ltn12.source.string('adipiscing elit, sed do eiusmod tempor '),
        ltn12.source.string('incididunt ut labore et dolore magna aliqua.'),
        ltn12.source.error('boom!'),
        ltn12.source.string('unreachable')
    ),
    ltn12.filter.chain(
        function(chunk) return chunk and chunk:upper() end,
        mime.wrap('text', 16)
    )
)

local bf = buffet_resty.new(src)
repeat
    local data, err, partial = bf:receive('*l')
    print(data or partial)
    if err then print(err) end
until err

print(buffet.get_iterator_error(bf))

--- Output:

LOREM IPSUM DOLO
R SIT AMET, CONS
ECTETUR ADIPISCI
NG ELIT, SED DO
EIUSMOD TEMPOR I
NCIDIDUNT UT LAB
ORE ET DOLORE MA
GNA ALIQUA.
closed
boom!
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] lua-buffet

Pierre Chapuis
On Fri, Nov 20, 2020, at 18:45, Dmitry Meyer wrote:

> I didn't do anything special to support ltn12, but at first glance it
> "just works":

Great, even better. Thanks for this library!