Luasocket + Luasec not retrieving data from https api

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Luasocket + Luasec not retrieving data from https api

lua.greatwolf
Hi all,

I'm running into some problems when I try to query data from an exchange api through https when
using luasockets + luasec. When I perform the same query using python I get the expected data back
but when done with lua I keep getting back "{"error":"Invalid command."}".

To make this issue easier to reproduce I've created a dummy account on the exchange along with the
associated secret api keys used in the scripts below.

Lua test query:    http://pastebin.com/Lt5PjwnN
Python test query: http://pastebin.com/PzgGm7TU

The api doc for the exchange in question: https://poloniex.com/api

I've checked the usual things like urlencoding and sha512 hmac signatures and they both match
between the two scripts. For the life of me, I cannot figure out why this works fine in python but
not in lua!

Is there some difference in the way luasec and luasockets is performing the https POST request that
differs from the python version using urllib? What could be causing the problem?

Thanks!



Reply | Threaded
Open this post in threaded view
|

Re: Luasocket + Luasec not retrieving data from https api

lua.greatwolf
After much investigation, I think I made some progress. It looks like I have to include "Content-Type" and "Content-Length" in the header for the POST request. So now instead of getting "{"error":"Invalid command."}", I get "{"error":"Invalid API key/secret pair."}".

The hmac-sha512 digest is computed from the secret and post data request sent to the server. Since I get the same hash as in python, I'm guessing somehow the server end isn't getting the post data correctly.

Anyone have any ideas?

Here's the updated script so far:

  http://pastebin.com/xBVNbucT

The main change is in the header table:

  headers =
  {
    Sign = signature,
    Key = key,
    ["Content-Type"] = "application/x-www-form-urlencoded",
    ["Content-Length"] = #post_data
  },

As a sidenote, I've manually constructed the POST request using the chrome app addon called "POSTMAN" as a test. The query successfully goes through when using the following 3 header-fields: Key, Sign, Content-Type with content type being "application/x-www-form-urlencoded". Filling the body of the request with "nonce=1399116375497&command=returnBalances" in raw form.

So I'm stumped as to why luasocket+luasec doesn't want to work.
Reply | Threaded
Open this post in threaded view
|

Re: Luasocket + Luasec not retrieving data from https api

Peng Zhicheng
On 05/04/2014 03:18 PM, lua.greatwolf wrote:

> After much investigation, I think I made some progress. It looks like I have
> to include "Content-Type" and "Content-Length" in the header for the POST
> request. So now instead of getting "{"error":"Invalid command."}", I get
> "{"error":"Invalid API key/secret pair."}".
>
> The hmac-sha512 digest is computed from the secret and post data request
> sent to the server. Since I get the same hash as in python, I'm guessing
> somehow the server end isn't getting the post data correctly.
>
> Anyone have any ideas?
>
> Here's the updated script so far:
>
>    http://pastebin.com/xBVNbucT
>
> The main change is in the header table:
>
>    headers =
>    {
>      Sign = signature,
>      Key = key,
>      ["Content-Type"] = "application/x-www-form-urlencoded",
>      ["Content-Length"] = #post_data
>    },
>
> As a sidenote, I've manually constructed the POST request using the chrome
> app addon called "POSTMAN" as a test. The query successfully goes through
> when using the following 3 header-fields: Key, Sign, Content-Type with
> content type being "application/x-www-form-urlencoded". Filling the body of
> the request with "nonce=1399116375497&command=returnBalances" in raw form.
>
> So I'm stumped as to why luasocket+luasec doesn't want to work.
>
>
>
I don't know the answer but I could provide some debugging tricks I often used.

Usually when I encountered such problems, I would write some code as a "fake" server,
whose only task was to dump the request the client sent, to aid dubugging.
for HTTP requests, this could be trivially done using luasocket.
but I don't know the SSL thing, so I would use a ready-made web server supporting HTTPS.

I don't use python, so I don't have full Python installed in my system. but I had no problem
run your Lua code. the dumped request was like this:
----------------8x-------------------------------------------
  2: REQUEST: POST
  3: HEADERS:
  5: Content-Type: application/x-www-form-urlencoded
  5: Host: localhost
  5: key: 89QK21JO-8O72SN4U-ZUR4CGE5-TSGNT63S
  5: sign: a97b404e4610b893a8bbaf5f8a458b5bef24b1b67bd279785380667944821c8258d25de9b32843168be5c46c8d79904a753969062cbc9e1c682fd394037a94b3
  5: TE: trailers
  5: User-Agent: LuaSocket 3.0-rc1
  5: Content-Length: 42
  5: Connection: close, TE
  7: PARAMETERS:
  9: nonce=1399116375497
  9: command=returnBalances
----------------8x-------------------------------------------

the "fake" server script is very simple:
----------------8x-------------------------------------------
<?lsp
trace("REQUEST: " .. request:method())
trace("HEADERS:")
for k, v in pairs(request:header()) do
    trace(k .. ": " .. v)
end
trace("PARAMETERS:")
for k, v in request:datapairs() do
    trace(k .. "=" .. v)
end
?>
<html><head><title>OK</title></head></html>
----------------8x-------------------------------------------

I used mako server[http://makoserver.net] to do the testing. it is very easy to use.
and would work out of the box, just download, unzip, and run. yet, it is quite powerful.

to use the the above code, just save it as a .lsp file, e.g. $HOME/tmp/dump.lsp
then run the following command:
   mako -l::$HOME/tmp
the page is now served at the URL https://localhost:9443/dump.lsp