C#/NLua + string.pack - Binary Message Not What I Expect

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

C#/NLua + string.pack - Binary Message Not What I Expect

Russell Haley
Hi,

My project is a serial port console written in C# that uses an embedded Lua interpreter via NLua. I have a lua script to send a binary messages out the serial port. I am using string.pack to create the message. My lua script:

local bytes = {0x5AA5}
local cable_info_msg = ""

for i,v in pairs(bytes) do
    print (v)
    print(string.format("%x",tonumber(v)))
    cable_info_msg = cable_info_msg .. string.pack("<h", v)
    num =  string.unpack("<h",cable_info_msg)
    if not num then print ('nil')
    else
        print(cable_info_msg)
        print(num)
        print(string.format("%x",num))    
    end
end
--A C# function...
WriteRemote(cable_info_msg, false)

-------------------------------------------------------------------
My console output from the script seems to be consistent with what I expect: 

-- Input
23205
5aa5
-- Output
ÑZ
23205
5aa5
---------------------------------------------------------------------
My C# WriteRemote is simple: 
        public void WriteRemote(string data, bool appendLineEnding = true)
        {
            if(_logging)
            {
                _scriptLog.WriteLine(data);
            }
            data = appendLineEnding ? data + _lineEnding : data;
            byte[] bytes = Encoding.UTF8.GetBytes(data);
            src.Write(bytes,0,bytes.Length);
        }
And the Lua/C# bindings are simple too:

        private Lua NewEnv()
        {
            Lua env = new Lua();
            env["WriteConsole"] = new Action<string>(WriteConsole);
            env["ReadConsole"] = new Func<string>(ReadConsole);
            env["WriteRemote"] = new Action<string,bool>(WriteRemote);
...                
            return env;
        }

When I look at the final bytes I am about to send on the C# debugger, the value seems to have been mangled (truncated?) to 0x3F5A. That value is also what I get on the other end of the serial port but I was expecting 0xA55A.

At this point I have two candidates for the cause:
- I'm an idiot and this is perfectly explainable behaviour (most probable)
- Something bad is happening in the translation between Lua and C#
-?

Does anyone have a Lua explanation why I would be getting different values sent out the port than what I am expecting? I'd like to rule that out before I start investigating NLua.

All input welcome. Thanks,

Russ


Reply | Threaded
Open this post in threaded view
|

Re: C#/NLua + string.pack - Binary Message Not What I Expect

Russell Haley


On Thu, Oct 17, 2019 at 12:00 PM Russell Haley <[hidden email]> wrote:
Hi,

My project is a serial port console written in C# that uses an embedded Lua interpreter via NLua. I have a lua script to send a binary messages out the serial port. I am using string.pack to create the message. My lua script:

local bytes = {0x5AA5}
local cable_info_msg = ""

for i,v in pairs(bytes) do
    print (v)
    print(string.format("%x",tonumber(v)))
    cable_info_msg = cable_info_msg .. string.pack("<h", v)
    num =  string.unpack("<h",cable_info_msg)
    if not num then print ('nil')
    else
        print(cable_info_msg)
        print(num)
        print(string.format("%x",num))    
    end
end
--A C# function...
WriteRemote(cable_info_msg, false)

-------------------------------------------------------------------
My console output from the script seems to be consistent with what I expect: 

-- Input
23205
5aa5
-- Output
ÑZ
23205
5aa5
---------------------------------------------------------------------
My C# WriteRemote is simple: 
        public void WriteRemote(string data, bool appendLineEnding = true)
        {
            if(_logging)
            {
                _scriptLog.WriteLine(data);
            }
            data = appendLineEnding ? data + _lineEnding : data;
            byte[] bytes = Encoding.UTF8.GetBytes(data);
            src.Write(bytes,0,bytes.Length);
        }
And the Lua/C# bindings are simple too:

        private Lua NewEnv()
        {
            Lua env = new Lua();
            env["WriteConsole"] = new Action<string>(WriteConsole);
            env["ReadConsole"] = new Func<string>(ReadConsole);
            env["WriteRemote"] = new Action<string,bool>(WriteRemote);
...                
            return env;
        }

When I look at the final bytes I am about to send on the C# debugger, the value seems to have been mangled (truncated?) to 0x3F5A. That value is also what I get on the other end of the serial port but I was expecting 0xA55A.

At this point I have two candidates for the cause:
- I'm an idiot and this is perfectly explainable behaviour (most probable)
- Something bad is happening in the translation between Lua and C#
-?

Does anyone have a Lua explanation why I would be getting different values sent out the port than what I am expecting? I'd like to rule that out before I start investigating NLua.

All input welcome. Thanks,

Russ

Sorry, the source code for the project is here: https://github.com/RussellHaley/NLuaSerialConsole

Russ 
Reply | Threaded
Open this post in threaded view
|

Re: C#/NLua + string.pack - Binary Message Not What I Expect

Sean Conner
In reply to this post by Russell Haley
It was thus said that the Great Russell Haley once stated:
> Hi,
>
> My project is a serial port console written in C# that uses an embedded Lua
> interpreter via NLua. I have a lua script to send a binary messages out the
> serial port. I am using string.pack to create the message. My lua script:
>
> local bytes = {0x5AA5}

  [ snip ]

> My C# WriteRemote is simple:
>         public void WriteRemote(string data, bool appendLineEnding = true)
>         {
>             if(_logging)
>             {
>                 _scriptLog.WriteLine(data);
>             }
>             data = appendLineEnding ? data + _lineEnding : data;
>             byte[] bytes = Encoding.UTF8.GetBytes(data);
                                      ^^^^

The string you are generating, 0xA5,0x5A, is not a valid UTF-8 string.
Could that perhaps be the issue?

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: C#/NLua + string.pack - Binary Message Not What I Expect

Russell Haley


On Thu, Oct 17, 2019 at 1:41 PM Sean Conner <[hidden email]> wrote:
It was thus said that the Great Russell Haley once stated:
> Hi,
>
> My project is a serial port console written in C# that uses an embedded Lua
> interpreter via NLua. I have a lua script to send a binary messages out the
> serial port. I am using string.pack to create the message. My lua script:
>
> local bytes = {0x5AA5}

  [ snip ]

> My C# WriteRemote is simple:
>         public void WriteRemote(string data, bool appendLineEnding = true)
>         {
>             if(_logging)
>             {
>                 _scriptLog.WriteLine(data);
>             }
>             data = appendLineEnding ? data + _lineEnding : data;
>             byte[] bytes = Encoding.UTF8.GetBytes(data);
                                      ^^^^

The string you are generating, 0xA5,0x5A, is not a valid UTF-8 string.
Could that perhaps be the issue?

  -spc

It very well could be. I'll look into this tomorrow. Thanks Sean!

Russ 
Reply | Threaded
Open this post in threaded view
|

Re: C#/NLua + string.pack - Binary Message Not What I Expect

Vinicius Jarina
Hello Russel

NLua will use the current Encoding to convert the Lua string to C# string.

There was a minor issue on NLua where wasn't possible to pass a Lua string to a byte [] parameter on C#.

I've fixed the issue. Here is a example how to pass binary data from Lua side to C# side.


Thank you

On Thu, Oct 17, 2019 at 8:01 PM Russell Haley <[hidden email]> wrote:


On Thu, Oct 17, 2019 at 1:41 PM Sean Conner <[hidden email]> wrote:
It was thus said that the Great Russell Haley once stated:
> Hi,
>
> My project is a serial port console written in C# that uses an embedded Lua
> interpreter via NLua. I have a lua script to send a binary messages out the
> serial port. I am using string.pack to create the message. My lua script:
>
> local bytes = {0x5AA5}

  [ snip ]

> My C# WriteRemote is simple:
>         public void WriteRemote(string data, bool appendLineEnding = true)
>         {
>             if(_logging)
>             {
>                 _scriptLog.WriteLine(data);
>             }
>             data = appendLineEnding ? data + _lineEnding : data;
>             byte[] bytes = Encoding.UTF8.GetBytes(data);
                                      ^^^^

The string you are generating, 0xA5,0x5A, is not a valid UTF-8 string.
Could that perhaps be the issue?

  -spc

It very well could be. I'll look into this tomorrow. Thanks Sean!

Russ 
Reply | Threaded
Open this post in threaded view
|

Re: C#/NLua + string.pack - Binary Message Not What I Expect

Russell Haley


On Thu, Oct 17, 2019 at 10:34 PM Vinicius Jarina <[hidden email]> wrote:
Hello Russel

NLua will use the current Encoding to convert the Lua string to C# string.

There was a minor issue on NLua where wasn't possible to pass a Lua string to a byte [] parameter on C#.

I've fixed the issue. Here is a example how to pass binary data from Lua side to C# side.


Thank you

That's fantastic. It works seamlessly in my test jig. This is going to be perfect for my serial console!

using System;
using System.Linq;
using NLua;

namespace NLuaConsoleTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Starting Lua");

            Lua lua = new Lua();
            lua["WriteBinary"] = (Action<byte[]>)WriteBinary;
            lua.DoString(@"
print('testing')
local t = {0x5aa5, 0x0600}
str = string.pack('<h<h',t[1], t[2])
print(str)
WriteBinary(str)
");

            Console.ReadLine();
        }

        private static void WriteBinary(byte[] buffer)
        {
            foreach(var b in buffer)
            {
                Console.WriteLine("{0:X}", b);
            }
        }
    }
}

Starting Lua
testing
ÑZ
A5
5A
0
6

Russ  

On Thu, Oct 17, 2019 at 8:01 PM Russell Haley <[hidden email]> wrote:


On Thu, Oct 17, 2019 at 1:41 PM Sean Conner <[hidden email]> wrote:
It was thus said that the Great Russell Haley once stated:
> Hi,
>
> My project is a serial port console written in C# that uses an embedded Lua
> interpreter via NLua. I have a lua script to send a binary messages out the
> serial port. I am using string.pack to create the message. My lua script:
>
> local bytes = {0x5AA5}

  [ snip ]

> My C# WriteRemote is simple:
>         public void WriteRemote(string data, bool appendLineEnding = true)
>         {
>             if(_logging)
>             {
>                 _scriptLog.WriteLine(data);
>             }
>             data = appendLineEnding ? data + _lineEnding : data;
>             byte[] bytes = Encoding.UTF8.GetBytes(data);
                                      ^^^^

The string you are generating, 0xA5,0x5A, is not a valid UTF-8 string.
Could that perhaps be the issue?

  -spc

It very well could be. I'll look into this tomorrow. Thanks Sean!

Russ