cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Russell Haley
Hello, 

Some months back I started running a Minecraft server (Java) on FreeBSD. There are two issues that I've run into:
1) The Minecraft client (game) automatically updates and will not run against servers of prior revisions without changing client settings.  
2) The Minecraft 'server' jar uses stdin and stdout. That means (in my understanding) that the process must be kept in the foreground for interaction. The current solution I've found (on the Minecraft wiki) requires screen/tmux to allow one to run a server instance from ssh. 

The solution I now envision is to use lua-http and cqueues to create a 'controller' that updates the server.jar file and runs the application without the need for the process to be in the foreground. The parts I see needing are:
- A routine that checks for updates and downloads the jar files
- A 'thread' for running the minecraft server.jar file in java. This 'thread' would re-direct stdin and stdout. This 'thread' should also be able to stop and start the running jar file. 
- A websocket server to funnel the captured IO to a webpage for interactivity.

So far, I have roughed in the server.jar checks and download, and the websocket server for communications. The code can be found here: https://github.com/RussellHaley/minecraft-runner/mc-runner.lua

What I don't know/understand yet if it's possible to run/control a process from Lua so that I can grab stdin/stdout? I am hoping there is something in cqueues to do this, but once again thought I'd send out a general request for ideas. 

Any input would be grand. 

Thanks!
Russ
Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Petite Abeille


> On Sep 12, 2018, at 9:30 PM, Russell Haley <[hidden email]> wrote:
>
> uses stdin and stdout. That means (in my understanding) that the process must be kept in the foreground for interaction

Could you perhaps redirect stdin and stdout in the first place? And then perhaps read & write to them directly for ‘interaction’ purpose?

E.g.:

os.execute( 'mkfifo output' )

local aWriter = assert( io.popen( 'sqlplus -S /NOLOG > output', 'w' ) )
local aReader = assert( io.open( 'output', 'r' ) )

assert( aWriter:write( 'connect schema/pass@instance', '\n' ) )
assert( aWriter:flush() )

assert( aWriter:write( 'prompt -8<-;', '\n' ) )
assert( aWriter:write( 'select * from user_users;', '\n' ) )
assert( aWriter:write( 'prompt ->8-;', '\n' ) )
assert( aWriter:flush() )

repeat
 local aLine = aReader:read()
 print( '*', aLine, aLine:len() )
until not aLine or aLine == '->8-'


Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Daurnimator
In reply to this post by Russell Haley
On 12 September 2018 at 12:30, Russell Haley <[hidden email]> wrote:
> What I don't know/understand yet if it's possible to run/control a process
> from Lua so that I can grab stdin/stdout?

Use https://github.com/daurnimator/lua-spawn to start programs
See https://github.com/daurnimator/lua-spawn/blob/master/examples/pipes.lua
for an example of controlling stdin/stdout of a child.

> I am hoping there is something in
> cqueues to do this, but once again thought I'd send out a general request
> for ideas.

You should be able to use cqueues.socket.dup or cqueues.socket.fdopen
to use normal cqueues socket functions with most types of file
descriptors.

Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Petite Abeille


> On Sep 12, 2018, at 11:02 PM, Daurnimator <[hidden email]> wrote:
>
> Use https://github.com/daurnimator/lua-spawn to start programs

^ • io.popen only allows interaction with one of stdin or stdout

Wouldn’t a bit of mkfifo go a long way?
Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Sean Conner
In reply to this post by Russell Haley
It was thus said that the Great Russell Haley once stated:

> Hello,
>
> Some months back I started running a Minecraft server (Java) on FreeBSD.
> There are two issues that I've run into:
> 1) The Minecraft client (game) automatically updates and will not run
> against servers of prior revisions without changing client settings.
> 2) The Minecraft 'server' jar uses stdin and stdout. That means (in my
> understanding) that the process must be kept in the foreground for
> interaction. The current solution I've found (on the Minecraft wiki)
> requires screen/tmux to allow one to run a server instance from ssh.

  No, you can run the process as:

  GenericUnixPrompt> java -jar BlahObjectThing </dev/null >/dev/null 2>/dev/null &

That will run your server in the background with stdin, stdout and stderr
all pointing to /dev/null (the bit bucket).  

> The solution I now envision is to use lua-http and cqueues to create a
> 'controller' that updates the server.jar file and runs the application
> without the need for the process to be in the foreground. The parts I see
> needing are:
> - A routine that checks for updates and downloads the jar files
> - A 'thread' for running the minecraft server.jar file in java. This
> 'thread' would re-direct stdin and stdout. This 'thread' should also be
> able to stop and start the running jar file.
> - A websocket server to funnel the captured IO to a webpage for
> interactivity.

  Avoid system level threads for this.  Mixing system level threads
(pthreads) and running another process (via fork()) do not mix [1].  So
you'll end up with Lua coroutines and a sepearate process.  Let's handle
that first.

        function start_minecraft()
          local pid,err = fork()
         
          if pid then
     if pid == 0 then -- child process
            redirect(stdin,"/dev/null") -- or a file or a pipe
            redirect(stdout,"/dev/null")
            redirect(stderr,"/dev/null")
            exec("java",{ "-jar" , "minecraft.jar" })
            _exit(1) -- exec() failed, we exit
          end
       
          -- mindcraft has PID of child
          -- it is running, and so are we.
       
          return pid,err
        end
       
  stdin, stdout and stderr can be redirected to a file, a pipe, a named
pipe, what have you.  I'm justing using "/dev/null" as an example for now.
You could use a file that you serve up through HTTP to see what happens; a
pipe to get the last N bytes of logging, etc.  That's up to you.

  I should note that the functions fork(), redirect(), exec() and _exit()
need to be adjusted to whatever module you use that provides such calls
(redirect() may be called dup() or dup2() or something like that).

  The main process (as I would do it) would then use coroutines (or Lua
threads) to handle things.  One coroutine to periodically check for an
updated jar file, and if so:

        while true do
          info = stat("myjarfile.jar")
          if info.st_mtime ~= original.st_mtime then
            raise(minecraft_pid,SIG_TERM)
            status = wait(minecraft_pid)
            minecraft_pid = start_minecraft()
            original.st_mtime = info.st_mtime
          end
          wait_for(some_amount_of_time)
        end

> So far, I have roughed in the server.jar checks and download, and the
> websocket server for communications. The code can be found here:
> https://github.com/RussellHaley/minecraft-runner/mc-runner.lua
>
> What I don't know/understand yet if it's possible to run/control a process
> from Lua so that I can grab stdin/stdout? I am hoping there is something in
> cqueues to do this, but once again thought I'd send out a general request
> for ideas.
>
> Any input would be grand.

  You need something that handles networking, coroutines and some POSIX
functions (fork(), wait(), etc).  

  -spc

[1] It probably *can* be done, but it's certainly above my pay grade,
        and I write servers for a living.

Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Russell Haley
In reply to this post by Daurnimator


On Wed, Sep 12, 2018 at 2:03 PM Daurnimator <[hidden email]> wrote:
On 12 September 2018 at 12:30, Russell Haley <[hidden email]> wrote:
> What I don't know/understand yet if it's possible to run/control a process
> from Lua so that I can grab stdin/stdout?

Use https://github.com/daurnimator/lua-spawn to start programs
See https://github.com/daurnimator/lua-spawn/blob/master/examples/pipes.lua
for an example of controlling stdin/stdout of a child.

> I am hoping there is something in
> cqueues to do this, but once again thought I'd send out a general request
> for ideas.

You should be able to use cqueues.socket.dup or cqueues.socket.fdopen
to use normal cqueues socket functions with most types of file
descriptors.


Thanks Daurnimator. I must say you maintain an impressive body of work.  

Russ
Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Russell Haley
In reply to this post by Petite Abeille


On Wed, Sep 12, 2018 at 1:56 PM Petite Abeille <[hidden email]> wrote:


> On Sep 12, 2018, at 9:30 PM, Russell Haley <[hidden email]> wrote:
>
> uses stdin and stdout. That means (in my understanding) that the process must be kept in the foreground for interaction

Could you perhaps redirect stdin and stdout in the first place? And then perhaps read & write to them directly for ‘interaction’ purpose?

E.g.:

os.execute( 'mkfifo output' )

local aWriter = assert( io.popen( 'sqlplus -S /NOLOG > output', 'w' ) )
local aReader = assert( io.open( 'output', 'r' ) )

assert( aWriter:write( 'connect schema/pass@instance', '\n' ) )
assert( aWriter:flush() )

assert( aWriter:write( 'prompt -8<-;', '\n' ) )
assert( aWriter:write( 'select * from user_users;', '\n' ) )
assert( aWriter:write( 'prompt ->8-;', '\n' ) )
assert( aWriter:flush() )

repeat
 local aLine = aReader:read()
 print( '*', aLine, aLine:len() )
until not aLine or aLine == '->8-'

mkfifo looks interesting. Thanks for this. 
Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Russell Haley
In reply to this post by Sean Conner


On Wed, Sep 12, 2018 at 2:17 PM Sean Conner <[hidden email]> wrote:
It was thus said that the Great Russell Haley once stated:
> Hello,
>
> Some months back I started running a Minecraft server (Java) on FreeBSD.
> There are two issues that I've run into:
> 1) The Minecraft client (game) automatically updates and will not run
> against servers of prior revisions without changing client settings.
> 2) The Minecraft 'server' jar uses stdin and stdout. That means (in my
> understanding) that the process must be kept in the foreground for
> interaction. The current solution I've found (on the Minecraft wiki)
> requires screen/tmux to allow one to run a server instance from ssh.

  No, you can run the process as:

  GenericUnixPrompt> java -jar BlahObjectThing </dev/null >/dev/null 2>/dev/null &

That will run your server in the background with stdin, stdout and stderr
all pointing to /dev/null (the bit bucket).  

> The solution I now envision is to use lua-http and cqueues to create a
> 'controller' that updates the server.jar file and runs the application
> without the need for the process to be in the foreground. The parts I see
> needing are:
> - A routine that checks for updates and downloads the jar files
> - A 'thread' for running the minecraft server.jar file in java. This
> 'thread' would re-direct stdin and stdout. This 'thread' should also be
> able to stop and start the running jar file.
> - A websocket server to funnel the captured IO to a webpage for
> interactivity.

  Avoid system level threads for this.  Mixing system level threads
(pthreads) and running another process (via fork()) do not mix [1].  So
you'll end up with Lua coroutines and a sepearate process.  Let's handle
that first.

        function start_minecraft()
          local pid,err = fork()

          if pid then
            if pid == 0 then -- child process
            redirect(stdin,"/dev/null") -- or a file or a pipe
            redirect(stdout,"/dev/null")
            redirect(stderr,"/dev/null")
            exec("java",{ "-jar" , "minecraft.jar" })
            _exit(1) -- exec() failed, we exit
          end

          -- mindcraft has PID of child
          -- it is running, and so are we.

          return pid,err
        end

  stdin, stdout and stderr can be redirected to a file, a pipe, a named
pipe, what have you.  I'm justing using "/dev/null" as an example for now.
You could use a file that you serve up through HTTP to see what happens; a
pipe to get the last N bytes of logging, etc.  That's up to you.

  I should note that the functions fork(), redirect(), exec() and _exit()
need to be adjusted to whatever module you use that provides such calls
(redirect() may be called dup() or dup2() or something like that).

  The main process (as I would do it) would then use coroutines (or Lua
threads) to handle things.  One coroutine to periodically check for an
updated jar file, and if so:

        while true do
          info = stat("myjarfile.jar")
          if info.st_mtime ~= original.st_mtime then
            raise(minecraft_pid,SIG_TERM)
            status = wait(minecraft_pid)
            minecraft_pid = start_minecraft()
            original.st_mtime = info.st_mtime
          end
          wait_for(some_amount_of_time)
        end

Thanks Conman! 
 
> So far, I have roughed in the server.jar checks and download, and the
> websocket server for communications. The code can be found here:
> https://github.com/RussellHaley/minecraft-runner/mc-runner.lua
>
> What I don't know/understand yet if it's possible to run/control a process
> from Lua so that I can grab stdin/stdout? I am hoping there is something in
> cqueues to do this, but once again thought I'd send out a general request
> for ideas.
>
> Any input would be grand.

  You need something that handles networking, coroutines and some POSIX
functions (fork(), wait(), etc). 

  -spc

[1]     It probably *can* be done, but it's certainly above my pay grade,
        and I write servers for a living.

Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

aryajur
I just used lpty http://tset.de/lpty/index.html for controlling a process as if in a separate terminal window in linux. It was quite easy and extremely useful.

Milind

On Wed, Sep 12, 2018, 10:33 PM Russell Haley <[hidden email]> wrote:


On Wed, Sep 12, 2018 at 2:17 PM Sean Conner <[hidden email]> wrote:
It was thus said that the Great Russell Haley once stated:
> Hello,
>
> Some months back I started running a Minecraft server (Java) on FreeBSD.
> There are two issues that I've run into:
> 1) The Minecraft client (game) automatically updates and will not run
> against servers of prior revisions without changing client settings.
> 2) The Minecraft 'server' jar uses stdin and stdout. That means (in my
> understanding) that the process must be kept in the foreground for
> interaction. The current solution I've found (on the Minecraft wiki)
> requires screen/tmux to allow one to run a server instance from ssh.

  No, you can run the process as:

  GenericUnixPrompt> java -jar BlahObjectThing </dev/null >/dev/null 2>/dev/null &

That will run your server in the background with stdin, stdout and stderr
all pointing to /dev/null (the bit bucket).  

> The solution I now envision is to use lua-http and cqueues to create a
> 'controller' that updates the server.jar file and runs the application
> without the need for the process to be in the foreground. The parts I see
> needing are:
> - A routine that checks for updates and downloads the jar files
> - A 'thread' for running the minecraft server.jar file in java. This
> 'thread' would re-direct stdin and stdout. This 'thread' should also be
> able to stop and start the running jar file.
> - A websocket server to funnel the captured IO to a webpage for
> interactivity.

  Avoid system level threads for this.  Mixing system level threads
(pthreads) and running another process (via fork()) do not mix [1].  So
you'll end up with Lua coroutines and a sepearate process.  Let's handle
that first.

        function start_minecraft()
          local pid,err = fork()

          if pid then
            if pid == 0 then -- child process
            redirect(stdin,"/dev/null") -- or a file or a pipe
            redirect(stdout,"/dev/null")
            redirect(stderr,"/dev/null")
            exec("java",{ "-jar" , "minecraft.jar" })
            _exit(1) -- exec() failed, we exit
          end

          -- mindcraft has PID of child
          -- it is running, and so are we.

          return pid,err
        end

  stdin, stdout and stderr can be redirected to a file, a pipe, a named
pipe, what have you.  I'm justing using "/dev/null" as an example for now.
You could use a file that you serve up through HTTP to see what happens; a
pipe to get the last N bytes of logging, etc.  That's up to you.

  I should note that the functions fork(), redirect(), exec() and _exit()
need to be adjusted to whatever module you use that provides such calls
(redirect() may be called dup() or dup2() or something like that).

  The main process (as I would do it) would then use coroutines (or Lua
threads) to handle things.  One coroutine to periodically check for an
updated jar file, and if so:

        while true do
          info = stat("myjarfile.jar")
          if info.st_mtime ~= original.st_mtime then
            raise(minecraft_pid,SIG_TERM)
            status = wait(minecraft_pid)
            minecraft_pid = start_minecraft()
            original.st_mtime = info.st_mtime
          end
          wait_for(some_amount_of_time)
        end

Thanks Conman! 
 
> So far, I have roughed in the server.jar checks and download, and the
> websocket server for communications. The code can be found here:
> https://github.com/RussellHaley/minecraft-runner/mc-runner.lua
>
> What I don't know/understand yet if it's possible to run/control a process
> from Lua so that I can grab stdin/stdout? I am hoping there is something in
> cqueues to do this, but once again thought I'd send out a general request
> for ideas.
>
> Any input would be grand.

  You need something that handles networking, coroutines and some POSIX
functions (fork(), wait(), etc). 

  -spc

[1]     It probably *can* be done, but it's certainly above my pay grade,
        and I write servers for a living.

Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

William Ahern
In reply to this post by Russell Haley
On Wed, Sep 12, 2018 at 12:30:58PM -0700, Russell Haley wrote:
<snip>
> What I don't know/understand yet if it's possible to run/control a process
> from Lua so that I can grab stdin/stdout? I am hoping there is something in
> cqueues to do this, but once again thought I'd send out a general request
> for ideas.

cqueues can be used to concurrently wait on raw descriptor events and
signals. But for low-level access to Unix process control APIs you need
something like luaposix or my lunix module.

The following regression script in lunix tests the behavior of PTYs and
process groups.

  https://github.com/wahern/lunix/blob/master/regress/0-ctty-sighup.lua

Specifically, it checks that SIGHUP is automatically sent to all the child
processes when the session leader dies, which is what should happen when the
controlling terminal is properly configured. (TTYs and process management
are intricately related because the evolution of job control in Unix made
the TTY driver responsible for sending various signals. See
http://www.linusakesson.net/programming/tty/)

The script shows how to use fork(2), pipe(2), and various process management
APIs exposed by lunix. Translating the I/O, signal, and timeout handling to
cqueues should [hopefully] be fairly straight-forward.

Normally you don't need to worry about PTYs and sessions if you're simply
trying to invoke a helper program and process stdin/stdout. But if you're
running as a daemon then its useful so that child processes are
automatically killed if the daemon crashes or terminates, especially if the
child process isn't in a simple loop reading stdin and writing to stdout
such that it would quickly receive EOF or SIGPIPE when the parent dies.

Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Russell Haley


On Thu, Sep 13, 2018 at 11:59 AM William Ahern <[hidden email]> wrote:
On Wed, Sep 12, 2018 at 12:30:58PM -0700, Russell Haley wrote:
<snip>
> What I don't know/understand yet if it's possible to run/control a process
> from Lua so that I can grab stdin/stdout? I am hoping there is something in
> cqueues to do this, but once again thought I'd send out a general request
> for ideas.

cqueues can be used to concurrently wait on raw descriptor events and
signals. But for low-level access to Unix process control APIs you need
something like luaposix or my lunix module.

The following regression script in lunix tests the behavior of PTYs and
process groups.

  https://github.com/wahern/lunix/blob/master/regress/0-ctty-sighup.lua

Specifically, it checks that SIGHUP is automatically sent to all the child
processes when the session leader dies, which is what should happen when the
controlling terminal is properly configured. (TTYs and process management
are intricately related because the evolution of job control in Unix made
the TTY driver responsible for sending various signals. See
http://www.linusakesson.net/programming/tty/)

The script shows how to use fork(2), pipe(2), and various process management
APIs exposed by lunix. Translating the I/O, signal, and timeout handling to
cqueues should [hopefully] be fairly straight-forward.

Normally you don't need to worry about PTYs and sessions if you're simply
trying to invoke a helper program and process stdin/stdout. But if you're
running as a daemon then its useful so that child processes are
automatically killed if the daemon crashes or terminates, especially if the
child process isn't in a simple loop reading stdin and writing to stdout
such that it would quickly receive EOF or SIGPIPE when the parent dies.

Thanks Bill. One of the things I thought of doing with cqueues was create an init/processes monitor, or augment one with Lua (https://github.com/mheily/jobd) so I appreciate this information. Not that I have any business writing init systems. :-/

Russ
Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Russell Haley
In reply to this post by aryajur


On Thu, Sep 13, 2018 at 7:17 AM Milind Gupta <[hidden email]> wrote:
I just used lpty http://tset.de/lpty/index.html for controlling a process as if in a separate terminal window in linux. It was quite easy and extremely useful.

Milind

Thanks Milind! I use Gunnars debug.lua sometimes. http://tset.de/debug.lua/index.html
So many choices!!

Russ
On Wed, Sep 12, 2018, 10:33 PM Russell Haley <[hidden email]> wrote:


On Wed, Sep 12, 2018 at 2:17 PM Sean Conner <[hidden email]> wrote:
It was thus said that the Great Russell Haley once stated:
> Hello,
>
> Some months back I started running a Minecraft server (Java) on FreeBSD.
> There are two issues that I've run into:
> 1) The Minecraft client (game) automatically updates and will not run
> against servers of prior revisions without changing client settings.
> 2) The Minecraft 'server' jar uses stdin and stdout. That means (in my
> understanding) that the process must be kept in the foreground for
> interaction. The current solution I've found (on the Minecraft wiki)
> requires screen/tmux to allow one to run a server instance from ssh.

  No, you can run the process as:

  GenericUnixPrompt> java -jar BlahObjectThing </dev/null >/dev/null 2>/dev/null &

That will run your server in the background with stdin, stdout and stderr
all pointing to /dev/null (the bit bucket).  

> The solution I now envision is to use lua-http and cqueues to create a
> 'controller' that updates the server.jar file and runs the application
> without the need for the process to be in the foreground. The parts I see
> needing are:
> - A routine that checks for updates and downloads the jar files
> - A 'thread' for running the minecraft server.jar file in java. This
> 'thread' would re-direct stdin and stdout. This 'thread' should also be
> able to stop and start the running jar file.
> - A websocket server to funnel the captured IO to a webpage for
> interactivity.

  Avoid system level threads for this.  Mixing system level threads
(pthreads) and running another process (via fork()) do not mix [1].  So
you'll end up with Lua coroutines and a sepearate process.  Let's handle
that first.

        function start_minecraft()
          local pid,err = fork()

          if pid then
            if pid == 0 then -- child process
            redirect(stdin,"/dev/null") -- or a file or a pipe
            redirect(stdout,"/dev/null")
            redirect(stderr,"/dev/null")
            exec("java",{ "-jar" , "minecraft.jar" })
            _exit(1) -- exec() failed, we exit
          end

          -- mindcraft has PID of child
          -- it is running, and so are we.

          return pid,err
        end

  stdin, stdout and stderr can be redirected to a file, a pipe, a named
pipe, what have you.  I'm justing using "/dev/null" as an example for now.
You could use a file that you serve up through HTTP to see what happens; a
pipe to get the last N bytes of logging, etc.  That's up to you.

  I should note that the functions fork(), redirect(), exec() and _exit()
need to be adjusted to whatever module you use that provides such calls
(redirect() may be called dup() or dup2() or something like that).

  The main process (as I would do it) would then use coroutines (or Lua
threads) to handle things.  One coroutine to periodically check for an
updated jar file, and if so:

        while true do
          info = stat("myjarfile.jar")
          if info.st_mtime ~= original.st_mtime then
            raise(minecraft_pid,SIG_TERM)
            status = wait(minecraft_pid)
            minecraft_pid = start_minecraft()
            original.st_mtime = info.st_mtime
          end
          wait_for(some_amount_of_time)
        end

Thanks Conman! 
 
> So far, I have roughed in the server.jar checks and download, and the
> websocket server for communications. The code can be found here:
> https://github.com/RussellHaley/minecraft-runner/mc-runner.lua
>
> What I don't know/understand yet if it's possible to run/control a process
> from Lua so that I can grab stdin/stdout? I am hoping there is something in
> cqueues to do this, but once again thought I'd send out a general request
> for ideas.
>
> Any input would be grand.

  You need something that handles networking, coroutines and some POSIX
functions (fork(), wait(), etc). 

  -spc

[1]     It probably *can* be done, but it's certainly above my pay grade,
        and I write servers for a living.

Reply | Threaded
Open this post in threaded view
|

Re: cqueues/lua-http 'process controller' for Minecraft server in FreeBSD

Russell Haley


On Fri, Sep 14, 2018 at 10:08 PM Russell Haley <[hidden email]> wrote:


On Thu, Sep 13, 2018 at 7:17 AM Milind Gupta <[hidden email]> wrote:
I just used lpty http://tset.de/lpty/index.html for controlling a process as if in a separate terminal window in linux. It was quite easy and extremely useful.

Milind

Thanks Milind! I use Gunnars debug.lua sometimes. http://tset.de/debug.lua/index.html
So many choices!!


Hmmm... I tried mkfifo and the read blocks. I can't have a blocking read on cqueues without a timeout. lpty looks like a nice solution. I'll try it tomorrow.

local function open_mc(pipe_name)
  os.execute( string.format('%s %s', mkfifo ', pipe_name )

  local aWriter = assert( io.popen( 'java -Xmx1024M -Xms1024M -jar minecraft/minecraft_server.1.13.1.jar nogui > output', 'w' ) )
  local aReader = assert( io.open( 'output', 'r' ) )


  repeat
   local aLine = aReader:read()
   print( '*', aLine, aLine:len() )
  
  until not aLine
  os.execute( string.format('%s %s', mkfifo ', pipe_name )
end

open_mc()

Russ
On Wed, Sep 12, 2018, 10:33 PM Russell Haley <[hidden email]> wrote:


On Wed, Sep 12, 2018 at 2:17 PM Sean Conner <[hidden email]> wrote:
It was thus said that the Great Russell Haley once stated:
> Hello,
>
> Some months back I started running a Minecraft server (Java) on FreeBSD.
> There are two issues that I've run into:
> 1) The Minecraft client (game) automatically updates and will not run
> against servers of prior revisions without changing client settings.
> 2) The Minecraft 'server' jar uses stdin and stdout. That means (in my
> understanding) that the process must be kept in the foreground for
> interaction. The current solution I've found (on the Minecraft wiki)
> requires screen/tmux to allow one to run a server instance from ssh.

  No, you can run the process as:

  GenericUnixPrompt> java -jar BlahObjectThing </dev/null >/dev/null 2>/dev/null &

That will run your server in the background with stdin, stdout and stderr
all pointing to /dev/null (the bit bucket).  

> The solution I now envision is to use lua-http and cqueues to create a
> 'controller' that updates the server.jar file and runs the application
> without the need for the process to be in the foreground. The parts I see
> needing are:
> - A routine that checks for updates and downloads the jar files
> - A 'thread' for running the minecraft server.jar file in java. This
> 'thread' would re-direct stdin and stdout. This 'thread' should also be
> able to stop and start the running jar file.
> - A websocket server to funnel the captured IO to a webpage for
> interactivity.

  Avoid system level threads for this.  Mixing system level threads
(pthreads) and running another process (via fork()) do not mix [1].  So
you'll end up with Lua coroutines and a sepearate process.  Let's handle
that first.

        function start_minecraft()
          local pid,err = fork()

          if pid then
            if pid == 0 then -- child process
            redirect(stdin,"/dev/null") -- or a file or a pipe
            redirect(stdout,"/dev/null")
            redirect(stderr,"/dev/null")
            exec("java",{ "-jar" , "minecraft.jar" })
            _exit(1) -- exec() failed, we exit
          end

          -- mindcraft has PID of child
          -- it is running, and so are we.

          return pid,err
        end

  stdin, stdout and stderr can be redirected to a file, a pipe, a named
pipe, what have you.  I'm justing using "/dev/null" as an example for now.
You could use a file that you serve up through HTTP to see what happens; a
pipe to get the last N bytes of logging, etc.  That's up to you.

  I should note that the functions fork(), redirect(), exec() and _exit()
need to be adjusted to whatever module you use that provides such calls
(redirect() may be called dup() or dup2() or something like that).

  The main process (as I would do it) would then use coroutines (or Lua
threads) to handle things.  One coroutine to periodically check for an
updated jar file, and if so:

        while true do
          info = stat("myjarfile.jar")
          if info.st_mtime ~= original.st_mtime then
            raise(minecraft_pid,SIG_TERM)
            status = wait(minecraft_pid)
            minecraft_pid = start_minecraft()
            original.st_mtime = info.st_mtime
          end
          wait_for(some_amount_of_time)
        end

Thanks Conman! 
 
> So far, I have roughed in the server.jar checks and download, and the
> websocket server for communications. The code can be found here:
> https://github.com/RussellHaley/minecraft-runner/mc-runner.lua
>
> What I don't know/understand yet if it's possible to run/control a process
> from Lua so that I can grab stdin/stdout? I am hoping there is something in
> cqueues to do this, but once again thought I'd send out a general request
> for ideas.
>
> Any input would be grand.

  You need something that handles networking, coroutines and some POSIX
functions (fork(), wait(), etc). 

  -spc

[1]     It probably *can* be done, but it's certainly above my pay grade,
        and I write servers for a living.