Building torch7

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Building torch7

Dibyendu Majumdar
Hi,

Has anyone successfully built Torch7 (https://github.com/torch/torch7)
with Lua 5.3?

Thanks and Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Italo Maia

2018-01-30 19:54 GMT-03:00 Dibyendu Majumdar <[hidden email]>:
Hi,

Has anyone successfully built Torch7 (https://github.com/torch/torch7)
with Lua 5.3?

Thanks and Regards
Dibyendu




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================
Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Dibyendu Majumdar
2018-01-31 1:52 GMT+00:00 Italo Maia <[hidden email]>:
> I have; https://github.com/italomaia/torch-docker
>
>> Has anyone successfully built Torch7 (https://github.com/torch/torch7)
>> with Lua 5.3?
>>


Okay thank you.

Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

p. shkadzko
In reply to this post by Dibyendu Majumdar
Can you be more specific? 
I am using ‎Torch on a daily basis.

Sent from my BlackBerry 10 smartphone.
  Original Message  
From: Dibyendu Majumdar
Sent: Tuesday, January 30, 2018 23:54
To: Lua mailing list
Reply To: Lua mailing list
Subject: Building torch7

Hi,

Has anyone successfully built Torch7 (https://github.com/torch/torch7)
with Lua 5.3?

Thanks and Regards
Dibyendu


Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

p. shkadzko
In reply to this post by Italo Maia
Hi, your dockerimage won't work with models that require nnx and dpnn or the packages that are not installed by standard install.sh script. It is also incompatible with the older code that uses rnn.‎ 

Sent from my BlackBerry 10 smartphone.
From: Italo Maia
Sent: Wednesday, January 31, 2018 02:52
To: Lua mailing list
Reply To: Lua mailing list
Subject: Re: Building torch7


2018-01-30 19:54 GMT-03:00 Dibyendu Majumdar <[hidden email]>:
Hi,

Has anyone successfully built Torch7 (https://github.com/torch/torch7)
with Lua 5.3?

Thanks and Regards
Dibyendu




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================

Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Italo Maia
How do I install those?

2018-01-31 8:31 GMT-03:00 p. shkadzko <[hidden email]>:
Hi, your dockerimage won't work with models that require nnx and dpnn or the packages that are not installed by standard install.sh script. It is also incompatible with the older code that uses rnn.‎ 

Sent from my BlackBerry 10 smartphone.
From: Italo Maia
Sent: Wednesday, January 31, 2018 02:52
To: Lua mailing list
Reply To: Lua mailing list
Subject: Re: Building torch7


2018-01-30 19:54 GMT-03:00 Dibyendu Majumdar <[hidden email]>:
Hi,

Has anyone successfully built Torch7 (https://github.com/torch/torch7)
with Lua 5.3?

Thanks and Regards
Dibyendu




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================
Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

p. shkadzko
Here is a list of additional Lua and Torch dependencies that I use.
You can pick whatever you feel you might need.

# install torch
RUN git clone https://github.com/torch/distro.git /torch --recursive \
    && cd /torch \
    && ./install.sh \
    && cd ..

# install torch deps
RUN /torch/install/bin/luarocks install dpnn \
    && /torch/install/bin/luarocks install optim \
    && /torch/install/bin/luarocks install cunn \ 
    && /torch/install/bin/luarocks install cudnn \
    && /torch/install/bin/luarocks install luautf8 \
    && /torch/install/bin/luarocks install penlight \
    && /torch/install/bin/luarocks install moses \
    && /torch/install/bin/luarocks install torchx \
    && /torch/install/bin/luarocks install lua-cjson \
    && /torch/install/bin/luarocks install csv \
    && /torch/install/bin/luarocks install autograd \
    && /torch/install/bin/luarocks install dataload \
    && /torch/install/bin/luarocks install torchnet \
    && /torch/install/bin/luarocks install cutorch \
    && /torch/install/bin/luarocks install https://raw.githubusercontent.com/Kaixhin/torch-pastalog/master/rocks/pastalog-scm-1.rockspec \
    && git clone https://github.com/Element-Research/rnn.git
    && cd rnn \
    && /torch/install/bin/luarocks make rocks/rnn-scm-1.rockspec \
    && cd .. \

The latest Torch does not support old rnn code that uses Element-Research classes.
In order to make you older recurrent models work you have to install Element-Research/rnn package separately otherwise just use "/torch/install/bin/luarocks rnn".

Best,
Pavel


On Wed, Jan 31, 2018 at 12:33 PM, Italo Maia <[hidden email]> wrote:
How do I install those?

2018-01-31 8:31 GMT-03:00 p. shkadzko <[hidden email]>:
Hi, your dockerimage won't work with models that require nnx and dpnn or the packages that are not installed by standard install.sh script. It is also incompatible with the older code that uses rnn.‎ 

Sent from my BlackBerry 10 smartphone.
From: Italo Maia
Sent: Wednesday, January 31, 2018 02:52
To: Lua mailing list
Reply To: Lua mailing list
Subject: Re: Building torch7


2018-01-30 19:54 GMT-03:00 Dibyendu Majumdar <[hidden email]>:
Hi,

Has anyone successfully built Torch7 (https://github.com/torch/torch7)
with Lua 5.3?

Thanks and Regards
Dibyendu




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================

Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Dibyendu Majumdar
In reply to this post by p. shkadzko
On 31 January 2018 at 08:00, p. shkadzko <[hidden email]> wrote:
> Can you be more specific?
> I am using ‎Torch on a daily basis.
>

Hi, it is good to know that torch works with Lua 5.3. I was able
successfully build torch7 (the core library only) for Ravi which is a
Lua 5.3 derivative. However when trying to run a test I face a couple
of issues. The first one is this strange call:

require "paths"
paths.require "libtorch"

The paths.lua script is generated from:
https://github.com/torch/torch7/blob/master/paths.lua.in

I must be missing something here.

Second issue I am still investigating. While initializing torch I get
an error that I was surprised to see - it is do with exceeding the
number of max upvalues; this is set to 120 in Ravi. I have to work out
whether this is what it appears or something else.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Paul K-2
> require "paths"
> paths.require "libtorch"

> The paths.lua script is generated from:
> https://github.com/torch/torch7/blob/master/paths.lua.in
> I must be missing something here.

I think it's a different paths module. I'd expect it to be this one:
https://github.com/torch/paths

Paul.

On Wed, Jan 31, 2018 at 1:09 PM, Dibyendu Majumdar
<[hidden email]> wrote:

> On 31 January 2018 at 08:00, p. shkadzko <[hidden email]> wrote:
>> Can you be more specific?
>> I am using ‎Torch on a daily basis.
>>
>
> Hi, it is good to know that torch works with Lua 5.3. I was able
> successfully build torch7 (the core library only) for Ravi which is a
> Lua 5.3 derivative. However when trying to run a test I face a couple
> of issues. The first one is this strange call:
>
> require "paths"
> paths.require "libtorch"
>
> The paths.lua script is generated from:
> https://github.com/torch/torch7/blob/master/paths.lua.in
>
> I must be missing something here.
>
> Second issue I am still investigating. While initializing torch I get
> an error that I was surprised to see - it is do with exceeding the
> number of max upvalues; this is set to 120 in Ravi. I have to work out
> whether this is what it appears or something else.
>
> Regards
> Dibyendu
>

Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Dibyendu Majumdar
In reply to this post by Dibyendu Majumdar
On 31 January 2018 at 21:09, Dibyendu Majumdar <[hidden email]> wrote:
> Second issue I am still investigating. While initializing torch I get
> an error that I was surprised to see - it is do with exceeding the
> number of max upvalues; this is set to 120 in Ravi. I have to work out
> whether this is what it appears or something else.
>

Okay I found the issue here - in debug builds of Ravi I have 'ltests'
turned ON. Unfortunately this messes with some Lua settings. When apps
are linked against Ravi they don't see the messed up settings .. and
bang! the LUA_REGISTRYINDEX is not where the app thinks it is.
I guess I have have to make 'ltests' an explicit feature and not part
of all debug builds.

If you don't know what 'ltests' is then it is part of Lua's internal
testing infrastructure. When it is enabled it does a bunch of internal
checks and also exposes a library that can be used in tests cases. It
also amends some Lua settings making them more aggressive - hence the
problem above.


Regards

Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Dibyendu Majumdar
In reply to this post by Paul K-2
On 31 January 2018 at 22:03, Paul K <[hidden email]> wrote:

>> require "paths"
>> paths.require "libtorch"
>
>> The paths.lua script is generated from:
>> https://github.com/torch/torch7/blob/master/paths.lua.in
>> I must be missing something here.
>
> I think it's a different paths module. I'd expect it to be this one:
> https://github.com/torch/paths
>

Ah okay thanks! Strange that there is 'paths.lua' in the Lua script folder.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Paige DePol
In reply to this post by Dibyendu Majumdar
Dibyendu Majumdar <[hidden email]> wrote:

> On 31 January 2018 at 21:09, Dibyendu Majumdar <[hidden email]> wrote:
>> Second issue I am still investigating. While initializing torch I get
>> an error that I was surprised to see - it is do with exceeding the
>> number of max upvalues; this is set to 120 in Ravi. I have to work out
>> whether this is what it appears or something else.
>
> Okay I found the issue here - in debug builds of Ravi I have 'ltests'
> turned ON. Unfortunately this messes with some Lua settings. When apps
> are linked against Ravi they don't see the messed up settings .. and
> bang! the LUA_REGISTRYINDEX is not where the app thinks it is.
> I guess I have have to make 'ltests' an explicit feature and not part
> of all debug builds.
>
> If you don't know what 'ltests' is then it is part of Lua's internal
> testing infrastructure. When it is enabled it does a bunch of internal
> checks and also exposes a library that can be used in tests cases. It
> also amends some Lua settings making them more aggressive - hence the
> problem above.

LUA_REGISTRYINDEX is set relative to LUAI_MAXSTACK, normally the max
stack define is set to 15k for 32-bit or 1kk otherwise. Under 'ltests'
it is defined as 50k. Perhaps just changing this value to the default
value would allow you to keep 'ltests' enabled when linking?

~Paige




Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Italo Maia
In reply to this post by p. shkadzko
Hello shkadzko, thanks for the feedback. A few of those rocks do install. Had some CUDA dependency issues, though.

2018-01-31 9:57 GMT-03:00 p. shkadzko <[hidden email]>:
Here is a list of additional Lua and Torch dependencies that I use.
You can pick whatever you feel you might need.

# install torch
RUN git clone https://github.com/torch/distro.git /torch --recursive \
    && cd /torch \
    && ./install.sh \
    && cd ..

# install torch deps
RUN /torch/install/bin/luarocks install dpnn \
    && /torch/install/bin/luarocks install optim \
    && /torch/install/bin/luarocks install cunn \ 
    && /torch/install/bin/luarocks install cudnn \
    && /torch/install/bin/luarocks install luautf8 \
    && /torch/install/bin/luarocks install penlight \
    && /torch/install/bin/luarocks install moses \
    && /torch/install/bin/luarocks install torchx \
    && /torch/install/bin/luarocks install lua-cjson \
    && /torch/install/bin/luarocks install csv \
    && /torch/install/bin/luarocks install autograd \
    && /torch/install/bin/luarocks install dataload \
    && /torch/install/bin/luarocks install torchnet \
    && /torch/install/bin/luarocks install cutorch \
    && /torch/install/bin/luarocks install https://raw.githubusercontent.com/Kaixhin/torch-pastalog/master/rocks/pastalog-scm-1.rockspec \
    && git clone https://github.com/Element-Research/rnn.git
    && cd rnn \
    && /torch/install/bin/luarocks make rocks/rnn-scm-1.rockspec \
    && cd .. \

The latest Torch does not support old rnn code that uses Element-Research classes.
In order to make you older recurrent models work you have to install Element-Research/rnn package separately otherwise just use "/torch/install/bin/luarocks rnn".

Best,
Pavel


On Wed, Jan 31, 2018 at 12:33 PM, Italo Maia <[hidden email]> wrote:
How do I install those?

2018-01-31 8:31 GMT-03:00 p. shkadzko <[hidden email]>:
Hi, your dockerimage won't work with models that require nnx and dpnn or the packages that are not installed by standard install.sh script. It is also incompatible with the older code that uses rnn.‎ 

Sent from my BlackBerry 10 smartphone.
From: Italo Maia
Sent: Wednesday, January 31, 2018 02:52
To: Lua mailing list
Reply To: Lua mailing list
Subject: Re: Building torch7


2018-01-30 19:54 GMT-03:00 Dibyendu Majumdar <[hidden email]>:
Hi,

Has anyone successfully built Torch7 (https://github.com/torch/torch7)
with Lua 5.3?

Thanks and Regards
Dibyendu




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================
Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

p. shkadzko
In reply to this post by Dibyendu Majumdar
I would not recommend building it for 5.3 since there might be build and speed issues. I highly doubt you'll need 5.3 functionality anyway. For playing around its ok. Good to know that Ravi can be used for Torch, yet it won't be faster for sure.

Sent from my BlackBerry 10 smartphone.
  Original Message  
From: Dibyendu Majumdar
Sent: Wednesday, January 31, 2018 22:09
To: Lua mailing list
Reply To: Lua mailing list
Subject: Re: Building torch7

On 31 January 2018 at 08:00, p. shkadzko <[hidden email]> wrote:
> Can you be more specific?
> I am using ‎Torch on a daily basis.
>

Hi, it is good to know that torch works with Lua 5.3. I was able
successfully build torch7 (the core library only) for Ravi which is a
Lua 5.3 derivative. However when trying to run a test I face a couple
of issues. The first one is this strange call:

require "paths"
paths.require "libtorch"

The paths.lua script is generated from:
https://github.com/torch/torch7/blob/master/paths.lua.in

I must be missing something here.

Second issue I am still investigating. While initializing torch I get
an error that I was surprised to see - it is do with exceeding the
number of max upvalues; this is set to 120 in Ravi. I have to work out
whether this is what it appears or something else.

Regards
Dibyendu


Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

Dibyendu Majumdar
On 1 February 2018 at 06:59, p. shkadzko <[hidden email]> wrote:
> I would not recommend building it for 5.3 since there might be build and speed issues. I highly doubt you'll need 5.3 functionality anyway. For playing around its ok. Good to know that Ravi can be used for Torch, yet it won't be faster for sure.
>

I assume you are using Torch with LuaJIT? The Torch project is now
marked 'in maintenance' as they have moved to PyTorch I believe - have
you any experience with PyTorch? How does it compare in terms of
performance?

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Building torch7

p. shkadzko
PyTorch is basically a direct Torch reimplementation in Python with some architectural improvements.
I am not aware of any benchmarks but I would give a slight edge to PyTorch since it receives max devs attention today.
I don't think that any speed difference if it exists will make a huge difference provided you have a good GPU which you should anyway.
What would make a difference is the ability to move you neural network into production and it is much more difficult to do with Torch/Lua unfortunately.
PyTorch is not production friendly either but at least there are converters in development.

I really hoped that Torch would invite academia into Lua world and the whole Lua ecosystem receives a huge benefit from new users coming (I learnt Lua because of Torch).
But luajit and 5.3 disconnection put an end to these hopes and persuaded the whole dev team to migrate to Python asap.
I will continue using Torch because of Lua but afraid that eventually I will join the flock soon.

On Thu, Feb 1, 2018 at 8:25 AM, Dibyendu Majumdar <[hidden email]> wrote:
On 1 February 2018 at 06:59, p. shkadzko <[hidden email]> wrote:
> I would not recommend building it for 5.3 since there might be build and speed issues. I highly doubt you'll need 5.3 functionality anyway. For playing around its ok. Good to know that Ravi can be used for Torch, yet it won't be faster for sure.
>

I assume you are using Torch with LuaJIT? The Torch project is now
marked 'in maintenance' as they have moved to PyTorch I believe - have
you any experience with PyTorch? How does it compare in terms of
performance?

Regards
Dibyendu