Fosdem talk on pain of moving from Python 2.x to 3.x

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Fosdem talk on pain of moving from Python 2.x to 3.x

Dibyendu Majumdar
I just watched this - https://fosdem.org/2018/schedule/event/python3.

The talk goes into details of how challenging it was to move the
Python user base from 2.6 to 3.x. It seems that the process needed
active help and support from the Python team. It made me think of the
pain in the Lua community in migrating from 5.1 to 5.3. I recently
installed the new developer edition of Redhat Enterprise Server - of
course the stock Lua is 5.1 still.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Vadi
We've been using Lua 5.1 as a tool to give our users scripting functionality for nearly 10 years now. It's working well, but as our motto is not to break backwards compatibility, it seems at this point we are eternally stuck with 5.1 as I'm not seeing much by the way of migration tools being available. There are just custom patches to future Lua versions that add backwards compatibility, but building a non-standard Lua version is another whole world of pain.

Is anyone in a similar position? How are you looking at migrating to newer Lua versions?

On Wed, Feb 7, 2018 at 11:12 PM Dibyendu Majumdar <[hidden email]> wrote:
I just watched this - https://fosdem.org/2018/schedule/event/python3.

The talk goes into details of how challenging it was to move the
Python user base from 2.6 to 3.x. It seems that the process needed
active help and support from the Python team. It made me think of the
pain in the Lua community in migrating from 5.1 to 5.3. I recently
installed the new developer edition of Redhat Enterprise Server - of
course the stock Lua is 5.1 still.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Apeng.Xiong
when I want to migrate to the new Lua, I faced the big problem is "table.getn" is no longer exist. so I still use Lua 5.1.


From: Vadim Peretokin <[hidden email]>
To: Lua mailing list <[hidden email]>,
Date: 2018/02/08 14:05
Subject: Re: Fosdem talk on pain of moving from Python 2.x to 3.x





We've been using Lua 5.1 as a tool to give our users scripting functionality for nearly 10 years now. It's working well, but as our motto is not to break backwards compatibility, it seems at this point we are eternally stuck with 5.1 as I'm not seeing much by the way of migration tools being available. There are just custom patches to future Lua versions that add backwards compatibility, but building a non-standard Lua version is another whole world of pain.

Is anyone in a similar position? How are you looking at migrating to newer Lua versions?

On Wed, Feb 7, 2018 at 11:12 PM Dibyendu Majumdar <mobile@...> wrote:
I just watched this - https://fosdem.org/2018/schedule/event/python3.

The talk goes into details of how challenging it was to move the
Python user base from 2.6 to 3.x. It seems that the process needed
active help and support from the Python team. It made me think of the
pain in the Lua community in migrating from 5.1 to 5.3. I recently
installed the new developer edition of Redhat Enterprise Server - of
course the stock Lua is 5.1 still.

Regards
Dibyendu


Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Dirk Laurie-2
2018-02-08 8:18 GMT+02:00 <[hidden email]>:
>
> when I want to migrate to the new Lua, I faced the big problem is "table.getn" is no longer exist. so I still use Lua 5.1.
>

That is not Lua 5.1, that is Lua 5.0. To get it under 5.1, you need to
have compiled with the compatibiliy option,[1] which is the default,
so that's why people think table.getn is 5.1.

Anyway, it is the easiest possible thing to supply.

table.getn = function(t) return #t end

[1] "Function table.setn was deprecated. Function table.getn
corresponds to the new length operator (#); use the operator instead
of the function." From the "Changes in the Libraries" section of the
Lua 5.1 manual.

Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Apeng.Xiong
Thanks for sharing, I know how to do if I want to solve "table.getn" issue to use new version Lua.


From: Dirk Laurie <[hidden email]>
To: Lua mailing list <[hidden email]>,
Date: 2018/02/08 14:47
Subject: Re: Fosdem talk on pain of moving from Python 2.x to 3.x





2018-02-08 8:18 GMT+02:00 <[hidden email]>:
>
> when I want to migrate to the new Lua, I faced the big problem is "table.getn" is no longer exist. so I still use Lua 5.1.
>

That is not Lua 5.1, that is Lua 5.0. To get it under 5.1, you need to
have compiled with the compatibiliy option,[1] which is the default,
so that's why people think table.getn is 5.1.

Anyway, it is the easiest possible thing to supply.

table.getn = function(t) return #t end

[1] "Function table.setn was deprecated. Function table.getn
corresponds to the new length operator (#); use the operator instead
of the function." From the "Changes in the Libraries" section of the
Lua 5.1 manual.



Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Marc Balmer
In reply to this post by Dibyendu Majumdar


> Am 07.02.2018 um 23:11 schrieb Dibyendu Majumdar <[hidden email]>:
>
> I just watched this - https://fosdem.org/2018/schedule/event/python3.
>
> The talk goes into details of how challenging it was to move the
> Python user base from 2.6 to 3.x. It seems that the process needed
> active help and support from the Python team. It made me think of the
> pain in the Lua community in migrating from 5.1 to 5.3. I recently
> installed the new developer edition of Redhat Enterprise Server - of
> course the stock Lua is 5.1 still.

It is easy enough to install Lua 5.3 from source, though.

Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Pierre Chapuis
In reply to this post by Vadi
On Thu, Feb 8, 2018, at 07:05, Vadim Peretokin wrote:
We've been using Lua 5.1 as a tool to give our users scripting functionality for nearly 10 years now. It's working well, but as our motto is not to break backwards compatibility, it seems at this point we are eternally stuck with 5.1 as I'm not seeing much by the way of migration tools being available. There are just custom patches to future Lua versions that add backwards compatibility, but building a non-standard Lua version is another whole world of pain.
Is anyone in a similar position? How are you looking at migrating to newer Lua versions?

A solution would be to embed both. If the script can work with Lua 5.3, use 5.3, otherwise fall back to 5.1.

If possible, measure how often that fallback happens. If users migrate and it almost never happens, you can think about retiring 5.1 entirely.

If you embed Lua in C, I suppose you will get symbol conflicts, so you will have to modify the source of one of the versions to prefix exported symbols.
Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Luiz Henrique de Figueiredo
> If you embed Lua in C, I suppose you will get symbol conflicts, so you will
> have to modify the source of one of the versions to prefix exported symbols.

I have done this once or twice with success. I can dig up the tools
I've used if there is any interest.

Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Vadi
In reply to this post by Pierre Chapuis

On Thu, Feb 8, 2018 at 3:55 PM Pierre Chapuis <[hidden email]> wrote:
On Thu, Feb 8, 2018, at 07:05, Vadim Peretokin wrote:
We've been using Lua 5.1 as a tool to give our users scripting functionality for nearly 10 years now. It's working well, but as our motto is not to break backwards compatibility, it seems at this point we are eternally stuck with 5.1 as I'm not seeing much by the way of migration tools being available. There are just custom patches to future Lua versions that add backwards compatibility, but building a non-standard Lua version is another whole world of pain.
Is anyone in a similar position? How are you looking at migrating to newer Lua versions?

A solution would be to embed both. If the script can work with Lua 5.3, use 5.3, otherwise fall back to 5.1.

If possible, measure how often that fallback happens. If users migrate and it almost never happens, you can think about retiring 5.1 entirely.

If you embed Lua in C, I suppose you will get symbol conflicts, so you will have to modify the source of one of the versions to prefix exported symbols.

Did not think that was possible. That is pretty cool, would be interested.
 
Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Viacheslav Usov
In reply to this post by Luiz Henrique de Figueiredo
On Thu, Feb 8, 2018 at 4:04 PM, Luiz Henrique de Figueiredo <[hidden email]> wrote:
> If you embed Lua in C, I suppose you will get symbol conflicts, so you will
> have to modify the source of one of the versions to prefix exported symbols.

I have done this once or twice with success. I can dig up the tools
I've used if there is any interest.


The following suggestion/request has been sitting in my drafts for some time.

A make-time option to include Lua API pointers in the Lua state structure, so that, for example, lua_absindex could be alternatively called as, for example, L->api->absindex(L, -1) and a related make-time option that defines all API functions as macros such that, for for example,  lua_absindex(L, -1) expands into  L->api->absindex(L, -1). The purpose of this is manifold, including (1) a possibility to host Lua as a statically linked library while loading external C-language Lua libraries and (2) eventually a possibility to host multiple versions of Lua.

Is that just a wild fantasy?

Cheers,
V.

Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Heinrich Hartmann
It should be mentioned in that context, that LuaJIT targets Lua 5.1
and has no intention to be fully 5.3 compatible in the future:

https://www.reddit.com/r/lua/comments/2zutj8/mike_pall_luajit_dislikes_lua_53/

So if you want extra speed, you have to stick with 5.1 for the time being.

Heinrich



On 8 February 2018 at 17:40, Viacheslav Usov <[hidden email]> wrote:
On Thu, Feb 8, 2018 at 4:04 PM, Luiz Henrique de Figueiredo <[hidden email]> wrote:
> If you embed Lua in C, I suppose you will get symbol conflicts, so you will
> have to modify the source of one of the versions to prefix exported symbols.

I have done this once or twice with success. I can dig up the tools
I've used if there is any interest.


The following suggestion/request has been sitting in my drafts for some time.

A make-time option to include Lua API pointers in the Lua state structure, so that, for example, lua_absindex could be alternatively called as, for example, L->api->absindex(L, -1) and a related make-time option that defines all API functions as macros such that, for for example,  lua_absindex(L, -1) expands into  L->api->absindex(L, -1). The purpose of this is manifold, including (1) a possibility to host Lua as a statically linked library while loading external C-language Lua libraries and (2) eventually a possibility to host multiple versions of Lua.

Is that just a wild fantasy?

Cheers,
V.




--
Dr. Heinrich Hartmann
Am Bökel 3
32351 Stemwede Levern

Mobile: <a href="tel:01525%203638134" value="+4915253638134" style="color:rgb(17,85,204)" target="_blank">+49 1525 363 8134
Landline: <a href="tel:05745%209200511" value="+4957459200511" style="color:rgb(17,85,204)" target="_blank">+49 5745 9200511
Reply | Threaded
Open this post in threaded view
|

Re: Fosdem talk on pain of moving from Python 2.x to 3.x

Chris Jones
In reply to this post by Dibyendu Majumdar
Hey

On 7 February 2018 at 22:11, Dibyendu Majumdar <[hidden email]> wrote:
It made me think of the pain in the Lua community in migrating from 5.1 to 5.3. I recently

I agree with your summary that Python's transition needed help from the core python developers, and I think it might be worth teasing out the two specific changes that helped increase traction in my interpretation of his talk:

 * Backporting newer features into 2.x
 * Emitting runtime deprecation warnings
 
As Victor said, these let people migrate over time - they could get their codebase into a state where it works with Python 3.x, while still using 2.x so there is no single painful migration.

I don't know if Lua has any more breaking changes in its future (and it would be nice to know, maybe if the development happened more in the open like a proper open source community project :) but perhaps these two aspects could be considered if there are any such changes coming :)

--
Cheers,

Chris