[ANN] Ravi 1.0 Beta5 release

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] Ravi 1.0 Beta5 release

Dibyendu Majumdar
Ravi (http://ravilang.org) is a dialect of Lua with a JIT backend and
limited optional static typing. I am pleased to announce that the 1.0
Beta-5 release of Ravi is now available.

The main highlights of this release are:

* Bug fixes to the Generational GC back-ported from Lua 5.4
* Support for LLVM has been removed in favour of the Lightweight MIR JIT backend
* Various updates to MIR JIT backend
* Ravi is now being tested on X86-64 and ARM64 architectures
* There is an experimental new compiler for Ravi that is available as
a preview only. More details of this new compiler can be found at:
https://github.com/dibyendumajumdar/ravi-compiler.


For general information regarding Ravi please visit:
* https://github.com/dibyendumajumdar/ravi

A distribution of Ravi with a small set of libraries is available at:

https://github.com/dibyendumajumdar/Suravi

All feedback welcome!

Thanks and Regards
Dibyendu
v
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

v
How does removal of LLVM backend affect platform support? Did you
managed to get MIR to work on platforms where you previously used LLVM?

From my point of view, it was nice to have more mature even if heavy
LLVM backend. MIR seems to view itself as highly experimental project
at the current state of its development.

On Sun, 2021-01-03 at 16:51 +0000, Dibyendu Majumdar wrote:

> Ravi (http://ravilang.org) is a dialect of Lua with a JIT backend and
> limited optional static typing. I am pleased to announce that the 1.0
> Beta-5 release of Ravi is now available.
>
> The main highlights of this release are:
>
> * Bug fixes to the Generational GC back-ported from Lua 5.4
> * Support for LLVM has been removed in favour of the Lightweight MIR
> JIT backend
> * Various updates to MIR JIT backend
> * Ravi is now being tested on X86-64 and ARM64 architectures
> * There is an experimental new compiler for Ravi that is available as
> a preview only. More details of this new compiler can be found at:
> https://github.com/dibyendumajumdar/ravi-compiler.
>
>
> For general information regarding Ravi please visit:
> * https://github.com/dibyendumajumdar/ravi
>
> A distribution of Ravi with a small set of libraries is available at:
>
> https://github.com/dibyendumajumdar/Suravi
>
> All feedback welcome!
>
> Thanks and Regards
> Dibyendu

--
v <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

Dibyendu Majumdar
On Mon, 4 Jan 2021 at 01:35, v <[hidden email]> wrote:
>
> How does removal of LLVM backend affect platform support? Did you
> managed to get MIR to work on platforms where you previously used LLVM?
>

Was there a particular platform you had in mind?
I had been testing on X86-64 only.
MIR supports X86-64, ARM64, PPC64, s390x. It now also works on
Windows, in addition to Mac OSX and Linux.

> From my point of view, it was nice to have more mature even if heavy
> LLVM backend. MIR seems to view itself as highly experimental project
> at the current state of its development.

The MIR project is actually quite stable and is due to have 1.0
release pretty soon.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

Russell Haley
In reply to this post by Dibyendu Majumdar


On Sun, Jan 3, 2021 at 11:53 AM Dibyendu Majumdar <[hidden email]> wrote:
Ravi (http://ravilang.org) is a dialect of Lua with a JIT backend and
limited optional static typing. I am pleased to announce that the 1.0
Beta-5 release of Ravi is now available.

The main highlights of this release are:

* Bug fixes to the Generational GC back-ported from Lua 5.4
* Support for LLVM has been removed in favour of the Lightweight MIR JIT backend
Hi Dibyendu, your work is very impressive. You might have shared this resource before, but I found a presentation about the MIR JIT (by the author): https://developers.redhat.com/blog/2020/01/20/mir-a-lightweight-jit-compiler-project/

The difference in SLOC between LLVM and MIR is hilarious: 4.13 Million LOC vs 16,000. How big is the base Ravi install?

Have you done any benchmarking between Ravi (MIR) and LuaJIT?

Is it possible to just swap in Ravi where one would use Lua or are there caveats? For instance, can I run standard lua scripts and lua rocks without static types?

I look forward to testing this soon! You could also post this message on LuaForum.com. We are getting a nice collection of Lua distributions (WinLua, RiscLua, etc) - it would be cool to add a new dialect like Ravi. Perhaps I will ask Maksumić if we can start a resource page to list all of them.

Russ

* Various updates to MIR JIT backend
* Ravi is now being tested on X86-64 and ARM64 architectures
* There is an experimental new compiler for Ravi that is available as
a preview only. More details of this new compiler can be found at:
https://github.com/dibyendumajumdar/ravi-compiler.


For general information regarding Ravi please visit:
* https://github.com/dibyendumajumdar/ravi

A distribution of Ravi with a small set of libraries is available at:

https://github.com/dibyendumajumdar/Suravi

All feedback welcome!

Thanks and Regards
Dibyendu
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

Dibyendu Majumdar
On Mon, 4 Jan 2021 at 20:31, Russell Haley <[hidden email]> wrote:
>
> The difference in SLOC between LLVM and MIR is hilarious: 4.13 Million LOC vs 16,000. How big is the base Ravi install?
>

Indeed. It takes seconds to build Ravi with MIR - it used to take
several minutes to build LLVM. Also resulting binaries are small.

MIR is an unbelievable project. All small JIT backends I tried before
were very bad at optimization. MIR manages to get close to 2x gcc or
clang even with its small size and thus less features. I guess it
helps that the developer of MIR is the maintainer of gcc register
allocation.


> Have you done any benchmarking between Ravi (MIR) and LuaJIT?

No, but LuaJIT will always win in the general case, as Ravi needs type
annotations for good performance. Until I can implement function
specializations with predicted types, it will be hard to beat LuaJIT.

>
> Is it possible to just swap in Ravi where one would use Lua or are there caveats? For instance, can I run standard lua scripts and lua rocks without static types?
>

Yes but there are some caveats. I guess if you are running standard
Lua code, there is not much benefit.


Regards
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

Italo Maia
Hey man! Good to know ravi is getting really warm in the oven! Regarding the changes to regular lua, I've got some questions:
  • I see it introduces some types. Most notably, to me, would be integer and number. Why not integer and float? 
  • Another question: if integer and number are initialized with 0, why not initialize arrays too? Or even dictionaries?
  • Ravi arrays seem to add a bit of complexity to the language, making it a bit harder to parse a lua program to ravi with all the optional optimizations. How much of a performance boost arrays add to an overall program compared to regular (ravi optimized) tables?
  • Ravi seems to remove nil from the spotlight a bit, it seems, with exception to string, closure and user-defined types. This could also be a concern when migrating projects to ravi. Not sure if a big concern but for sure something to keep in mind. Would there be plans to go further, like changing the regular table's behavior regarding setting a key to nil and adding a new method to tables? Or even back down? What do you think is the purpose of nil in a fully ravi optimized program?
  • Any chance fixed size array declaration would be in the queue for us humble folks? (aka: local array: integer[4] = {2, 3, 4, 5})
  • Given arrays are a thing for ravi, how does immutability sound in your book? Quite an interesting feature with arrays. We can already do it using standard lua kung fu, but just checking how it sounds to the ears.
  • How "local by default" variables sound to you?
Some of these questions really leap away from Lua but, curiosity usually bests me on january (and subsequent months). Cheers!

Em seg., 4 de jan. de 2021 às 23:39, Dibyendu Majumdar <[hidden email]> escreveu:
On Mon, 4 Jan 2021 at 20:31, Russell Haley <[hidden email]> wrote:
>
> The difference in SLOC between LLVM and MIR is hilarious: 4.13 Million LOC vs 16,000. How big is the base Ravi install?
>

Indeed. It takes seconds to build Ravi with MIR - it used to take
several minutes to build LLVM. Also resulting binaries are small.

MIR is an unbelievable project. All small JIT backends I tried before
were very bad at optimization. MIR manages to get close to 2x gcc or
clang even with its small size and thus less features. I guess it
helps that the developer of MIR is the maintainer of gcc register
allocation.


> Have you done any benchmarking between Ravi (MIR) and LuaJIT?

No, but LuaJIT will always win in the general case, as Ravi needs type
annotations for good performance. Until I can implement function
specializations with predicted types, it will be hard to beat LuaJIT.

>
> Is it possible to just swap in Ravi where one would use Lua or are there caveats? For instance, can I run standard lua scripts and lua rocks without static types?
>

Yes but there are some caveats. I guess if you are running standard
Lua code, there is not much benefit.


Regards


--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

Dibyendu Majumdar
On Tue, 5 Jan 2021 at 00:51, Italo Maia <[hidden email]> wrote:
>

> I see it introduces some types. Most notably, to me, would be integer and number. Why not integer and float?

I guess that 'number' is more consistent with existing Lua terminology.

> Another question: if integer and number are initialized with 0, why not initialize arrays too? Or even dictionaries?

In Ravi you can create initialized number / integer arrays of a given
size using a library function.

> Ravi arrays seem to add a bit of complexity to the language, making it a bit harder to parse a lua program to ravi with all the optional optimizations. How much of a performance boost arrays add to an overall program compared to regular (ravi optimized) tables?

Lua tables simply cannot perform like native arrays. Ravi arrays are
like native arrays. Moreover these arrays do not need the garbage
collector to scan them.
Originally I added these arrays because I was looking at using Ravi in
a financial analytics app.

> Ravi seems to remove nil from the spotlight a bit, it seems, with exception to string, closure and user-defined types. This could also be a concern when migrating projects to ravi. Not sure if a big concern but for sure something to keep in mind. Would there be plans to go further, like changing the regular table's behavior regarding setting a key to nil and adding a new method to tables? Or even back down? What do you think is the purpose of nil in a fully ravi optimized program?

Nil values are actually not great for a JIT compiler as it introduces
the need for Nil checks everywhere.Lua team have decided to subclass
Nil so that is probably the way it will evolve. I have to assess the
impact of that before I can implement it in Ravi.

Here is the definition in Lua 5.4

/* Standard nil */
#define LUA_VNIL        makevariant(LUA_TNIL, 0)

/* Empty slot (which might be different from a slot containing nil) */
#define LUA_VEMPTY      makevariant(LUA_TNIL, 1)

/* Value returned for a key not found in a table (absent key) */
#define LUA_VABSTKEY    makevariant(LUA_TNIL, 2)


> Any chance fixed size array declaration would be in the queue for us humble folks? (aka: local array: integer[4] = {2, 3, 4, 5})

You can kind of do that via a library function except that the array
is initialized to a single default value.
In Ravi arrays can be fixed size. But not Lua tables, although I think
it is fairly easy to add a flag in the table to ensure it is a fixed
size array. It is just work that needs doing.
I think an easier way to do above in Ravi would be to create the array
and then call a library function to make it fixed size.

> Given arrays are a thing for ravi, how does immutability sound in your book? Quite an interesting feature with arrays. We can already do it using standard lua kung fu, but just checking how it sounds to the ears.

Arrays and tables can be easily made immutable to be honest - and I
thought about that as a way of optimizing a table. For instance after
populating a table, you could invoke a library function that makes the
table immutable and optimizes it for performance - avoiding hash
collisions for example. Again this is just work.

> How "local by default" variables sound to you?

Perhaps there could be a command line flag to enable that behaviour.
But it would probably create incompatibilities with existing code.

Thanks for the feedback!

Regards
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

Russell Haley
In reply to this post by Dibyendu Majumdar


On Mon, Jan 4, 2021 at 2:39 PM Dibyendu Majumdar <[hidden email]> wrote:
On Mon, 4 Jan 2021 at 20:31, Russell Haley <[hidden email]> wrote:
>
> The difference in SLOC between LLVM and MIR is hilarious: 4.13 Million LOC vs 16,000. How big is the base Ravi install?
>

Indeed. It takes seconds to build Ravi with MIR - it used to take
several minutes to build LLVM. Also resulting binaries are small.

MIR is an unbelievable project. All small JIT backends I tried before
were very bad at optimization. MIR manages to get close to 2x gcc or
clang even with its small size and thus less features. I guess it
helps that the developer of MIR is the maintainer of gcc register
allocation.


> Have you done any benchmarking between Ravi (MIR) and LuaJIT?

No, but LuaJIT will always win in the general case, as Ravi needs type
annotations for good performance. Until I can implement function
specializations with predicted types, it will be hard to beat LuaJIT.
I'm not really into competing in these regards, I was more thinking of LuaJIT as a benchmark to gauge Ravi. 
 

>
> Is it possible to just swap in Ravi where one would use Lua or are there caveats? For instance, can I run standard lua scripts and lua rocks without static types?
>

Yes but there are some caveats. I guess if you are running standard
Lua code, there is not much benefit.
I have watched the time and effort you have put into Ravi and I can say without hesitation that you are an inspiring person. Permit me (forgive me) for one moment to use "active listening" and re-write your response:

"No, but LuaJIT will always win..."
Ravi is a new piece of technology and still has many optimizations available. I have taken Ravi as far as a single human being can with the time available, and now it is up to the community to give me feedback and help invest in this great project that can benefit everyone, perhaps even Mike Pall and the LuaJIT team. MIR is a general purpose JIT so our community investment in Ravi also benefits *other languages*. The MIR author, Vladimir Makarov, is hoping to implement Ruby on top of MIR starting this year. It would be great if someone could take the time to do some benchmarking for me especially if they can use a comparison tool or website to post results against multiple languages.

"Yes but there are some caveats. I guess if you are running standard Lua code, there is not much benefit."

There are some caveats, but most code should run well. To take advantage of Ravi's features, one would need to use annotations and static typing [just a guess]. Again, if the community can test drive Ravi and give me feedback we could see where other adjustments can improve all use cases.


Regards

I won't have time to drill into your source code so I shall ask instead: 
- Did you re-use any Lua.org code? Can you guess on a percentage? (just out of curiosity)
- Were you forced to write any MIR text? What was the interface with MIR like? Or is it just standard C? 
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

Dibyendu Majumdar
Hi,

On Tue, 5 Jan 2021 at 18:40, Russell Haley <[hidden email]> wrote:

> - Did you re-use any Lua.org code? Can you guess on a percentage? (just out of curiosity)

Of course - depends on the component we are talking about. There is a
new compiler that is largely being written from the ground up, but
even there the lexer is pretty much a hacked version from Lua, and
there are other bits and pieces that are borrowed too.

> - Were you forced to write any MIR text? What was the interface with MIR like? Or is it just standard C?

Fortunately MIR provides a C front-end, so you can feed in C code and
let MIR convert it. I like this because:

a) Much less effort to maintain it. I had written up the LLVM backend
by hand, using the LLVM api, and it made it very difficult to maintain
and change it. (So I am glad to see the end of the LLVM backend!)

b) It is relatively straight-forward to also support AOT compilation
when you have C output.

Regards
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

Italo Maia
In reply to this post by Dibyendu Majumdar
> Perhaps there could be a command line flag to enable that behaviour.
But it would probably create incompatibilities with existing code.

As a ravi program is not compatible with lua, so that could go as a nice ravi only feature.

Regarding arrays ...
If ravi arrays are not that close to tables, why not give them their own syntax? Declaring arrays like tables when they don't behave the same in multiple levels might make code harder to read and write. 

I guess that 'number' is more consistent with existing Lua terminology.
Is the number type in ravi just like the number type in lua (int or float)? 

Regarding nil ...
So, that definition is so that the "set nil to remove element" feature for tables would work, I presume. To me, that is one of the Lua bits that can create confusion and make programs harder to understand. See the example:

> x = {3,4,5}
> #x
3
> x[2] = nil
> #x
3
> for k, v in pairs(x) do print(k, v) end
1 3
3 5

With ravi arrays, I assume the last line would be something like this:

> for k, v in pairs(x) do print(k, v) end
1 3
2 nil
3 5

Correct? Or setting nil would break things because nil is not of the type integer?

Is there an official ravi docker image?

Em ter., 5 de jan. de 2021 às 15:37, Dibyendu Majumdar <[hidden email]> escreveu:
On Tue, 5 Jan 2021 at 00:51, Italo Maia <[hidden email]> wrote:
>

> I see it introduces some types. Most notably, to me, would be integer and number. Why not integer and float?

I guess that 'number' is more consistent with existing Lua terminology.

> Another question: if integer and number are initialized with 0, why not initialize arrays too? Or even dictionaries?

In Ravi you can create initialized number / integer arrays of a given
size using a library function.

> Ravi arrays seem to add a bit of complexity to the language, making it a bit harder to parse a lua program to ravi with all the optional optimizations. How much of a performance boost arrays add to an overall program compared to regular (ravi optimized) tables?

Lua tables simply cannot perform like native arrays. Ravi arrays are
like native arrays. Moreover these arrays do not need the garbage
collector to scan them.
Originally I added these arrays because I was looking at using Ravi in
a financial analytics app.

> Ravi seems to remove nil from the spotlight a bit, it seems, with exception to string, closure and user-defined types. This could also be a concern when migrating projects to ravi. Not sure if a big concern but for sure something to keep in mind. Would there be plans to go further, like changing the regular table's behavior regarding setting a key to nil and adding a new method to tables? Or even back down? What do you think is the purpose of nil in a fully ravi optimized program?

Nil values are actually not great for a JIT compiler as it introduces
the need for Nil checks everywhere.Lua team have decided to subclass
Nil so that is probably the way it will evolve. I have to assess the
impact of that before I can implement it in Ravi.

Here is the definition in Lua 5.4

/* Standard nil */
#define LUA_VNIL        makevariant(LUA_TNIL, 0)

/* Empty slot (which might be different from a slot containing nil) */
#define LUA_VEMPTY      makevariant(LUA_TNIL, 1)

/* Value returned for a key not found in a table (absent key) */
#define LUA_VABSTKEY    makevariant(LUA_TNIL, 2)


> Any chance fixed size array declaration would be in the queue for us humble folks? (aka: local array: integer[4] = {2, 3, 4, 5})

You can kind of do that via a library function except that the array
is initialized to a single default value.
In Ravi arrays can be fixed size. But not Lua tables, although I think
it is fairly easy to add a flag in the table to ensure it is a fixed
size array. It is just work that needs doing.
I think an easier way to do above in Ravi would be to create the array
and then call a library function to make it fixed size.

> Given arrays are a thing for ravi, how does immutability sound in your book? Quite an interesting feature with arrays. We can already do it using standard lua kung fu, but just checking how it sounds to the ears.

Arrays and tables can be easily made immutable to be honest - and I
thought about that as a way of optimizing a table. For instance after
populating a table, you could invoke a library function that makes the
table immutable and optimizes it for performance - avoiding hash
collisions for example. Again this is just work.

> How "local by default" variables sound to you?

Perhaps there could be a command line flag to enable that behaviour.
But it would probably create incompatibilities with existing code.

Thanks for the feedback!

Regards


--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi 1.0 Beta5 release

Dibyendu Majumdar
On Mon, 11 Jan 2021 at 10:25, Italo Maia <[hidden email]> wrote:
>
> > Perhaps there could be a command line flag to enable that behaviour.
> But it would probably create incompatibilities with existing code.
>
> As a ravi program is not compatible with lua, so that could go as a nice ravi only feature.

Making unqualified variables local by default is problematic because
then you need a way to declare globals like Python has.
A  better solution is probably to just issue warnings.

>
> Regarding arrays ...
> If ravi arrays are not that close to tables, why not give them their own syntax? Declaring arrays like tables when they don't behave the same in multiple levels might make code harder to read and write.
>

Ravi arrays appear as regular tables in Lua code. Also adding new
syntax for arrays is not that easy.

> I guess that 'number' is more consistent with existing Lua terminology.
> Is the number type in ravi just like the number type in lua (int or float)?
>

It is - I guess I stuck to the terms 'number' and 'integer' based on
the C typedefs in Lua (lua_Number, lua_Integer).


> Regarding nil ...
> So, that definition is so that the "set nil to remove element" feature for tables would work, I presume. To me, that is one of the Lua bits that can create confusion and make programs harder to understand. See the example:
>
> > x = {3,4,5}
> > #x
> 3
> > x[2] = nil
> > #x
> 3
> > for k, v in pairs(x) do print(k, v) end
> 1 3
> 3 5
>
> With ravi arrays, I assume the last line would be something like this:
>
> > for k, v in pairs(x) do print(k, v) end
> 1 3
> 2 nil
> 3 5
>
> Correct? Or setting nil would break things because nil is not of the type integer?

No, because a ravi array is integer[] or double[] - neither allow nils.

>
> Is there an official ravi docker image?
>

There was but I need to check if docker has deleted it. There is a
Dockerfile included so you can build an image easily.


Regards