API for String substitution (excluding regex)

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

API for String substitution (excluding regex)

Rv Rv
The Lua API string.gsub will also take a pattern as an argument. I am assuming that because of the assumption that a pattern may be present as an argument, there will be additional processing that will be done every time a string comparison is made.  If we know for sure that the argument is going to be a string, is there way to avoid this additional performance penalty. I was looking for an API that operates only on string like strcpy  in C , but could not find anything
Is there any faster way to do string substitution in Lua apart from using string.gsub.
Reply | Threaded
Open this post in threaded view
|

Re: API for String substitution (excluding regex)

Luiz Henrique de Figueiredo
> The Lua API string.gsub will also take a pattern as an argument. I am assuming that because of the assumption that a pattern may be present as an argument, there will be additional processing that will be done every time a string comparison is made.

It seems that you think string.gsub operates in place and modifies the
given string. Despite the name, it does not. So, no, this does not
affect string comparison.

> Is there any faster way to do string substitution in Lua apart from using string.gsub.

What problem are you really trying to solve?

Reply | Threaded
Open this post in threaded view
|

Re: API for String substitution (excluding regex)

Marco Mastropaolo
In reply to this post by Rv Rv

If I remember correctly, the first thing gsub does is checking if the second argument is a pattern or not and, if not, it goes on with a direct replace. Thus the performance impact is minimal in that case -a linear scan over the second arg string.

This if I understood the question correctly, if not, sorry :)

Il 03/nov/2014 06:09 "Rv Rv" <[hidden email]> ha scritto:
The Lua API string.gsub will also take a pattern as an argument. I am assuming that because of the assumption that a pattern may be present as an argument, there will be additional processing that will be done every time a string comparison is made.  If we know for sure that the argument is going to be a string, is there way to avoid this additional performance penalty. I was looking for an API that operates only on string like strcpy  in C , but could not find anything
Is there any faster way to do string substitution in Lua apart from using string.gsub.
Reply | Threaded
Open this post in threaded view
|

Re: API for String substitution (excluding regex)

Philipp Janda
Am 03.11.2014 um 09:32 schröbte Marco Mastropaolo:
> If I remember correctly, the first thing gsub does is checking if the
> second argument is a pattern or not and, if not, it goes on with a direct
> replace. Thus the performance impact is minimal in that case -a linear scan
> over the second arg string.

AFAICS, only `string.find` does that (if the `plain` argument is falsish).

>
> This if I understood the question correctly, if not, sorry :)
> Il 03/nov/2014 06:09 "Rv Rv" <[hidden email]> ha scritto:
>
>> The Lua API string.gsub will also take a pattern as an argument. I am
>> assuming that because of the assumption that a pattern may be present as an
>> argument, there will be additional processing that will be done every time
>> a string comparison is made.  If we know for sure that the argument is
>> going to be a string, is there way to avoid this additional performance
>> penalty. I was looking for an API that operates only on string like strcpy
>> in C , but could not find anything
>> Is there any faster way to do string substitution in Lua apart from using
>> string.gsub.
>>

There has been a discussion about a similar `plain` option for
`string.gsub` not long ago[1]. You could implement something like that
in C quite easily (using Lua's `luaL_Buffer` API), but AFAIK noone has
done so yet. You can try one of the real regex libraries[2] for Lua -- a
regular expression compiled to a finite state machine could be faster
than an interpreted Lua pattern. And of course it's always worth
checking out LPeg[3] (although I doubt that it will be faster in this case).

Always assuming that string substitution is actually a bottleneck in
your program ...


Philipp


   [1]: http://lua-users.org/lists/lua-l/2014-08/msg00584.html
   [2]: http://rrthomas.github.io/lrexlib/
   [3]: http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html



Reply | Threaded
Open this post in threaded view
|

Re: API for String substitution (excluding regex)

Marco Mastropaolo
>> AFAICS, only `string.find` does that (if the `plain` argument is falsish).

You are right, I didn't - in fact - remember correctly and confused it with the implementation of string.find.

On Mon, Nov 3, 2014 at 10:31 AM, Philipp Janda <[hidden email]> wrote:
Am 03.11.2014 um 09:32 schröbte Marco Mastropaolo:
If I remember correctly, the first thing gsub does is checking if the
second argument is a pattern or not and, if not, it goes on with a direct
replace. Thus the performance impact is minimal in that case -a linear scan
over the second arg string.

AFAICS, only `string.find` does that (if the `plain` argument is falsish).


This if I understood the question correctly, if not, sorry :)
Il 03/nov/2014 06:09 "Rv Rv" <[hidden email]> ha scritto:

The Lua API string.gsub will also take a pattern as an argument. I am
assuming that because of the assumption that a pattern may be present as an
argument, there will be additional processing that will be done every time
a string comparison is made.  If we know for sure that the argument is
going to be a string, is there way to avoid this additional performance
penalty. I was looking for an API that operates only on string like strcpy
in C , but could not find anything
Is there any faster way to do string substitution in Lua apart from using
string.gsub.


There has been a discussion about a similar `plain` option for `string.gsub` not long ago[1]. You could implement something like that in C quite easily (using Lua's `luaL_Buffer` API), but AFAIK noone has done so yet. You can try one of the real regex libraries[2] for Lua -- a regular expression compiled to a finite state machine could be faster than an interpreted Lua pattern. And of course it's always worth checking out LPeg[3] (although I doubt that it will be faster in this case).

Always assuming that string substitution is actually a bottleneck in your program ...


Philipp


  [1]: http://lua-users.org/lists/lua-l/2014-08/msg00584.html
  [2]: http://rrthomas.github.io/lrexlib/
  [3]: http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html




Reply | Threaded
Open this post in threaded view
|

Re: API for String substitution (excluding regex)

Rv Rv
In reply to this post by Marco Mastropaolo
Thanks Marco. That was the query. Thanks for your prompt answer.


On Monday, 3 November 2014 2:02 PM, Marco Mastropaolo <[hidden email]> wrote:


If I remember correctly, the first thing gsub does is checking if the second argument is a pattern or not and, if not, it goes on with a direct replace. Thus the performance impact is minimal in that case -a linear scan over the second arg string.
This if I understood the question correctly, if not, sorry :)
Il 03/nov/2014 06:09 "Rv Rv" <[hidden email]> ha scritto:
The Lua API string.gsub will also take a pattern as an argument. I am assuming that because of the assumption that a pattern may be present as an argument, there will be additional processing that will be done every time a string comparison is made.  If we know for sure that the argument is going to be a string, is there way to avoid this additional performance penalty. I was looking for an API that operates only on string like strcpy  in C , but could not find anything
Is there any faster way to do string substitution in Lua apart from using string.gsub.