Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

孙世龙 sunshilong
Hi, list

I want to use a self-defined memory acquirement function instead of
using realloc(3) in the l_alloc function.
Since I am a newbie, I don't know if there are some potential problems.
Could somebody shed some light on this problem?
Thank you for your attention to this matter.

For your convenience, here is the original l_alloc() function for lua.
static void *l_alloc (void *ud, void *ptr, size_t osize, size_t nsize) {
  (void)ud; (void)osize;  /* not used */
  if (nsize == 0) {
    free(ptr);
    return NULL;
  }
  else
    return realloc(ptr, nsize);
}

Best regards.
Sunshilong
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Stefan-2
Am 10.08.2020 um 10:32 schrieb 孙世龙 sunshilong:
> Hi, list
>
> I want to use a self-defined memory acquirement function instead of
> using realloc(3) in the l_alloc function.
> Since I am a newbie, I don't know if there are some potential problems.
> Could somebody shed some light on this problem?
> Thank you for your attention to this matter.

I think Lua relies on the behaviour that the realloc-like function
does not return a different pointer (copy) for shrinking an existing
block. Maybe that could be a problem for some allocators.
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Viacheslav Usov
On Mon, Aug 10, 2020 at 2:34 PM Stefan <[hidden email]> wrote:

> I think Lua relies on the behaviour that the realloc-like function
> does not return a different pointer (copy) for shrinking an existing
> block. Maybe that could be a problem for some allocators.

Such a behavior is not documented for realloc() either by the ISO C
Standard or by POSIX. Many modern allocators use different pools of
memory blocks for dissimilar allocation sizes, say one block for up to
16 byte allocations and another for 16 to 32, etc. Therefore shrinking
an allocation in size, if it is to have any effect, would necessarily
move it into another block of memory. To use a practical example,
tcmalloc may move allocations when shrinking.

I'd be surprised if Lua really assumed this. What it does assume, and
that is documented, is "that the allocator never fails when osize >=
nsize.", which not an intrinsically safe assumption about realloc(),
but which can be assured on top of it in lua_Alloc(). The above
mentioned tcmalloc, by looking at its code, seems capable of returning
a null pointer when shrinking.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Roberto Ierusalimschy
In reply to this post by Stefan-2
> I think Lua relies on the behaviour that the realloc-like function
> does not return a different pointer (copy) for shrinking an existing
> block. Maybe that could be a problem for some allocators.

Lua does not rely on that. The only assumption was that shrinking
an existing block could not fail, but it can move the block without
problems. Actually, the test suite in Lua has a mode that forces every
single reallocation (growing, shrinking, or to the same size!) to move
the block.

Lua 5.4 has removed this restriction of "no fail when shrinking".

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Stefan-2
Am 10.08.2020 um 16:34 schrieb Roberto Ierusalimschy:
>> I think Lua relies on the behaviour that the realloc-like function
>> does not return a different pointer (copy) for shrinking an existing
>> block. Maybe that could be a problem for some allocators.
>
> Lua does not rely on that. The only assumption was that shrinking
> an existing block could not fail, but it can move the block without
> problems. Actually, the test suite in Lua has a mode that forces every
> single reallocation (growing, shrinking, or to the same size!) to move
> the block.

My test program was flawed. Sorry.

>
> Lua 5.4 has removed this restriction of "no fail when shrinking".
>
> -- Roberto
>

Wow, so this is why it is no longer in the manual.
I wonder if there is any system with C support left that can't
run Lua :)
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Roberto Ierusalimschy
> > Lua 5.4 has removed this restriction of "no fail when shrinking".
> >
> > -- Roberto
> >
>
> Wow, so this is why it is no longer in the manual.
> I wonder if there is any system with C support left that can't
> run Lua :)

As I pointed out in my previous message, it is very easy to modify
l_alloc to circumvent this restriction in previous versions of Lua.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Philippe Verdy-2
In reply to this post by Stefan-2
I don't think thank shrinking or keeping the same size may fail: if a better location can't be used (because the memory pool where it would fit can't be augmented or modified, the best thing to do is to keep the existing pointer, and just mark the end as "supposedly" unused (even if that unused part is not reallocatable for something else as that part would need to be added in a free pool.
The interface allows recording the effective size that is used in an allocated block, so that the end of block may or may not be reused later (immediately or not, when it will be possible to add the free part in a free pool, which may occur later when possible and where there will be an opportunity to do it).
Moving blocks  when shrinking or redefining its size is allowed to move it, but is not forced to do it or to make a real deallocation immediately: the reported available free size may not change immediately.
But returning NULL for shrinking an allocated block or setting it to the same size is a non-sense that would cause more problems than what it would attempt to solve.

Le lun. 10 août 2020 à 17:33, Stefan <[hidden email]> a écrit :
Am 10.08.2020 um 16:34 schrieb Roberto Ierusalimschy:
>> I think Lua relies on the behaviour that the realloc-like function
>> does not return a different pointer (copy) for shrinking an existing
>> block. Maybe that could be a problem for some allocators.
>
> Lua does not rely on that. The only assumption was that shrinking
> an existing block could not fail, but it can move the block without
> problems. Actually, the test suite in Lua has a mode that forces every
> single reallocation (growing, shrinking, or to the same size!) to move
> the block.

My test program was flawed. Sorry.

>
> Lua 5.4 has removed this restriction of "no fail when shrinking".
>
> -- Roberto
>

Wow, so this is why it is no longer in the manual.
I wonder if there is any system with C support left that can't
run Lua :)
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Andrew Gierth
>>>>> "Philippe" == Philippe Verdy <[hidden email]> writes:

 Philippe> I don't think thank shrinking or keeping the same size may
 Philippe> fail

It doesn't matter what you think. What matters is what the language
standards say, and what they say is that realloc() is permitted to fail
when shrinking a block. (Some realloc implementations never fail in that
case, but many in common use do.)

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

孙世龙 sunshilong
In reply to this post by Stefan-2
>Lua relies on the behaviour that the realloc-like function
>does not return a different pointer (copy) for shrinking an existing
>block.
What problems may occur if this rule(or advice) is broken?

On Mon, Aug 10, 2020 at 8:34 PM Stefan <[hidden email]> wrote:

>
> Am 10.08.2020 um 10:32 schrieb 孙世龙 sunshilong:
> > Hi, list
> >
> > I want to use a self-defined memory acquirement function instead of
> > using realloc(3) in the l_alloc function.
> > Since I am a newbie, I don't know if there are some potential problems.
> > Could somebody shed some light on this problem?
> > Thank you for your attention to this matter.
>
> I think Lua relies on the behaviour that the realloc-like function
> does not return a different pointer (copy) for shrinking an existing
> block. Maybe that could be a problem for some allocators.
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

孙世龙 sunshilong
Lua relies on the behaviour that the realloc-like function
>does not return a different pointer (copy) for shrinking an existing
>block.
What problems may occur if this rule(or advice) is broken?
Furthermore, realloc(3) provided by stand C does not guarantee it and
realloc(3) returns a different address indeed even if shrinking the
memory(for details, see
https://groups.google.com/g/comp.std.c/c/dKkZ1qG10Hk?pli=1).
S

On Tue, Aug 11, 2020 at 10:19 AM 孙世龙 sunshilong <[hidden email]> wrote:

>
> >Lua relies on the behaviour that the realloc-like function
> >does not return a different pointer (copy) for shrinking an existing
> >block.
> What problems may occur if this rule(or advice) is broken?
>
> On Mon, Aug 10, 2020 at 8:34 PM Stefan <[hidden email]> wrote:
> >
> > Am 10.08.2020 um 10:32 schrieb 孙世龙 sunshilong:
> > > Hi, list
> > >
> > > I want to use a self-defined memory acquirement function instead of
> > > using realloc(3) in the l_alloc function.
> > > Since I am a newbie, I don't know if there are some potential problems.
> > > Could somebody shed some light on this problem?
> > > Thank you for your attention to this matter.
> >
> > I think Lua relies on the behaviour that the realloc-like function
> > does not return a different pointer (copy) for shrinking an existing
> > block. Maybe that could be a problem for some allocators.
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

孙世龙 sunshilong
In reply to this post by Roberto Ierusalimschy
Thank you for your clarification and guidance.

>Lua 5.4 has removed this restriction of "no fail when shrinking".
Why is there such a restriction for the older version?
Could you please explain that in more detail?

>Actually, the test suite in Lua has a mode that forces every
> single reallocation (growing, shrinking, or to the same size!) to move
> the block.
What are the purposes for providing this mode(i.e.
forcing every single reallocation to move the block)? For security?
When should I use this mode?

Thank you for your attention to this matter.
Best Regards.
Sunshilong


On Mon, Aug 10, 2020 at 10:36 PM Roberto Ierusalimschy
<[hidden email]> wrote:

>
> > I think Lua relies on the behaviour that the realloc-like function
> > does not return a different pointer (copy) for shrinking an existing
> > block. Maybe that could be a problem for some allocators.
>
> Lua does not rely on that. The only assumption was that shrinking
> an existing block could not fail, but it can move the block without
> problems. Actually, the test suite in Lua has a mode that forces every
> single reallocation (growing, shrinking, or to the same size!) to move
> the block.
>
> Lua 5.4 has removed this restriction of "no fail when shrinking".
>
> -- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

孙世龙 sunshilong
In reply to this post by Andrew Gierth
>What matters is what the language
>standards say, and what they say is that realloc() is permitted to fail
>when shrinking a block. (Some realloc implementations never fail in that
>case, but many in common use do.)
Could you please please give me some references or documents for me to
go through?
What about malloc(3) provided by the stand C library?

Best Regards.
Sunshilong

On Tue, Aug 11, 2020 at 10:10 AM Andrew Gierth
<[hidden email]> wrote:

>
> >>>>> "Philippe" == Philippe Verdy <[hidden email]> writes:
>
>  Philippe> I don't think thank shrinking or keeping the same size may
>  Philippe> fail
>
> It doesn't matter what you think. What matters is what the language
> standards say, and what they say is that realloc() is permitted to fail
> when shrinking a block. (Some realloc implementations never fail in that
> case, but many in common use do.)
>
> --
> Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Sean Conner
It was thus said that the Great 孙世龙 sunshilong once stated:
> >What matters is what the language
> >standards say, and what they say is that realloc() is permitted to fail
> >when shrinking a block. (Some realloc implementations never fail in that
> >case, but many in common use do.)
> Could you please please give me some references or documents for me to
> go through?
> What about malloc(3) provided by the stand C library?

  Nothing better than the C Standard document itself.  They're pretty easy
to find on the Internet (I found the C89, C99 and C11 standards).

  But the descriptions are small enough to quote here.  For malloc():

        #include <stdlib.h>
        void *malloc(size_t size);

        The malloc function allocates spae for an object whose size is
        speificied by size and whose value in indeterminate.

        The malloc function returns either a null pointer or a pointer to
        the allocated space.

(wording is the same for C89, C99 and C11).  For free():

        #include <stdlib.h>
        void free(void *ptr);

        The free function causes the space pointed to by ptr to be
        deallocated, that is, made available for further allocation.  If ptr
        is a null pointer, no action occurs.  Otherwise, if the argument
        does not match a pointer earlier returned by the calloc, malloc, or
        realloc funciton, or if space as been deallocated by a call to free
        or realloc, the behavior is undefined.

        The free function returns no value.

(wording is the same for C89, C99 and C11).  And finally realloc()---the
wording changed between C89 and C99/C11, so first, C89:

        #include <stdlib.h>
        void *realloc(void *ptr,size_t size);

        The realloc function changes the size of the object pointed to by
        ptr to the size specified by size.  The contents of the object shall
        be unchanged up to the lesser of the new and old sizes.  If the new
        size is larger, the value of the newly allocated portion of the
        object is indeterminate.  If ptr is a null pointer, the realloc
        function behaves like the malloc function for the specified size.
        Otherwise, if ptr does not match a pointer earlier return by calloc,
        malloc, or realloc function, or if the space has been deallocated by
        a call to the free or realloc function, the behavior is undefined.
        If the space cannot be allocated, the object pointed to by ptr is
        unchanged.  If size is zero and ptr is not a null pointer, the
        object it points to is freed.

        The realloc function returns either a null pointer or a pointer to
        the possibly moved allocated space.

and finally C99/C11 (wording is the same for both):

        #include <stdlib.h>
        void *realloc(void *ptr,size_t size);

        The realloc function deallocates the old object pointed to by ptr
        and returns a pointer to a new object that has the size specified by
        size.  The contents of the new object shall be the same as that of
        the old object prior to deallocation, up to the lesser of the new
        and old sizes.  Any bytes in the new object beyond the size of the
        old object have indeterminate values.

        If ptr is a null pointer, the realloc function behhaves like the
        malloc function for the specfified size.  Otherwise, if ptr does not
        match a pointer earlier returned by the calloc, malloc, or realloc
        function, or if the space has deallocated by a call to the free or
        realloc function, the behavior is undefined.  If memory for the new
        object cannot be allocated, the old object is not deallocated and
        its value is unchanged.

        The realloc function returns a pointer to the new object (which may
        have the same value as a pointer to the old object), or a null
        pointer if the new object could not be allocated.

  -spc (any typos are mine, it's best to refer to the actual documents)
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

孙世龙 sunshilong
Thank you for your generous help.

>> >What matters is what the language
>> >standards say, and what they say is that realloc() is permitted to fail
>> >when shrinking a block. (Some realloc implementations never fail in that
>> >case, but many in common use do.)
>> Could you please please give me some references or documents for me to
>> go through?
>> What about malloc(3) provided by the stand C library?
>The realloc function returns a pointer to the new object (which may
>have the same value as a pointer to the old object), or a null
>pointer if the new object could not be allocated.

Thank you for your generous help.
But it still can't answer the question:
Is realloc(provided by stand C library) permitted to fail when
shrinking a block?

Best Regards.
Sunshilong

On Tue, Aug 11, 2020 at 12:34 PM Sean Conner <[hidden email]> wrote:

>
> It was thus said that the Great 孙世龙 sunshilong once stated:
> > >What matters is what the language
> > >standards say, and what they say is that realloc() is permitted to fail
> > >when shrinking a block. (Some realloc implementations never fail in that
> > >case, but many in common use do.)
> > Could you please please give me some references or documents for me to
> > go through?
> > What about malloc(3) provided by the stand C library?
>
>   Nothing better than the C Standard document itself.  They're pretty easy
> to find on the Internet (I found the C89, C99 and C11 standards).
>
>   But the descriptions are small enough to quote here.  For malloc():
>
>         #include <stdlib.h>
>         void *malloc(size_t size);
>
>         The malloc function allocates spae for an object whose size is
>         speificied by size and whose value in indeterminate.
>
>         The malloc function returns either a null pointer or a pointer to
>         the allocated space.
>
> (wording is the same for C89, C99 and C11).  For free():
>
>         #include <stdlib.h>
>         void free(void *ptr);
>
>         The free function causes the space pointed to by ptr to be
>         deallocated, that is, made available for further allocation.  If ptr
>         is a null pointer, no action occurs.  Otherwise, if the argument
>         does not match a pointer earlier returned by the calloc, malloc, or
>         realloc funciton, or if space as been deallocated by a call to free
>         or realloc, the behavior is undefined.
>
>         The free function returns no value.
>
> (wording is the same for C89, C99 and C11).  And finally realloc()---the
> wording changed between C89 and C99/C11, so first, C89:
>
>         #include <stdlib.h>
>         void *realloc(void *ptr,size_t size);
>
>         The realloc function changes the size of the object pointed to by
>         ptr to the size specified by size.  The contents of the object shall
>         be unchanged up to the lesser of the new and old sizes.  If the new
>         size is larger, the value of the newly allocated portion of the
>         object is indeterminate.  If ptr is a null pointer, the realloc
>         function behaves like the malloc function for the specified size.
>         Otherwise, if ptr does not match a pointer earlier return by calloc,
>         malloc, or realloc function, or if the space has been deallocated by
>         a call to the free or realloc function, the behavior is undefined.
>         If the space cannot be allocated, the object pointed to by ptr is
>         unchanged.  If size is zero and ptr is not a null pointer, the
>         object it points to is freed.
>
>         The realloc function returns either a null pointer or a pointer to
>         the possibly moved allocated space.
>
> and finally C99/C11 (wording is the same for both):
>
>         #include <stdlib.h>
>         void *realloc(void *ptr,size_t size);
>
>         The realloc function deallocates the old object pointed to by ptr
>         and returns a pointer to a new object that has the size specified by
>         size.  The contents of the new object shall be the same as that of
>         the old object prior to deallocation, up to the lesser of the new
>         and old sizes.  Any bytes in the new object beyond the size of the
>         old object have indeterminate values.
>
>         If ptr is a null pointer, the realloc function behhaves like the
>         malloc function for the specfified size.  Otherwise, if ptr does not
>         match a pointer earlier returned by the calloc, malloc, or realloc
>         function, or if the space has deallocated by a call to the free or
>         realloc function, the behavior is undefined.  If memory for the new
>         object cannot be allocated, the old object is not deallocated and
>         its value is unchanged.
>
>         The realloc function returns a pointer to the new object (which may
>         have the same value as a pointer to the old object), or a null
>         pointer if the new object could not be allocated.
>
>   -spc (any typos are mine, it's best to refer to the actual documents)
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Sean Conner
It was thus said that the Great 孙世龙 sunshilong once stated:

> Thank you for your generous help.
>
> >> >What matters is what the language
> >> >standards say, and what they say is that realloc() is permitted to fail
> >> >when shrinking a block. (Some realloc implementations never fail in that
> >> >case, but many in common use do.)
> >> Could you please please give me some references or documents for me to
> >> go through?
> >> What about malloc(3) provided by the stand C library?
> >The realloc function returns a pointer to the new object (which may
> >have the same value as a pointer to the old object), or a null
> >pointer if the new object could not be allocated.
>
> Thank you for your generous help.
> But it still can't answer the question:
> Is realloc(provided by stand C library) permitted to fail when
> shrinking a block?

  None of the C standards I quoted state that realloc() cannot fail when
newsize is less than oldsize.  If it's not outright stated, you cannot rely
upon it being true.  The standard does NOT state that ralloc() CANNOT fail
when the newsize is less than the oldsize.  So in that sense, yes, realloc()
CAN fail when newsize is less than oldsize.  You might never encounter a
system where that happens (much like you will probably never come across a
system with sign-magnitude integer arithmetic that the C standard allows),
but that doesn't mean it doesn't exist somewhere.

  It also means that realloc() can return the SAME pointer when growing a
block (per the wording in the C99/C11 standard---read it closely---it never
says WHEN it might return the same pointer, only that it MAY):

        #include <stdlib.h>
        void *realloc(void *ptr,size_t size);

        The realloc function deallocates the old object pointed to by ptr
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        and returns a pointer to a new object that has the size specified by
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        size.  The contents of the new object shall be the same as that of
        ^^^^
        the old object prior to deallocation, up to the lesser of the new
        and old sizes.  Any bytes in the new object beyond the size of the
        old object have indeterminate values.

        If ptr is a null pointer, the realloc function behhaves like the
        malloc function for the specfified size.  Otherwise, if ptr does not
        match a pointer earlier returned by the calloc, malloc, or realloc
        function, or if the space has deallocated by a call to the free or
        realloc function, the behavior is undefined.  If memory for the new
        object cannot be allocated, the old object is not deallocated and
        its value is unchanged.

        The realloc function returns a pointer to the new object (which may
                                                                  ^^^^^^^^^
        have the same value as a pointer to the old object), or a null
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        pointer if the new object could not be allocated.

(emphasis added).  The language is such that any number of implementations
can be used and still be considered valid.

  All you can really rely upon is checking the return pointer from realloc()
and handling the case when it returns NULL.  Do NOT assume that shrinking a
block will always succeed.  Nor assume that the pointer returned from
realloc() is the same one passed in.  

  -spc (Also unstated is what malloc(0), realloc(NULL,0), will return)
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Andrew Gierth
>>>>> "Sean" == Sean Conner <[hidden email]> writes:

 Sean> You might never encounter a system where that happens (much like
 Sean> you will probably never come across a system with sign-magnitude
 Sean> integer arithmetic that the C standard allows),

Systems where realloc() can fail to shrink a block are not rare (as I
pointed out when this topic came up before) - in particular jemalloc,
which is the default malloc on FreeBSD, can fail this way, as can any
system which maintains a distinction between large and small blocks
(which is a fairly common design approach for malloc).

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

孙世龙 sunshilong
In reply to this post by Sean Conner
Hi, Sean Conner, Adrian,Roberto

I see. I agree with you.
Realloc(3) can fail when shrinking the block.

It seems that Lua has made such an assumption.

As per the documentation(i.e.
https://www.lua.org/manual/5.3/manual.html#lua_Alloc),
which says[emphasise mine]:
Note that Standard C ensures that free(NULL) has no effect and that
realloc(NULL,size) is equivalent to malloc(size).
This code assumes that realloc does not fail when shrinking a block.
(Although Standard C does not ensure this behavior, it seems to be a
safe assumption.)

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

On Tue, Aug 11, 2020 at 5:36 PM Sean Conner <[hidden email]> wrote:

>
> It was thus said that the Great 孙世龙 sunshilong once stated:
> > Thank you for your generous help.
> >
> > >> >What matters is what the language
> > >> >standards say, and what they say is that realloc() is permitted to fail
> > >> >when shrinking a block. (Some realloc implementations never fail in that
> > >> >case, but many in common use do.)
> > >> Could you please please give me some references or documents for me to
> > >> go through?
> > >> What about malloc(3) provided by the stand C library?
> > >The realloc function returns a pointer to the new object (which may
> > >have the same value as a pointer to the old object), or a null
> > >pointer if the new object could not be allocated.
> >
> > Thank you for your generous help.
> > But it still can't answer the question:
> > Is realloc(provided by stand C library) permitted to fail when
> > shrinking a block?
>
>   None of the C standards I quoted state that realloc() cannot fail when
> newsize is less than oldsize.  If it's not outright stated, you cannot rely
> upon it being true.  The standard does NOT state that ralloc() CANNOT fail
> when the newsize is less than the oldsize.  So in that sense, yes, realloc()
> CAN fail when newsize is less than oldsize.  You might never encounter a
> system where that happens (much like you will probably never come across a
> system with sign-magnitude integer arithmetic that the C standard allows),
> but that doesn't mean it doesn't exist somewhere.
>
>   It also means that realloc() can return the SAME pointer when growing a
> block (per the wording in the C99/C11 standard---read it closely---it never
> says WHEN it might return the same pointer, only that it MAY):
>
>         #include <stdlib.h>
>         void *realloc(void *ptr,size_t size);
>
>         The realloc function deallocates the old object pointed to by ptr
>                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>         and returns a pointer to a new object that has the size specified by
>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>         size.  The contents of the new object shall be the same as that of
>         ^^^^
>         the old object prior to deallocation, up to the lesser of the new
>         and old sizes.  Any bytes in the new object beyond the size of the
>         old object have indeterminate values.
>
>         If ptr is a null pointer, the realloc function behhaves like the
>         malloc function for the specfified size.  Otherwise, if ptr does not
>         match a pointer earlier returned by the calloc, malloc, or realloc
>         function, or if the space has deallocated by a call to the free or
>         realloc function, the behavior is undefined.  If memory for the new
>         object cannot be allocated, the old object is not deallocated and
>         its value is unchanged.
>
>         The realloc function returns a pointer to the new object (which may
>                                                                   ^^^^^^^^^
>         have the same value as a pointer to the old object), or a null
>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>         pointer if the new object could not be allocated.
>
> (emphasis added).  The language is such that any number of implementations
> can be used and still be considered valid.
>
>   All you can really rely upon is checking the return pointer from realloc()
> and handling the case when it returns NULL.  Do NOT assume that shrinking a
> block will always succeed.  Nor assume that the pointer returned from
> realloc() is the same one passed in.
>
>   -spc (Also unstated is what malloc(0), realloc(NULL,0), will return)
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Andrew Gierth
>>>>> "孙世龙" == 孙世龙 sunshilong <[hidden email]> writes:

 孙世龙> Hi, Sean Conner, Adrian,Roberto
 孙世龙> I see. I agree with you.
 孙世龙> Realloc(3) can fail when shrinking the block.

 孙世龙> It seems that Lua has made such an assumption.

In 5.3, yes. I pointed out this bug on this very mailing list in
November 2017 and it has not been fixed in 5.3, only in 5.4.

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Gerhard Sittig-2
In reply to this post by 孙世龙 sunshilong
On Tue, 2020-08-11 at 11:49 +0800, 孙世龙 sunshilong wrote:
>
> >What matters is what the language
> >standards say, and what they say is that realloc() is permitted to fail
> >when shrinking a block. (Some realloc implementations never fail in that
> >case, but many in common use do.)
> Could you please please give me some references or documents for me to
> go through?
> What about malloc(3) provided by the stand C library?

There is no such thing as _the_ malloc(3) provided by _the_
standard C library. That's the very point.

It doesn't matter what the specific setup that you happen to use
locally happens to do in this specific version and configuration
on this specific platform that you use today. You just cannot
rely on any of these local details unless they officially are
granted to you. Which they aren't in this case.

And even if you see a specific implementation dependent behaviour
today in your local setup, you still cannot rely on it to be
present tomorrow in another version of this very software or in a
different configuration on that platform of yours or on another
platform with the same version and configuration of the software
component.

Or as an alternative: Remain aware that you rely on something
that you were not given. And accept when your assumption doesn't
hold. This can be as valid as sticking to portable code, just
needs to be done consciously. Your decision.


virtually yours
Gerhard Sittig
--
     If you don't understand or are scared by any of the above
             ask your parents or an adult to help you.
Reply | Threaded
Open this post in threaded view
|

Re: Are there some potential problems if I use a self-defined memory acquirement function instead of realloc(3) in the l_alloc function?

Roberto Ierusalimschy
In reply to this post by 孙世龙 sunshilong
> >Lua relies on the behaviour that the realloc-like function
> >does not return a different pointer (copy) for shrinking an existing
> >block.

That is not true. Where did you read that?

-- Roberto
12