• Memory allocator

    From Ayush Anshul@21:1/5 to All on Thu Sep 23 07:46:27 2021
    Consider me a C++ noob and go easy on me :|

    Suppose I have a package code and a product code on top of it.
    I have control only over the package code side of things. My
    memory is allocated/monitored via product code. Can
    I override new/malloc on my side to get all allocations done via
    that product code function or something similar

    One way I know is that they override global new/malloc on their end.
    Other way is I write custom allocator, and use it everywhere(where I
    have there allocator function registered as a callback).

    Is there some other easier way?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alf P. Steinbach@21:1/5 to Ayush Anshul on Thu Sep 23 18:07:04 2021
    On 23 Sep 2021 16:46, Ayush Anshul wrote:
    Consider me a C++ noob and go easy on me :|

    Suppose I have a package code and a product code on top of it.
    I have control only over the package code side of things. My
    memory is allocated/monitored via product code. Can
    I override new/malloc on my side to get all allocations done via
    that product code function or something similar

    One way I know is that they override global new/malloc on their end.
    Other way is I write custom allocator, and use it everywhere(where I
    have there allocator function registered as a callback).

    Is there some other easier way?


    At the C++ level you can define `operator new` and `operator delete`
    (plus array variants) in each class.

    For a Windows DLL you can override `malloc` internally in the DLL, but a
    Unix shared library is more like a static library, not separate from the
    client code, so you can't do that with a shared library.

    An alternative to "overriding" is to base the library code on an
    allocator specified by the client code, and reasonably defaulted.

    No matter what you decide to do, make absolutely utterly sure that
    there's no reasonable way that client code can simply do `delete p;`
    where `p` was obtained from your library, allocated in special way.

    Probably you need to specialize `std::default_delete` to support `std::unique_ptr` for your classes.

    - Alf

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Lurndal@21:1/5 to Alf P. Steinbach on Thu Sep 23 17:15:12 2021
    "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
    On 23 Sep 2021 16:46, Ayush Anshul wrote:
    Consider me a C++ noob and go easy on me :|

    Suppose I have a package code and a product code on top of it.
    I have control only over the package code side of things. My
    memory is allocated/monitored via product code. Can
    I override new/malloc on my side to get all allocations done via
    that product code function or something similar

    One way I know is that they override global new/malloc on their end.
    Other way is I write custom allocator, and use it everywhere(where I
    have there allocator function registered as a callback).

    Is there some other easier way?


    At the C++ level you can define `operator new` and `operator delete`
    (plus array variants) in each class.

    For a Windows DLL you can override `malloc` internally in the DLL, but a
    Unix shared library is more like a static library, not separate from the >client code, so you can't do that with a shared library.

    By why criteria do you consider a unix shared object to be anything like
    a static library?

    "malloc" is a "weak" symbol in all modern unix systems, which means that
    the symbol can be overridden by the application if desired.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alf P. Steinbach@21:1/5 to Scott Lurndal on Thu Sep 23 21:59:47 2021
    On 23 Sep 2021 19:15, Scott Lurndal wrote:
    "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
    On 23 Sep 2021 16:46, Ayush Anshul wrote:
    Consider me a C++ noob and go easy on me :|

    Suppose I have a package code and a product code on top of it.
    I have control only over the package code side of things. My
    memory is allocated/monitored via product code. Can
    I override new/malloc on my side to get all allocations done via
    that product code function or something similar

    One way I know is that they override global new/malloc on their end.
    Other way is I write custom allocator, and use it everywhere(where I
    have there allocator function registered as a callback).

    Is there some other easier way?


    At the C++ level you can define `operator new` and `operator delete`
    (plus array variants) in each class.

    For a Windows DLL you can override `malloc` internally in the DLL, but a
    Unix shared library is more like a static library, not separate from the
    client code, so you can't do that with a shared library.

    By why criteria do you consider a unix shared object to be anything like
    a static library?

    "malloc" is a "weak" symbol in all modern unix systems, which means that
    the symbol can be overridden by the application if desired.

    The OP is interested in overriding it locally in his library (to be
    precise the OP calls it a "package", I presume that's a library).

    A Windows DLL supports that local override where the library uses one
    variant of `malloc` while the client code, the app, uses another.

    I believe, but please correct me if I'm wrong, that you can't do that
    with a Unix shared library?

    - Alf

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Lurndal@21:1/5 to Alf P. Steinbach on Fri Sep 24 14:58:53 2021
    "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
    On 23 Sep 2021 19:15, Scott Lurndal wrote:
    "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
    On 23 Sep 2021 16:46, Ayush Anshul wrote:
    Consider me a C++ noob and go easy on me :|

    Suppose I have a package code and a product code on top of it.
    I have control only over the package code side of things. My
    memory is allocated/monitored via product code. Can
    I override new/malloc on my side to get all allocations done via
    that product code function or something similar

    One way I know is that they override global new/malloc on their end.
    Other way is I write custom allocator, and use it everywhere(where I
    have there allocator function registered as a callback).

    Is there some other easier way?


    At the C++ level you can define `operator new` and `operator delete`
    (plus array variants) in each class.

    For a Windows DLL you can override `malloc` internally in the DLL, but a >>> Unix shared library is more like a static library, not separate from the >>> client code, so you can't do that with a shared library.

    By why criteria do you consider a unix shared object to be anything like
    a static library?

    "malloc" is a "weak" symbol in all modern unix systems, which means that
    the symbol can be overridden by the application if desired.

    The OP is interested in overriding it locally in his library (to be
    precise the OP calls it a "package", I presume that's a library).

    A Windows DLL supports that local override where the library uses one
    variant of `malloc` while the client code, the app, uses another.

    I believe, but please correct me if I'm wrong, that you can't do that
    with a Unix shared library?


    For every symbol declared as "weak" there is a corresponding "strong"
    symbol which starts with an underscore. The dynamic shared object
    (dso) can use the strong version if it wishes to prevent the
    application from overriding its malloc function.

    (or read, or write, or lseek, etc).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alf P. Steinbach@21:1/5 to Scott Lurndal on Fri Sep 24 18:55:35 2021
    On 24 Sep 2021 16:58, Scott Lurndal wrote:
    "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
    On 23 Sep 2021 19:15, Scott Lurndal wrote:
    "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
    On 23 Sep 2021 16:46, Ayush Anshul wrote:
    Consider me a C++ noob and go easy on me :|

    Suppose I have a package code and a product code on top of it.
    I have control only over the package code side of things. My
    memory is allocated/monitored via product code. Can
    I override new/malloc on my side to get all allocations done via
    that product code function or something similar

    One way I know is that they override global new/malloc on their end. >>>>> Other way is I write custom allocator, and use it everywhere(where I >>>>> have there allocator function registered as a callback).

    Is there some other easier way?


    At the C++ level you can define `operator new` and `operator delete`
    (plus array variants) in each class.

    For a Windows DLL you can override `malloc` internally in the DLL, but a >>>> Unix shared library is more like a static library, not separate from the >>>> client code, so you can't do that with a shared library.

    By why criteria do you consider a unix shared object to be anything like >>> a static library?

    "malloc" is a "weak" symbol in all modern unix systems, which means that >>> the symbol can be overridden by the application if desired.

    The OP is interested in overriding it locally in his library (to be
    precise the OP calls it a "package", I presume that's a library).

    A Windows DLL supports that local override where the library uses one
    variant of `malloc` while the client code, the app, uses another.

    I believe, but please correct me if I'm wrong, that you can't do that
    with a Unix shared library?


    For every symbol declared as "weak" there is a corresponding "strong"
    symbol which starts with an underscore. The dynamic shared object
    (dso) can use the strong version if it wishes to prevent the
    application from overriding its malloc function.

    (or read, or write, or lseek, etc).


    Well, that's something else (e.g. it won't affect calls to `malloc` from `operator new`, and it only supports the app overriding, not the library overriding), but I learned something. Thanks. :)

    - Alf

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Lurndal@21:1/5 to Alf P. Steinbach on Fri Sep 24 17:23:59 2021
    "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
    On 24 Sep 2021 16:58, Scott Lurndal wrote:
    "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
    On 23 Sep 2021 19:15, Scott Lurndal wrote:
    "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
    On 23 Sep 2021 16:46, Ayush Anshul wrote:
    Consider me a C++ noob and go easy on me :|

    Suppose I have a package code and a product code on top of it.
    I have control only over the package code side of things. My
    memory is allocated/monitored via product code. Can
    I override new/malloc on my side to get all allocations done via
    that product code function or something similar

    One way I know is that they override global new/malloc on their end. >>>>>> Other way is I write custom allocator, and use it everywhere(where I >>>>>> have there allocator function registered as a callback).

    Is there some other easier way?


    At the C++ level you can define `operator new` and `operator delete` >>>>> (plus array variants) in each class.

    For a Windows DLL you can override `malloc` internally in the DLL, but a >>>>> Unix shared library is more like a static library, not separate from the >>>>> client code, so you can't do that with a shared library.

    By why criteria do you consider a unix shared object to be anything like >>>> a static library?

    "malloc" is a "weak" symbol in all modern unix systems, which means that >>>> the symbol can be overridden by the application if desired.

    The OP is interested in overriding it locally in his library (to be
    precise the OP calls it a "package", I presume that's a library).

    A Windows DLL supports that local override where the library uses one
    variant of `malloc` while the client code, the app, uses another.

    I believe, but please correct me if I'm wrong, that you can't do that
    with a Unix shared library?


    For every symbol declared as "weak" there is a corresponding "strong"
    symbol which starts with an underscore. The dynamic shared object
    (dso) can use the strong version if it wishes to prevent the
    application from overriding its malloc function.

    (or read, or write, or lseek, etc).


    Well, that's something else (e.g. it won't affect calls to `malloc` from >`operator new`, and it only supports the app overriding, not the library >overriding), but I learned something. Thanks. :)

    Wouldn't it be possible for the library to simply build its
    own allocator[*] and/or overload 'operator new' for the cases
    where it wants to control its own allocation?

    [*] Based on sbrk/mmap/whatever-windows-equivalent-exists-if-nay

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)