[ub] memcpy blessing a new object of statically unknown type

[David Krauss]

Porting my std::function library <https://github.com/potswa/cxx_function/> to MSVC, some UB reared its head. The function wrapper class uses an aligned_storage blob for its call target under the small-object optimization. When the target is small and trivially-copyable, the copy/move constructors use memcpy on the blob to avoid determining the dynamic type of the target.

MSVC seems to do stricter alias analysis than GCC or Clang. The effect of the memcpy is not subsequently observed unless its both its source and destination operands are of the appropriate type (before they’re implicitly cast to void*).

// Given:
erasure_base & erasure()
{ return reinterpret_cast< erasure_base & >( this->storage ); }

// Inside copy/move constructor:
// (1) well-behaved:
std::memcpy( & erasure(), & o.erasure(), sizeof (storage) );
// (2) crashes after erasure().foo observes stale bytes:
std::memcpy( & erasure(), & storage, sizeof (storage) );

In retrospect, (1) is certainly more kosher, but shouldn’t (2) work as well?

This doesn’t seem to rise to the level of a bug report, but it might be relevant to the new memcpy lifetime-initiation rules. Will they ascribe some static type to the source and destination bytes?

More worryingly, erasure_base is a base class. When the memcpy replaces one derived object with another of the same type, what’s the guarantee that the overwritten value won’t be observed? This problem doesn’t manifest so far, but it seems only a matter of time. No easy solution comes to mind, nor any practical workaround for type-erasure libraries. The tough solution is for alias analysis to somehow work without considering the static types passed to memcpy.

I’m no expert on alias analysis, so I don’t know whether GCC and Clang are more clever, or less aggressive. Is it luck that the same bug hasn’t appeared on other platforms? Any insights about the upcoming rules?

- thanks,
D

[Hubert Tong]

My two cents: even when memcpy becomes capable of initiating the lifetime
of (and initializing) an object in an "aligned_storage blob", the kosher
way of accessing that object would require std::launder.

Now, I am not sure if it is a copy/paste error, my misunderstanding of the
code, or what, but it seems that (2) crashes because "&erasure()" and
"&storage" overlap (and if the memcpy UB was not enough, it probably does
not do what you intended).

-- HT

On Fri, Jan 8, 2016 at 10:15 AM, David Krauss <***@me.com> wrote:

> Porting my std::function library <https://github.com/potswa/cxx_function/> to
> MSVC, some UB reared its head. The function wrapper class uses an
> aligned_storage blob for its call target under the small-object
> optimization. When the target is small and trivially-copyable, the
> copy/move constructors use memcpy on the blob to avoid determining the
> dynamic type of the target.
>
> MSVC seems to do stricter alias analysis than GCC or Clang. The effect of
> the memcpy is not subsequently observed unless its both its source and
> destination operands are of the appropriate type (before they’re implicitly
> cast to void*).
>
> // Given:
> erasure_base & erasure()
> { return reinterpret_cast< erasure_base & >( this->storage ); }
>
> // Inside copy/move constructor:
> // (1) well-behaved:
> std::memcpy( & erasure(), & o.erasure(), sizeof (storage) );
> // (2) crashes after erasure().foo observes stale bytes:
> std::memcpy( & erasure(), & storage, sizeof (storage) );
>
> In retrospect, (1) is certainly more kosher, but shouldn’t (2) work as
> well?
>
> This doesn’t seem to rise to the level of a bug report, but it might be
> relevant to the new memcpy lifetime-initiation rules. Will they ascribe
> some static type to the source and destination bytes?
>
> More worryingly, erasure_base is a base class. When the memcpy replaces
> one derived object with another of the same type, what’s the guarantee that
> the overwritten value won’t be observed? This problem doesn’t manifest so
> far, but it seems only a matter of time. No easy solution comes to mind,
> nor any practical workaround for type-erasure libraries. The tough solution
> is for alias analysis to somehow work without considering the static types
> passed to memcpy.
>
> I’m no expert on alias analysis, so I don’t know whether GCC and Clang are
> more clever, or less aggressive. Is it luck that the same bug hasn’t
> appeared on other platforms? Any insights about the upcoming rules?
>
> - thanks,
> D
>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
>

[David Krauss]

> On 2016–01–09, at 1:54 AM, Hubert Tong <***@gmail.com> wrote:
>
> My two cents: even when memcpy becomes capable of initiating the lifetime of (and initializing) an object in an "aligned_storage blob", the kosher way of accessing that object would require std::launder.

Then the question becomes, does std::launder require a static type? I was under the impression that it does. In type erasure, memcpy needs to support polymorphism.

> Now, I am not sure if it is a copy/paste error, my misunderstanding of the code, or what, but it seems that (2) crashes because "&erasure()" and "&storage" overlap (and if the memcpy UB was not enough, it probably does not do what you intended).

Oops, copy-paste error. It should say &o.storage. (I just double-checked, the post does differ from the defective test.)

It’s a good thing you mentioned it, because I ran some more tests and now it looks like an inlining issue, not cleverness in alias analysis. Apparently it works whenever both memcpy arguments are function calls; it doesn’t matter what type they cast to.

So, my bugfix didn’t clarify types at all; it merely obfuscated the object identities. The port works, but it’s still as UB as ever.

[Hubert Tong]

On Fri, Jan 8, 2016 at 9:41 PM, David Krauss <***@me.com> wrote:

>
> On 2016–01–09, at 1:54 AM, Hubert Tong <***@gmail.com>
> wrote:
>
> My two cents: even when memcpy becomes capable of initiating the lifetime
> of (and initializing) an object in an "aligned_storage blob", the kosher
> way of accessing that object would require std::launder.
>
>
> Then the question becomes, does std::launder require a static type? I was
> under the impression that it does. In type erasure, memcpy needs to
> support polymorphism.
>
Yes, std::launder requires a static type; however, it does not limit the
ability of memcpy to operate without knowing the type of the object being
copied. The std::launder call is involved *after* the completion of memcpy
to access the object that the memcpy initialized.


> Now, I am not sure if it is a copy/paste error, my misunderstanding of the
> code, or what, but it seems that (2) crashes because "&erasure()" and
> "&storage" overlap (and if the memcpy UB was not enough, it probably does
> not do what you intended).
>
>
> Oops, copy-paste error. It should say &o.storage. (I just double-checked,
> the post does differ from the defective test.)
>
> It’s a good thing you mentioned it, because I ran some more tests and now
> it looks like an inlining issue, not cleverness in alias analysis.
> Apparently it works whenever both memcpy arguments are function calls; it
> doesn’t matter what type they cast to.
>
> So, my bugfix didn’t clarify types at all; it merely obfuscated the object
> identities. The port works, but it’s still as UB as ever.
>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
>

[Patrice Roy]

std::launder() is one of the scariest things I've seen, but it's part of
the pleasure of working on WG21 :)

2016-01-08 22:35 GMT-05:00 Hubert Tong <***@gmail.com>:

> On Fri, Jan 8, 2016 at 9:41 PM, David Krauss <***@me.com> wrote:
>
>>
>> On 2016–01–09, at 1:54 AM, Hubert Tong <***@gmail.com>
>> wrote:
>>
>> My two cents: even when memcpy becomes capable of initiating the lifetime
>> of (and initializing) an object in an "aligned_storage blob", the kosher
>> way of accessing that object would require std::launder.
>>
>>
>> Then the question becomes, does std::launder require a static type? I
>> was under the impression that it does. In type erasure, memcpy needs to
>> support polymorphism.
>>
> Yes, std::launder requires a static type; however, it does not limit the
> ability of memcpy to operate without knowing the type of the object being
> copied. The std::launder call is involved *after* the completion of memcpy
> to access the object that the memcpy initialized.
>
>
>> Now, I am not sure if it is a copy/paste error, my misunderstanding of
>> the code, or what, but it seems that (2) crashes because "&erasure()" and
>> "&storage" overlap (and if the memcpy UB was not enough, it probably does
>> not do what you intended).
>>
>>
>> Oops, copy-paste error. It should say &o.storage. (I just
>> double-checked, the post does differ from the defective test.)
>>
>> It’s a good thing you mentioned it, because I ran some more tests and now
>> it looks like an inlining issue, not cleverness in alias analysis.
>> Apparently it works whenever both memcpy arguments are function calls;
>> it doesn’t matter what type they cast to.
>>
>> So, my bugfix didn’t clarify types at all; it merely obfuscated the
>> object identities. The port works, but it’s still as UB as ever.
>>
>>
>> _______________________________________________
>> ub mailing list
>> ***@isocpp.open-std.org
>> http://www.open-std.org/mailman/listinfo/ub
>>
>>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
>

[Gabriel Dos Reis]

I suspect the situation is analogous to reinterpret_cast vs. C-style cast: not so long ago, people would freak out when they see reinterpret_cast directly expressing the same intent an ambiguous C-style cast was previously used for. Of course, people would abuse; but I rather see them use std::launder and police them, than they doing other crazy hacks that are much harder to audit for correctness.

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Patrice Roy
Sent: Friday, January 8, 2016 7:51 PM
To: WG21 UB study group <***@open-std.org>
Subject: Re: [ub] memcpy blessing a new object of statically unknown type

std::launder() is one of the scariest things I've seen, but it's part of the pleasure of working on WG21 :)

2016-01-08 22:35 GMT-05:00 Hubert Tong <***@gmail.com<mailto:***@gmail.com>>:
On Fri, Jan 8, 2016 at 9:41 PM, David Krauss <***@me.com<mailto:***@me.com>> wrote:

On 2016–01–09, at 1:54 AM, Hubert Tong <***@gmail.com<mailto:***@gmail.com>> wrote:

My two cents: even when memcpy becomes capable of initiating the lifetime of (and initializing) an object in an "aligned_storage blob", the kosher way of accessing that object would require std::launder.

Then the question becomes, does std::launder require a static type? I was under the impression that it does. In type erasure, memcpy needs to support polymorphism.
Yes, std::launder requires a static type; however, it does not limit the ability of memcpy to operate without knowing the type of the object being copied. The std::launder call is involved after the completion of memcpy to access the object that the memcpy initialized.


Now, I am not sure if it is a copy/paste error, my misunderstanding of the code, or what, but it seems that (2) crashes because "&erasure()" and "&storage" overlap (and if the memcpy UB was not enough, it probably does not do what you intended).

Oops, copy-paste error. It should say &o.storage. (I just double-checked, the post does differ from the defective test.)

It’s a good thing you mentioned it, because I ran some more tests and now it looks like an inlining issue, not cleverness in alias analysis. Apparently it works whenever both memcpy arguments are function calls; it doesn’t matter what type they cast to.

So, my bugfix didn’t clarify types at all; it merely obfuscated the object identities. The port works, but it’s still as UB as ever.


_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.open-std.org%2fmailman%2flistinfo%2fub&data=01%7c01%7cgdr%40microsoft.com%7cf061719c225842af4c3a08d318a80f50%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=XeiYc1%2fXuTGiLnvXg8W41qgi0Jdv4AvAoie7ur0%2bcfc%3d>


_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.open-std.org%2fmailman%2flistinfo%2fub&data=01%7c01%7cgdr%40microsoft.com%7cf061719c225842af4c3a08d318a80f50%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=XeiYc1%2fXuTGiLnvXg8W41qgi0Jdv4AvAoie7ur0%2bcfc%3d>

[David Krauss]

> On 2016–01–09, at 11:35 AM, Hubert Tong <***@gmail.com> wrote:
>
> Yes, std::launder requires a static type; however, it does not limit the ability of memcpy to operate without knowing the type of the object being copied. The std::launder call is involved after the completion of memcpy to access the object that the memcpyinitialized.

Right, but in type erasure, the static type must be determined by inspecting the blob somehow. (Stashing a discriminating value elsewhere is one solution, but it’s more common and often more efficient to use an abstract base class or a discriminator inside a union.)

My library would launder the erasure_base subobject to retrieve its dispatch table, but then it’d be stuck. Dispatching to a derived class would lead back to UB.

One workaround could be to launder the same address repeatedly as the type becomes better resolved. For example, the call wrapper could launder a base class address, then perform an indirect call, then the callee could launder again to the derived class. For the common case of virtual dynamic dispatch, this sounds like it would incur UB before first line of the callee. My library doesn’t use virtual, but similar ones do. If only complete objects can be laundered, devirtualization could kick in
 or launder could refuse to handle an abstract class at all. The workaround would also imply an excessive number of derived-type launder calls, which could compromise optimization by suggesting that bitwise manipulations are occurring when none are. Reloading the dispatch table pointer costs cycles.

Perhaps a second style of laundering could implement a compromise. First, auto &header = *launder(header_ptr) gets a fully-formed header object from a blob, and then auto &whole = launder_extend<whole_type>(header) revises the object identity to make header a subobject sharing its address with another already-fully-formed object of type whole. (For example, header could be a base, a union member, or an initial struct member.) The launder_extend function differs in that it acts only if its argument was believed to be a complete object (i.e. fresh from launder), and it only launders the remainder of the new complete object. To solve the virtual issue, do not let launder imply that its result is most-derived. Perhaps, let virtual dispatch implicitly do launder_extend.

This scheme leaves launder open-ended so a polymorphic object or union can be used, yet still laundered further. A simple implementation can opt to treat launder_extend the same as launder.

Example:
struct discriminator { int value; };
struct foo { discriminator d; int i; };
struct bar { discriminator d; float f; };
union foobar { foo a; bar b; };
struct baz : discriminator { double x; };
struct bad : discriminator { virtual ~ bad(); };

void unpack( discriminator * p ) {
std::launder( p ); // OK: now we can access p.
int disc = p->value;
if ( disc == 0 ) {
auto & f = std::launder_extend< foo >( * p ); // OK: now we can access a foo.
int q = p->value; // Load may be elided. Value is already in disc, equal to zero.
auto & fb = std::launder_extend< foobar >( * f ); // OK: a further extension to a super-object.
auto & f2 = std::launder_extend< foo >( * p ); // OK, no-op: p was already extended, becoming a subobject.
auto & b = std::launder_extend< bar >( * p ); // UB: there’s already a different object there.
auto & i = std::launder_extend< int >( * p ); // Library precondition violation: invalid object extension.
} else if ( disc == 1 ) {
baz & z = std::launder_extend< baz >( * p ); // OK, but implementation-dependent in theory.
} else if ( disc == 2 ) {
// Library precondition violation: no discriminator subobject shares an address with class bad.
bad & x = std::launder_extend< bad >( * p );
}
}

[Hubert Tong]

On Sat, Jan 9, 2016 at 1:35 AM, David Krauss <***@me.com> wrote:

>
> On 2016–01–09, at 11:35 AM, Hubert Tong <***@gmail.com>
> wrote:
>
> Yes, std::launder requires a static type; however, it does not limit the
> ability of memcpy to operate without knowing the type of the object being
> copied. The std::launder call is involved *after* the completion of memcpy
> to access the object that the memcpyinitialized.
>
>
> Right, but in type erasure, the static type must be determined by
> inspecting the blob somehow. (Stashing a discriminating value elsewhere is
> one solution, but it’s more common and often more efficient to use an
> abstract base class or a discriminator inside a union.)
>
> My library would launder the erasure_base subobject to retrieve its
> dispatch table, but then it’d be stuck. Dispatching to a derived class
> would lead back to UB.
>
While std::launder requires a static (as in, bound at compile-time) type,
it does not require that the object retrieved is a complete object.
Following the std::launder, using plain-old static_cast to convert from the
base to the derived type is fine (assuming the static_cast would otherwise
be well-formed).


>
> One workaround could be to launder the same address repeatedly as the type
> becomes better resolved. For example, the call wrapper could launder a base
> class address, then perform an indirect call, then the callee could launder
> again to the derived class. For the common case of virtual dynamic
> dispatch, this sounds like it would incur UB before first line of the
> callee. My library doesn’t use virtual, but similar ones do. If only
> complete objects can be laundered, devirtualization could kick in
 or
> launder could refuse to handle an abstract class at all. The workaround
> would also imply an excessive number of derived-type launder calls, which
> could compromise optimization by suggesting that bitwise manipulations are
> occurring when none are. Reloading the dispatch table pointer costs cycles.
>
> Perhaps a second style of laundering could implement a compromise. First, auto
> &header = *launder(header_ptr) gets a fully-formed header object from a
> blob, and then auto &whole = launder_extend<whole_type>(header) revises
> the object identity to make header a subobject sharing its address with
> another already-fully-formed object of type whole. (For example, header could
> be a base, a union member, or an initial struct member.) The
> launder_extend function differs in that it acts only if its argument was
> believed to be a complete object (i.e. fresh from launder), and it only
> launders the remainder of the new complete object. To solve the virtual issue,
> do not let launder imply that its result is most-derived. Perhaps, let
> virtual dispatch implicitly do launder_extend.
>
> This scheme leaves launder open-ended so a polymorphic object or union
> can be used, yet still laundered further. A simple implementation can opt
> to treat launder_extend the same as launder.
>
> Example:
> struct discriminator { int value; };
> struct foo { discriminator d; int i; };
> struct bar { discriminator d; float f; };
> union foobar { foo a; bar b; };
> struct baz : discriminator { double x; };
> struct bad : discriminator { virtual ~ bad(); };
>
> void unpack( discriminator * p ) {
> std::launder( p ); // OK: now we can access p.
>
Launder does not cause side-effects. This line does nothing.
Try:
discriminator *pp = std::launder(p); // OK: pp points to a discriminator
object if the preconditions of launder are met


> int disc = p->value;
>
Use pp, but yes.


> if ( disc == 0 ) {
> auto & f = std::launder_extend< foo >( * p ); // OK: now we can
> access a foo.
>
auto &f = *reinterpret_cast<foo *>(pp); // OK: we had a valid pointer to
the first non-static data member of a standard-layout struct (refer to CWG
notes from Kona)

int q = p->value; // Load may be elided. Value is already in disc,
> equal to zero.
>
Use pp, but yes.

auto & fb = std::launder_extend< foobar >( * f ); // OK: a further
> extension to a super-object.
>
auto &fb = reinterpret_cast<foobar &>(f); // OK: we have a valid "pointer"
to a non-static data member of a union (refer to CWG notes from Kona)
or
auto &fb = *reinterpret_cast<foobar *>(pp); // OK: the applies transitivity
rules

auto & f2 = std::launder_extend< foo >( * p ); // OK, no-op: p was
> already extended, becoming a subobject.
>
Same as f.

auto & b = std::launder_extend< bar >( * p ); // UB: there’s
> already a different object there.
>
I think the issue of non-active members of a union was still in flux during
Kona.


> auto & i = std::launder_extend< int >( * p ); // Library
> precondition violation: invalid object extension.
>
auto &i = *reinterpret_cast<int *>(pp); // OK


> } else if ( disc == 1 ) {
> baz & z = std::launder_extend< baz >( * p ); // OK, but
> implementation-dependent in theory.
>
baz &z = *static_cast<baz *>(pp); // OK


> } else if ( disc == 2 ) {
>
This case is likely to go wrong before it gets here:
bad b;
discriminator *pb = reinterpret_cast<discriminator *>(&b); // result
unspecified


> // Library precondition violation: no discriminator subobject
> shares an address with class bad.
> bad & x = std::launder_extend< bad >( * p );
> }
> }
>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
>

[Gabriel Dos Reis]

My assumption has always been that std::launder is a zero-overhead abstraction (really a no-op at the machine level) whole sole purpose is to inform the static semantics elaborator about the lifetime and type of a given storage. I hope we aren’t considering anything more complicated than that.

-- Gaby

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of David Krauss
Sent: Friday, January 8, 2016 10:36 PM
To: WG21 UB study group <***@open-std.org>
Cc: ***@gmail.com
Subject: Re: [ub] memcpy blessing a new object of statically unknown type


On 2016–01–09, at 11:35 AM, Hubert Tong <***@gmail.com<mailto:***@gmail.com>> wrote:

Yes, std::launder requires a static type; however, it does not limit the ability of memcpy to operate without knowing the type of the object being copied. The std::launder call is involved after the completion of memcpy to access the object that the memcpyinitialized.

Right, but in type erasure, the static type must be determined by inspecting the blob somehow. (Stashing a discriminating value elsewhere is one solution, but it’s more common and often more efficient to use an abstract base class or a discriminator inside a union.)

My library would launder the erasure_base subobject to retrieve its dispatch table, but then it’d be stuck. Dispatching to a derived class would lead back to UB.

One workaround could be to launder the same address repeatedly as the type becomes better resolved. For example, the call wrapper could launder a base class address, then perform an indirect call, then the callee could launder again to the derived class. For the common case of virtual dynamic dispatch, this sounds like it would incur UB before first line of the callee. My library doesn’t use virtual, but similar ones do. If only complete objects can be laundered, devirtualization could kick in
 or launder could refuse to handle an abstract class at all. The workaround would also imply an excessive number of derived-type launder calls, which could compromise optimization by suggesting that bitwise manipulations are occurring when none are. Reloading the dispatch table pointer costs cycles.

Perhaps a second style of laundering could implement a compromise. First, auto &header = *launder(header_ptr) gets a fully-formed header object from a blob, and then auto &whole = launder_extend<whole_type>(header) revises the object identity to make header a subobject sharing its address with another already-fully-formed object of type whole. (For example, header could be a base, a union member, or an initial struct member.) The launder_extend function differs in that it acts only if its argument was believed to be a complete object (i.e. fresh from launder), and it only launders the remainder of the new complete object. To solve the virtual issue, do not let launder imply that its result is most-derived. Perhaps, let virtual dispatch implicitly do launder_extend.

This scheme leaves launder open-ended so a polymorphic object or union can be used, yet still laundered further. A simple implementation can opt to treat launder_extend the same as launder.

Example:
struct discriminator { int value; };
struct foo { discriminator d; int i; };
struct bar { discriminator d; float f; };
union foobar { foo a; bar b; };
struct baz : discriminator { double x; };
struct bad : discriminator { virtual ~ bad(); };

void unpack( discriminator * p ) {
std::launder( p ); // OK: now we can access p.
int disc = p->value;
if ( disc == 0 ) {
auto & f = std::launder_extend< foo >( * p ); // OK: now we can access a foo.
int q = p->value; // Load may be elided. Value is already in disc, equal to zero.
auto & fb = std::launder_extend< foobar >( * f ); // OK: a further extension to a super-object.
auto & f2 = std::launder_extend< foo >( * p ); // OK, no-op: p was already extended, becoming a subobject.
auto & b = std::launder_extend< bar >( * p ); // UB: there’s already a different object there.
auto & i = std::launder_extend< int >( * p ); // Library precondition violation: invalid object extension.
} else if ( disc == 1 ) {
baz & z = std::launder_extend< baz >( * p ); // OK, but implementation-dependent in theory.
} else if ( disc == 2 ) {
// Library precondition violation: no discriminator subobject shares an address with class bad.
bad & x = std::launder_extend< bad >( * p );
}
}

[Patrice Roy]

I know it is. I just hope it stays hidden deep where those who know about
it are... essentially those who read this mailing list :)

2016-01-09 16:42 GMT-05:00 Gabriel Dos Reis <***@microsoft.com>:

> My assumption has always been that std::launder is a zero-overhead
> abstraction (really a no-op at the machine level) whole sole purpose is to
> inform the static semantics elaborator about the lifetime and type of a
> given storage. I hope we aren’t considering anything more complicated than
> that.
>
>
>
> -- Gaby
>
>
>
> *From:* ub-***@open-std.org [mailto:ub-***@open-std.org] *On
> Behalf Of *David Krauss
> *Sent:* Friday, January 8, 2016 10:36 PM
> *To:* WG21 UB study group <***@open-std.org>
> *Cc:* ***@gmail.com
> *Subject:* Re: [ub] memcpy blessing a new object of statically unknown
> type
>
>
>
>
>
> On 2016–01–09, at 11:35 AM, Hubert Tong <***@gmail.com>
> wrote:
>
>
>
> Yes, std::launder requires a static type; however, it does not limit the
> ability of memcpy to operate without knowing the type of the object being
> copied. The std::launder call is involved *after* the completion of memcpy
> to access the object that the memcpyinitialized.
>
>
>
> Right, but in type erasure, the static type must be determined by
> inspecting the blob somehow. (Stashing a discriminating value elsewhere is
> one solution, but it’s more common and often more efficient to use an
> abstract base class or a discriminator inside a union.)
>
>
>
> My library would launder the erasure_base subobject to retrieve its
> dispatch table, but then it’d be stuck. Dispatching to a derived class
> would lead back to UB.
>
>
>
> One workaround could be to launder the same address repeatedly as the type
> becomes better resolved. For example, the call wrapper could launder a base
> class address, then perform an indirect call, then the callee could launder
> again to the derived class. For the common case of virtual dynamic
> dispatch, this sounds like it would incur UB before first line of the
> callee. My library doesn’t use virtual, but similar ones do. If only
> complete objects can be laundered, devirtualization could kick in
 or
> launder could refuse to handle an abstract class at all. The workaround
> would also imply an excessive number of derived-type launder calls, which
> could compromise optimization by suggesting that bitwise manipulations are
> occurring when none are. Reloading the dispatch table pointer costs cycles.
>
>
>
> Perhaps a second style of laundering could implement a compromise. First, auto
> &header = *launder(header_ptr) gets a fully-formed header object from a
> blob, and then auto &whole = launder_extend<whole_type>(header) revises
> the object identity to make header a subobject sharing its address with
> another already-fully-formed object of type whole. (For example, header could
> be a base, a union member, or an initial struct member.) The
> launder_extend function differs in that it acts only if its argument was
> believed to be a complete object (i.e. fresh from launder), and it only
> launders the remainder of the new complete object. To solve the virtual issue,
> do not let launder imply that its result is most-derived. Perhaps, let
> virtual dispatch implicitly do launder_extend.
>
>
>
> This scheme leaves launder open-ended so a polymorphic object or union
> can be used, yet still laundered further. A simple implementation can opt
> to treat launder_extend the same as launder.
>
>
>
> Example:
>
> struct discriminator { int value; };
>
> struct foo { discriminator d; int i; };
>
> struct bar { discriminator d; float f; };
>
> union foobar { foo a; bar b; };
>
> struct baz : discriminator { double x; };
>
> struct bad : discriminator { virtual ~ bad(); };
>
>
>
> void unpack( discriminator * p ) {
>
> std::launder( p ); // OK: now we can access p.
>
> int disc = p->value;
>
> if ( disc == 0 ) {
>
> auto & f = std::launder_extend< foo >( * p ); // OK: now we can
> access a foo.
>
> int q = p->value; // Load may be elided. Value is already in disc,
> equal to zero.
>
> auto & fb = std::launder_extend< foobar >( * f ); // OK: a further
> extension to a super-object.
>
> auto & f2 = std::launder_extend< foo >( * p ); // OK, no-op: p was
> already extended, becoming a subobject.
>
> auto & b = std::launder_extend< bar >( * p ); // UB: there’s
> already a different object there.
>
> auto & i = std::launder_extend< int >( * p ); // Library
> precondition violation: invalid object extension.
>
> } else if ( disc == 1 ) {
>
> baz & z = std::launder_extend< baz >( * p ); // OK, but
> implementation-dependent in theory.
>
> } else if ( disc == 2 ) {
>
> // Library precondition violation: no discriminator subobject
> shares an address with class bad.
>
> bad & x = std::launder_extend< bad >( * p );
>
> }
>
> }
>
>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
>

[Gabriel Dos Reis]

Fully agreed!


-------- Original message --------
From: Patrice Roy <***@gmail.com>
Date: 1/9/2016 5:36 PM (GMT-08:00)
To: WG21 UB study group <***@open-std.org>
Subject: Re: [ub] memcpy blessing a new object of statically unknown type

I know it is. I just hope it stays hidden deep where those who know about it are... essentially those who read this mailing list :)

2016-01-09 16:42 GMT-05:00 Gabriel Dos Reis <***@microsoft.com<mailto:***@microsoft.com>>:
My assumption has always been that std::launder is a zero-overhead abstraction (really a no-op at the machine level) whole sole purpose is to inform the static semantics elaborator about the lifetime and type of a given storage. I hope we aren’t considering anything more complicated than that.

-- Gaby

From: ub-***@open-std.org<mailto:ub-***@open-std.org> [mailto:ub-***@open-std.org<mailto:ub-***@open-std.org>] On Behalf Of David Krauss
Sent: Friday, January 8, 2016 10:36 PM
To: WG21 UB study group <***@open-std.org<mailto:***@open-std.org>>
Cc: ***@gmail.com<mailto:***@gmail.com>
Subject: Re: [ub] memcpy blessing a new object of statically unknown type


On 2016–01–09, at 11:35 AM, Hubert Tong <***@gmail.com<mailto:***@gmail.com>> wrote:

Yes, std::launder requires a static type; however, it does not limit the ability of memcpy to operate without knowing the type of the object being copied. The std::launder call is involved after the completion of memcpy to access the object that the memcpyinitialized.

Right, but in type erasure, the static type must be determined by inspecting the blob somehow. (Stashing a discriminating value elsewhere is one solution, but it’s more common and often more efficient to use an abstract base class or a discriminator inside a union.)

My library would launder the erasure_base subobject to retrieve its dispatch table, but then it’d be stuck. Dispatching to a derived class would lead back to UB.

One workaround could be to launder the same address repeatedly as the type becomes better resolved. For example, the call wrapper could launder a base class address, then perform an indirect call, then the callee could launder again to the derived class. For the common case of virtual dynamic dispatch, this sounds like it would incur UB before first line of the callee. My library doesn’t use virtual, but similar ones do. If only complete objects can be laundered, devirtualization could kick in… or launder could refuse to handle an abstract class at all. The workaround would also imply an excessive number of derived-type launder calls, which could compromise optimization by suggesting that bitwise manipulations are occurring when none are. Reloading the dispatch table pointer costs cycles.

Perhaps a second style of laundering could implement a compromise. First, auto &header = *launder(header_ptr) gets a fully-formed header object from a blob, and then auto &whole = launder_extend<whole_type>(header) revises the object identity to make header a subobject sharing its address with another already-fully-formed object of type whole. (For example, header could be a base, a union member, or an initial struct member.) The launder_extend function differs in that it acts only if its argument was believed to be a complete object (i.e. fresh from launder), and it only launders the remainder of the new complete object. To solve the virtual issue, do not let launder imply that its result is most-derived. Perhaps, let virtual dispatch implicitly do launder_extend.

This scheme leaves launder open-ended so a polymorphic object or union can be used, yet still laundered further. A simple implementation can opt to treat launder_extend the same as launder.

Example:
struct discriminator { int value; };
struct foo { discriminator d; int i; };
struct bar { discriminator d; float f; };
union foobar { foo a; bar b; };
struct baz : discriminator { double x; };
struct bad : discriminator { virtual ~ bad(); };

void unpack( discriminator * p ) {
std::launder( p ); // OK: now we can access p.
int disc = p->value;
if ( disc == 0 ) {
auto & f = std::launder_extend< foo >( * p ); // OK: now we can access a foo.
int q = p->value; // Load may be elided. Value is already in disc, equal to zero.
auto & fb = std::launder_extend< foobar >( * f ); // OK: a further extension to a super-object.
auto & f2 = std::launder_extend< foo >( * p ); // OK, no-op: p was already extended, becoming a subobject.
auto & b = std::launder_extend< bar >( * p ); // UB: there’s already a different object there.
auto & i = std::launder_extend< int >( * p ); // Library precondition violation: invalid object extension.
} else if ( disc == 1 ) {
baz & z = std::launder_extend< baz >( * p ); // OK, but implementation-dependent in theory.
} else if ( disc == 2 ) {
// Library precondition violation: no discriminator subobject shares an address with class bad.
bad & x = std::launder_extend< bad >( * p );
}
}

[David Krauss]

> On 2016–01–10, at 5:42 AM, Gabriel Dos Reis <***@microsoft.com> wrote:
>
> My assumption has always been that std::launder is a zero-overhead abstraction (really a no-op at the machine level) whole sole purpose is to inform the static semantics elaborator about the lifetime and type of a given storage. I hope we aren’t considering anything more complicated than that.

This perfectly expresses my expectation as well. But now it looks slightly different: The type remains malleable, and rather than marking an area of storage, launder provides a single blessed reference to an unbounded region. (If reinterpret_cast and downcasts are valid on the result of launder, what’s its connection to the type system? Why can’t it operate on void*?)

And, it’s not exactly zero-overhead, as its purpose is to disable an optimization which might be harmless. For example, take N4303 <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4303.html>’s motivating case in std::optional. Statically, optional’s contained object might have been created by placement-new, so the accessors always need to launder their return value. This would seem to make optional slower than “native” objects.

namespace std {
template< typename t >
t & optional< t >::operator * ()
{ return * launder( & this->storage.object ); }
}

void foo() {
int i;
std::optional< int > o = 3;

i = *o; // Must load *o from memory because address was laundered.
i = *o + *o; // Reload twice more. Will optimizers second-guess launder?
}

Likewise, MSVC appears to be remembering the value inside my wrapper’s NSDM embedded storage. It’s a beneficial optimization to the common case of a sequence of accesses, so it would be nice not to lose it.

It would also be easier-to-use if the assignment operator could inform the compiler when a lifetime begins, as opposed to the accessors providing notification that a new lifetime might have already begun. But, I suppose that’s not feasible, or we wouldn’t need launder in the first place.