Discussion:
[ub] type punning through congruent base class?
(too old to reply)
Fabio Fracassi
2014-01-06 09:26:58 UTC
Permalink
<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>
<div>Hello UB-Experts,</div>

<div>&nbsp;</div>

<div>as far as I can see the following (static_cast) is UB:</div>

<div>&nbsp;</div>

<div>struct B {</div>

<div>&nbsp; int i;</div>

<div>};</div>

<div>&nbsp;</div>

<div>struct D : B {</div>

<div>&nbsp; void foo() { /* access B::i */ }</div>

<div>};</div>

<div>&nbsp;</div>

<div>B b;</div>

<div>static_cast&lt;D&amp;&gt;(b).foo();</div>

<div>&nbsp;</div>

<div>because of [expr.static.cast] clause 11</div>

<div>&nbsp;</div>

<div>first question: is my assessment of the situation correct or is this use legal?</div>

<div>if it is not (legal): could we make it legal or would we run afoul of the aliasing rules?</div>

<div>&nbsp;</div>

<div>best regards</div>

<div>&nbsp;</div>

<div>Fabio</div>
</div></div></body></html>
Gabriel Dos Reis
2014-01-06 14:33:23 UTC
Permalink
"Fabio Fracassi" <***@gmx.net> writes:

| Hello UB-Experts,
|  
| as far as I can see the following (static_cast) is UB:
|  
| struct B {
|   int i;
| };
|  
| struct D : B {
|   void foo() { /* access B::i */ }
| };
|  
| B b;
| static_cast<D&>(b).foo();
|  
| because of [expr.static.cast] clause 11
|  
| first question: is my assessment of the situation correct or is this
| use legal?
| if it is not (legal): could we make it legal or would we run afoul of
| the aliasing rules?

Yes, it is undefined behavior -- you're referring to an inexistent
(sub)object of type D.

Why can't you create an object of type D if your program needs it? Why
is it a hardship?

-- Gaby
Fabio Fracassi
2014-01-06 15:34:56 UTC
Permalink
> From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
> Behalf Of Gabriel Dos Reis
> Sent: Montag, 6. Januar 2014 15:33
>
> "Fabio Fracassi" <***@gmx.net> writes:
>
> | Hello UB-Experts,
> |
> | as far as I can see the following (static_cast) is UB:
> |
> | struct B {
> | int i;
> | };
> |
> | struct D : B {
> | void foo() { /* access B::i */ }
> | };
> |
> | B b;
> | static_cast<D&>(b).foo();
> |
> | because of [expr.static.cast] clause 11
> |
> | first question: is my assessment of the situation correct or is this
> | use legal?
> | if it is not (legal): could we make it legal or would we run afoul of
> | the aliasing rules?
>
> Yes, it is undefined behavior -- you're referring to an inexistent
> (sub)object of type D.
>
> Why can't you create an object of type D if your program needs it? Why
> is it a hardship?
>

It would enable us to extend or change the interface of a class without
copying or moving the underlying object (think mixin without additional data).
Of course we could make our "extention members" (like foo() in the example above)
free functions which take a B&, but the grouping and the access to protected
members would simplify the design.

But use case for this example aside (for now), I'd still be curious if we were to
allow this, would it interfere with aliasing?

best regards

Fabio
Gabriel Dos Reis
2014-01-06 19:33:11 UTC
Permalink
| -----Original Message-----
| From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of
| Fabio Fracassi
| Sent: Monday, January 6, 2014 7:35 AM
| To: 'WG21 UB study group'
| Subject: Re: [ub] type punning through congruent base class?
|
| > From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
| > Behalf Of Gabriel Dos Reis
| > Sent: Montag, 6. Januar 2014 15:33
| >
| > "Fabio Fracassi" <***@gmx.net> writes:
| >
| > | Hello UB-Experts,
| > |
| > | as far as I can see the following (static_cast) is UB:
| > |
| > | struct B {
| > | int i;
| > | };
| > |
| > | struct D : B {
| > | void foo() { /* access B::i */ }
| > | };
| > |
| > | B b;
| > | static_cast<D&>(b).foo();
| > |
| > | because of [expr.static.cast] clause 11
| > |
| > | first question: is my assessment of the situation correct or is this
| > | use legal?
| > | if it is not (legal): could we make it legal or would we run afoul of
| > | the aliasing rules?
| >
| > Yes, it is undefined behavior -- you're referring to an inexistent
| > (sub)object of type D.
| >
| > Why can't you create an object of type D if your program needs it? Why
| > is it a hardship?
| >
|
| It would enable us to extend or change the interface of a class without
| copying or moving the underlying object (think mixin without additional data).
| Of course we could make our "extention members" (like foo() in the example above)
| free functions which take a B&, but the grouping and the access to protected
| members would simplify the design.
|
| But use case for this example aside (for now), I'd still be curious if we were to
| allow this, would it interfere with aliasing?
|
| best regards
|
| Fabio

If Jason's workaround is insufficient for your needs, then I believe you are looking for fundamentally altering the C++ object model. That is a language design and largely philosophical issue ("what is the computing model of the language?"), well beyond "undefined behavior". At a personal level, I wouldn't take a design (like this) just because I know how to solve it technically.

-- Gaby
Ville Voutilainen
2014-01-06 16:11:36 UTC
Permalink
On 6 January 2014 17:34, Fabio Fracassi <***@gmx.net> wrote:
> It would enable us to extend or change the interface of a class without
> copying or moving the underlying object (think mixin without additional data).

Why don't you just construct D with a B&?
Fabio Fracassi
2014-01-06 17:10:38 UTC
Permalink
> On 6 January 2014 17:34, Fabio Fracassi <***@gmx.net> wrote:
> > It would enable us to extend or change the interface of a class
> > without copying or moving the underlying object (think mixin without
> additional data).
>
> Why don't you just construct D with a B&?

Because I do not want a new object ... I want to use the existing one
(over which I have no (or do not want to take) control) with the extended
interface. Something along these lines:

struct fooable_vec : std::vector<int> {
void foo1();
void foo2() const;
};

auto make_fooable(std::vector& v) { return static_cast<fooable_vec&>(v); }

void do_stuff(fooable_vec& fv) {
fv.foo1();
fv.foo2();
}

auto v = std::vector<int>(1'000'000);
std::iota(v.begin(), v.end(),1);
...
do_stuff(make_fooable(v));
...

I do not want v to be copied or moved, and I do not want fooable_vec to
contain an indirection either.

best regards

Fabio
Jean-Marc Bourguet
2014-01-06 16:17:36 UTC
Permalink
On Mon, 6 Jan 2014 18:11:36 +0200, Ville Voutilainen
<***@gmail.com> wrote:
> On 6 January 2014 17:34, Fabio Fracassi <***@gmx.net> wrote:
>> It would enable us to extend or change the interface of a class without
>> copying or moving the underlying object (think mixin without additional data).
>
> Why don't you just construct D with a B&?

The only reason I can think is to access protected members, but I'll
argue that's
a good reason not to allow that.

Yours,

--
Jean-Marc
Fabio Fracassi
2013-12-19 17:52:27 UTC
Permalink
Hello UB-Experts,

as far as I can see the following (static_cast) is UB:

struct B {
int i;
};

struct D : B {
void foo() { /* access B::i */ }
};

B b;
static_cast<D&>(b).foo();

because of [expr.static.cast] clause 11

first question: is my assessment of the situation correct or is this use legal?
if it is not (legal): could we make it legal or would we run afoul of the aliasing rules?

best regards

Fabio

--
Fabio Fracassi | ***@think-cell.com<mailto:***@think-cell.com>
Software Engineer

________________________________
think-cell Sales GmbH & Co. KG http://www.think-cell.com<http://www.think-cell.com/>
Chausseestr. 8/E phone / fax +49 30 666473-10 / -19
10115 Berlin, Germany US phone / fax +1 800 891 8091 / +1 212 504 3039
Amtsgericht Berlin-Charlottenburg, HRA 44531 | European Union VAT Id DE815233792
General partner: think-cell Operations GmbH | Amtsgericht Berlin-Charlottenburg, HRB 129917
Directors: Dr. Markus Hannebauer, Dr. Arno Sch?dl
Jason Merrill
2014-01-06 18:22:24 UTC
Permalink
On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
> if it is not (legal): could we make it legal or would we run afoul of
> the aliasing rules?

The access is not allowed by the aliasing rules in 3.10. But it seems
that this would be:

struct B {
int i;
};

struct D {
B bmem;
void foo() { /* access bmem.i */ }
};

B b;
reinterpret_cast<D&>(b).foo();

because B is a non-static data member of D, and 9.2/19 guarantees that
the address of D::bmem is the same as the address of the D object.

Jason
Jeffrey Yasskin
2014-01-06 19:46:11 UTC
Permalink
On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com> wrote:
> On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
>> if it is not (legal): could we make it legal or would we run afoul of
>> the aliasing rules?
>
> The access is not allowed by the aliasing rules in 3.10. But it seems
> that this would be:
>
> struct B {
> int i;
> };
>
> struct D {
> B bmem;
> void foo() { /* access bmem.i */ }
> };
>
> B b;
> reinterpret_cast<D&>(b).foo();
>
> because B is a non-static data member of D, and 9.2/19 guarantees that
> the address of D::bmem is the same as the address of the D object.

The NaCl compiler broke code using this pattern, and we had to fix it
with https://codereview.chromium.org/10579030 and
https://codereview.chromium.org/11078014. We couldn't ever find a
clear place in the standard where it either allowed the pattern or
disallowed it, and I didn't get a chance to dig into the compiler to
figure out what transformation was breaking the code, so it's possible
it was a compiler bug instead of a code bug, but the pattern was
sketchy enough that we changed the code. 9.2/19 doesn't obviously
apply because no D object was ever created.

However, I don't think the above anecdote should prevent this group
from changing the rules to allow this use.

Jeffrey
Richard Smith
2014-01-06 23:45:13 UTC
Permalink
On Mon, Jan 6, 2014 at 3:44 PM, Richard Smith <***@google.com>wrote:

> On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com> wrote:
>
>> On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
>> > if it is not (legal): could we make it legal or would we run afoul of
>> > the aliasing rules?
>>
>> The access is not allowed by the aliasing rules in 3.10. But it seems
>> that this would be:
>>
>> struct B {
>> int i;
>> };
>>
>> struct D {
>> B bmem;
>> void foo() { /* access bmem.i */ }
>> };
>>
>> B b;
>> reinterpret_cast<D&>(b).foo();
>>
>> because B is a non-static data member of D, and 9.2/19 guarantees that
>> the address of D::bmem is the same as the address of the D object.
>
>
> How is that fundamentally different? 9.3.1/2 makes that UB too, if
> 'reinterpret_cast<D&>(b)' does not refer to an object of type 'b'.
>

... an object of type 'D'. Sorry!


> And within D::foo, the implicit this->bmem would have the same problem.
>
>
> If I might play devil's advocate for a moment...
>
> struct B { int i; };
> struct D : B {
> void foo();
> };
>
> B b;
>
> I claim this line starts the lifetime of an object of type D. Per
> [basic.life]p1, the lifetime of an object of type 'D' begins when storage
> with the proper alignment and size for type T is obtained (which "B b"
> happens to satisfy). The object does not have non-trivial initialization,
> so the second bullet does not apply.
>
> (This is the same argument that makes this valid:
>
> D *p = (D*)malloc(sizeof(D));
> p->foo();
>
> ... so any counterargument will need to explain why the two cases are
> fundamentally different.)
>
> Then:
>
> reinterpret_cast<D&>(b).foo();
>
> ... is valid, because the cast produces the same memory address, and that
> memory address contains an object of type 'D' (as claimed above).
>
Richard Smith
2014-01-06 23:44:29 UTC
Permalink
On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com> wrote:

> On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
> > if it is not (legal): could we make it legal or would we run afoul of
> > the aliasing rules?
>
> The access is not allowed by the aliasing rules in 3.10. But it seems
> that this would be:
>
> struct B {
> int i;
> };
>
> struct D {
> B bmem;
> void foo() { /* access bmem.i */ }
> };
>
> B b;
> reinterpret_cast<D&>(b).foo();
>
> because B is a non-static data member of D, and 9.2/19 guarantees that
> the address of D::bmem is the same as the address of the D object.


How is that fundamentally different? 9.3.1/2 makes that UB too, if
'reinterpret_cast<D&>(b)' does not refer to an object of type 'b'. And
within D::foo, the implicit this->bmem would have the same problem.


If I might play devil's advocate for a moment...

struct B { int i; };
struct D : B {
void foo();
};

B b;

I claim this line starts the lifetime of an object of type D. Per
[basic.life]p1, the lifetime of an object of type 'D' begins when storage
with the proper alignment and size for type T is obtained (which "B b"
happens to satisfy). The object does not have non-trivial initialization,
so the second bullet does not apply.

(This is the same argument that makes this valid:

D *p = (D*)malloc(sizeof(D));
p->foo();

... so any counterargument will need to explain why the two cases are
fundamentally different.)

Then:

reinterpret_cast<D&>(b).foo();

... is valid, because the cast produces the same memory address, and that
memory address contains an object of type 'D' (as claimed above).
Herb Sutter
2014-01-15 21:48:26 UTC
Permalink
Richard, I'm not sure I understand your position... Given the following complete program ...


struct B { int i; };
struct D : B { };

int main() {
B b; // line X
}

... are you actually saying that line X starts the lifetime of an object of type D? or just setting up a strawman? (Sorry, I really couldn't tell.)

If yes, then given the following complete program ...

struct C { long long i; };

int main() {
C c; // line Y
}

... are you saying that line Y could start the lifetime of an object of type D (which is not mentioned in the code), double, shared_ptr<widget>, or any other type than C, as long as the size of that other type is the same or less than sizeof(C)?

Herb



________________________________
From: ub-***@open-std.org <ub-***@open-std.org> on behalf of Richard Smith <***@google.com>
Sent: Monday, January 6, 2014 3:44 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com<mailto:***@redhat.com>> wrote:
On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
> if it is not (legal): could we make it legal or would we run afoul of
> the aliasing rules?

The access is not allowed by the aliasing rules in 3.10. But it seems
that this would be:

struct B {
int i;
};

struct D {
B bmem;
void foo() { /* access bmem.i */ }
};

B b;
reinterpret_cast<D&>(b).foo();

because B is a non-static data member of D, and 9.2/19 guarantees that
the address of D::bmem is the same as the address of the D object.

How is that fundamentally different? 9.3.1/2 makes that UB too, if 'reinterpret_cast<D&>(b)' does not refer to an object of type 'b'. And within D::foo, the implicit this->bmem would have the same problem.


If I might play devil's advocate for a moment...

struct B { int i; };
struct D : B {
void foo();
};

B b;

I claim this line starts the lifetime of an object of type D. Per [basic.life]p1, the lifetime of an object of type 'D' begins when storage with the proper alignment and size for type T is obtained (which "B b" happens to satisfy). The object does not have non-trivial initialization, so the second bullet does not apply.

(This is the same argument that makes this valid:

D *p = (D*)malloc(sizeof(D));
p->foo();

... so any counterargument will need to explain why the two cases are fundamentally different.)

Then:

reinterpret_cast<D&>(b).foo();

... is valid, because the cast produces the same memory address, and that memory address contains an object of type 'D' (as claimed above).
Richard Smith
2014-01-15 23:08:57 UTC
Permalink
On Wed, Jan 15, 2014 at 1:48 PM, Herb Sutter <***@microsoft.com> wrote:

> Richard, I'm not sure I understand your position... Given the following
> complete program ...
>
>
> struct B { int i; };
> struct D : B { };
>
> int main() {
> B b; // line X
> }
>
> ... are you actually saying that line X starts the lifetime of an object
> of type D? or just setting up a strawman? (Sorry, I really couldn't tell.)
>
> If yes, then given the following complete program ...
>
> struct C { long long i; };
>
> int main() {
> C c; // line Y
> }
>
> ... are you saying that line Y could start the lifetime of an object of
> type D (which is not mentioned in the code), double, shared_ptr<widget>, or
> any other type than C, as long as the size of that other type is the same
> or less than sizeof(C)?
>

I think my position is more nuanced. There are a set of cases that I think
people expect to have defined behavior, and the standard does not currently
distinguish those cases from your cases above, as far as I can tell -- if
using malloc to create an object "with trivial initialization" is
acceptable, then it appears that *any* way of producing the
appropriately-sized-and-aligned storage is acceptable.


I would expect that that we would have consensus that the lifetime of an
object of type D *should* start at some point in this code (and more
specifically that the code has defined behavior):

B *p = (B*)malloc(sizeof(B));
p->i = 0;

I would expect to also have consensus that the same is true here:

alignof(B) char buffer[sizeof(B)];
B *p = (B*)buffer;
p->i = 0;

(These expectations are based on the vast amount of existing code that
relies upon these behaviors, not on the wording of the standard.)

So, that's "what people probably expect". Next, "what the standard says".
Consider 3.8/1:

"The lifetime of an object of type T begins when:
— storage with the proper alignment and size for type T is obtained, and
— if the object has non-trivial initialization, its initialization is
complete."

This requires us to answer three questions: (1) is there an object of type
B in the above snippets, (2) when is storage for it obtained, and (3) does
it have non-trivial initialization? It seems that "yes, before the storage
is used as an object of type B, no" is a self-consistent set of answers,
and one that gives the program defined behavior. There are also sets of
answers that give the program undefined behavior, and the standard doesn't
give us direction in how we might pick a set of answers to these questions.

There seem to be two obvious ways forward: either (a) if we can pick
answers to these questions such that the program has defined behavior, then
the program has defined behavior, or (b) if we can pick answers to these
questions such that the program has undefined behavior, then the program
has undefined behavior.

If we want to align "what people probably expect" with "what the standard
says", it seems we need to either change the above rule, or accept
interpretation (a), under which the program above is valid, as is any other
program where 'buffer' obtains storage of the appropriate size and
alignment for an object of type D. (Option (a) also matches the behavior of
current optimizing compilers, as far as I'm aware.)


I've spent quite some time thinking about and discussing this problem and
related issues (such as, under what circumstances does a pointer point to
an object, when do two pointers alias, ...), and I personally think that
the best approach here is to embrace option (a) above: if there exist a
consistent set of choices of object lifetimes such that the program has
defined behavior, then the program has the behavior implied by that set. (I
have a sketch proof that such behavior is the same for *every* such
consistent set, aside from deviations caused by unspecified values and
other pre-existing sources of nondeterminism.) In essence, the implication
of this is that objects' lifetimes would start just in time to avoid
undefined behavior.

Put another way, yes, I personally think this code should have defined
behavior:

C c; // #1
static_assert(sizeof(C) >= sizeof(D) && alignof(C) >= alignof(D), "");
D *p = (D*)&c;
d->i = 0; // #2

... and the lifetime of a D object at address &c should start at some point
between lines #1 and #2. (Naturally, the lifetime of the C object ended
before this happened.) Moreover, this is something that plenty of existing
C++ code relies on.

*From:* ub-***@open-std.org <ub-***@open-std.org> on behalf of
> Richard Smith <***@google.com>
> *Sent:* Monday, January 6, 2014 3:44 PM
>
> *To:* WG21 UB study group
> *Subject:* Re: [ub] type punning through congruent base class?
>
> On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com> wrote:
>
>> On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
>> > if it is not (legal): could we make it legal or would we run afoul of
>> > the aliasing rules?
>>
>> The access is not allowed by the aliasing rules in 3.10. But it seems
>> that this would be:
>>
>> struct B {
>> int i;
>> };
>>
>> struct D {
>> B bmem;
>> void foo() { /* access bmem.i */ }
>> };
>>
>> B b;
>> reinterpret_cast<D&>(b).foo();
>>
>> because B is a non-static data member of D, and 9.2/19 guarantees that
>> the address of D::bmem is the same as the address of the D object.
>
>
> How is that fundamentally different? 9.3.1/2 makes that UB too, if
> 'reinterpret_cast<D&>(b)' does not refer to an object of type 'b'. And
> within D::foo, the implicit this->bmem would have the same problem.
>
>
> If I might play devil's advocate for a moment...
>
> struct B { int i; };
> struct D : B {
> void foo();
> };
>
> B b;
>
> I claim this line starts the lifetime of an object of type D. Per
> [basic.life]p1, the lifetime of an object of type 'D' begins when storage
> with the proper alignment and size for type T is obtained (which "B b"
> happens to satisfy). The object does not have non-trivial initialization,
> so the second bullet does not apply.
>
> (This is the same argument that makes this valid:
>
> D *p = (D*)malloc(sizeof(D));
> p->foo();
>
> ... so any counterargument will need to explain why the two cases are
> fundamentally different.)
>
> Then:
>
> reinterpret_cast<D&>(b).foo();
>
> ... is valid, because the cast produces the same memory address, and
> that memory address contains an object of type 'D' (as claimed above).
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
>
Gabriel Dos Reis
2014-01-15 23:40:41 UTC
Permalink
I suspect I would have to strongly disagree with the model put forward. Even if we attempted to make C++ look more like C, I don’t see anything that would make p->i = 0; starts the lifetime of a D object.

It has been the very foundation of C++ since day 1 that an object’s lifetime starts after its constructors ends and ends right before its destructor starts. Then, there was a little special case made for C. In my view, the way to resolve these questions isn’t to make C++ look more like C; it is to make the exceptions closer to the founding principle.

-- Gaby

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Richard Smith
Sent: Wednesday, January 15, 2014 3:09 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Wed, Jan 15, 2014 at 1:48 PM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:

Richard, I'm not sure I understand your position... Given the following complete program ...


struct B { int i; };
struct D : B { };

int main() {
B b; // line X
}

... are you actually saying that line X starts the lifetime of an object of type D? or just setting up a strawman? (Sorry, I really couldn't tell.)

If yes, then given the following complete program ...

struct C { long long i; };

int main() {
C c; // line Y
}

... are you saying that line Y could start the lifetime of an object of type D (which is not mentioned in the code), double, shared_ptr<widget>, or any other type than C, as long as the size of that other type is the same or less than sizeof(C)?

I think my position is more nuanced. There are a set of cases that I think people expect to have defined behavior, and the standard does not currently distinguish those cases from your cases above, as far as I can tell -- if using malloc to create an object "with trivial initialization" is acceptable, then it appears that *any* way of producing the appropriately-sized-and-aligned storage is acceptable.


I would expect that that we would have consensus that the lifetime of an object of type D *should* start at some point in this code (and more specifically that the code has defined behavior):

B *p = (B*)malloc(sizeof(B));
p->i = 0;

I would expect to also have consensus that the same is true here:

alignof(B) char buffer[sizeof(B)];
B *p = (B*)buffer;
p->i = 0;

(These expectations are based on the vast amount of existing code that relies upon these behaviors, not on the wording of the standard.)

So, that's "what people probably expect". Next, "what the standard says". Consider 3.8/1:

"The lifetime of an object of type T begins when:
— storage with the proper alignment and size for type T is obtained, and
— if the object has non-trivial initialization, its initialization is complete."

This requires us to answer three questions: (1) is there an object of type B in the above snippets, (2) when is storage for it obtained, and (3) does it have non-trivial initialization? It seems that "yes, before the storage is used as an object of type B, no" is a self-consistent set of answers, and one that gives the program defined behavior. There are also sets of answers that give the program undefined behavior, and the standard doesn't give us direction in how we might pick a set of answers to these questions.

There seem to be two obvious ways forward: either (a) if we can pick answers to these questions such that the program has defined behavior, then the program has defined behavior, or (b) if we can pick answers to these questions such that the program has undefined behavior, then the program has undefined behavior.

If we want to align "what people probably expect" with "what the standard says", it seems we need to either change the above rule, or accept interpretation (a), under which the program above is valid, as is any other program where 'buffer' obtains storage of the appropriate size and alignment for an object of type D. (Option (a) also matches the behavior of current optimizing compilers, as far as I'm aware.)


I've spent quite some time thinking about and discussing this problem and related issues (such as, under what circumstances does a pointer point to an object, when do two pointers alias, ...), and I personally think that the best approach here is to embrace option (a) above: if there exist a consistent set of choices of object lifetimes such that the program has defined behavior, then the program has the behavior implied by that set. (I have a sketch proof that such behavior is the same for *every* such consistent set, aside from deviations caused by unspecified values and other pre-existing sources of nondeterminism.) In essence, the implication of this is that objects' lifetimes would start just in time to avoid undefined behavior.

Put another way, yes, I personally think this code should have defined behavior:

C c; // #1
static_assert(sizeof(C) >= sizeof(D) && alignof(C) >= alignof(D), "");
D *p = (D*)&c;
d->i = 0; // #2

... and the lifetime of a D object at address &c should start at some point between lines #1 and #2. (Naturally, the lifetime of the C object ended before this happened.) Moreover, this is something that plenty of existing C++ code relies on.

From: ub-***@open-std.org<mailto:ub-***@open-std.org> <ub-***@open-std.org<mailto:ub-***@open-std.org>> on behalf of Richard Smith <***@google.com<mailto:***@google.com>>
Sent: Monday, January 6, 2014 3:44 PM

To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com<mailto:***@redhat.com>> wrote:
On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
> if it is not (legal): could we make it legal or would we run afoul of
> the aliasing rules?
The access is not allowed by the aliasing rules in 3.10. But it seems
that this would be:

struct B {
int i;
};

struct D {
B bmem;
void foo() { /* access bmem.i */ }
};

B b;
reinterpret_cast<D&>(b).foo();

because B is a non-static data member of D, and 9.2/19 guarantees that
the address of D::bmem is the same as the address of the D object.

How is that fundamentally different? 9.3.1/2 makes that UB too, if 'reinterpret_cast<D&>(b)' does not refer to an object of type 'b'. And within D::foo, the implicit this->bmem would have the same problem.


If I might play devil's advocate for a moment...

struct B { int i; };
struct D : B {
void foo();
};

B b;

I claim this line starts the lifetime of an object of type D. Per [basic.life]p1, the lifetime of an object of type 'D' begins when storage with the proper alignment and size for type T is obtained (which "B b" happens to satisfy). The object does not have non-trivial initialization, so the second bullet does not apply.

(This is the same argument that makes this valid:

D *p = (D*)malloc(sizeof(D));
p->foo();

... so any counterargument will need to explain why the two cases are fundamentally different.)

Then:

reinterpret_cast<D&>(b).foo();

... is valid, because the cast produces the same memory address, and that memory address contains an object of type 'D' (as claimed above).

_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub
Herb Sutter
2014-01-16 00:32:11 UTC
Permalink
I would expect that that we would have consensus that the lifetime of an object of type D *should* start at some point in this code (and more specifically that the code has defined behavior):

B *p = (B*)malloc(sizeof(B));
p->i = 0;


(blink) D is not even mentioned, so why would we expect that? Can you explain?

If the standard said the lifetime of a D object started somewhere here, I would wonder if there’s a defect in the lifetime wording. That would seem to violate sensible notions of type safety.

Herb


From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Richard Smith
Sent: Wednesday, January 15, 2014 3:09 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Wed, Jan 15, 2014 at 1:48 PM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:

Richard, I'm not sure I understand your position... Given the following complete program ...


struct B { int i; };
struct D : B { };

int main() {
B b; // line X
}

... are you actually saying that line X starts the lifetime of an object of type D? or just setting up a strawman? (Sorry, I really couldn't tell.)

If yes, then given the following complete program ...

struct C { long long i; };

int main() {
C c; // line Y
}

... are you saying that line Y could start the lifetime of an object of type D (which is not mentioned in the code), double, shared_ptr<widget>, or any other type than C, as long as the size of that other type is the same or less than sizeof(C)?

I think my position is more nuanced. There are a set of cases that I think people expect to have defined behavior, and the standard does not currently distinguish those cases from your cases above, as far as I can tell -- if using malloc to create an object "with trivial initialization" is acceptable, then it appears that *any* way of producing the appropriately-sized-and-aligned storage is acceptable.


I would expect that that we would have consensus that the lifetime of an object of type D *should* start at some point in this code (and more specifically that the code has defined behavior):

B *p = (B*)malloc(sizeof(B));
p->i = 0;

I would expect to also have consensus that the same is true here:

alignof(B) char buffer[sizeof(B)];
B *p = (B*)buffer;
p->i = 0;

(These expectations are based on the vast amount of existing code that relies upon these behaviors, not on the wording of the standard.)

So, that's "what people probably expect". Next, "what the standard says". Consider 3.8/1:

"The lifetime of an object of type T begins when:
— storage with the proper alignment and size for type T is obtained, and
— if the object has non-trivial initialization, its initialization is complete."

This requires us to answer three questions: (1) is there an object of type B in the above snippets, (2) when is storage for it obtained, and (3) does it have non-trivial initialization? It seems that "yes, before the storage is used as an object of type B, no" is a self-consistent set of answers, and one that gives the program defined behavior. There are also sets of answers that give the program undefined behavior, and the standard doesn't give us direction in how we might pick a set of answers to these questions.

There seem to be two obvious ways forward: either (a) if we can pick answers to these questions such that the program has defined behavior, then the program has defined behavior, or (b) if we can pick answers to these questions such that the program has undefined behavior, then the program has undefined behavior.

If we want to align "what people probably expect" with "what the standard says", it seems we need to either change the above rule, or accept interpretation (a), under which the program above is valid, as is any other program where 'buffer' obtains storage of the appropriate size and alignment for an object of type D. (Option (a) also matches the behavior of current optimizing compilers, as far as I'm aware.)


I've spent quite some time thinking about and discussing this problem and related issues (such as, under what circumstances does a pointer point to an object, when do two pointers alias, ...), and I personally think that the best approach here is to embrace option (a) above: if there exist a consistent set of choices of object lifetimes such that the program has defined behavior, then the program has the behavior implied by that set. (I have a sketch proof that such behavior is the same for *every* such consistent set, aside from deviations caused by unspecified values and other pre-existing sources of nondeterminism.) In essence, the implication of this is that objects' lifetimes would start just in time to avoid undefined behavior.

Put another way, yes, I personally think this code should have defined behavior:

C c; // #1
static_assert(sizeof(C) >= sizeof(D) && alignof(C) >= alignof(D), "");
D *p = (D*)&c;
d->i = 0; // #2

... and the lifetime of a D object at address &c should start at some point between lines #1 and #2. (Naturally, the lifetime of the C object ended before this happened.) Moreover, this is something that plenty of existing C++ code relies on.

From: ub-***@open-std.org<mailto:ub-***@open-std.org> <ub-***@open-std.org<mailto:ub-***@open-std.org>> on behalf of Richard Smith <***@google.com<mailto:***@google.com>>
Sent: Monday, January 6, 2014 3:44 PM

To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com<mailto:***@redhat.com>> wrote:
On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
> if it is not (legal): could we make it legal or would we run afoul of
> the aliasing rules?
The access is not allowed by the aliasing rules in 3.10. But it seems
that this would be:

struct B {
int i;
};

struct D {
B bmem;
void foo() { /* access bmem.i */ }
};

B b;
reinterpret_cast<D&>(b).foo();

because B is a non-static data member of D, and 9.2/19 guarantees that
the address of D::bmem is the same as the address of the D object.

How is that fundamentally different? 9.3.1/2 makes that UB too, if 'reinterpret_cast<D&>(b)' does not refer to an object of type 'b'. And within D::foo, the implicit this->bmem would have the same problem.


If I might play devil's advocate for a moment...

struct B { int i; };
struct D : B {
void foo();
};

B b;

I claim this line starts the lifetime of an object of type D. Per [basic.life]p1, the lifetime of an object of type 'D' begins when storage with the proper alignment and size for type T is obtained (which "B b" happens to satisfy). The object does not have non-trivial initialization, so the second bullet does not apply.

(This is the same argument that makes this valid:

D *p = (D*)malloc(sizeof(D));
p->foo();

... so any counterargument will need to explain why the two cases are fundamentally different.)

Then:

reinterpret_cast<D&>(b).foo();

... is valid, because the cast produces the same memory address, and that memory address contains an object of type 'D' (as claimed above).

_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub
Richard Smith
2014-01-16 00:51:55 UTC
Permalink
On Wed, Jan 15, 2014 at 4:32 PM, Herb Sutter <***@microsoft.com> wrote:

> I would expect that that we would have consensus that the lifetime of an
> object of type D *should* start at some point in this code (and more
> specifically that the code has defined behavior):
>
>
>
> B *p = (B*)malloc(sizeof(B));
>
> p->i = 0;
>
>
>
>
>
> (blink) D is not even mentioned, so why would we expect that? Can you
> explain?
>

Sorry, editing error, I meant 'B' here.


> If the standard said the lifetime of a D object started somewhere here, I
> would wonder if there’s a defect in the lifetime wording. That would seem
> to violate sensible notions of type safety.
>

Playing devil's advocate again: it would be entirely unobservable whether
an object of type 'B' or 'D' actually had its lifetime start here. Why
would you be surprised if you got a 'D' instead of a 'B'? Indeed, if later
I wrote:

D *q = (D*)p; // ha ha, it was a D object the whole time!
int n = q->i;

... why should that /not/ work? How is this any different from if I cast to
D* before I used the object as a B?


> *From:* ub-***@open-std.org [mailto:ub-***@open-std.org] *On
> Behalf Of *Richard Smith
> *Sent:* Wednesday, January 15, 2014 3:09 PM
>
> *To:* WG21 UB study group
> *Subject:* Re: [ub] type punning through congruent base class?
>
>
>
> On Wed, Jan 15, 2014 at 1:48 PM, Herb Sutter <***@microsoft.com>
> wrote:
>
> Richard, I'm not sure I understand your position... Given the following
> complete program ...
>
>
>
> struct B { int i; };
>
> struct D : B { };
>
>
>
> int main() {
>
> B b; // line X
>
> }
>
>
>
> ... are you actually saying that line X starts the lifetime of an object
> of type D? or just setting up a strawman? (Sorry, I really couldn't tell.)
>
>
>
> If yes, then given the following complete program ...
>
>
>
> struct C { long long i; };
>
>
>
> int main() {
>
> C c; // line Y
>
> }
>
>
>
> ... are you saying that line Y could start the lifetime of an object of
> type D (which is not mentioned in the code), double, shared_ptr<widget>, or
> any other type than C, as long as the size of that other type is the same
> or less than sizeof(C)?
>
>
>
> I think my position is more nuanced. There are a set of cases that I think
> people expect to have defined behavior, and the standard does not currently
> distinguish those cases from your cases above, as far as I can tell -- if
> using malloc to create an object "with trivial initialization" is
> acceptable, then it appears that *any* way of producing the
> appropriately-sized-and-aligned storage is acceptable.
>
>
>
>
>
> I would expect that that we would have consensus that the lifetime of an
> object of type D *should* start at some point in this code (and more
> specifically that the code has defined behavior):
>
>
>
> B *p = (B*)malloc(sizeof(B));
>
> p->i = 0;
>
>
>
> I would expect to also have consensus that the same is true here:
>
>
>
> alignof(B) char buffer[sizeof(B)];
>
> B *p = (B*)buffer;
>
> p->i = 0;
>
>
>
> (These expectations are based on the vast amount of existing code that
> relies upon these behaviors, not on the wording of the standard.)
>
>
>
> So, that's "what people probably expect". Next, "what the standard says".
> Consider 3.8/1:
>
>
>
> "The lifetime of an object of type T begins when:
>
> — storage with the proper alignment and size for type T is obtained, and
>
> — if the object has non-trivial initialization, its initialization is
> complete."
>
>
>
> This requires us to answer three questions: (1) is there an object of type
> B in the above snippets, (2) when is storage for it obtained, and (3) does
> it have non-trivial initialization? It seems that "yes, before the storage
> is used as an object of type B, no" is a self-consistent set of answers,
> and one that gives the program defined behavior. There are also sets of
> answers that give the program undefined behavior, and the standard doesn't
> give us direction in how we might pick a set of answers to these questions.
>
>
>
> There seem to be two obvious ways forward: either (a) if we can pick
> answers to these questions such that the program has defined behavior, then
> the program has defined behavior, or (b) if we can pick answers to these
> questions such that the program has undefined behavior, then the program
> has undefined behavior.
>
>
>
> If we want to align "what people probably expect" with "what the standard
> says", it seems we need to either change the above rule, or accept
> interpretation (a), under which the program above is valid, as is any other
> program where 'buffer' obtains storage of the appropriate size and
> alignment for an object of type D. (Option (a) also matches the behavior of
> current optimizing compilers, as far as I'm aware.)
>
>
>
>
>
> I've spent quite some time thinking about and discussing this problem and
> related issues (such as, under what circumstances does a pointer point to
> an object, when do two pointers alias, ...), and I personally think that
> the best approach here is to embrace option (a) above: if there exist a
> consistent set of choices of object lifetimes such that the program has
> defined behavior, then the program has the behavior implied by that set. (I
> have a sketch proof that such behavior is the same for *every* such
> consistent set, aside from deviations caused by unspecified values and
> other pre-existing sources of nondeterminism.) In essence, the implication
> of this is that objects' lifetimes would start just in time to avoid
> undefined behavior.
>
>
>
> Put another way, yes, I personally think this code should have defined
> behavior:
>
>
>
> C c; // #1
>
> static_assert(sizeof(C) >= sizeof(D) && alignof(C) >= alignof(D), "");
>
> D *p = (D*)&c;
>
> d->i = 0; // #2
>
>
>
> ... and the lifetime of a D object at address &c should start at some
> point between lines #1 and #2. (Naturally, the lifetime of the C object
> ended before this happened.) Moreover, this is something that plenty of
> existing C++ code relies on.
>
>
>
> *From:* ub-***@open-std.org <ub-***@open-std.org> on behalf of
> Richard Smith <***@google.com>
> *Sent:* Monday, January 6, 2014 3:44 PM
>
>
> *To:* WG21 UB study group
> *Subject:* Re: [ub] type punning through congruent base class?
>
>
>
> On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com> wrote:
>
> On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
> > if it is not (legal): could we make it legal or would we run afoul of
> > the aliasing rules?
>
> The access is not allowed by the aliasing rules in 3.10. But it seems
> that this would be:
>
>
> struct B {
> int i;
> };
>
> struct D {
>
> B bmem;
> void foo() { /* access bmem.i */ }
> };
>
> B b;
> reinterpret_cast<D&>(b).foo();
>
> because B is a non-static data member of D, and 9.2/19 guarantees that
> the address of D::bmem is the same as the address of the D object.
>
>
>
> How is that fundamentally different? 9.3.1/2 makes that UB too, if
> 'reinterpret_cast<D&>(b)' does not refer to an object of type 'b'. And
> within D::foo, the implicit this->bmem would have the same problem.
>
>
>
>
>
> If I might play devil's advocate for a moment...
>
>
>
> struct B { int i; };
>
> struct D : B {
>
> void foo();
>
> };
>
>
>
> B b;
>
>
>
> I claim this line starts the lifetime of an object of type D. Per
> [basic.life]p1, the lifetime of an object of type 'D' begins when storage
> with the proper alignment and size for type T is obtained (which "B b"
> happens to satisfy). The object does not have non-trivial initialization,
> so the second bullet does not apply.
>
>
>
> (This is the same argument that makes this valid:
>
>
>
> D *p = (D*)malloc(sizeof(D));
>
> p->foo();
>
>
>
> ... so any counterargument will need to explain why the two cases are
> fundamentally different.)
>
>
>
> Then:
>
>
>
> reinterpret_cast<D&>(b).foo();
>
>
>
> ... is valid, because the cast produces the same memory address, and that
> memory address contains an object of type 'D' (as claimed above).
>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
>
Gabriel Dos Reis
2014-01-16 18:04:06 UTC
Permalink
Casting a pointer from T* to U*, in and by itself, does not necessarily establish or end a lifetime.

-- Gaby

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Richard Smith
Sent: Wednesday, January 15, 2014 4:52 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Wed, Jan 15, 2014 at 4:32 PM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:
I would expect that that we would have consensus that the lifetime of an object of type D *should* start at some point in this code (and more specifically that the code has defined behavior):

B *p = (B*)malloc(sizeof(B));
p->i = 0;


(blink) D is not even mentioned, so why would we expect that? Can you explain?

Sorry, editing error, I meant 'B' here.

If the standard said the lifetime of a D object started somewhere here, I would wonder if there’s a defect in the lifetime wording. That would seem to violate sensible notions of type safety.

Playing devil's advocate again: it would be entirely unobservable whether an object of type 'B' or 'D' actually had its lifetime start here. Why would you be surprised if you got a 'D' instead of a 'B'? Indeed, if later I wrote:

D *q = (D*)p; // ha ha, it was a D object the whole time!
int n = q->i;

... why should that /not/ work? How is this any different from if I cast to D* before I used the object as a B?

From: ub-***@open-std.org<mailto:ub-***@open-std.org> [mailto:ub-***@open-std.org<mailto:ub-***@open-std.org>] On Behalf Of Richard Smith
Sent: Wednesday, January 15, 2014 3:09 PM

To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Wed, Jan 15, 2014 at 1:48 PM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:

Richard, I'm not sure I understand your position... Given the following complete program ...


struct B { int i; };
struct D : B { };

int main() {
B b; // line X
}

... are you actually saying that line X starts the lifetime of an object of type D? or just setting up a strawman? (Sorry, I really couldn't tell.)

If yes, then given the following complete program ...

struct C { long long i; };

int main() {
C c; // line Y
}

... are you saying that line Y could start the lifetime of an object of type D (which is not mentioned in the code), double, shared_ptr<widget>, or any other type than C, as long as the size of that other type is the same or less than sizeof(C)?

I think my position is more nuanced. There are a set of cases that I think people expect to have defined behavior, and the standard does not currently distinguish those cases from your cases above, as far as I can tell -- if using malloc to create an object "with trivial initialization" is acceptable, then it appears that *any* way of producing the appropriately-sized-and-aligned storage is acceptable.


I would expect that that we would have consensus that the lifetime of an object of type D *should* start at some point in this code (and more specifically that the code has defined behavior):

B *p = (B*)malloc(sizeof(B));
p->i = 0;

I would expect to also have consensus that the same is true here:

alignof(B) char buffer[sizeof(B)];
B *p = (B*)buffer;
p->i = 0;

(These expectations are based on the vast amount of existing code that relies upon these behaviors, not on the wording of the standard.)

So, that's "what people probably expect". Next, "what the standard says". Consider 3.8/1:

"The lifetime of an object of type T begins when:
— storage with the proper alignment and size for type T is obtained, and
— if the object has non-trivial initialization, its initialization is complete."

This requires us to answer three questions: (1) is there an object of type B in the above snippets, (2) when is storage for it obtained, and (3) does it have non-trivial initialization? It seems that "yes, before the storage is used as an object of type B, no" is a self-consistent set of answers, and one that gives the program defined behavior. There are also sets of answers that give the program undefined behavior, and the standard doesn't give us direction in how we might pick a set of answers to these questions.

There seem to be two obvious ways forward: either (a) if we can pick answers to these questions such that the program has defined behavior, then the program has defined behavior, or (b) if we can pick answers to these questions such that the program has undefined behavior, then the program has undefined behavior.

If we want to align "what people probably expect" with "what the standard says", it seems we need to either change the above rule, or accept interpretation (a), under which the program above is valid, as is any other program where 'buffer' obtains storage of the appropriate size and alignment for an object of type D. (Option (a) also matches the behavior of current optimizing compilers, as far as I'm aware.)


I've spent quite some time thinking about and discussing this problem and related issues (such as, under what circumstances does a pointer point to an object, when do two pointers alias, ...), and I personally think that the best approach here is to embrace option (a) above: if there exist a consistent set of choices of object lifetimes such that the program has defined behavior, then the program has the behavior implied by that set. (I have a sketch proof that such behavior is the same for *every* such consistent set, aside from deviations caused by unspecified values and other pre-existing sources of nondeterminism.) In essence, the implication of this is that objects' lifetimes would start just in time to avoid undefined behavior.

Put another way, yes, I personally think this code should have defined behavior:

C c; // #1
static_assert(sizeof(C) >= sizeof(D) && alignof(C) >= alignof(D), "");
D *p = (D*)&c;
d->i = 0; // #2

... and the lifetime of a D object at address &c should start at some point between lines #1 and #2. (Naturally, the lifetime of the C object ended before this happened.) Moreover, this is something that plenty of existing C++ code relies on.

From: ub-***@open-std.org<mailto:ub-***@open-std.org> <ub-***@open-std.org<mailto:ub-***@open-std.org>> on behalf of Richard Smith <***@google.com<mailto:***@google.com>>
Sent: Monday, January 6, 2014 3:44 PM

To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com<mailto:***@redhat.com>> wrote:
On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
> if it is not (legal): could we make it legal or would we run afoul of
> the aliasing rules?
The access is not allowed by the aliasing rules in 3.10. But it seems
that this would be:

struct B {
int i;
};

struct D {
B bmem;
void foo() { /* access bmem.i */ }
};

B b;
reinterpret_cast<D&>(b).foo();

because B is a non-static data member of D, and 9.2/19 guarantees that
the address of D::bmem is the same as the address of the D object.

How is that fundamentally different? 9.3.1/2 makes that UB too, if 'reinterpret_cast<D&>(b)' does not refer to an object of type 'b'. And within D::foo, the implicit this->bmem would have the same problem.


If I might play devil's advocate for a moment...

struct B { int i; };
struct D : B {
void foo();
};

B b;

I claim this line starts the lifetime of an object of type D. Per [basic.life]p1, the lifetime of an object of type 'D' begins when storage with the proper alignment and size for type T is obtained (which "B b" happens to satisfy). The object does not have non-trivial initialization, so the second bullet does not apply.

(This is the same argument that makes this valid:

D *p = (D*)malloc(sizeof(D));
p->foo();

... so any counterargument will need to explain why the two cases are fundamentally different.)

Then:

reinterpret_cast<D&>(b).foo();

... is valid, because the cast produces the same memory address, and that memory address contains an object of type 'D' (as claimed above).

_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub

_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub
Matt Austern
2014-01-16 18:29:46 UTC
Permalink
On Thu, Jan 16, 2014 at 10:04 AM, Gabriel Dos Reis <***@microsoft.com>wrote:

> Casting a pointer from T* to U*, in and by itself, does not necessarily
> establish or end a lifetime.
>

Indeed. Richard's messages make me a little uneasy about just what does
begin the lifetime of a POD object, though. I'm thinking of:
struct B { int x; }; // 1
void* p = malloc(sizeof(B)); // 2
B* pb = static_cast<B*>(p); //3
pb->x = 17; // 4

I take it as obvious that the lifetime of an object of type B has begun
somewhere in this code snippet. Now the question is: which specific line
begins that lifetime? As you say, casting a pointer doesn't begin or end a
lifetime; I think we can rule out line 3 as the one that begins the
lifetime of that B object. Line 1 doesn't look too promising either.

A literal reading of the standard ([basic.life]/1) suggests that it's line
2:
"The lifetime of an object of type T begins when:
— storage with the proper alignment and size for type T is obtained, and
— if the object has non-trivial initialization, its initialization is
complete."

In line 2 we have obtained storage with the proper alignment and size for
type B, and the second bullet item doesn't apply since B has no non-trivial
initialization. None of the other lines are relevant to the beginning of
the lifetime.

But that's a little disturbing too. If we've got:
struct X { int x; }; // 1'
struct Y { int y; }; // 2'
void* p = malloc(sizeof(X)); // 3'
then does line 3' mean that the lifetime of some object has begun? Again a
literal reading of that same standard text suggests yes. We have obtained
storage with the proper alignment and size for type X, so the lifetime of
an object of type X has begun. Exactly the same argument means that the
lifetime of an object of type Y has begun, and similarly for every other
type, mentioned or not, whose size is equal to sizeof(X).

That seems like a crazy conclusion, but I don't see how to reject it
without also rejecting the idea that the lifetime of an object of type B
has been established somewhere in my first code snippet. What all of this
suggests to me is that the concept of object lifetime needs a little more
thought for types that don't have initialization.

--Matt
Gabriel Dos Reis
2014-01-16 19:04:52 UTC
Permalink
| -----Original Message-----
| From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
| Behalf Of Matt Austern
| Sent: Thursday, January 16, 2014 10:30 AM
| To: WG21 UB study group
| Subject: Re: [ub] type punning through congruent base class?
|
| On Thu, Jan 16, 2014 at 10:04 AM, Gabriel Dos Reis <***@microsoft.com
| <mailto:***@microsoft.com> > wrote:
|
|
| > Casting a pointer from T* to U*, in and by itself, does not necessarily
| > establish or end a lifetime.
|
|
| Indeed. Richard's messages make me a little uneasy about just what does
| begin the lifetime of a POD object, though. I'm thinking of:
| struct B { int x; }; // 1
| void* p = malloc(sizeof(B)); // 2
| B* pb = static_cast<B*>(p); //3
| pb->x = 17; // 4
|
| I take it as obvious that the lifetime of an object of type B has begun
| somewhere in this code snippet. Now the question is: which specific line
| begins that lifetime? As you say, casting a pointer doesn't begin or end a
| lifetime; I think we can rule out line 3 as the one that begins the lifetime of
| that B object. Line 1 doesn't look too promising either.

Well, in fact I don't take it obvious that the lifetime of an object has even begun!
I don't even see that or object has been constructed or initialized.

| A literal reading of the standard ([basic.life]/1) suggests that it's line 2:
| "The lifetime of an object of type T begins when:
| — storage with the proper alignment and size for type T is obtained, and
| — if the object has non-trivial initialization, its initialization is complete."
|
| In line 2 we have obtained storage with the proper alignment and size for type
| B, and the second bullet item doesn't apply since B has no non-trivial
| initialization. None of the other lines are relevant to the beginning of the
| lifetime.
|
| But that's a little disturbing too. If we've got:
| struct X { int x; }; // 1'
| struct Y { int y; }; // 2'
| void* p = malloc(sizeof(X)); // 3'
| then does line 3' mean that the lifetime of some object has begun? Again a
| literal reading of that same standard text suggests yes. We have obtained
| storage with the proper alignment and size for type X, so the lifetime of an
| object of type X has begun. Exactly the same argument means that the
| lifetime of an object of type Y has begun, and similarly for every other type,
| mentioned or not, whose size is equal to sizeof(X).
|
| That seems like a crazy conclusion, but I don't see how to reject it without also
| rejecting the idea that the lifetime of an object of type B has been established
| somewhere in my first code snippet. What all of this suggests to me is that the
| concept of object lifetime needs a little more thought for types that don't have
| initialization.
|
| --Matt

I suspect part of the disturbance or incomfort is that we might be reading too much into the paragraph about obtaining storage or appropriate alignment and size.
As I said above, I don't see an object of type B being established -- this isn't argumentation for the sake of argumentation.

I think we should go back to the general principle - which works well for general objects - that an object lifetime starts right after its construct finishes and ends right before its destructor starts. I don't think we want to compromise that.

Of course, C-style data types (I'm focusing on what we used to call POD structs) have constructors and destructors - even when we call deem them trivial.
Declared objects should follow the same lifetime rule as any other object class type. That takes care of many of Richard's previous examples. It is even close to C's -- if we feel that we need to make a case for C-compatibility. What remains is the case of undeclared objects, that is an object with dynamic storage: (a) either a new-expression fires up the constructor (however trivial); or (b) a use memcpy or memmove has been applied to copy an object representation into that storage from a living object.

Note that, even when in C's model, it is not clear that memberwise assignment to fields automatically establishes lifetime for "the" complete object -- if there is one.

Note 1: at this point, I'm not considering the case where the storage of an object has been reused because I am assuming it isn't under dispute that it ends the lifetime of the object (if any) that previously occupied that storage. And that this is the same for all complete object types.

Note 2: I am not suggesting to adopt C's model - I'm merely mentioning
Herb Sutter
2014-01-16 19:44:25 UTC
Permalink
> | struct B { int x; }; // 1
> | void* p = malloc(sizeof(B)); // 2
> | B* pb = static_cast<B*>(p); //3
> | pb->x = 17; // 4
> |
> | I take it as obvious that the lifetime of an object of type B has
> | begun somewhere in this code snippet. Now the question is: which
> | specific line begins that lifetime? As you say, casting a pointer
> | doesn't begin or end a lifetime; I think we can rule out line 3 as the
> | one that begins the lifetime of that B object. Line 1 doesn't look too
> promising either.
>
> Well, in fact I don't take it obvious that the lifetime of an object has even
> begun!
> I don't even see that or object has been constructed or initialized.

Agreed. I would expect line 4 to be at least unspecified behavior and probably undefined behavior.


> I think we should go back to the general principle - which works well for
> general objects - that an object lifetime starts right after its construct finishes
> and ends right before its destructor starts. I don't think we want to
> compromise that.

Exactly, this is essential. It seems the confusion is about whether we say this clearly for trivially-constructible types. If we don't,
James Dennett
2014-01-16 20:01:26 UTC
Permalink
On Thu, Jan 16, 2014 at 11:44 AM, Herb Sutter <***@microsoft.com> wrote:
>> | struct B { int x; }; // 1
>> | void* p = malloc(sizeof(B)); // 2
>> | B* pb = static_cast<B*>(p); //3
>> | pb->x = 17; // 4
>> |
>> | I take it as obvious that the lifetime of an object of type B has
>> | begun somewhere in this code snippet. Now the question is: which
>> | specific line begins that lifetime? As you say, casting a pointer
>> | doesn't begin or end a lifetime; I think we can rule out line 3 as the
>> | one that begins the lifetime of that B object. Line 1 doesn't look too
>> promising either.
>>
>> Well, in fact I don't take it obvious that the lifetime of an object has even
>> begun!
>> I don't even see that or object has been constructed or initialized.
>
> Agreed. I would expect line 4 to be at least unspecified behavior and probably undefined behavior.

I feel like I must have missed part of the conversation.

We want to utterly break compatibility with C (and C-like C++ code) here?

struct B { int x };
struct B* p = (B*) malloc(sizeof(B));
p->x = 17;

This (modulo the cast) has been how C has handled dynamic allocation
of structs approximately forever. There are no constructors in C,
structs don't get initialized, their fields just get assigned to.
When compiled as C++, no constructor is called here either. And we've
allowed that.

The issue we're dealing with here is that C allowed writing to a
suitably-aligned chunk of memory (from malloc(), at least) as if it
were an object. There is no process whereby the storage _becomes_ an
object. You just use it.

We can't adopt a stricter model without breaking existing code. I'm
fine with breaking existing code, except that I suspect we'd break so
much that it would never all get fixed, and it's a silent breaking
change in many cases.

-- James
Herb Sutter
2014-01-16 20:17:32 UTC
Permalink
> >> | struct B { int x; }; // 1
> >> | void* p = malloc(sizeof(B)); // 2
> >> | B* pb = static_cast<B*>(p); //3
> >> | pb->x = 17; // 4
> >> |
> >> | I take it as obvious that the lifetime of an object of type B has
> >> | begun somewhere in this code snippet. Now the question is: which
> >> | specific line begins that lifetime? As you say, casting a pointer
> >> | doesn't begin or end a lifetime; I think we can rule out line 3 as
> >> | the one that begins the lifetime of that B object. Line 1 doesn't
> >> | look too
> >> promising either.
> >>
> >> Well, in fact I don't take it obvious that the lifetime of an object
> >> has even begun!
> >> I don't even see that or object has been constructed or initialized.
> >
> > Agreed. I would expect line 4 to be at least unspecified behavior and
> > probably undefined behavior.

OK, let me back this off to just "I would expect that in this code no lifetime of any object has begun."

> I feel like I must have missed part of the conversation.
>
> We want to utterly break compatibility with C (and C-like C++ code) here?
>
> struct B { int x };
> struct B* p = (B*) malloc(sizeof(B));
> p->x = 17;

FWIW, std::vector does similarly internally, except it actually calls constructor in-place. I'm fine with that.

What I'm not fine with is the claim that the "lifetime" of anything began here. That doesn't make sense, and it matters to me because lifetime is fundamental.

As I pointed out in other email that might have been delayed by moderation, if in the above code any lifetime began, then we have a contradiction in the standard because we would be in the untenable position that two objects can have the same address:

struct B { int x };
void* p = malloc(sizeof(B));

B* pb = (B*)p;
pb->x = 17;

short* ps = (short*)p
*ps = 17;

None of this code can be viewed as starting a lifetime. Otherwise, proof by contradiction (meaning contradiction in the standard): B and short are both trivially-constructible. If any of these lines start a lifetime of either a B or a short on the grounds that they are trivially-constructible, then this code must start the life of *both* a B and a short, and then *pb and *ps have the same address, which is a contradiction. Therefore none of these lines start a lifetime, QED.

Am I missing something?

Herb
David Vandevoorde
2014-01-16 20:24:17 UTC
Permalink
On Jan 16, 2014, at 3:17 PM, Herb Sutter <***@microsoft.com> wrote:
[...]
> As I pointed out in other email that might have been delayed by moderation, if in the above code any lifetime began, then we have a contradiction in the standard because we would be in the untenable position that two objects can have the same address:
>
> struct B { int x };
> void* p = malloc(sizeof(B));
>
> B* pb = (B*)p;
> pb->x = 17;
>
> short* ps = (short*)p
> *ps = 17;
>
> None of this code can be viewed as starting a lifetime. Otherwise, proof by contradiction (meaning contradiction in the standard): B and short are both trivially-constructible. If any of these lines start a lifetime of either a B or a short on the grounds that they are trivially-constructible, then this code must start the life of *both* a B and a short, and then *pb and *ps have the same address, which is a contradiction. Therefore none of these lines start a lifetime, QED.
>
> Am I missing something?

I think so: 3.8/1 also says that the lifetime of an object ends if its storage is reused. So when you write to *ps, you've terminated the lifetime of *pb.

Daveed
James Dennett
2014-01-16 20:46:57 UTC
Permalink
On Thu, Jan 16, 2014 at 12:24 PM, David Vandevoorde <***@edg.com> wrote:
>
> On Jan 16, 2014, at 3:17 PM, Herb Sutter <***@microsoft.com> wrote:
> [...]
>> As I pointed out in other email that might have been delayed by moderation, if in the above code any lifetime began, then we have a contradiction in the standard because we would be in the untenable position that two objects can have the same address:
>>
>> struct B { int x };
>> void* p = malloc(sizeof(B));
>>
>> B* pb = (B*)p;
>> pb->x = 17;
>>
>> short* ps = (short*)p
>> *ps = 17;
>>
>> None of this code can be viewed as starting a lifetime. Otherwise, proof by contradiction (meaning contradiction in the standard): B and short are both trivially-constructible. If any of these lines start a lifetime of either a B or a short on the grounds that they are trivially-constructible, then this code must start the life of *both* a B and a short, and then *pb and *ps have the same address, which is a contradiction. Therefore none of these lines start a lifetime, QED.
>> Am I missing something?
>
> I think so: 3.8/1 also says that the lifetime of an object ends if its storage is reused. So when you write to *ps, you've terminated the lifetime of *pb.

If the lifetime started when the storage was allocated, then there's
still a window up to the write to *ps where the lifetimes of both
objects have started and not ended. (And we can assume that the
lifetime of objects of _every_ type compatible with the size and
alignment of the storage started at the same address at the same time,
because the standard kind of says so.)

C tries to have wording around "effective types", which change the
types based on assignments and memcpy/memmove: "If a value is stored
into an object having no declared type through an lvalue having a type
that is not a character type, then the type of the lvalue becomes the
effective type of the object for that access and for subsequent
accesses that do not modify the stored value. If a value is copied
into an object having no declared type using memcpy or memmove, or is
copied as an array of character type, then the effective type of the
modified object for that access and for subsequent accesses that do
not modify the value is the effective type of the object from which
the value is copied, if it has one. For all other accesses to an
object having no declared type, the effective type of the object is
simply the type of the lvalue used for the access."

In C++ we'd like to say that _assignment_ must be to an object that
already exists, which pushes us towards the only other possible
candidate for starting the lifetime, which is the allocation. That
doesn't work out well either.

Trying to merge the model of C (effective type established by
declarations or by assignments or memcpy/memmove, no explicit
initialization/destruction step) with the newer model of C++ (lifetime
controlled by initialization/destruction) is decidedly non-trivial.

-- James
Gabriel Dos Reis
2014-01-16 22:10:06 UTC
Permalink
[James Dennett]

| In C++ we'd like to say that _assignment_ must be to an object that
| already exists,

Bingo! Except that it is not a wish, it is something we have always done since day 1 :-)

And we have always carefully made a distinction between assignment and other things that may start lifetime.

| which pushes us towards the only other possible
| candidate for starting the lifetime, which is the allocation. That
| doesn't work out well either.

Again, I think we may be reading too much into the 'allocation' blurb.

|
| Trying to merge the model of C (effective type established by
| declarations or by assignments or memcpy/memmove, no explicit
| initialization/destruction step) with the newer model of C++ (lifetime
| controlled by initialization/destruction) is decidedly non-trivial.

I agree with that -- but it isn't impossible.

It is fundamental and worth doing.
There are reasons to conceive that future C++ implementations would be more "checking" than past implementations were -- the movement has already begun, with several implementations now offering some form of "undefined behavior sanitizers". This in turn was in response to more and more sophisticated translators that start exploiting just about every bit of the language spec. There are surely more reasons; I'm just scratching the surface here.

-- Gaby
Jens Maurer
2014-01-16 21:48:59 UTC
Permalink
On 01/16/2014 09:17 PM, Herb Sutter wrote:
>>>> | struct B { int x; }; // 1
>>>> | void* p = malloc(sizeof(B)); // 2
>>>> | B* pb = static_cast<B*>(p); //3
>>>> | pb->x = 17; // 4
>>>> |
>>>> | I take it as obvious that the lifetime of an object of type B has
>>>> | begun somewhere in this code snippet.

>>>> Well, in fact I don't take it obvious that the lifetime of an object
>>>> has even begun!
>>>> I don't even see that or object has been constructed or initialized.
>>>
>>> Agreed. I would expect line 4 to be at least unspecified behavior and
>>> probably undefined behavior.
>
> OK, let me back this off to just "I would expect that in this code no lifetime of any object has begun."

So, a subsequent read of "pb->x" would then be undefined behavior
according to 3.8p5 bullet 2?

This seems to break C compatibility, since the code above (after replacing
the static_cast) certainly works as expected in C.

I have no objections to someone rewriting 3.8 basic.life to suit feelings
about the intuitive meaning of "lifetime", but let's please have a holistic
approach in a paper.

Jens
Gabriel Dos Reis
2014-01-16 22:14:01 UTC
Permalink
| >>> Agreed. I would expect line 4 to be at least unspecified behavior and
| >>> probably undefined behavior.
| >
| > OK, let me back this off to just "I would expect that in this code no lifetime
| of any object has begun."
|
| So, a subsequent read of "pb->x" would then be undefined behavior
| according to 3.8p5 bullet 2?

In the C world, the storage at "&pb->x" has effective type "int".

| This seems to break C compatibility, since the code above (after replacing
| the static_cast) certainly works as expected in C.
|
| I have no objections to someone rewriting 3.8 basic.life to suit feelings
| about the intuitive meaning of "lifetime", but let's please have a holistic
| approach in a paper.
Gabriel Dos Reis
2014-01-16 20:53:54 UTC
Permalink
| -----Original Message-----
| From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
| Behalf Of James Dennett
| Sent: Thursday, January 16, 2014 12:01 PM
| To: WG21 UB study group
| Cc: Bjarne Stroustrup
| Subject: Re: [ub] type punning through congruent base class?
|
| On Thu, Jan 16, 2014 at 11:44 AM, Herb Sutter <***@microsoft.com>
| wrote:
| >> | struct B { int x; }; // 1
| >> | void* p = malloc(sizeof(B)); // 2
| >> | B* pb = static_cast<B*>(p); //3
| >> | pb->x = 17; // 4
| >> |
| >> | I take it as obvious that the lifetime of an object of type B has
| >> | begun somewhere in this code snippet. Now the question is: which
| >> | specific line begins that lifetime? As you say, casting a pointer
| >> | doesn't begin or end a lifetime; I think we can rule out line 3 as the
| >> | one that begins the lifetime of that B object. Line 1 doesn't look too
| >> promising either.
| >>
| >> Well, in fact I don't take it obvious that the lifetime of an object has even
| >> begun!
| >> I don't even see that or object has been constructed or initialized.
| >
| > Agreed. I would expect line 4 to be at least unspecified behavior and
| probably undefined behavior.
|
| I feel like I must have missed part of the conversation.
|
| We want to utterly break compatibility with C (and C-like C++ code) here?

This is the first time you are mentioning and I feel like I was following the discussion, yet I've read anything to that effect.

I think it is also important to look at what the C standards actually say on this matter (e.g. object lifetime) before concluding that there is an effort to utterly break compatibility with C.

Both languages have a notion of 'lifetime' that they more or less define accurately or completely. It is not clear that C could just adopt C++'s notion and be OK or that C++ could just adopt C's and be OK. And in fact, neither did. What they have done is to translate some perception of what they want into their own models. So, when we make claim that there is attempt to break C compatibility, we might want to be more precise about the part of the translation we are worried about.

C's object model is based on a certain notion of lifetime (with some *expected* guarantees, that isn't obvious are held) and notion "effective type". C++ doesn't use effective type; rather it uses the notion of 'dynamic type' which is tightly coupled with the notion of constructor (which C does not have.)

| struct B { int x };
| struct B* p = (B*) malloc(sizeof(B));
| p->x = 17;
|
| This (modulo the cast) has been how C has handled dynamic allocation
| of structs approximately forever. There are no constructors in C,
| structs don't get initialized, their fields just get assigned to.

Could you walk us through the C standards and explain what you believe this program fragment is supposed to do and what actually is at the address pointed by p?
I think that will be a very illuminating exercise -- and would possibly help clear some confusions.

| When compiled as C++, no constructor is called here either. And we've
| allowed that.

We have to allow something and define something. What I feel strongly about is the current interpretation that once a blob of memory is obtained with a given alignment and size, then an object's life time has begun. I don't think that is OK.

Even if we take the C model, it is not clear what is there. We can infer that after the statement 'p->x = 17;' the storage at address '&p-x' has effective type 'int', but that does not say much about the rest. I -suspect- C is OK with that because fundamentally, its object model is structural.


| The issue we're dealing with here is that C allowed writing to a
| suitably-aligned chunk of memory (from malloc(), at least) as if it
| were an object. There is no process whereby the storage _becomes_ an
| object. You just use it.

really? Which parts of the C standards support that?


| We can't adopt a stricter model without breaking existing code. I'm
| fine with breaking existing code, except that I suspect we'd break so
| much that it would never all get fixed, and it's a silent breaking
| change in many cases.

I think we may be getting ahead of ourselves a little bit here.

-- Gaby
Kazutoshi Satoda
2014-01-17 03:57:23 UTC
Permalink
On 2014/01/17 5:53 +0900, Gabriel Dos Reis wrote:
> | struct B { int x };
> | struct B* p = (B*) malloc(sizeof(B));
> | p->x = 17;
> |
> | This (modulo the cast) has been how C has handled dynamic allocation
> | of structs approximately forever. There are no constructors in C,
> | structs don't get initialized, their fields just get assigned to.
>
> Could you walk us through the C standards and explain what you believe this program fragment is supposed to do and what actually is at the address pointed by p?
> I think that will be a very illuminating exercise -- and would possibly help clear some confusions.

Let me try the exercise. I'm referring WG14 N1570.
http://www.open-std.org/jtc1/sc22/wg14/www/standards.html


p points an object (a region of data storage, 3.15) which is allocated
by malloc(). The object has allocated storage duration (6.2.4) and its
lifetime starts at the allocation in malloc(), and ends at deallocation
in free() (7.22.3).

"p->x" yields an lvalue of type int which designates an object at
address pointed by ((char*)p + offsetof(B, x)). There is no rule about
the effective type of the object pointed by p to evaluate the "->"
expression. (6.5.2.3)

"p->x = 17" makes the effective type of the object between &p->x and
(&p->x + 1) becomes int (6.5 p6), and stores value 17 (a value
representation of int which represents 17) into that object (6.5.16).

(Note)
In C, there is no such a thing like "an object of type int" unlike in
C++ where an object has a type and the type is determined when the
object is created. In C, "object" is a merely a region of data storage,
and may be labeled by an effective type which is determined by ongoing
or previous access to the object at a time.

If "struct B x = *p" follows the above code, the access (lvalue
conversion, 6.3.2.1 p2) on *p, which is allowed by the aliasing rule
(6.5 p7, bullet 5 "an aggregate or union ..."), changes the effective
type of the object at p.


I hope this is correct and does help.

--
k_satoda
Kazutoshi Satoda
2014-01-17 04:21:55 UTC
Permalink
On 2014/01/17 12:57 +0900, Kazutoshi Satoda wrote:
> (Note)
> In C, there is no such a thing like "an object of type int" unlike in
> C++ where an object has a type and the type is determined when the
> object is created. In C, "object" is a merely a region of data storage,
> and may be labeled by an effective type which is determined by ongoing
> or previous access to the object at a time.
>
> If "struct B x = *p" follows the above code, the access (lvalue
> conversion, 6.3.2.1 p2) on *p, which is allowed by the aliasing rule
> (6.5 p7, bullet 5 "an aggregate or union ..."), changes the effective
> type of the object at p.

Uh, sorry. I was wrong here. Non-modifying access to an object doesn't
change the effective type of subsequent access to the object. And the
aliasing rule does not apply here.

Let me correct the last paragraph.

If "struct B x = {0}; *p = x;" follows the above code, the write on *p
changes the effective type of the object at p (from int to B) for
subsequent non-modifying access of that object. (6.5 p6, about an
object having no declared type)

--
k_satoda
Gabriel Dos Reis
2014-01-17 05:04:43 UTC
Permalink
Kazutoshi Satoda <***@f2.dion.ne.jp> writes:

| On 2014/01/17 12:57 +0900, Kazutoshi Satoda wrote:
| > (Note)
| > In C, there is no such a thing like "an object of type int" unlike in
| > C++ where an object has a type and the type is determined when the
| > object is created. In C, "object" is a merely a region of data storage,
| > and may be labeled by an effective type which is determined by ongoing
| > or previous access to the object at a time.
| >
| > If "struct B x = *p" follows the above code, the access (lvalue
| > conversion, 6.3.2.1 p2) on *p, which is allowed by the aliasing rule
| > (6.5 p7, bullet 5 "an aggregate or union ..."), changes the effective
| > type of the object at p.
|
| Uh, sorry. I was wrong here. Non-modifying access to an object doesn't
| change the effective type of subsequent access to the object. And the
| aliasing rule does not apply here.

Aha, but what about 6.5/6

[...] For all other accesses to an object having no declared type,
the effective type of the object is simply the type of the lvalue use
for the access.

?

| Let me correct the last paragraph.
|
| If "struct B x = {0}; *p = x;" follows the above code, the write on *p
| changes the effective type of the object at p (from int to B) for
| subsequent non-modifying access of that object. (6.5 p6, about an
| object having no declared type)

C++ has a simpler notation for that:

new (p) B{ 0 };

but we have been there before :-)

-- Gaby
Gabriel Dos Reis
2014-01-17 04:58:31 UTC
Permalink
Kazutoshi Satoda <***@f2.dion.ne.jp> writes:

| On 2014/01/17 5:53 +0900, Gabriel Dos Reis wrote:
| > | struct B { int x };
| > | struct B* p = (B*) malloc(sizeof(B));
| > | p->x = 17;
| > |
| > | This (modulo the cast) has been how C has handled dynamic allocation
| > | of structs approximately forever. There are no constructors in C,
| > | structs don't get initialized, their fields just get assigned to.
| >
| > Could you walk us through the C standards and explain what you
| > believe this program fragment is supposed to do and what actually is
| > at the address pointed by p?
| > I think that will be a very illuminating exercise -- and would
| > possibly help clear some confusions.
|
| Let me try the exercise. I'm referring WG14 N1570.
| http://www.open-std.org/jtc1/sc22/wg14/www/standards.html
|
|
| p points an object (a region of data storage, 3.15) which is allocated
| by malloc(). The object has allocated storage duration (6.2.4) and its
| lifetime starts at the allocation in malloc(), and ends at deallocation
| in free() (7.22.3).
|
| "p->x" yields an lvalue of type int which designates an object at
| address pointed by ((char*)p + offsetof(B, x)). There is no rule about
| the effective type of the object pointed by p to evaluate the "->"
| expression. (6.5.2.3)
|
| "p->x = 17" makes the effective type of the object between &p->x and
| (&p->x + 1) becomes int (6.5 p6), and stores value 17 (a value
| representation of int which represents 17) into that object (6.5.16).

Agreed. In particular, at this point, we still do not have an object of
type B -- from C's perspective.

| (Note)
| In C, there is no such a thing like "an object of type int" unlike in
| C++ where an object has a type and the type is determined when the
| object is created. In C, "object" is a merely a region of data storage,
| and may be labeled by an effective type which is determined by ongoing
| or previous access to the object at a time.

Yes. This goes back to the comment I made:

# Even if we take the C model, it is not clear what is there. We can
# infer that after the statement 'p->x = 17;' the storage at address
# &p-x' has effective type 'int', but that does not say much about the
# rest. I -suspect- C is OK with that because fundamentally, its object
# model is structural.

| If "struct B x = *p" follows the above code, the access (lvalue
| conversion, 6.3.2.1 p2) on *p, which is allowed by the aliasing rule
| (6.5 p7, bullet 5 "an aggregate or union ..."), changes the effective
| type of the object at p.

Indeed. In particular, the mere write to p->x does not imply that there
is an object of effective type B at p. Another important difference
between C and C++ is that the dynamic type (the closest approximation of
'effective type') of an object does not change during its lifetime.

Going back to constructor, what C++ has done is to automatically
"bless" or perform the conceptial equivalent of access that stamps the
effective type of the object. That also highlights why C++ is carefully
in what can be done between the timeframe where storage is obtained and
initialization is complete (e.g. the effective type/dynamic type is
stamped.)

Now, in most code fragments that have been shown so far (except of
course your last addition) there has been no attempt to access the
storage at p as B, yet it has been claimed that an object of type B existed
there. I don't believe that is correct.

Bonus points: after

struct B x = *p;

what is the effective type of the object at p and &p->x? B and int or B
and B? Can we even give a meaning to the question?

-- Gaby
Tony Van Eerd
2014-01-17 18:05:36 UTC
Permalink
>
> Going back to constructor, what C++ has done is to automatically
> "bless" or perform the conceptial equivalent of access that stamps the
> effective type of the object. That also highlights why C++ is
> carefully in what can be done between the timeframe where storage is obtained and
> initialization is complete (e.g. the effective type/dynamic type is
> stamped.)
>

I think that is the way forward. After the constructor, the memory has been blessed/stamped/whatever.

The "object" returned from malloc is a "sequence of N unsigned char objects" (purposely quoting from 3.9's definition of 'object representation'). ie ready to be an object of whatever type, but not yet blessed beyond the base type unsigned char. (You could try to say it isn't even an array of unsigned char, but I don't think that will help.)

You then need to go at least some distance down the road of C's "effective type".

Tony
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.
Kazutoshi Satoda
2014-01-18 18:23:15 UTC
Permalink
On 2014/01/17 13:58 +0900, Gabriel Dos Reis wrote:
> Kazutoshi Satoda <***@f2.dion.ne.jp> writes:
> | On 2014/01/17 5:53 +0900, Gabriel Dos Reis wrote:
> | > On 2014/01/17 5:01 +0900, James Dennett wrote:
> | > | struct B { int x };
> | > | struct B* p = (B*) malloc(sizeof(B));
> | > | p->x = 17;
> | > |
> | > | This (modulo the cast) has been how C has handled dynamic allocation
> | > | of structs approximately forever. There are no constructors in C,
> | > | structs don't get initialized, their fields just get assigned to.
(snip)
> | "p->x = 17" makes the effective type of the object between &p->x and
> | (&p->x + 1) becomes int (6.5 p6), and stores value 17 (a value
> | representation of int which represents 17) into that object (6.5.16).
>
> Agreed. In particular, at this point, we still do not have an object of
> type B -- from C's perspective.
>
> | (Note)
> | In C, there is no such a thing like "an object of type int" unlike in
> | C++ where an object has a type and the type is determined when the
> | object is created. In C, "object" is a merely a region of data storage,
> | and may be labeled by an effective type which is determined by ongoing
> | or previous access to the object at a time.
>
> Yes. This goes back to the comment I made:
>
> # Even if we take the C model, it is not clear what is there. We can
> # infer that after the statement 'p->x = 17;' the storage at address
> # &p-x' has effective type 'int', but that does not say much about the
> # rest. I -suspect- C is OK with that because fundamentally, its object
> # model is structural.

There is an example which (I think) illustrates a real-world problem
caused by this assertion "the effective type of the object is just int
and not B." Please see this code and the result with gcc 4.8.2 (-O2).
http://melpon.org/wandbox/permlink/BJQkgZDmbsHEp71j

struct A { int a; };
struct B { int a; double b; };

int f(struct A* a, struct B* b)
{
b->a = 123;
a->a = 456;
return b->a;
}

Here, the compiler performs an optimization and returns constant 123
from f(), assuming that two unrelated struct types never alias even if
they have common initial members. I think this optimization is intended
to be allowed by the aliasing rule in both C and C++.

But to allow this optimization in the above case, it is required to be
OK that a compiler can assume the type of object at b is B, and the type
of object at a is A, provided just the two assignment and one read to
their member.

Then, it seems a de facto rule that an evaluation of member access
operator solely (even when no actual access follow) implies the dynamic
type (effective type in C) of the object designated by the left operand
("the object expression" in C++ term) to be a type which satisfies the
aliasing rule as if the object is accessed by the static type of the
object expression.

Is this rule acceptable to be in the standard? For C++, it would be like
that:

Change 3.10 p10 (C++) from:
If a program attempts to access the stored value of an object
through a glvalue of other than one of the following types the
behavior is undefined:
to
If a program attempts to access the stored value of an object
<ins>or evaluates a member access expression of non-static member
(including interpretation of overloaded operators)</ins> through
a glvalue of other than one of the following types the behavior is
undefined:

And define "reuse" to include (along with assignment to a glvalue of a
scalar type, memcpy, etc) an evaluation of member access expression for
trivially constructible class types, to take such a evaluation as a
start of object lifetime with a certain type.

With these rules, the above optimization is said to be OK because if
"a->a = 456" was reusing the storage at b, the following access "return
b->a" is undefined as it read from an object of type B which was killed
by the reuse. Thus a compiler can assume that "a->a = 456" is not
reusing b.


(going back to analysis in C)
> | If "struct B x = *p" follows the above code, the access (lvalue
> | conversion, 6.3.2.1 p2) on *p, which is allowed by the aliasing rule
> | (6.5 p7, bullet 5 "an aggregate or union ..."), changes the effective
> | type of the object at p.
(snip)
> Bonus points: after
>
> struct B x = *p;
>
> what is the effective type of the object at p and &p->x? B and int or B
> and B? Can we even give a meaning to the question?
...(from my follow up)
> | Uh, sorry. I was wrong here. Non-modifying access to an object doesn't
> | change the effective type of subsequent access to the object. And the
> | aliasing rule does not apply here.
>
> Aha, but what about 6.5/6
>
> [...] For all other accesses to an object having no declared type,
> the effective type of the object is simply the type of the lvalue use
> for the access.
>
> ?

A storing access determines the effective type for subsequent
non-modifying accesses. And such subsequent non-modifying accesses do
not drop into "all other accesses", I think.

Then, after "struct B x = *p", the effective type of the object at p
and &p->x are both still int.

--
k_satoda
Gabriel Dos Reis
2014-01-19 02:54:10 UTC
Permalink
| -----Original Message-----
| From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
| Behalf Of Kazutoshi Satoda
| Sent: Saturday, January 18, 2014 10:23 AM
| To: WG21 UB study group
| Cc: Bjarne Stroustrup
| Subject: Re: [ub] type punning through congruent base class?
|
| On 2014/01/17 13:58 +0900, Gabriel Dos Reis wrote:
| > Kazutoshi Satoda <***@f2.dion.ne.jp> writes:
| > | On 2014/01/17 5:53 +0900, Gabriel Dos Reis wrote:
| > | > On 2014/01/17 5:01 +0900, James Dennett wrote:
| > | > | struct B { int x };
| > | > | struct B* p = (B*) malloc(sizeof(B));
| > | > | p->x = 17;
| > | > |
| > | > | This (modulo the cast) has been how C has handled dynamic allocation
| > | > | of structs approximately forever. There are no constructors in C,
| > | > | structs don't get initialized, their fields just get assigned to.
| (snip)
| > | "p->x = 17" makes the effective type of the object between &p->x and
| > | (&p->x + 1) becomes int (6.5 p6), and stores value 17 (a value
| > | representation of int which represents 17) into that object (6.5.16).
| >
| > Agreed. In particular, at this point, we still do not have an object of
| > type B -- from C's perspective.
| >
| > | (Note)
| > | In C, there is no such a thing like "an object of type int" unlike in
| > | C++ where an object has a type and the type is determined when the
| > | object is created. In C, "object" is a merely a region of data storage,
| > | and may be labeled by an effective type which is determined by ongoing
| > | or previous access to the object at a time.
| >
| > Yes. This goes back to the comment I made:
| >
| > # Even if we take the C model, it is not clear what is there. We can
| > # infer that after the statement 'p->x = 17;' the storage at address
| > # &p-x' has effective type 'int', but that does not say much about the
| > # rest. I -suspect- C is OK with that because fundamentally, its object
| > # model is structural.
|
| There is an example which (I think) illustrates a real-world problem
| caused by this assertion "the effective type of the object is just int
| and not B." Please see this code and the result with gcc 4.8.2 (-O2).
| http://melpon.org/wandbox/permlink/BJQkgZDmbsHEp71j
|
| struct A { int a; };
| struct B { int a; double b; };
|
| int f(struct A* a, struct B* b)
| {
| b->a = 123;
| a->a = 456;
| return b->a;
| }
|
| Here, the compiler performs an optimization and returns constant 123
| from f(), assuming that two unrelated struct types never alias even if
| they have common initial members. I think this optimization is intended
| to be allowed by the aliasing rule in both C and C++.

Yes, I think this form of optimization -- type-based alias analysis -- is intended to be allowed in C++ and we definitely want to retain it. C's rules for type equivalence and type alias are on occasions a bit obscure for me to penetrate properly without tripping over. C11 fixed several bugs related to type aliasing and lifetime, but I don't know whether all C compilers were updated to the latest rules.

>
Kazutoshi Satoda
2014-01-19 08:12:37 UTC
Permalink
On 2014/01/19 11:54 +0900, Gabriel Dos Reis wrote:
> On 2014/01/19 3:23 +0900, Kazutoshi Satoda wrote:
> | There is an example which (I think) illustrates a real-world problem
> | caused by this assertion "the effective type of the object is just int
> | and not B." Please see this code and the result with gcc 4.8.2 (-O2).
> | http://melpon.org/wandbox/permlink/BJQkgZDmbsHEp71j
> |
> | struct A { int a; };
> | struct B { int a; double b; };
> |
> | int f(struct A* a, struct B* b)
> | {
> | b->a = 123;
> | a->a = 456;
> | return b->a;
> | }
(snip)
>
Herb Sutter
2014-01-16 19:41:25 UTC
Permalink
Matt quoted:

"The lifetime of an object of type T begins when:
— storage with the proper alignment and size for type T is obtained, and
— if the object has non-trivial initialization, its initialization is complete."

Perhaps the necessary fix is to strike “if the object has non-trivial initialization” here. I don’t see why trivial constructors are special – sure, they don’t do anything, but they are notionally important.

A bag of memory of alignment and size suitable for T is-not-a T object until its (possibly trivial) ctor is called on that memory – it seems wrong for that to be true for all T except trivially-constructible T’s (with implicit universal type-punning for all types <= sizeof(T)).

Herb
Ville Voutilainen
2014-01-16 19:45:19 UTC
Permalink
On 16 January 2014 21:41, Herb Sutter <***@microsoft.com> wrote:
> Matt quoted:
>
>
>
> "The lifetime of an object of type T begins when:
>
> — storage with the proper alignment and size for type T is obtained, and
>
> — if the object has non-trivial initialization, its initialization is
> complete."
>
>
>
> Perhaps the necessary fix is to strike “if the object has non-trivial
> initialization” here. I don’t see why trivial constructors are special –
> sure, they don’t do anything, but they are notionally important.
>
>
>
> A bag of memory of alignment and size suitable for T is-not-a T object until
> its (possibly trivial) ctor is called on that memory – it seems wrong for
> that to be true for all T except trivially-constructible T’s (with implicit
> universal type-punning for all types <= sizeof(T)).


Well, as far as I understand, trivial constructors might not be called at all.
Herb Sutter
2014-01-16 19:50:26 UTC
Permalink
> Well, as far as I understand, trivial constructors might not be called at all.

Maybe "might not be called" is part of the bug then. Isn't the right model that trivial ctors are called, but "might do nothing"?

Herb
Gabriel Dos Reis
2014-01-16 20:00:50 UTC
Permalink
| -----Original Message-----
| From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
| Behalf Of Herb Sutter
| Sent: Thursday, January 16, 2014 11:50 AM
| To: WG21 UB study group
| Subject: Re: [ub] type punning through congruent base class?
|
| > Well, as far as I understand, trivial constructors might not be called at all.
|
| Maybe "might not be called" is part of the bug then. Isn't the right model that
| trivial ctors are called, but "might do nothing"?

Yes, agreed.

-- Gaby
James Dennett
2014-01-16 20:05:57 UTC
Permalink
On Thu, Jan 16, 2014 at 12:00 PM, Gabriel Dos Reis <***@microsoft.com> wrote:
>
>
> | -----Original Message-----
> | From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
> | Behalf Of Herb Sutter
> | Sent: Thursday, January 16, 2014 11:50 AM
> | To: WG21 UB study group
> | Subject: Re: [ub] type punning through congruent base class?
> |
> | > Well, as far as I understand, trivial constructors might not be called at all.
> |
> | Maybe "might not be called" is part of the bug then. Isn't the right model that
> | trivial ctors are called, but "might do nothing"?
>
> Yes, agreed.

That may be part of a proposed model, but it's not part of the
existing model, which (for compatibility) is based on C's model, where
fields of a (necessarily trivially-constructible) struct can be
written to as soon as suitable memory for that struct is obtained.

If C++ weren't based on C, it would likely have a model where you
could not access an object without an initialization operation to
create an object from raw memory. But C allows use of raw memory as
objects, and I don't hear anyone suggesting how we can accommodate
that while avoiding its horrible consequences. We could choose _not_
to accommodate it (i.e., require the initialization step that creates
an object from raw memory), but it's a significant change that
invalidates massive amounts of existing code.

-- James
Herb Sutter
2014-01-16 21:10:19 UTC
Permalink
> > | > Well, as far as I understand, trivial constructors might not be called at all.
> > |
> > | Maybe "might not be called" is part of the bug then. Isn't the right
> > | model that trivial ctors are called, but "might do nothing"?
> >
> > Yes, agreed.
>
> That may be part of a proposed model, but it's not part of the existing model,
> which (for compatibility) is based on C's model, where fields of a (necessarily
> trivially-constructible) struct can be written to as soon as suitable memory for
> that struct is obtained.

Ah, I see the problem, thanks James and Daveed. Let me try to summarize what I've learned so far.

In C++, "storage" and "lifetime" don't mean the same thing for a non-trivially-constructible type.

But in C the word "lifetime" has a different meaning than C++. I have C99 handy, but it should be the same in C11, and "lifetime" and "storage duration" are the same thing in C:

6.2.4/1: An object has a storage duration that determines its lifetime. There are three storage durations: static, automatic, and allocated. Allocated storage is described in 7.20.3.

6.2.4/2: The lifetime of an object is the portion of program execution during which storage is guaranteed to be reserved for it.

7.20.3: The order and contiguity of storage allocated by successive calls to the calloc, malloc, and realloc functions is unspecified. The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object and then used to access such an object or an array of such objects in the space allocated (until the space is explicitly deallocated). The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object. The pointer returned points to the start (lowest byte address) of the allocated space.

But one thing that follows here is that in C, "the object" is the raw memory. Right? This follows from the above three quotes.


So is that different in C++? As Daveed pointed out:

> > Am I missing something?
>
> I think so: 3.8/1 also says that the lifetime of an object ends if its storage is
> reused

"...or released" -- okay. I always thought reused meant deallocated and reallocated, but since it says "reused or released" it does seem to follow that reused means what Daveed said:

> . So when you write to *ps, you've terminated the lifetime of *pb.

OK, that's C++. The word "reused" doesn't appear anywhere in the C99 standard at least.


Trying to summarize:

So was the goal of the 3.8/1 wording of "reuse" to acknowledge (and be compatible with) the C memory-scribbling, while still being able to say that even for raw storage and PODs a given piece of storage has a well-defined type at any given point in time (namely the one it was last written to as)? And does "reused" clearly say that "written-to" part if that's the intent?

And so for raw memory and PODs, storage as a type T begins when you read or write it as a T, and ends when you read or write as some other type U, as long as T and U are PODs that fit in the storage (incl. size and alignment)?

And dare I ask: Um, what about using part of the same allocated buffer to hold a T and another part to hold a U, such as malloc(1000) and read/write an int at offset 10 and read/write a short at offset 314 -- they're the same allocation, so do we even have any way to talk about that?

Herb
Richard Smith
2014-01-16 21:48:18 UTC
Permalink
On 16 January 2014 13:10, Herb Sutter <***@microsoft.com> wrote:

> > > | > Well, as far as I understand, trivial constructors might not be
> called at all.
> > > |
> > > | Maybe "might not be called" is part of the bug then. Isn't the right
> > > | model that trivial ctors are called, but "might do nothing"?
> > >
> > > Yes, agreed.
> >
> > That may be part of a proposed model, but it's not part of the existing
> model,
> > which (for compatibility) is based on C's model, where fields of a
> (necessarily
> > trivially-constructible) struct can be written to as soon as suitable
> memory for
> > that struct is obtained.
>
> Ah, I see the problem, thanks James and Daveed. Let me try to summarize
> what I've learned so far.
>
> In C++, "storage" and "lifetime" don't mean the same thing for a
> non-trivially-constructible type.
>
> But in C the word "lifetime" has a different meaning than C++. I have C99
> handy, but it should be the same in C11, and "lifetime" and "storage
> duration" are the same thing in C:
>
> 6.2.4/1: An object has a storage duration that determines its
> lifetime. There are three storage durations: static, automatic, and
> allocated. Allocated storage is described in 7.20.3.
>
> 6.2.4/2: The lifetime of an object is the portion of program
> execution during which storage is guaranteed to be reserved for it.
>
> 7.20.3: The order and contiguity of storage allocated by
> successive calls to the calloc, malloc, and realloc functions is
> unspecified. The pointer returned if the allocation succeeds is suitably
> aligned so that it may be assigned to a pointer to any type of object and
> then used to access such an object or an array of such objects in the space
> allocated (until the space is explicitly deallocated). The lifetime of an
> allocated object extends from the allocation until the deallocation. Each
> such allocation shall yield a pointer to an object disjoint from any other
> object. The pointer returned points to the start (lowest byte address) of
> the allocated space.
>
> But one thing that follows here is that in C, "the object" is the raw
> memory. Right? This follows from the above three quotes.
>
>
> So is that different in C++?


There's one other rule that I'd overlooked in my prior messages.
[intro.object]/1 presents this beautiful model: "An object is a region of
storage. [...] An object is created by a definition (3.1), by a
new-expression (5.3.4) or by the implementation (12.2) when needed.[...] An
object has a storage duration (3.7) which influences its lifetime (3.8). An
object has a type (3.9)."

Trouble is, this model gives a vast quantity of real-world C++ code
undefined behavior, including any C-like code that uses the 'malloc'
technique described in this thread. This breaks compatibility with C and
with real-world C++ so thoroughly that I think it's not a tenable position,
and that the rule must be revised. Prior to this thread, I had assumed that
we would easily reach consensus on that, but it seems that's not the case.

As Daveed pointed out:
>
> > > Am I missing something?
> >
> > I think so: 3.8/1 also says that the lifetime of an object ends if its
> storage is
> > reused
>
> "...or released" -- okay. I always thought reused meant deallocated and
> reallocated, but since it says "reused or released" it does seem to follow
> that reused means what Daveed said:
>
> > . So when you write to *ps, you've terminated the lifetime of *pb.
>
> OK, that's C++. The word "reused" doesn't appear anywhere in the C99
> standard at least.
>
>
> Trying to summarize:
>
> So was the goal of the 3.8/1 wording of "reuse" to acknowledge (and be
> compatible with) the C memory-scribbling, while still being able to say
> that even for raw storage and PODs a given piece of storage has a
> well-defined type at any given point in time (namely the one it was last
> written to as)? And does "reused" clearly say that "written-to" part if
> that's the intent?
>
> And so for raw memory and PODs, storage as a type T begins when you read
> or write it as a T, and ends when you read or write as some other type U,
> as long as T and U are PODs that fit in the storage (incl. size and
> alignment)?
>
> And dare I ask: Um, what about using part of the same allocated buffer to
> hold a T and another part to hold a U, such as malloc(1000) and read/write
> an int at offset 10 and read/write a short at offset 314 -- they're the
> same allocation, so do we even have any way to talk about that?
>
> Herb
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
Jens Maurer
2014-01-16 22:13:02 UTC
Permalink
On 01/16/2014 10:48 PM, Richard Smith wrote:
> There's one other rule that I'd overlooked in my prior messages.
> [intro.object]/1 presents this beautiful model: "An object is a
> region of storage. [...] An object is created by a definition (3.1),
> by a new-expression (5.3.4) or by the implementation (12.2) when
> needed.[...] An object has a storage duration (3.7) which influences
> its lifetime (3.8). An object has a type (3.9)."
>
> Trouble is, this model gives a vast quantity of real-world C++ code
> undefined behavior, including any C-like code that uses the 'malloc'
> technique described in this thread.

Why? Applying the term "object" to something doesn't mean it's
actually usable (outside of its lifetime); see 3.8p5.

Put differently: 1.8p1 doesn't say anything about "lifetime", it
just says "object", which is a fairly vacuous term by itself.

(I concur with your earlier point that the C++ wording 3.8p1
"storage with the proper alignment and size for type T is obtained"
isn't good enough, and we might need something akin to the
C model of effective type, or apply the term "dynamic type"
expressly for the memory-scribbling case.)

Jens
Richard Smith
2014-01-16 22:34:04 UTC
Permalink
On 16 January 2014 14:13, Jens Maurer <***@gmx.net> wrote:

> On 01/16/2014 10:48 PM, Richard Smith wrote:
> > There's one other rule that I'd overlooked in my prior messages.
> > [intro.object]/1 presents this beautiful model: "An object is a
> > region of storage. [...] An object is created by a definition (3.1),
> > by a new-expression (5.3.4) or by the implementation (12.2) when
> > needed.[...] An object has a storage duration (3.7) which influences
> > its lifetime (3.8). An object has a type (3.9)."
> >
> > Trouble is, this model gives a vast quantity of real-world C++ code
> > undefined behavior, including any C-like code that uses the 'malloc'
> > technique described in this thread.
>
> Why? Applying the term "object" to something doesn't mean it's
> actually usable (outside of its lifetime); see 3.8p5.
>

Because:

int *p = (int*)malloc(sizeof(int));
*p = 0;

... does not contain a definition of an 'int', not a new-expression, nor a
temporary. Therefore, by 1.8/1, it does not create an object of type 'int'.
Therefore the assignment (presumably) has undefined behavior.


> Put differently: 1.8p1 doesn't say anything about "lifetime", it
> just says "object", which is a fairly vacuous term by itself.
>
> (I concur with your earlier point that the C++ wording 3.8p1
> "storage with the proper alignment and size for type T is obtained"
> isn't good enough, and we might need something akin to the
> C model of effective type, or apply the term "dynamic type"
> expressly for the memory-scribbling case.)
>
> Jens
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
Gabriel Dos Reis
2014-01-16 22:42:06 UTC
Permalink
| (I concur with your earlier point that the C++ wording 3.8p1
| "storage with the proper alignment and size for type T is obtained"
| isn't good enough, and we might need something akin to the
| C model of effective type, or apply the term "dynamic type"
| expressly for the memory-scribbling case.)

I am unconvinced by the notion of the effective type and I see no reason to deviate from "dynamic type".
On the other hand, we definitely need to define more cases -- see for example my earlier that suggested to consider certain uses of memcpy() as having the moral semantics of constructor call -- matching C's transfer of effective type.

-- Gaby
Gabriel Dos Reis
2014-01-16 22:39:13 UTC
Permalink
I had always assumed that you were always mindful of that paragraph – which explains part of my puzzlement at your previous claims. It is so central the entire C++ object model. Thanks for clarifying.

We may have to tweak this paragraph, but I don’t think the root cause is here. It is elsewhere – the very paragraph under discussion in previous messages.

By the way, my concern isn’t driven by theory. I would like to make sure that future high quality C++ implementations can actually include instrumentation of lifetime and aliasing issues – the way implementations currently do with “address sanitizers”, “signed integer arithmetic overflow”, etc.

-- Gaby

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Richard Smith
Sent: Thursday, January 16, 2014 1:48 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On 16 January 2014 13:10, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:
> > | > Well, as far as I understand, trivial constructors might not be called at all.
> > |
> > | Maybe "might not be called" is part of the bug then. Isn't the right
> > | model that trivial ctors are called, but "might do nothing"?
> >
> > Yes, agreed.
>
> That may be part of a proposed model, but it's not part of the existing model,
> which (for compatibility) is based on C's model, where fields of a (necessarily
> trivially-constructible) struct can be written to as soon as suitable memory for
> that struct is obtained.
Ah, I see the problem, thanks James and Daveed. Let me try to summarize what I've learned so far.

In C++, "storage" and "lifetime" don't mean the same thing for a non-trivially-constructible type.

But in C the word "lifetime" has a different meaning than C++. I have C99 handy, but it should be the same in C11, and "lifetime" and "storage duration" are the same thing in C:

6.2.4/1: An object has a storage duration that determines its lifetime. There are three storage durations: static, automatic, and allocated. Allocated storage is described in 7.20.3.

6.2.4/2: The lifetime of an object is the portion of program execution during which storage is guaranteed to be reserved for it.

7.20.3: The order and contiguity of storage allocated by successive calls to the calloc, malloc, and realloc functions is unspecified. The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object and then used to access such an object or an array of such objects in the space allocated (until the space is explicitly deallocated). The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object. The pointer returned points to the start (lowest byte address) of the allocated space.

But one thing that follows here is that in C, "the object" is the raw memory. Right? This follows from the above three quotes.


So is that different in C++?

There's one other rule that I'd overlooked in my prior messages. [intro.object]/1 presents this beautiful model: "An object is a region of storage. [...] An object is created by a definition (3.1), by a new-expression (5.3.4) or by the implementation (12.2) when needed.[...] An object has a storage duration (3.7) which influences its lifetime (3.8). An object has a type (3.9)."

Trouble is, this model gives a vast quantity of real-world C++ code undefined behavior, including any C-like code that uses the 'malloc' technique described in this thread. This breaks compatibility with C and with real-world C++ so thoroughly that I think it's not a tenable position, and that the rule must be revised. Prior to this thread, I had assumed that we would easily reach consensus on that, but it seems that's not the case.

As Daveed pointed out:

> > Am I missing something?
>
> I think so: 3.8/1 also says that the lifetime of an object ends if its storage is
> reused
"...or released" -- okay. I always thought reused meant deallocated and reallocated, but since it says "reused or released" it does seem to follow that reused means what Daveed said:

> . So when you write to *ps, you've terminated the lifetime of *pb.
OK, that's C++. The word "reused" doesn't appear anywhere in the C99 standard at least.


Trying to summarize:

So was the goal of the 3.8/1 wording of "reuse" to acknowledge (and be compatible with) the C memory-scribbling, while still being able to say that even for raw storage and PODs a given piece of storage has a well-defined type at any given point in time (namely the one it was last written to as)? And does "reused" clearly say that "written-to" part if that's the intent?

And so for raw memory and PODs, storage as a type T begins when you read or write it as a T, and ends when you read or write as some other type U, as long as T and U are PODs that fit in the storage (incl. size and alignment)?

And dare I ask: Um, what about using part of the same allocated buffer to hold a T and another part to hold a U, such as malloc(1000) and read/write an int at offset 10 and read/write a short at offset 314 -- they're the same allocation, so do we even have any way to talk about that?

Herb

_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub
Richard Smith
2014-01-16 22:55:12 UTC
Permalink
On 16 January 2014 14:39, Gabriel Dos Reis <***@microsoft.com> wrote:

> I had always assumed that you were always mindful of that paragraph –
> which explains part of my puzzlement at your previous claims. It is so
> central the entire C++ object model. Thanks for clarifying.
>
>
>
> We may have to tweak this paragraph, but I don’t think the root cause is
> here. It is elsewhere – the very paragraph under discussion in previous
> messages.
>
>
>
> By the way, my concern isn’t driven by theory. I would like to make sure
> that future high quality C++ implementations can actually include
> instrumentation of lifetime and aliasing issues – the way implementations
> currently do with “address sanitizers”, “signed integer arithmetic
> overflow”, etc.
>

That's also my goal; I started digging into this precisely because I was
designing an object lifetime sanitizer -- this very rapidly led to the
observation that the model described in 1.8/1 is untenable as a basis for
such a sanitizer, because it does not match the model used by programmers
and implementers -- such a sanitizer would diagnose problems in a large
quantity of real code, in which people assume they can obtain appropriate
storage any way they like, and use it as an object of type T, so long as T
is suitably trivial.

*From:* ub-***@open-std.org [mailto:ub-***@open-std.org] *On
> Behalf Of *Richard Smith
> *Sent:* Thursday, January 16, 2014 1:48 PM
>
> *To:* WG21 UB study group
> *Subject:* Re: [ub] type punning through congruent base class?
>
>
>
> On 16 January 2014 13:10, Herb Sutter <***@microsoft.com> wrote:
>
> > > | > Well, as far as I understand, trivial constructors might not be
> called at all.
> > > |
> > > | Maybe "might not be called" is part of the bug then. Isn't the right
> > > | model that trivial ctors are called, but "might do nothing"?
> > >
> > > Yes, agreed.
> >
> > That may be part of a proposed model, but it's not part of the existing
> model,
> > which (for compatibility) is based on C's model, where fields of a
> (necessarily
> > trivially-constructible) struct can be written to as soon as suitable
> memory for
> > that struct is obtained.
>
> Ah, I see the problem, thanks James and Daveed. Let me try to summarize
> what I've learned so far.
>
> In C++, "storage" and "lifetime" don't mean the same thing for a
> non-trivially-constructible type.
>
> But in C the word "lifetime" has a different meaning than C++. I have C99
> handy, but it should be the same in C11, and "lifetime" and "storage
> duration" are the same thing in C:
>
> 6.2.4/1: An object has a storage duration that determines its
> lifetime. There are three storage durations: static, automatic, and
> allocated. Allocated storage is described in 7.20.3.
>
> 6.2.4/2: The lifetime of an object is the portion of program
> execution during which storage is guaranteed to be reserved for it.
>
> 7.20.3: The order and contiguity of storage allocated by
> successive calls to the calloc, malloc, and realloc functions is
> unspecified. The pointer returned if the allocation succeeds is suitably
> aligned so that it may be assigned to a pointer to any type of object and
> then used to access such an object or an array of such objects in the space
> allocated (until the space is explicitly deallocated). The lifetime of an
> allocated object extends from the allocation until the deallocation. Each
> such allocation shall yield a pointer to an object disjoint from any other
> object. The pointer returned points to the start (lowest byte address) of
> the allocated space.
>
> But one thing that follows here is that in C, "the object" is the raw
> memory. Right? This follows from the above three quotes.
>
>
> So is that different in C++?
>
>
>
> There's one other rule that I'd overlooked in my prior messages.
> [intro.object]/1 presents this beautiful model: "An object is a region of
> storage. [...] An object is created by a definition (3.1), by a
> new-expression (5.3.4) or by the implementation (12.2) when needed.[...] An
> object has a storage duration (3.7) which influences its lifetime (3.8). An
> object has a type (3.9)."
>
>
>
> Trouble is, this model gives a vast quantity of real-world C++ code
> undefined behavior, including any C-like code that uses the 'malloc'
> technique described in this thread. This breaks compatibility with C and
> with real-world C++ so thoroughly that I think it's not a tenable position,
> and that the rule must be revised. Prior to this thread, I had assumed that
> we would easily reach consensus on that, but it seems that's not the case.
>
>
>
> As Daveed pointed out:
>
>
> > > Am I missing something?
> >
> > I think so: 3.8/1 also says that the lifetime of an object ends if its
> storage is
> > reused
>
> "...or released" -- okay. I always thought reused meant deallocated and
> reallocated, but since it says "reused or released" it does seem to follow
> that reused means what Daveed said:
>
> > . So when you write to *ps, you've terminated the lifetime of *pb.
>
> OK, that's C++. The word "reused" doesn't appear anywhere in the C99
> standard at least.
>
>
> Trying to summarize:
>
> So was the goal of the 3.8/1 wording of "reuse" to acknowledge (and be
> compatible with) the C memory-scribbling, while still being able to say
> that even for raw storage and PODs a given piece of storage has a
> well-defined type at any given point in time (namely the one it was last
> written to as)? And does "reused" clearly say that "written-to" part if
> that's the intent?
>
> And so for raw memory and PODs, storage as a type T begins when you read
> or write it as a T, and ends when you read or write as some other type U,
> as long as T and U are PODs that fit in the storage (incl. size and
> alignment)?
>
> And dare I ask: Um, what about using part of the same allocated buffer to
> hold a T and another part to hold a U, such as malloc(1000) and read/write
> an int at offset 10 and read/write a short at offset 314 -- they're the
> same allocation, so do we even have any way to talk about that?
>
> Herb
>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org
> http://www.open-std.org/mailman/listinfo/ub
>
Gabriel Dos Reis
2014-01-16 22:58:16 UTC
Permalink
Again, just to reassure you: we are dealing with massive real world existing codes on our side :-)

-- Gaby

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Richard Smith
Sent: Thursday, January 16, 2014 2:55 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On 16 January 2014 14:39, Gabriel Dos Reis <***@microsoft.com<mailto:***@microsoft.com>> wrote:
I had always assumed that you were always mindful of that paragraph – which explains part of my puzzlement at your previous claims. It is so central the entire C++ object model. Thanks for clarifying.

We may have to tweak this paragraph, but I don’t think the root cause is here. It is elsewhere – the very paragraph under discussion in previous messages.

By the way, my concern isn’t driven by theory. I would like to make sure that future high quality C++ implementations can actually include instrumentation of lifetime and aliasing issues – the way implementations currently do with “address sanitizers”, “signed integer arithmetic overflow”, etc.

That's also my goal; I started digging into this precisely because I was designing an object lifetime sanitizer -- this very rapidly led to the observation that the model described in 1.8/1 is untenable as a basis for such a sanitizer, because it does not match the model used by programmers and implementers -- such a sanitizer would diagnose problems in a large quantity of real code, in which people assume they can obtain appropriate storage any way they like, and use it as an object of type T, so long as T is suitably trivial.

From: ub-***@open-std.org<mailto:ub-***@open-std.org> [mailto:ub-***@open-std.org<mailto:ub-***@open-std.org>] On Behalf Of Richard Smith
Sent: Thursday, January 16, 2014 1:48 PM

To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On 16 January 2014 13:10, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:
> > | > Well, as far as I understand, trivial constructors might not be called at all.
> > |
> > | Maybe "might not be called" is part of the bug then. Isn't the right
> > | model that trivial ctors are called, but "might do nothing"?
> >
> > Yes, agreed.
>
> That may be part of a proposed model, but it's not part of the existing model,
> which (for compatibility) is based on C's model, where fields of a (necessarily
> trivially-constructible) struct can be written to as soon as suitable memory for
> that struct is obtained.
Ah, I see the problem, thanks James and Daveed. Let me try to summarize what I've learned so far.

In C++, "storage" and "lifetime" don't mean the same thing for a non-trivially-constructible type.

But in C the word "lifetime" has a different meaning than C++. I have C99 handy, but it should be the same in C11, and "lifetime" and "storage duration" are the same thing in C:

6.2.4/1: An object has a storage duration that determines its lifetime. There are three storage durations: static, automatic, and allocated. Allocated storage is described in 7.20.3.

6.2.4/2: The lifetime of an object is the portion of program execution during which storage is guaranteed to be reserved for it.

7.20.3: The order and contiguity of storage allocated by successive calls to the calloc, malloc, and realloc functions is unspecified. The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object and then used to access such an object or an array of such objects in the space allocated (until the space is explicitly deallocated). The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object. The pointer returned points to the start (lowest byte address) of the allocated space.

But one thing that follows here is that in C, "the object" is the raw memory. Right? This follows from the above three quotes.


So is that different in C++?

There's one other rule that I'd overlooked in my prior messages. [intro.object]/1 presents this beautiful model: "An object is a region of storage. [...] An object is created by a definition (3.1), by a new-expression (5.3.4) or by the implementation (12.2) when needed.[...] An object has a storage duration (3.7) which influences its lifetime (3.8). An object has a type (3.9)."

Trouble is, this model gives a vast quantity of real-world C++ code undefined behavior, including any C-like code that uses the 'malloc' technique described in this thread. This breaks compatibility with C and with real-world C++ so thoroughly that I think it's not a tenable position, and that the rule must be revised. Prior to this thread, I had assumed that we would easily reach consensus on that, but it seems that's not the case.

As Daveed pointed out:

> > Am I missing something?
>
> I think so: 3.8/1 also says that the lifetime of an object ends if its storage is
> reused
"...or released" -- okay. I always thought reused meant deallocated and reallocated, but since it says "reused or released" it does seem to follow that reused means what Daveed said:

> . So when you write to *ps, you've terminated the lifetime of *pb.
OK, that's C++. The word "reused" doesn't appear anywhere in the C99 standard at least.


Trying to summarize:

So was the goal of the 3.8/1 wording of "reuse" to acknowledge (and be compatible with) the C memory-scribbling, while still being able to say that even for raw storage and PODs a given piece of storage has a well-defined type at any given point in time (namely the one it was last written to as)? And does "reused" clearly say that "written-to" part if that's the intent?

And so for raw memory and PODs, storage as a type T begins when you read or write it as a T, and ends when you read or write as some other type U, as long as T and U are PODs that fit in the storage (incl. size and alignment)?

And dare I ask: Um, what about using part of the same allocated buffer to hold a T and another part to hold a U, such as malloc(1000) and read/write an int at offset 10 and read/write a short at offset 314 -- they're the same allocation, so do we even have any way to talk about that?

Herb

_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub


_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub
Jens Maurer
2014-01-16 22:06:09 UTC
Permalink
On 01/16/2014 10:10 PM, Herb Sutter wrote:
>>> | > Well, as far as I understand, trivial constructors might not be called at all.
>>> |
>>> | Maybe "might not be called" is part of the bug then. Isn't the right
>>> | model that trivial ctors are called, but "might do nothing"?
>>>
>>> Yes, agreed.
>>
>> That may be part of a proposed model, but it's not part of the existing model,
>> which (for compatibility) is based on C's model, where fields of a (necessarily
>> trivially-constructible) struct can be written to as soon as suitable memory for
>> that struct is obtained.
>
> Ah, I see the problem, thanks James and Daveed. Let me try to summarize what I've learned so far.
>
> In C++, "storage" and "lifetime" don't mean the same thing for a non-trivially-constructible type.

Yes, except we're mostly not talking about these here.

> But in C the word "lifetime" has a different meaning than C++.

For the subset of C++ that uses only PODs, the meaning
of "lifetime" is supposed to be the same, in my understanding.

> I have
> C99 handy, but it should be the same in C11, and "lifetime" and
> "storage duration" are the same thing in C:
>
> 6.2.4/1: An object has a storage duration that determines its
> lifetime. There are three storage durations: static, automatic, and
> allocated. Allocated storage is described in 7.20.3.
>
> 6.2.4/2: The lifetime of an object is the portion of program
> execution during which storage is guaranteed to be reserved for it.

That corresponds to the "reused or released" phrasing in C++.

> 7.20.3: The order and contiguity of storage allocated by successive
> calls to the calloc, malloc, and realloc functions is unspecified.
> The pointer returned if the allocation succeeds is suitably aligned
> so that it may be assigned to a pointer to any type of object and
> then used to access such an object or an array of such objects in the
> space allocated (until the space is explicitly deallocated). The
> lifetime of an allocated object extends from the allocation until the
> deallocation. Each such allocation shall yield a pointer to an object
> disjoint from any other object. The pointer returned points to the
> start (lowest byte address) of the allocated space.
>
> But one thing that follows here is that in C, "the object" is the raw
> memory. Right? This follows from the above three quotes.

Same in C++: 1.8p1: "An object is a region of storage."

> So was the goal of the 3.8/1 wording of "reuse" to acknowledge (and
> be compatible with) the C memory-scribbling,

That's my understanding.

> while still being able
> to say that even for raw storage and PODs a given piece of storage
> has a well-defined type at any given point in time (namely the one it
> was last written to as)? And does "reused" clearly say that
> "written-to" part if that's the intent?

Well, "reused" is just used to specify that the lifetime of an
object ends. It doesn't say that the lifetime of another object
necessarily starts.

> And so for raw memory and PODs, storage as a type T begins when you

That part doesn't make sense to me. I'm going to read "storage as
a type T" as "lifetime of an object of POD-type T", and ignore the
mentioning of PODs earlier in the sentence.

> read or write it as a T, and ends when you read or write as some
> other type U, as long as T and U are PODs that fit in the storage
> (incl. size and alignment)?

As far as I understand, that's the intent, in both C and C++.

> And dare I ask: Um, what about using part of the same allocated
> buffer to hold a T and another part to hold a U, such as malloc(1000)
> and read/write an int at offset 10 and read/write a short at offset
> 314 -- they're the same allocation, so do we even have any way to
> talk about that?

They're the same allocation, but different regions of storage, so
you're fine, in my opinion. (Implementing your own sub-allocators
with a malloc backend probably depends on this to work.)

Your particular example might have alignment issues, though, since
offset 10 is probably misaligned for an "int", and offset 314 might
be misaligned for a "short".

Thanks,
Jens
Jeffrey Yasskin
2014-01-16 22:53:00 UTC
Permalink
On Thu, Jan 16, 2014 at 2:06 PM, Jens Maurer <***@gmx.net> wrote:
> On 01/16/2014 10:10 PM, Herb Sutter wrote:
>> And dare I ask: Um, what about using part of the same allocated
>> buffer to hold a T and another part to hold a U, such as malloc(1000)
>> and read/write an int at offset 10 and read/write a short at offset
>> 314 -- they're the same allocation, so do we even have any way to
>> talk about that?
>
> They're the same allocation, but different regions of storage, so
> you're fine, in my opinion. (Implementing your own sub-allocators
> with a malloc backend probably depends on this to work.)

Mhmm; that matches my understanding. For some more litmus tests, how
about an object like:

struct S {
int x;
std::aligned_union<1, float>::type y;
};

If I write:
S* s = (S*)malloc(sizeof(S));
float* y = (float*)(&s->y);
*y = 5.0f;

Do I have defined behavior? Can I use *y as an object of type float,
or does it stay an object of type std::aligned_union<float>::type? (Am
I using "object" in the right way here?)

If I change that to:

S* s = new S;
float* y = (float*)(&s->y);
*y = 5.0f;

does anything change?

What if I use

S s;
float* y = (float*)(&s.y);
*y = 5.0f;

instead? My reading of C suggests that moving the object on to the
stack makes a difference because the object now has a "declared type",
but what do we want in C++?

Or, finally

S s;
new (&s.y) float();
float* y = (float*)(&s.y);

Types like llvm::SmallVector
(http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ADT/SmallVector.h?revision=190708&view=markup#l112)
do this one.

If some part of the answer is that std::aligned_union is "magic", does
that make the possible implementation at
http://en.cppreference.com/w/cpp/types/aligned_storage#Possible_implementation
incorrect?

Thanks,
Jeffrey
Ion Gaztañaga
2014-01-16 22:34:23 UTC
Permalink
El 16/01/2014 22:10, Herb Sutter wrote:
> And dare I ask: Um, what about using part of the same allocated
> buffer to hold a T and another part to hold a U, such as malloc(1000)
> and read/write an int at offset 10 and read/write a short at offset
> 314 -- they're the same allocation, so do we even have any way to
> talk about that?

Any existing memory allocator use this pattern (obtaining memory via
mmap/VirtualAlloc and buffers and writing metadata for management
purposes and splitting and coalescing chunks of memory writing new
metadata, all whithout calling any constructor and destructor).

We are missing a mechanism to support system programming techniques that
reuse storage, even for objects that have a constructors, just like we
support not explicitly calling the destructor in 3.8 [basic.life] p.4
"the program is not required to call the destructor explicitly before
the storage which the object occupies is reused or released".

Imagine we have a mechanism for that:

//Assume "storage" is properly aligned
//and big enough to store an object
//of type T
void* storage = malloc(...);
OR
void *storage = &one_object_of_type_X;
OR
void *storage = &array_of_type_Y[array_size/2];

//The lifetime of the object (if any)
//stored (even partially) in "storage" ends in the
//following line. The storage is "reused" and the
//lifetime of an object of type T is started.
//
//Old object's destructor is not called so any
//program that depends on the side effects
//produced by the destructor has undefined behavior
//(3.8 [basic.life] p.4)
//
//T's constructor is not called so any program
//that depends on the side effects produced by
//the constructor has undefined behavior.
T* t = as_if_ctor_was_already_called<T>(storage);
t->foo();

U* u = as_if_ctor_was_already_called<U>(storage);
u->foo();

V* v = as_if_ctor_was_already_called<V>(storage);
v->foo();

...

Now let me ask some questions...

a) Would this have some impact in the performance of the code produced
by current C++ compilers?

b) If a)'s answer is "no", could "as_if_ctor_was_already_called<T>" be
called "reintepret_cast<T*>"?

c) If b)'s answer is "yes", wouldn't this allow C compatibility?

Best,

Ion
Gabriel Dos Reis
2014-01-16 22:56:08 UTC
Permalink
| We are missing a mechanism to support system programming techniques that
| reuse storage, even for objects that have a constructor

Yes, currently, we are missing well-defined constructs to support external data structures -- that is a problem that many system-level components (OS, network, etc.) have.
Let's do it without compromising the foundations :-)

-- Gaby
Gabriel Dos Reis
2014-01-16 22:28:39 UTC
Permalink
| -----Original Message-----
| From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
| Behalf Of James Dennett
| Sent: Thursday, January 16, 2014 12:06 PM
| To: WG21 UB study group
| Subject: Re: [ub] type punning through congruent base class?
|
| On Thu, Jan 16, 2014 at 12:00 PM, Gabriel Dos Reis <***@microsoft.com>
| wrote:
| >
| >
| > | -----Original Message-----
| > | From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
| > | Behalf Of Herb Sutter
| > | Sent: Thursday, January 16, 2014 11:50 AM
| > | To: WG21 UB study group
| > | Subject: Re: [ub] type punning through congruent base class?
| > |
| > | > Well, as far as I understand, trivial constructors might not be called at
| all.
| > |
| > | Maybe "might not be called" is part of the bug then. Isn't the right model
| that
| > | trivial ctors are called, but "might do nothing"?
| >
| > Yes, agreed.
|
| That may be part of a proposed model, but it's not part of the
| existing model, which (for compatibility) is based on C's model, where
| fields of a (necessarily trivially-constructible) struct can be
| written to as soon as suitable memory for that struct is obtained.

There is only one person who knows what he wanted with he original model (see CC:) and I'm not speaking for him, nor am I pretending to.
However, based on extended past discussions on this very subject I suspect that the current wording does not reflect what is supposed to happen - and I suspect, based on this thread, that sentiment is shared by other people as well.

| If C++ weren't based on C, it would likely have a model where you
| could not access an object without an initialization operation to
| create an object from raw memory.

But it is not clear that the fact that it is based on C means the current wording is what want. I don't believe it is what the original author of C++ wanted.

| But C allows use of raw memory as
| objects, and I don't hear anyone suggesting how we can accommodate
| that while avoiding its horrible consequences. We could choose _not_
| to accommodate it (i.e., require the initialization step that creates
| an object from raw memory), but it's a significant change that
| invalidates massive amounts of existing code.

-- Gaby
Ville Voutilainen
2014-01-16 20:04:24 UTC
Permalink
On 16 January 2014 21:50, Herb Sutter <***@microsoft.com> wrote:
>> Well, as far as I understand, trivial constructors might not be called at all.
>
> Maybe "might not be called" is part of the bug then. Isn't the right model that trivial ctors are called, but "might do nothing"?


Maybe. I don't know how to word it right so that we say that
constructors are conceptually
called. I suppose you're heading to the right direction in the sense
that such wording
should clarify the situation so that you need to construct an object
to start its lifetime,
mere reference/pointer cast won't do. I don't know how this would interact with
suitably aligned buffers and standard-layout types. Perhaps it
interacts with the way
we want, so that just allocating a bag of chars won't begin the
lifetime of whatever
type that would fit into that bag. I also don't know whether this idea
has any impact
on the bane of my existence, aka unions.
Gabriel Dos Reis
2014-01-16 20:00:08 UTC
Permalink
| On 16 January 2014 21:41, Herb Sutter <***@microsoft.com> wrote:
| > Matt quoted:
| >
| >
| >
| > "The lifetime of an object of type T begins when:
| >
| > - storage with the proper alignment and size for type T is obtained, and
| >
| > - if the object has non-trivial initialization, its initialization is
| > complete."
| >
| >
| >
| > Perhaps the necessary fix is to strike "if the object has non-trivial
| > initialization" here. I don't see why trivial constructors are special -
| > sure, they don't do anything, but they are notionally important.
| >
| >
| >
| > A bag of memory of alignment and size suitable for T is-not-a T object until
| > its (possibly trivial) ctor is called on that memory - it seems wrong for
| > that to be true for all T except trivially-constructible T's (with implicit
| > universal type-punning for all types <= sizeof(T)).
|
|
| Well, as far as I understand, trivial constructors might not be called at all.

I think you might be referring to the cases where the default constructor does not necessarily initialize a builtin type (or some other C-style types), like in this case:

#include <string>
struct A {
std::string s;
int i;
};

int main() {
A a; // a.s initialized; but a.i has an indeterminate value
}

however, I think we do have object a, and subobjects a.s and a.i, with well-defined lifetimes.

-- Gaby
Matt Austern
2014-01-16 22:05:33 UTC
Permalink
On Thu, Jan 16, 2014 at 11:41 AM, Herb Sutter <***@microsoft.com> wrote:

> Matt quoted:
>
>
>
> "The lifetime of an object of type T begins when:
>
> — storage with the proper alignment and size for type T is obtained, and
>
> — if the object has non-trivial initialization, its initialization is
> complete."
>
>
>
> Perhaps the necessary fix is to strike “if the object has non-trivial
> initialization” here. I don’t see why trivial constructors are special –
> sure, they don’t do anything, but they are notionally important.
>
>
>
> A bag of memory of alignment and size suitable for T is-not-a T object
> until its (possibly trivial) ctor is called on that memory – it seems wrong
> for that to be true for all T except trivially-constructible T’s (with
> implicit universal type-punning for all types <= sizeof(T)).
>

That's an interesting possibility that I hadn't considered. It's definitely
appealing in some ways, but I think that it, too, would imply some pretty
big surgery in some pretty fundamental parts of the standard. So let's
consider the following code snippet:
struct MyPOD { int x; }; // 1
void* vp = malloc(sizeof(MyPOD)); // 2
MyPOD* p1 = static_cast<MyPOD*>(vp); // 3
MyPOD* p2 = new MyPOD; // 4

So the basic question I'd ask is: after line 4, what are you allowed to do
with p1 and what are allowed to do with p2, and how do the answers differ?
We've got a couple of choices.

First choice: we decide that you aren't allowed to do anything that
involves dereferencing p1, since p1 doesn't point to an object. (No
object's lifetime has begun in line 2 or line 3. I find that unappealing. I
don't think it's the status quo, either in the standard's text, or in what
compilers do, or in what programmers expect. If we forbade it and compiler
implementers took us seriously, it would break the world.

Second choice: we decide that you're allowed to do just the same things
with p1 as you are with p2. (Except, of course, that you're required to
free p1 versus deleting p2.) So *p2 is an object and *p1 isn't, but you're
allowed to use *p1 as if it was an object.

One consequence of the second choice: we'd need to go through the standard
and make sure that the words actually say what we want. I bet we don't
currently have the notion of a contiguous region of storage that isn't an
object of type MyPOD but can be used as if it was an object of type MyPOD.
And, of course, we'd probably have to figure out what the difference is
between an actual object of type MyPOD and a region of storage that isn't
an object but is allowed to be used as if it was an object: if there isn't
any difference between the two concepts then it's a little less clear why
we need both of them.

I'm still leaning toward thinking that, after line 4, both *p1 and *p2 are
objects of type MyPOD. I don't know how the standard should say that,
though. Some change, somewhere, seems called for.

--Matt
Jens Maurer
2014-01-16 22:21:12 UTC
Permalink
On 01/16/2014 11:05 PM, Matt Austern wrote:
> That's an interesting possibility that I hadn't considered. It's definitely appealing in some ways, but I think that it, too, would imply some pretty big surgery in some pretty fundamental parts of the standard. So let's consider the following code snippet:
> struct MyPOD { int x; }; // 1
> void* vp = malloc(sizeof(MyPOD)); // 2
> MyPOD* p1 = static_cast<MyPOD*>(vp); // 3
> MyPOD* p2 = new MyPOD; // 4
>
> So the basic question I'd ask is: after line 4, what are you allowed
> to do with p1 and what are allowed to do with p2, and how do the
> answers differ? We've got a couple of choices.
>
> First choice: we decide that you aren't allowed to do anything that
> involves dereferencing p1, since p1 doesn't point to an object.

Please don't confuse "object" with "lifetime".

1.8p1 says an object is a region of storage. Both *p1 and
*p2 satisfy that.

Further, 3.8p1 seems to say that the lifetime of both *p1 and *p2
has begun.

> Second choice: we decide that you're allowed to do just the same
> things with p1 as you are with p2. (Except, of course, that you're
> required to free p1 versus deleting p2.) So *p2 is an object and *p1
> isn't, but you're allowed to use *p1 as if it was an object.

Both are objects, the current C++ wording leaves no doubt about
that.

> I'm still leaning toward thinking that, after line 4, both *p1 and
> *p2 are objects of type MyPOD. I don't know how the standard should
> say that, though. Some change, somewhere, seems called for.

The bug is more that the lifetimes of objects of arbitrary
type T appear to begin per the words in 3.8p1, which seems wrong.

Jens
Matt Austern
2014-01-16 22:41:27 UTC
Permalink
On Thu, Jan 16, 2014 at 2:21 PM, Jens Maurer <***@gmx.net> wrote:

> On 01/16/2014 11:05 PM, Matt Austern wrote:
> > That's an interesting possibility that I hadn't considered. It's
> definitely appealing in some ways, but I think that it, too, would imply
> some pretty big surgery in some pretty fundamental parts of the standard.
> So let's consider the following code snippet:
> > struct MyPOD { int x; }; // 1
> > void* vp = malloc(sizeof(MyPOD)); // 2
> > MyPOD* p1 = static_cast<MyPOD*>(vp); // 3
> > MyPOD* p2 = new MyPOD; // 4
> >
> > So the basic question I'd ask is: after line 4, what are you allowed
> > to do with p1 and what are allowed to do with p2, and how do the
> > answers differ? We've got a couple of choices.
> >
> > First choice: we decide that you aren't allowed to do anything that
> > involves dereferencing p1, since p1 doesn't point to an object.
>
> Please don't confuse "object" with "lifetime".
>
> 1.8p1 says an object is a region of storage. Both *p1 and
> *p2 satisfy that.
>

That's an interesting observation. It's also true, of course, that
void* vp = malloc(12);
creates a region of storage and thus, by the definition in 1.8p1,
creates an object. I'm a little reluctant to say that this line
creates an object of type T, though, no matter what T is. Maybe
we have subtly different concepts of "object", "object of type T",
and "object whose lifetime has begun". Or maybe we need to
make those subtle distinctions even we aren't currently making
them.


>
> Further, 3.8p1 seems to say that the lifetime of both *p1 and *p2
> has begun.


I agree that 3.8p1 seems to say that. Not everyone seems to
agree that it says that, though; some people also seem to think
that even if it does say that, it shouldn't. I sympathize. That
reading of 3.8p1 has some surprising consequences. (But every
proposed alternative I can think of seems to have some
surprising consequences too.)

--Matt
Gabriel Dos Reis
2014-01-16 23:26:52 UTC
Permalink
Excellent summary! Thanks.

-- Gaby

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Matt Austern
Sent: Thursday, January 16, 2014 2:41 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Thu, Jan 16, 2014 at 2:21 PM, Jens Maurer <***@gmx.net<mailto:***@gmx.net>> wrote:
On 01/16/2014 11:05 PM, Matt Austern wrote:
> That's an interesting possibility that I hadn't considered. It's definitely appealing in some ways, but I think that it, too, would imply some pretty big surgery in some pretty fundamental parts of the standard. So let's consider the following code snippet:
> struct MyPOD { int x; }; // 1
> void* vp = malloc(sizeof(MyPOD)); // 2
> MyPOD* p1 = static_cast<MyPOD*>(vp); // 3
> MyPOD* p2 = new MyPOD; // 4
>
> So the basic question I'd ask is: after line 4, what are you allowed
> to do with p1 and what are allowed to do with p2, and how do the
> answers differ? We've got a couple of choices.
>
> First choice: we decide that you aren't allowed to do anything that
> involves dereferencing p1, since p1 doesn't point to an object.
Please don't confuse "object" with "lifetime".

1.8p1 says an object is a region of storage. Both *p1 and
*p2 satisfy that.

That's an interesting observation. It's also true, of course, that
void* vp = malloc(12);
creates a region of storage and thus, by the definition in 1.8p1,
creates an object. I'm a little reluctant to say that this line
creates an object of type T, though, no matter what T is. Maybe
we have subtly different concepts of "object", "object of type T",
and "object whose lifetime has begun". Or maybe we need to
make those subtle distinctions even we aren't currently making
them.


Further, 3.8p1 seems to say that the lifetime of both *p1 and *p2
has begun.

I agree that 3.8p1 seems to say that. Not everyone seems to
agree that it says that, though; some people also seem to think
that even if it does say that, it shouldn't. I sympathize. That
reading of 3.8p1 has some surprising consequences. (But every
proposed alternative I can think of seems to have some
surprising consequences too.)

--Matt
Herb Sutter
2014-01-16 19:38:25 UTC
Permalink
Richard, it cannot mean that (or if it does, IMO we have an obvious bug) for at least two specific reasons I can think of (below), besides the general reasons that it would not be sensical and would violate type safety.

First, objects must have unique addresses. Consider, still assuming B is trivially constructible:
void *p = malloc(sizeof(B));
B* pb = (B*)p;
pb->i = 0;
short* ps = (short*)p;
*ps = 0;
This cannot possibly be construed as starting the lifetime of a B object and a short object, else they would have the same address, which is illegal. Am I missing something?

Second, this function:

template<typename T> T* test() {
auto p = (T*)malloc(sizeof(T));
new (p) T{};
return p;
}

clearly returns a pointer to a T object for any default-constructible type T – and that has to be true whether or not T is trivially constructible.

If the current rules are as Richard says, then it would true that test() returns a pointer to a T, unless T is trivially constructible in which case it returns a pointer to anything of size T. That doesn’t make any sense, and if the rules say that now I’d say we have a defect.


The more I see about this, the more I think the lifetime wording (a) is correct for non-trivial-ctor cases and (b) is incomplete or incorrect for trivial-ctor cases.

In particular, untyped storage (such as returned from malloc) should never be considered of any type T until after the possibly-trivial T::T has run, such as via placement new. Note I include “possibly-trivial” there on purpose. If you don’t call a T ctor, you don’t have a T object – that has to be true. Otherwise don’t we have multiple contradictions in the standard?

“It’s not a T until the T ctor runs” has to be part of the rules somewhere. It is there for non-trivially-constructible types.

Do we have a defect here that we’re missing that rule for trivially-constructible types?

Herb


From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Richard Smith
Sent: Wednesday, January 15, 2014 4:52 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Wed, Jan 15, 2014 at 4:32 PM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:
I would expect that that we would have consensus that the lifetime of an object of type D *should* start at some point in this code (and more specifically that the code has defined behavior):

B *p = (B*)malloc(sizeof(B));
p->i = 0;


(blink) D is not even mentioned, so why would we expect that? Can you explain?

Sorry, editing error, I meant 'B' here.

If the standard said the lifetime of a D object started somewhere here, I would wonder if there’s a defect in the lifetime wording. That would seem to violate sensible notions of type safety.

Playing devil's advocate again: it would be entirely unobservable whether an object of type 'B' or 'D' actually had its lifetime start here. Why would you be surprised if you got a 'D' instead of a 'B'? Indeed, if later I wrote:

D *q = (D*)p; // ha ha, it was a D object the whole time!
int n = q->i;

... why should that /not/ work? How is this any different from if I cast to D* before I used the object as a B?

From: ub-***@open-std.org<mailto:ub-***@open-std.org> [mailto:ub-***@open-std.org<mailto:ub-***@open-std.org>] On Behalf Of Richard Smith
Sent: Wednesday, January 15, 2014 3:09 PM

To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Wed, Jan 15, 2014 at 1:48 PM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:

Richard, I'm not sure I understand your position... Given the following complete program ...


struct B { int i; };
struct D : B { };

int main() {
B b; // line X
}

... are you actually saying that line X starts the lifetime of an object of type D? or just setting up a strawman? (Sorry, I really couldn't tell.)

If yes, then given the following complete program ...

struct C { long long i; };

int main() {
C c; // line Y
}

... are you saying that line Y could start the lifetime of an object of type D (which is not mentioned in the code), double, shared_ptr<widget>, or any other type than C, as long as the size of that other type is the same or less than sizeof(C)?

I think my position is more nuanced. There are a set of cases that I think people expect to have defined behavior, and the standard does not currently distinguish those cases from your cases above, as far as I can tell -- if using malloc to create an object "with trivial initialization" is acceptable, then it appears that *any* way of producing the appropriately-sized-and-aligned storage is acceptable.


I would expect that that we would have consensus that the lifetime of an object of type D *should* start at some point in this code (and more specifically that the code has defined behavior):

B *p = (B*)malloc(sizeof(B));
p->i = 0;

I would expect to also have consensus that the same is true here:

alignof(B) char buffer[sizeof(B)];
B *p = (B*)buffer;
p->i = 0;

(These expectations are based on the vast amount of existing code that relies upon these behaviors, not on the wording of the standard.)

So, that's "what people probably expect". Next, "what the standard says". Consider 3.8/1:

"The lifetime of an object of type T begins when:
— storage with the proper alignment and size for type T is obtained, and
— if the object has non-trivial initialization, its initialization is complete."

This requires us to answer three questions: (1) is there an object of type B in the above snippets, (2) when is storage for it obtained, and (3) does it have non-trivial initialization? It seems that "yes, before the storage is used as an object of type B, no" is a self-consistent set of answers, and one that gives the program defined behavior. There are also sets of answers that give the program undefined behavior, and the standard doesn't give us direction in how we might pick a set of answers to these questions.

There seem to be two obvious ways forward: either (a) if we can pick answers to these questions such that the program has defined behavior, then the program has defined behavior, or (b) if we can pick answers to these questions such that the program has undefined behavior, then the program has undefined behavior.

If we want to align "what people probably expect" with "what the standard says", it seems we need to either change the above rule, or accept interpretation (a), under which the program above is valid, as is any other program where 'buffer' obtains storage of the appropriate size and alignment for an object of type D. (Option (a) also matches the behavior of current optimizing compilers, as far as I'm aware.)


I've spent quite some time thinking about and discussing this problem and related issues (such as, under what circumstances does a pointer point to an object, when do two pointers alias, ...), and I personally think that the best approach here is to embrace option (a) above: if there exist a consistent set of choices of object lifetimes such that the program has defined behavior, then the program has the behavior implied by that set. (I have a sketch proof that such behavior is the same for *every* such consistent set, aside from deviations caused by unspecified values and other pre-existing sources of nondeterminism.) In essence, the implication of this is that objects' lifetimes would start just in time to avoid undefined behavior.

Put another way, yes, I personally think this code should have defined behavior:

C c; // #1
static_assert(sizeof(C) >= sizeof(D) && alignof(C) >= alignof(D), "");
D *p = (D*)&c;
d->i = 0; // #2

... and the lifetime of a D object at address &c should start at some point between lines #1 and #2. (Naturally, the lifetime of the C object ended before this happened.) Moreover, this is something that plenty of existing C++ code relies on.

From: ub-***@open-std.org<mailto:ub-***@open-std.org> <ub-***@open-std.org<mailto:ub-***@open-std.org>> on behalf of Richard Smith <***@google.com<mailto:***@google.com>>
Sent: Monday, January 6, 2014 3:44 PM

To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On Mon, Jan 6, 2014 at 10:22 AM, Jason Merrill <***@redhat.com<mailto:***@redhat.com>> wrote:
On 01/06/2014 04:26 AM, Fabio Fracassi wrote:
> if it is not (legal): could we make it legal or would we run afoul of
> the aliasing rules?
The access is not allowed by the aliasing rules in 3.10. But it seems
that this would be:

struct B {
int i;
};

struct D {
B bmem;
void foo() { /* access bmem.i */ }
};

B b;
reinterpret_cast<D&>(b).foo();

because B is a non-static data member of D, and 9.2/19 guarantees that
the address of D::bmem is the same as the address of the D object.

How is that fundamentally different? 9.3.1/2 makes that UB too, if 'reinterpret_cast<D&>(b)' does not refer to an object of type 'b'. And within D::foo, the implicit this->bmem would have the same problem.


If I might play devil's advocate for a moment...

struct B { int i; };
struct D : B {
void foo();
};

B b;

I claim this line starts the lifetime of an object of type D. Per [basic.life]p1, the lifetime of an object of type 'D' begins when storage with the proper alignment and size for type T is obtained (which "B b" happens to satisfy). The object does not have non-trivial initialization, so the second bullet does not apply.

(This is the same argument that makes this valid:

D *p = (D*)malloc(sizeof(D));
p->foo();

... so any counterargument will need to explain why the two cases are fundamentally different.)

Then:

reinterpret_cast<D&>(b).foo();

... is valid, because the cast produces the same memory address, and that memory address contains an object of type 'D' (as claimed above).

_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub

_______________________________________________
ub mailing list
***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
http://www.open-std.org/mailman/listinfo/ub
James Dennett
2014-01-17 07:49:43 UTC
Permalink
On Thu, Jan 16, 2014 at 11:38 AM, Herb Sutter <***@microsoft.com> wrote:
> “It’s not a T until the T ctor runs” has to be part of the rules somewhere.

That's be nice, but it's hard to make it compatible with C, where you
can get a value of type T either by declaring one, or by copying an
existing one (via assignment, or via memcpy/memmove), and maybe in
other ways that I'm not remembering. Certainly C code assumes that
recreating the same bytes is sufficient to be allowed to use them as a
T object.

> It is there for non-trivially-constructible types.
>
> Do we have a defect here that we’re missing that rule for
> trivially-constructible types?

I think not, because (as I see it) such a rule would be a defect in
that it would invalidate much valid C/C++ code.

int *p = (int*)malloc(sizeof(int));
*p = 42;
*p = 0;

After this, there is an int, and p points to it, and at no time was
any constructor (trivial or otherwise) used. C doesn't have
constructors. C has regions of memory and effective types, and the
act of writing to a piece of memory sets its effective type.

We could say that each assignment above constructs an int first, but
then we'd also want to say that it destroys the previous int... except
that there wasn't an int there until the first assignment put one
there. It cannot be a precondition of "*p = 0" that p points at an
int, and it can't be a precondition that it doesn't. Both worked just
fine in C; assignment was memcpy, and gave the same object
representation, and hence the same value. (I seem to recall reading
C's rules and finding that they are also self-contradictory when read
in more depth, but I certainly lack motivation to try to fix that.)
Maybe we could say that "*p = 0" ends the lifetime of the
trivially-destructible object that was previously at p, if there was
one (but not one of type int). Seems fragile to me, but it
approximates what C permits.

I agree that what C++ says is untenable (allocating storage really
can't start the lifetime of objects of types that aren't even
expressed in code at the time the allocation happens), but C didn't
distinguish between overwriting a bunch of bytes and overwriting a
value of a type. C++ approximately mirrors C in saying that "An
object is a region of storage", but we only half mean it. We have two
notions of what objects are -- one mostly inherited from C, and a new,
arguably-cleaner one added by C++. Where the two collide we get
unpleasant quirks, like the issue raised some while ago (maybe by
Gaby) where p->~int() is explicitly a no-op, and doesn't end the
lifetime of the int.

-- James
Gabriel Dos Reis
2014-01-17 15:05:13 UTC
Permalink
| -----Original Message-----
| From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
| Behalf Of James Dennett
| Sent: Thursday, January 16, 2014 11:50 PM
| To: WG21 UB study group
| Subject: Re: [ub] type punning through congruent base class?
|
| On Thu, Jan 16, 2014 at 11:38 AM, Herb Sutter <***@microsoft.com>
| wrote:
| > “It’s not a T until the T ctor runs” has to be part of the rules somewhere.
|
| That's be nice, but it's hard to make it compatible with C, where you
| can get a value of type T either by declaring one, or by copying an
| existing one (via assignment, or via memcpy/memmove), and maybe in
| other ways that I'm not remembering.

I think we can make that work.
I already proposed that if you memcpy or memmove into a storage with dynamic storage then you're also transferring dynamic type. (That is close to C's transfer of effective type.)
The point is that we do have the conceptual notion of constructor; the question is what the notation is? For the vast majority of the cases, I think the rule has to be what Herb is suggesting. And then, we can take care of the rest, special cases. The next thing is to address external data structures -- even C does not appear to have answers for that, despite some programmers believing it does :-)

| Certainly C code assumes that
| recreating the same bytes is sufficient to be allowed to use them as a
| T object.
|
| > It is there for non-trivially-constructible types.
| >
| > Do we have a defect here that we’re missing that rule for
| > trivially-constructible types?
|
| I think not, because (as I see it) such a rule would be a defect in
| that it would invalidate much valid C/C++ code.
|
| int *p = (int*)malloc(sizeof(int));
| *p = 42;
| *p = 0;
|
| After this, there is an int, and p points to it, and at no time was
| any constructor (trivial or otherwise) used. C doesn't have
| constructors. C has regions of memory and effective types, and the
| act of writing to a piece of memory sets its effective type.

Yes, but we have to be careful about what we mean by that. Writing to individual members of what is supposed to be a structure does not set the effective type of the structure, until an access is made as a structure.

| We could say that each assignment above constructs an int first, but
| then we'd also want to say that it destroys the previous int... except
| that there wasn't an int there until the first assignment put one
| there. It cannot be a precondition of "*p = 0" that p points at an
| int, and it can't be a precondition that it doesn't. Both worked just
| fine in C; assignment was memcpy, and gave the same object
| representation, and hence the same value. (I seem to recall reading
| C's rules and finding that they are also self-contradictory when read
| in more depth, but I certainly lack motivation to try to fix that.)

We can say that the first write to *p sets the dynamic type to int, and everything else is assignment.

C's model isn't without its own problems. I believe Xavier Leroy (the author of CompCert) worked to make sense of most of it, but last time we discussed these issues (about 2 years ago, so a lot may have happened since then) we were still trying to understand what it really means in C to have a value of structure type -- this is in connection with modelling the semantics of calling a function that accepts or returns a structure by value. Xavier is on the list, so I trust he would chime in when he gets a chance to work his way through the blizzard of messages.

| Maybe we could say that "*p = 0" ends the lifetime of the
| trivially-destructible object that was previously at p, if there was
| one (but not one of type int). Seems fragile to me, but it
| approximates what C permits.

I think this works too, and may be a way out of this mess. It definitely needs more exploring.

| I agree that what C++ says is untenable (allocating storage really
| can't start the lifetime of objects of types that aren't even
| expressed in code at the time the allocation happens), but C didn't
| distinguish between overwriting a bunch of bytes and overwriting a
| value of a type.

Yup.

| C++ approximately mirrors C in saying that "An
| object is a region of storage", but we only half mean it. We have two
| notions of what objects are -- one mostly inherited from C, and a new,
| arguably-cleaner one added by C++.

I don't think that C++ has ever had a notion of object that didn't come with a type, e.g. constructor/initialization. And that was a deliberate design decision and fundamental principle. The way I've always heard Bjarne expressed this is: a constructor turns raw memory into object, and a destructor turns an object back to mere memory. Also, see slide page 16 of his talk "Foundations of C++" at ETAPS 2012:

http://cs.ioc.ee/etaps12/invited/stroustrup-slides.pdf


| Where the two collide we get
| unpleasant quirks, like the issue raised some while ago (maybe by
| Gaby) where p->~int() is explicitly a no-op, and doesn't end the
| lifetime of the int.

Yes, that is right. We need to fix that too -- it was an unfortunate optimization. We had a sustained discussion when I brought it up, b
David Krauss
2014-01-17 07:56:11 UTC
Permalink
On Jan 17, 2014, at 3:38 AM, Herb Sutter <***@microsoft.com> wrote:

> Richard, it cannot mean that (or if it does, IMO we have an obvious bug) for at least two specific reasons I can think of (below), besides the general reasons that it would not be sensical and would violate type safety.
>
> First, objects must have unique addresses. Consider, still assuming B is trivially constructible:
> void *p = malloc(sizeof(B));
> B* pb = (B*)p;
> pb->i = 0;
> short* ps = (short*)p;
> *ps = 0;
> This cannot possibly be construed as starting the lifetime of a B object and a short object, else they would have the same address, which is illegal. Am I missing something?


1.8/6 applies here:

Two objects that are not bit-fields may have the same address if one is a subobject of the other,


Even without that, it’s not clear what the meaning of a POD object is. You can arbitrarily say that the lifetimes of two aliased objects begin and end however you like, as the lifetime of a complete object does not seem to dictate the lifetime of its subobjects.

struct q { int m; };
struct r { int n; };

static_assert ( sizeof (q) == sizeof (int), “” );

int main() {
q * pq = (q *) malloc( sizeof (int) ); // Nothing here implies a lifetime.
* (int *) pq = 5; // Begin lifetime of int object.
r * pr = (r *) pq; // I decree that an r comes to life here.
std::cout << pr->n << ‘\n’; // The int is still alive, though.
// OK, now I want the r lifetime to end and a q lifetime to begin.
// No normative rule says not, because storage is already obtained.
// The int continues to live on as initialized.
std::cout << pq->m << ‘\n’;
}

The catch (or saving grace) is that the reinterpret_cast only works for the first subobject of a standard layout class or an array. Although, I don’t see why you couldn’t just assert that the offsets or member subobject pointer values match (as void*) in other cases.

Even if the lifetime of an object is passed on to its subobjects, the object representation must be shared by the ints and that guarantees identical values, or if you prefer, pre-initialization.

> The more I see about this, the more I think the lifetime wording (a) is correct for non-trivial-ctor cases and (b) is incomplete or incorrect for trivial-ctor cases.
> In particular, untyped storage (such as returned from malloc) should never be considered of any type T until after the possibly-trivial T::T has run, such as via placement new. Note I include “possibly-trivial” there on purpose. If you don’t call a T ctor, you don’t have a T object – that has to be true. Otherwise don’t we have multiple contradictions in the standard?

C++ conspicuously does not require a new-expression to begin the lifetime of a trivially-constructible class. I’ve always assumed this to be to bless malloc-based C code, and by extension libraries written in C. So, what meaning can be assigned to the constructor?

The current rules seem to be very loose, but the loophole should be tightened very carefully because a lot of code must depend on it.

I’m not sure whether my example above should be deprecated, but if so, there might be something about a minimal lifetime for an object, or some sort of end-of-lifetime event that is required between “aliased" uses of the same storage as different class types, which conceptually taints the object representation.

I wrote an earlier post to this thread about aliasing and lvalue-to-rvalue conversion but it wasn’t distributed (or bounced) by the listserv. Hopefully this message fares better.
David Krauss
2014-01-17 09:01:35 UTC
Permalink
> Even without that, it’s not clear what the meaning of a POD object is. You can arbitrarily say that the lifetimes

Sorry, not clear what the meaning of a *lifetime* of a POD object is. POD objects are pretty well defined :P
Gabriel Dos Reis
2014-01-17 12:42:33 UTC
Permalink
In C++, there is a relationship between the lifetime of a complete object and that of its subobjects - the only exception I know of is that of arrays, which has it th opposite way (wrong!) and is on my list of things to get fixed (already reported that in the past.) I don't understand the bit about new-expression.

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of David Krauss
Sent: Thursday, January 16, 2014 11:56 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?


On Jan 17, 2014, at 3:38 AM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:


Richard, it cannot mean that (or if it does, IMO we have an obvious bug) for at least two specific reasons I can think of (below), besides the general reasons that it would not be sensical and would violate type safety.

First, objects must have unique addresses. Consider, still assuming B is trivially constructible:
void *p = malloc(sizeof(B));
B* pb = (B*)p;
pb->i = 0;
short* ps = (short*)p;
*ps = 0;
This cannot possibly be construed as starting the lifetime of a B object and a short object, else they would have the same address, which is illegal. Am I missing something?


1.8/6 applies here:


1. Two objects that are not bit-fields may have the same address if one is a subobject of the other,

Even without that, it's not clear what the meaning of a POD object is. You can arbitrarily say that the lifetimes of two aliased objects begin and end however you like, as the lifetime of a complete object does not seem to dictate the lifetime of its subobjects.

struct q { int m; };
struct r { int n; };

static_assert ( sizeof (q) == sizeof (int), "" );

int main() {
q * pq = (q *) malloc( sizeof (int) ); // Nothing here implies a lifetime.
* (int *) pq = 5; // Begin lifetime of int object.
r * pr = (r *) pq; // I decree that an r comes to life here.
std::cout << pr->n << '\n'; // The int is still alive, though.
// OK, now I want the r lifetime to end and a q lifetime to begin.
// No normative rule says not, because storage is already obtained.
// The int continues to live on as initialized.
std::cout << pq->m << '\n';
}

The catch (or saving grace) is that the reinterpret_cast only works for the first subobject of a standard layout class or an array. Although, I don't see why you couldn't just assert that the offsets or member subobject pointer values match (as void*) in other cases.

Even if the lifetime of an object is passed on to its subobjects, the object representation must be shared by the ints and that guarantees identical values, or if you prefer, pre-initialization.


The more I see about this, the more I think the lifetime wording (a) is correct for non-trivial-ctor cases and (b) is incomplete or incorrect for trivial-ctor cases.
In particular, untyped storage (such as returned from malloc) should never be considered of any type T until after the possibly-trivial T::T has run, such as via placement new. Note I include "possibly-trivial" there on purpose. If you don't call a T ctor, you don't have a T object - that has to be true. Otherwise don't we have multiple contradictions in the standard?

C++ conspicuously does not require a new-expression to begin the lifetime of a trivially-constructible class. I've always assumed this to be to bless malloc-based C code, and by extension libraries written in C. So, what meaning can be assigned to the constructor?

The current rules seem to be very loose, but the loophole should be tightened very carefully because a lot of code must depend on it.

I'm not sure whether my example above should be deprecated, but if so, there might be something about a minimal lifetime for an object, or some sort of end-of-lifetime event that is required between "aliased" uses of the same storage as different class types, which conceptually taints the object representation.

I wrote an earlier post to this thread about aliasing and lvalue-to-rvalue conversion but it wasn't distributed (or bounced) by the listserv. Hopefully this message fares better.
David Krauss
2014-01-17 13:45:37 UTC
Permalink
On Jan 17, 2014, at 8:42 PM, Gabriel Dos Reis <***@microsoft.com> wrote:

> In C++, there is a relationship between the lifetime of a complete object and that of its subobjects – the only exception I know of is that of arrays, which has it th opposite way (wrong!) and is on my list of things to get fixed (already reported that in the past.) I don’t understand the bit about new-expression.

It sounds like both cases are covered by §3.8/2:
[ Note: The lifetime of an array object starts as soon as storage with proper size and alignment is obtained, and its lifetime ends when the storage which the array occupies is reused or released. 12.6.2 describes the lifetime of base and member subobjects. — end note ]

As for arrays, yes that sounds backwards. If the lifetime of the whole array starts before initialization finishes, that might imply something crazy like the ability to destroy all of it in the middle of initialization. (I don’t know what normative text makes the note correct, though.)

As for objects of class types, [class.base.init] 12.6.2 only seems to mention initialization, not lifetimes in the abstract sense, but I was only talking about objects with trivial initialization, for which initialization plays no part in determining lifetime.

Am I missing something else?
Gabriel Dos Reis
2014-01-17 13:53:58 UTC
Permalink
One of the things that bothers me with the array thing is that conceptually, an array is just like a structure with homogeneous elements (and we do actually tend to think of it that way in certain parts of the library, e.g. see std::complex). Furthermore, I don't see the point of this "exception".

As for an object type with "trivial initialization", I suspect that the general confusion comes from the fact for some reasons the standards text assumed that it could just "optimize" that case without much confusion. But evidence appear to suggest otherwise. Complexity has grown, with increased likelihood of inconsistency. What is the meaning of T{ } when T is a POD struct? Do you think there is no lifetime there?

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of David Krauss
Sent: Friday, January 17, 2014 5:46 AM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?


On Jan 17, 2014, at 8:42 PM, Gabriel Dos Reis <***@microsoft.com<mailto:***@microsoft.com>> wrote:


In C++, there is a relationship between the lifetime of a complete object and that of its subobjects - the only exception I know of is that of arrays, which has it th opposite way (wrong!) and is on my list of things to get fixed (already reported that in the past.) I don't understand the bit about new-expression.

It sounds like both cases are covered by §3.8/2:

1. [ Note: The lifetime of an array object starts as soon as storage with proper size and alignment is obtained, and its lifetime ends when the storage which the array occupies is reused or released. 12.6.2 describes the lifetime of base and member subobjects. - end note ]
As for arrays, yes that sounds backwards. If the lifetime of the whole array starts before initialization finishes, that might imply something crazy like the ability to destroy all of it in the middle of initialization. (I don't know what normative text makes the note correct, though.)

As for objects of class types, [class.base.init] 12.6.2 only seems to mention initialization, not lifetimes in the abstract sense, but I was only talking about objects with trivial initialization, for which initialization plays no part in determining lifetime.

Am I missing something else?
David Krauss
2014-01-17 15:16:19 UTC
Permalink
On Jan 17, 2014, at 9:53 PM, Gabriel Dos Reis <***@microsoft.com> wrote:

> As for an object type with “trivial initialization”, I suspect that the general confusion comes from the fact for some reasons the standards text assumed that it could just “optimize” that case without much confusion. But evidence appear to suggest otherwise. Complexity has grown, with increased likelihood of inconsistency. What is the meaning of T{ } when T is a POD struct? Do you think there is no lifetime there?


My impression (for what it’s worth) is that a lifetime is required to exist in a retroactive sense around every subobject access. But the lifetime of a POD object does nothing but grant the possibility of accessing a subobject, which in turn has no special properties or invariants by virtue of the complete object.

So it’s not to say there’s no lifetime there, but POD lifetime is a meaningless concept because you can always just pull one out of thin air. (T{} is value-initialized, and assigning those zeroes does imply subobject accesses to be sure.)

What evidence otherwise? It sounds like Richard is trying to push the envelope, but one would have to review existing verification/test tools to see if anyone has really tried to do anything with POD lifetimes. In a very real sense, POD objects are handed to any C++ program by the OS libraries in a fashion opaque to (outside the scope of) most verification tools. Programs need original input data to come out of thin air, and POD provides a way to structure such data while preserving its bytestream-like purity.
Gabriel Dos Reis
2014-01-17 16:37:58 UTC
Permalink
I really don't understand that you mean by 'lifetime is a meaningless concept for POD'. The C standards has had several aliasing bugs related to lifetime. I also do not understand "lifetime is required to exist in a retroactive sense around every subobject access." I think we have a lot of confusion already.

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of David Krauss
Sent: Friday, January 17, 2014 7:16 AM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?


On Jan 17, 2014, at 9:53 PM, Gabriel Dos Reis <***@microsoft.com<mailto:***@microsoft.com>> wrote:


As for an object type with "trivial initialization", I suspect that the general confusion comes from the fact for some reasons the standards text assumed that it could just "optimize" that case without much confusion. But evidence appear to suggest otherwise. Complexity has grown, with increased likelihood of inconsistency. What is the meaning of T{ } when T is a POD struct? Do you think there is no lifetime there?


My impression (for what it's worth) is that a lifetime is required to exist in a retroactive sense around every subobject access. But the lifetime of a POD object does nothing but grant the possibility of accessing a subobject, which in turn has no special properties or invariants by virtue of the complete object.

So it's not to say there's no lifetime there, but POD lifetime is a meaningless concept because you can always just pull one out of thin air. (T{} is value-initialized, and assigning those zeroes does imply subobject accesses to be sure.)

What evidence otherwise? It sounds like Richard is trying to push the envelope, but one would have to review existing verification/test tools to see if anyone has really tried to do anything with POD lifetimes. In a very real sense, POD objects are handed to any C++ program by the OS libraries in a fashion opaque to (outside the scope of) most verification tools. Programs need original input data to come out of thin air, and POD provides a way to structure such data while preserving its bytestream-like purity.
Matt Austern
2014-01-17 17:54:28 UTC
Permalink
On Thu, Jan 16, 2014 at 11:38 AM, Herb Sutter <***@microsoft.com> wrote:

> Richard, it cannot mean that (or if it does, IMO we have an obvious bug)
> for at least two specific reasons I can think of (below), besides the
> general reasons that it would not be sensical and would violate type safety.
>
>
>
> First, objects must have unique addresses. Consider, still assuming B is
> trivially constructible:
>
> void *p = malloc(sizeof(B));
>
> B* pb = (B*)p;
> pb->i = 0;
>
> short* ps = (short*)p;
> *ps = 0;
>
> This cannot possibly be construed as starting the lifetime of a B object
> and a short object, else they would have the same address, which is
> illegal. Am I missing something?
>
>
>
> Second, this function:
>
>
>
> template<typename T> T* test() {
>
> auto p = (T*)malloc(sizeof(T));
>
> new (p) T{};
>
> return p;
>
> }
>
>
>
> clearly returns a pointer to a T object for any default-constructible type
> T – and that has to be true whether or not T is trivially constructible.
>
>
>
> If the current rules are as Richard says, then it would true that test()
> returns a pointer to a T, unless T is trivially constructible in which case
> it returns a pointer to anything of size T. That doesn’t make any sense,
> and if the rules say that now I’d say we have a defect.
>
>
>
>
>
> The more I see about this, the more I think the lifetime wording (a) is
> correct for non-trivial-ctor cases and (b) is incomplete or incorrect for
> trivial-ctor cases.
>
>
>
> In particular, untyped storage (such as returned from malloc) should never
> be considered of any type T *until after the possibly-trivial T::T has
> run*, such as via placement new. Note I include “possibly-trivial” there
> on purpose. If you don’t call a T ctor, you don’t have a T object – that
> *has* to be true. Otherwise don’t we have multiple contradictions in the
> standard?
>
>
>
> “It’s not a T until the T ctor runs” has to be part of the rules
> somewhere. It is there for non-trivially-constructible types.
>
>
>
> Do we have a defect here that we’re missing that rule for
> trivially-constructible types?
>

I agree that something is missing from the rules. I'm not convinced that
"it's not a T until the T ctor runs" is the right fix. Mostly it's just
that I don't understand what the consequences of that rule would be, and
the standard would have to say something about those consequences. So
consider the following snippet:
struct T { int x; };
T x{17}; // 1
void* pv = malloc(sizeof(T)); // 2
T* p = static_cast<T*>(pv); // 3
*p = x; // 4

I'm trying to understand what your rule would imply for that snippet. Does
your rule imply that line 4 is undefined behavior (since, according to your
proposed rule, *p isn't an object of type T)? Or does your rule imply that
*p is an object of type T after line 4 (maybe an assignment might count as
the moral equivalent of a constructor)? Or does your rule imply that this
code snippet is well defined, and that *p may be used to do (some of) the
things you can do with an object of type T but that it is not in fact an
object of type T?

I'm afraid I'm just not sure this is a fruitful line of reasoning. All
three of those possible followups to your rule have odd consequences. Maybe
there's a fourth option I'm not seeing, or maybe it means we need a
different fix.

--Matt
Jeffrey Yasskin
2014-01-17 21:34:08 UTC
Permalink
Note that this post from Herb arrived after
http://www.open-std.org/pipermail/ub/2014-January/000418.html but was sent
before, so the thread got a little mixed up.

On Thu, Jan 16, 2014 at 11:38 AM, Herb Sutter <***@microsoft.com> wrote:

> Richard, it cannot mean that (or if it does, IMO we have an obvious bug)
> for at least two specific reasons I can think of (below), besides the
> general reasons that it would not be sensical and would violate type safety.
>
>
We do have an obvious bug in [basic.life]p1, "The lifetime of an object of
type T begins when storage with the proper alignment and size for type T is
obtained", if we interpret "obtained" as "obtained from the memory
allocator". Even with strict uses of placement-new to change the type of
memory, placement-new doesn't "obtain" any memory. If we interpret
"obtained" as just "the programmer intends a region of storage to be
available for a T", as I think Richard is suggesting, the bug is only that
we need the wording to be clearer.

First, objects must have unique addresses. Consider, still assuming B is
> trivially constructible:
>
> void *p = malloc(sizeof(B));
>

The lifetime of a B starts some time after-or-including the malloc() call
in the above line and the access of 'pb->i' two lines down. [basic.life]p5
("Before the lifetime of an object has started ... The program has undefined
behavior if ... the pointer is used to access a non-static data member")

The assignment to 'i' might start the lifetime of an 'int' subobject, but
that's not enough to make the use of 'pb->i' defined if no 'B's lifetime
has started.


> B* pb = (B*)p;
> pb->i = 0;
>

The lifetime of the B *ends* when its storage is re-used for the 'short'
([basic.life]p1 "The lifetime of an object of type T ends when ... the
storage which the object occupies is reused"), as Daveed said. This happens
some time after the access in the previous line, and the assignment two
lines down.


> short* ps = (short*)p;
> *ps = 0;
>
> This cannot possibly be construed as starting the lifetime of a B object
> and a short object, else they would have the same address, which is
> illegal. Am I missing something?
>

Both a B object and a short object have their lifetimes started in your
code snippet, but the lifetimes don't overlap.

Confusingly, the start of these lifetimes is *not* called out in any
particular line of code; it's implied by them. In particular, the casts
don't have any lifetime effects (contra the straw man at
http://www.open-std.org/pipermail/ub/2014-January/000406.html). The code
would be just as defined (or undefined) written as:

void *p = malloc(sizeof(B));

B* pb = (B*)p;
short* ps = (short*)p;
pb->i = 0;

*ps = 0;


As Matt alluded to in
http://www.open-std.org/pipermail/ub/2014-January/000456.html, it might be
possible to say that all lifetime effects are called out in explicit
expressions without breaking C compatibility, *if* we instead say that
accessing the members of objects with trivial constructors can be done
outside of the lifetime of such objects. I have no idea whether that would
be better or worse than saying that lifetime effects can be implied.


Jeffrey
Herb Sutter
2014-01-17 22:56:12 UTC
Permalink
> Note that this post from Herb arrived after http://www.open-std.org/pipermail/ub/2014-January/000418.html but was sent before, so the thread got a little mixed up.

Yes, I've been trying to reply less on this thread until that sync'ed back up. :)

From what I've learned in this thread, the (rough) intended C++ model for PODs (assuming memory of the right size/alignment) would seem to be "the lifetime of a B starts when you write to the memory as a B, and ends when you free the memory or write to the memory as different type." [Disclaimer: I'm not sure if "read from the memory as a B" also starts lifetime."]

I think we can do better, but it seems like that's the (rough) intent of the status quo, leaving aside the question of whether the wording actually says that.

*If* that is the (rough) intent, then in:

void *p = malloc(sizeof(B)); // 1
B* pb = (B*)p; // 2
pb->i = 0; // 3
short* ps = (short*)p; // 4
*ps = 0; // 5
free(p); // 6


I assume that the reasoning would be that:

* line 3 starts the lifetime of a B (we're writing to the bits of a B member, not just any int)
* line 5 ends the lifetime of that B and begins the lifetime of a short
* line 6 ends the lifetime of that short


Again ignoring whether this is desirable, is that (roughly) the intent of the current wording?


If yes, does the wording express it (a) accurately and (b) clearly?


Finally, regardless of the above answer, do we want to change anything about the legality or semantics of the above type-punning code, such as possibly having a "type-safe mode" where such code is somehow not allowed unless in an "extern "C-compat"" block or something?


Herb



________________________________
From: ub-***@open-std.org <ub-***@open-std.org> on behalf of Jeffrey Yasskin <***@google.com>
Sent: Friday, January 17, 2014 1:34 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

Note that this post from Herb arrived after http://www.open-std.org/pipermail/ub/2014-January/000418.html but was sent before, so the thread got a little mixed up.

On Thu, Jan 16, 2014 at 11:38 AM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:
Richard, it cannot mean that (or if it does, IMO we have an obvious bug) for at least two specific reasons I can think of (below), besides the general reasons that it would not be sensical and would violate type safety.

We do have an obvious bug in [basic.life]p1, "The lifetime of an object of type T begins when storage with the proper alignment and size for type T is obtained", if we interpret "obtained" as "obtained from the memory allocator". Even with strict uses of placement-new to change the type of memory, placement-new doesn't "obtain" any memory. If we interpret "obtained" as just "the programmer intends a region of storage to be available for a T", as I think Richard is suggesting, the bug is only that we need the wording to be clearer.

First, objects must have unique addresses. Consider, still assuming B is trivially constructible:
void *p = malloc(sizeof(B));

The lifetime of a B starts some time after-or-including the malloc() call in the above line and the access of 'pb->i' two lines down. [basic.life]p5 ("Before the lifetime of an object has started ... The program has undefined behavior if ... the pointer is used to access a non-static data member")

The assignment to 'i' might start the lifetime of an 'int' subobject, but that's not enough to make the use of 'pb->i' defined if no 'B's lifetime has started.

B* pb = (B*)p;
pb->i = 0;

The lifetime of the B *ends* when its storage is re-used for the 'short' ([basic.life]p1 "The lifetime of an object of type T ends when ... the storage which the object occupies is reused"), as Daveed said. This happens some time after the access in the previous line, and the assignment two lines down.

short* ps = (short*)p;
*ps = 0;
This cannot possibly be construed as starting the lifetime of a B object and a short object, else they would have the same address, which is illegal. Am I missing something?

Both a B object and a short object have their lifetimes started in your code snippet, but the lifetimes don't overlap.

Confusingly, the start of these lifetimes is *not* called out in any particular line of code; it's implied by them. In particular, the casts don't have any lifetime effects (contra the straw man at http://www.open-std.org/pipermail/ub/2014-January/000406.html). The code would be just as defined (or undefined) written as:

void *p = malloc(sizeof(B));
B* pb = (B*)p;
short* ps = (short*)p;
pb->i = 0;
*ps = 0;

As Matt alluded to in http://www.open-std.org/pipermail/ub/2014-January/000456.html, it might be possible to say that all lifetime effects are called out in explicit expressions without breaking C compatibility, *if* we instead say that accessing the members of objects with trivial constructors can be done outside of the lifetime of such objects. I have no idea whether that would be better or worse than saying that lifetime effects can be implied.

Jeffrey
Gabriel Dos Reis
2014-01-17 23:19:38 UTC
Permalink
| From what I've learned in this thread, the (rough) intended C++ model for
| PODs (assuming memory of the right size/alignment) would seem to be "the
| lifetime of a B starts when you write to the memory as a B, and ends when you
| free the memory or write to the memory as different type." [Disclaimer: I'm
| not sure if "read from the memory as a B" also starts lifetime."]

that would be close to what C does with effective types.

| I think we can do better,

Same here.

| but it seems like that's the (rough) intent of the
| status quo, leaving aside the question of whether the wording actually says
| that.
|
|
| *If* that is the (rough) intent, then in:
|
|
| void *p = malloc(sizeof(B)); // 1
|
| B* pb = (B*)p; // 2
|
| pb->i = 0; // 3
|
| short* ps = (short*)p; // 4
| *ps = 0; // 5
|
| free(p); // 6
|
|
|
|
| I assume that the reasoning would be that:
|
| * line 3 starts the lifetime of a B (we're writing to the bits of a B member,
| not just any int)

line 3 does not write as a B, so it wouldn't count -- just like in C.

| * line 5 ends the lifetime of that B and begins the lifetime of a short
| * line 6 ends the lifetime of that short

Agreed.

| Again ignoring whether this is desirable, is that (roughly) the intent of the
| current wording?

Almost: we never started the lifetime of a B object.

| If yes, does the wording express it (a) accurately and (b) clearly?

My understanding is "no" for both.

| Finally, regardless of the above answer, do we want to change anything about
| the legality or semantics of the above type-punning code, such as possibly
| having a "type-safe mode" where such code is somehow not allowed unless in
| an "extern "C-compat"" block or something?

This is a good question! Which definitely needs a paper :-)

My view is that if the storage isn't declared or accessed as a B, then it isn't a B .
And this is compatible with the C's standard
Richard Smith
2014-01-17 23:42:52 UTC
Permalink
On 17 January 2014 15:19, Gabriel Dos Reis <***@microsoft.com> wrote:
Gabriel Dos Reis
2014-01-17 23:49:33 UTC
Permalink
No, we would have exception for non-static data member access for unconstructed objects – this would be conceptually the same form of exception we grant in constructors for vcalls or typeids and even certain member access.

From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Richard Smith
Sent: Friday, January 17, 2014 3:43 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

On 17 January 2014 15:19, Gabriel Dos Reis <***@microsoft.com<mailto:***@microsoft.com>> wrote:
| From what I've learned in this thread, the (rough) intended C++ model for
| PODs (assuming memory of the right size/alignment) would seem to be "the
| lifetime of a B starts when you write to the memory as a B, and ends when you
| free the memory or write to the memory as different type." [Disclaimer: I'm
| not sure if "read from the memory as a B" also starts lifetime."]
that would be close to what C does with effective types.

| I think we can do better,
Same here.

| but it seems like that's the (rough) intent of the
| status quo, leaving aside the question of whether the wording actually says
| that.
|
|
| *If* that is the (rough) intent, then in:
|
|
| void *p = malloc(sizeof(B)); // 1
|
| B* pb = (B*)p; // 2
|
| pb->i = 0; // 3
|
| short* ps = (short*)p; // 4
| *ps = 0; // 5
|
| free(p); // 6
|
|
|
|
| I assume that the reasoning would be that:
|
| * line 3 starts the lifetime of a B (we're writing to the bits of a B member,
| not just any int)

line 3 does not write as a B, so it wouldn't count -- just like in C.

Would that not give 'pb->i = 0' undefined behavior, because 'pb' does not, in fact, point to a B object?


The only clear governing rule I can find is 3.8/5, but that only applies if the storage ever actually contains a B object. Consider:

void *p = malloc(sizeof(B));
B *pb = (B*)p;
pb->i = 0;
// ...
new (p) B;

In this example, "pb->i = 0" has undefined behavior, because we performed member access on a pointer that points to storage where a B object will later be constructed. It seems strange that the definedness of 'pb->i = 0' would depend on how the storage is used later.

| * line 5 ends the lifetime of that B and begins the lifetime of a short
| * line 6 ends the lifetime of that short

Agreed.

| Again ignoring whether this is desirable, is that (roughly) the intent of the
| current wording?
Almost: we never started the lifetime of a B object.

| If yes, does the wording express it (a) accurately and (b) clearly?
My understanding is "no" for both.

Hear, hear =)

| Finally, regardless of the above answer, do we want to change anything about
| the legality or semantics of the above type-punning code, such as possibly
| having a "type-safe mode" where such code is somehow not allowed unless in
| an "extern "C-compat"" block or something?
This is a good question! Which definitely needs a paper :-)

My view is that if the storage isn't declared or accessed as a B, then it isn't a B .

It seems you don't consider member access as a B to qualify here. Is that your intent?
David Krauss
2014-01-18 00:20:53 UTC
Permalink
On Jan 18, 2014, at 7:49 AM, Gabriel Dos Reis <***@microsoft.com> wrote:

> No, we would have exception for non-static data member access for unconstructed objects – this would be conceptually the same form of exception we grant in constructors for vcalls or typeids and even certain member access.

But how do you know what object is unconstructed? An unconstructed object pops into existence when you perform an access, and that’s enough to validate my awful example. This is what I meant by “retroactive.”

Inside a constructor, the type of *this is well-defined, and the exception is granted over a limited scope. Outside a constructor, it’s a free for all.

I don’t know about rules in C, but as far as I know in C++, you can just arbitrarily decide to reinterpret bytes of memory as one POD or another, as long as they are “congruent,” by mangling the concepts of lifetime and allocation.
Gabriel Dos Reis
2014-01-18 04:13:47 UTC
Permalink
| But how do you know what object is unconstructed?

I do not understand the question; could you clarify?

| An unconstructed object
| pops into existence when you perform an access, and that's enough to validate
| my awful example. This is what I meant by "retroactive."

I don't understand. We define what it means for a raw memory to be constructed as an object.


| Inside a constructor, the type of *this is well-defined, and the exception is
| granted over a limited scope. Outside a constructor, it's a free for all.

I don't understand this. I am saying that if you get dynamically allocated storage of proper alignment and size, then the first write access is, by definition, construction of an object of type given by the lvalue used to perform the write.

| I don't know about rules in C,

the relevant rules have been extensively referenced in this thread. There were a couple of messages specifically dedicated to that last night.

|but as far as I know in C++, you can just
| arbitrarily decide to reinterpret bytes of memory as one POD or another, as
| long as they are "congruent," by mangling the concepts of lifetime and
| allocation.
David Krauss
2014-01-18 05:06:24 UTC
Permalink
On Jan 18, 2014, at 12:13 PM, Gabriel Dos Reis <***@microsoft.com> wrote:

> | But how do you know what object is unconstructed?
>
> I do not understand the question; could you clarify?
>
> | An unconstructed object
> | pops into existence when you perform an access, and that's enough to validate
> | my awful example. This is what I meant by "retroactive."
>
> I don't understand. We define what it means for a raw memory to be constructed as an object.

You proposed that an access causes the memory to be treated from that point as an unconstructed POD object, just as access in a constructor. But that rule is already loose enough to render the idea of a lifetime essentially without effect.

The only criterion for having an accessible but not-yet-constructed POD object is trying to use it (as an overlay to memory that doesn’t already contain a non-trivially destructible object). Whatever you try to use, you have. That’s just the worst interpretation of the status quo.

> | Inside a constructor, the type of *this is well-defined, and the exception is
> | granted over a limited scope. Outside a constructor, it's a free for all.
>
> I don't understand this. I am saying that if you get dynamically allocated storage of proper alignment and size, then the first write access is, by definition, construction of an object of type given by the lvalue used to perform the write.

And then you can arbitrarily say that any point in the code is a user-defined dynamic allocator which enables a new lifetime to begin by reusing the storage. The comments in my earlier example emphasized this.

Perhaps what we need is a way to stop lifetimes from ending by reuse. Then requiring delete-expressions would be useful.

> | I don't know about rules in C,
>
> the relevant rules have been extensively referenced in this thread. There were a couple of messages specifically dedicated to that last night.

I know that if you try to alias structures in C, layout differences can bite you, but I don’t see what prevents you from doing so as long as layouts are the same (which can be checked using member-wise offsetof). “Congruence” is there in the title of this thread.

I haven’t seen anything like “effective type” for C structs or C++ POD class types. Aliasing rules only apply at lvalue-to-rvalue conversion of fundamental types.

So given the earlier discussion, I still don’t know what rules in C prevent you from punning congruent classes.
Gabriel Dos Reis
2014-01-18 10:00:56 UTC
Permalink
| -----Original Message-----
| From: ub-***@open-std.org [mailto:ub-***@open-std.org] On
| Behalf Of David Krauss
| Sent: Friday, January 17, 2014 9:06 PM
| To: WG21 UB study group
| Subject: Re: [ub] type punning through congruent base class?
|
|
| On Jan 18, 2014, at 12:13 PM, Gabriel Dos Reis <***@microsoft.com> wrote:
|
| > | But how do you know what object is unconstructed?
| >
| > I do not understand the question; could you clarify?
| >
| > | An unconstructed object
| > | pops into existence when you perform an access, and that's enough to
| validate
| > | my awful example. This is what I meant by "retroactive."
| >
| > I don't understand. We define what it means for a raw memory to be
| constructed as an object.
|
| You proposed that an access causes the memory to be treated from that point
| as an unconstructed POD object, just as access in a constructor. But that rule
| is already loose enough to render the idea of a lifetime essentially without
| effect.

The last sentence is your assertion. I am saying I do not understand it. Could you detail the reasoning?

| The only criterion for having an accessible but not-yet-constructed POD
| object is trying to use it (as an overlay to memory that doesn't already contain
| a non-trivially destructible object).

Why?

| Whatever you try to use, you have. That's just the worst interpretation of the status quo.

Again, I do not understand.

| > | Inside a constructor, the type of *this is well-defined, and the exception is
| > | granted over a limited scope. Outside a constructor, it's a free for all.
| >
| > I don't understand this. I am saying that if you get dynamically allocated
| storage of proper alignment and size, then the first write access is, by
| definition, construction of an object of type given by the lvalue used to
| perform the write.
|
| And then you can arbitrarily say that any point in the code is a user-defined
| dynamic allocator which enables a new lifetime to begin by reusing the
| storage.

I do not understand this.

| The comments in my earlier example emphasized this.
|
| Perhaps what we need is a way to stop lifetimes from ending by reuse.

Why? It is already an acquired concept.

| Then requiring delete-expressions would be useful.

We do not require that for non-POD object, why should that be required here?

| > | I don't know about rules in C,
| >
| > the relevant rules have been extensively referenced in this thread. There
| were a couple of messages specifically dedicated to that last night.
|
| I know that if you try to alias structures in C, layout differences can bite you,
| but I don't see what prevents you from doing so as long as layouts are the
| same (which can be checked using member-wise offsetof). "Congruence" is
| there in the title of this thread.

"congruent" is in the title but it is defined anywhere in C++ for classes. So, unless you are more precise about what you are saying, it is hard to understand.
Just having the same layout isn't sufficient to say "Oh, it is OK, go ahead and do whatever you want". On a defined platform, an IEEE-conformant double and a 64-bit long may have the same alignment and size; we aren't saying it is OK to alias them.

| I haven't seen anything like "effective type" for C structs or C++ POD class
| types.

Because I do not see the point of introducing 'effective type' when we have 'dynamic type'.
The whole point of saying that the object is constructed is to assert that its 'dynamic type' is now known (which corresponds to 'effective type' in C).
See my previous message(s) on this point.

| Aliasing rules only apply at lvalue-to-rvalue conversion of fundamental
| types.
|
| So given the earlier discussion, I still don't know what rules in C prevent you
| from punning congruent classes.

We are trying to define a missing case for C++ - not C.

-- Gaby
Jeffrey Yasskin
2014-01-18 00:21:02 UTC
Permalink
I'd like to complicate things further, in response to the idea that the
snippet involving 'B' and 'short' is type-punning and that we should
consider a type-safe mode. :)

First, consider whether the following code, intended to be completely
normal C code:

B* pb = (B*)malloc(sizeof(B));
pb->i = 0;
free(pb);
short* ps = (short*)malloc(sizeof(short));
*ps = 0;
free(ps);

looks like code that should be valid in this type-safe mode. If you'd like
to ban it, the rest of this post won't have anything interesting for you.

Now imagine I write a library (please forgive compile and logic errors, and
typos):

std::map<size_t, std::stack<void*>> size_classes = {{16, {}}, {32, {}},
...};

void* my_malloc(size_t size) {
auto size_class = size_classes.lower_bound(size);
assert(size_class != size_classes.end());
if (size_class->second.empty())
return malloc(size_class->first);
void* result = size_class->second.top();
size_class->second.pop();
return result;
}

void my_free(size_t size, void* block) {
size_classes.lower_bound(size)->second.push(block);
}

Then I use it like:

B* pb = (B*)my_malloc(sizeof(B));
pb->i = 0;
my_free(sizeof(B), pb);
short* ps = (short*)my_malloc(sizeof(short));
*ps = 0;
my_free(sizeof(short), ps);

Is this worse than the above malloc/free-based code? That is, can users
write wrappers around malloc and free?

But the compiler can inline the my_malloc use down to:

void *p = malloc(16); // Probably 16.
B* pb = (B*)p;
pb->i = 0;
short* ps = (short*)pb;
*ps = 0;

using knowledge of the behavior of std::map and std::stack, which is nearly
identical to the "type-punning" code. So how do we make the type-punning
invalid without breaking standard malloc-based code or user-written
libraries?

On Fri, Jan 17, 2014 at 2:56 PM, Herb Sutter <***@microsoft.com> wrote:
>> Note that this post from Herb arrived after
>> http://www.open-std.org/pipermail/ub/2014-January/000418.html but was
sent
>> before, so the thread got a little mixed up.
>
> Yes, I've been trying to reply less on this thread until that sync'ed back
> up. :)
>
Herb Sutter
2014-01-18 01:12:04 UTC
Permalink
For the first example: I’d be inclined to view it as type-unsafe because of the cast from void* to B*, not because of the malloc.

For the rest: At quick glance, the my_malloc/my_free internals look type safe to me, so the issue isn’t wrapping malloc/free per se. Rather, it’s the calling code that’s unsafe for casting from void* to B*, which is unchanged in the optimized version of the code.

So IMO the three examples are all the same because they all contain the unsafe cast from void* to B*, just moving it around a little.


Moving to a question you didn’t ask: What if my_malloc/my_free returned/took not void*, but my_class* (so a class-specific allocator, implemented using malloc/free)? That is:

std::map<size_t, std::stack<my_class*>> size_classes = {{16, {}}, {32, {}}, ...};

my_class* my_malloc(size_t size) {
auto size_class = size_classes.lower_bound(size);
assert(size_class != size_classes.end());
if (size_class->second.empty())
return (my_class*)malloc(size_class->first);
void* result = size_class->second.top();
size_class->second.pop();
return result;
}

void my_free(size_t size, my_class* block) {
size_classes.lower_bound(size)->second.push(block);
}

Then this would require some decoration around the (single) cast to my_class*, perhaps:

...
if (size_class->second.empty()) {
my_class* ret = nullptr;
extern “c-style” {
ret = (my_class*)malloc(size_class->first);
}
return ret;
}



And this doesn’t surprise me because I would expect the internals of an allocator to be a classic example of code that resorts to the explicit type-unsafe escape hatch (“extern “C-style” { }” block around or whatever).

Herb


From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of Jeffrey Yasskin
Sent: Friday, January 17, 2014 4:21 PM
To: WG21 UB study group
Subject: Re: [ub] type punning through congruent base class?

I'd like to complicate things further, in response to the idea that the snippet involving 'B' and 'short' is type-punning and that we should consider a type-safe mode. :)

First, consider whether the following code, intended to be completely normal C code:

B* pb = (B*)malloc(sizeof(B));
pb->i = 0;
free(pb);
short* ps = (short*)malloc(sizeof(short));
*ps = 0;
free(ps);

looks like code that should be valid in this type-safe mode. If you'd like to ban it, the rest of this post won't have anything interesting for you.

Now imagine I write a library (please forgive compile and logic errors, and typos):

std::map<size_t, std::stack<void*>> size_classes = {{16, {}}, {32, {}}, ...};

void* my_malloc(size_t size) {
auto size_class = size_classes.lower_bound(size);
assert(size_class != size_classes.end());
if (size_class->second.empty())
return malloc(size_class->first);
void* result = size_class->second.top();
size_class->second.pop();
return result;
}

void my_free(size_t size, void* block) {
size_classes.lower_bound(size)->second.push(block);
}

Then I use it like:

B* pb = (B*)my_malloc(sizeof(B));
pb->i = 0;
my_free(sizeof(B), pb);
short* ps = (short*)my_malloc(sizeof(short));
*ps = 0;
my_free(sizeof(short), ps);

Is this worse than the above malloc/free-based code? That is, can users write wrappers around malloc and free?

But the compiler can inline the my_malloc use down to:

void *p = malloc(16); // Probably 16.
B* pb = (B*)p;
pb->i = 0;
short* ps = (short*)pb;
*ps = 0;

using knowledge of the behavior of std::map and std::stack, which is nearly identical to the "type-punning" code. So how do we make the type-punning invalid without breaking standard malloc-based code or user-written libraries?

On Fri, Jan 17, 2014 at 2:56 PM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:
>> Note that this post from Herb arrived after
>> http://www.open-std.org/pipermail/ub/2014-January/000418.html but was sent
>> before, so the thread got a little mixed up.
>
> Yes, I've been trying to reply less on this thread until that sync'ed back
> up. :)
>
> From what I've learned in this thread, the (rough) intended C++ model for
> PODs (assuming memory of the right size/alignment) would seem to be "the
> lifetime of a B starts when you write to the memory as a B, and ends when
> you free the memory or write to the memory as different type." [Disclaimer:
> I'm not sure if "read from the memory as a B" also starts lifetime."]
>
> I think we can do better, but it seems like that's the (rough) intent of the
> status quo, leaving aside the question of whether the wording actually says
> that.
>
> *If* that is the (rough) intent, then in:
>
>> void *p = malloc(sizeof(B)); // 1
>>
>> B* pb = (B*)p; // 2
>>
>> pb->i = 0; // 3
>>
>> short* ps = (short*)p; // 4
>> *ps = 0; // 5
>>
>> free(p); // 6
>
>
> I assume that the reasoning would be that:
>
> line 3 starts the lifetime of a B (we're writing to the bits of a B member,
> not just any int)
> line 5 ends the lifetime of that B and begins the lifetime of a short
> line 6 ends the lifetime of that short
>
>
> Again ignoring whether this is desirable, is that (roughly) the intent of
> the current wording?
>
>
> If yes, does the wording express it (a) accurately and (b) clearly?
>
>
> Finally, regardless of the above answer, do we want to change anything about
> the legality or semantics of the above type-punning code, such as possibly
> having a "type-safe mode" where such code is somehow not allowed unless in
> an "extern "C-compat"" block or something?
>
>
> Herb
>
>
>
> ________________________________
> From: ub-***@open-std.org<mailto:ub-***@open-std.org> <ub-***@open-std.org<mailto:ub-***@open-std.org>> on behalf of Jeffrey
> Yasskin <***@google.com<mailto:***@google.com>>
> Sent: Friday, January 17, 2014 1:34 PM
>
> To: WG21 UB study group
> Subject: Re: [ub] type punning through congruent base class?
>
> Note that this post from Herb arrived after
> http://www.open-std.org/pipermail/ub/2014-January/000418.html but was sent
> before, so the thread got a little mixed up.
>
> On Thu, Jan 16, 2014 at 11:38 AM, Herb Sutter <***@microsoft.com<mailto:***@microsoft.com>> wrote:
>>
>> Richard, it cannot mean that (or if it does, IMO we have an obvious bug)
>> for at least two specific reasons I can think of (below), besides the
>> general reasons that it would not be sensical and would violate type safety.
>
>
> We do have an obvious bug in [basic.life]p1, "The lifetime of an object of
> type T begins when storage with the proper alignment and size for type T is
> obtained", if we interpret "obtained" as "obtained from the memory
> allocator". Even with strict uses of placement-new to change the type of
> memory, placement-new doesn't "obtain" any memory. If we interpret
> "obtained" as just "the programmer intends a region of storage to be
> available for a T", as I think Richard is suggesting, the bug is only that
> we need the wording to be clearer.
>
>> First, objects must have unique addresses. Consider, still assuming B is
>> trivially constructible:
>>
>> void *p = malloc(sizeof(B));
>
>
> The lifetime of a B starts some time after-or-including the malloc() call in
> the above line and the access of 'pb->i' two lines down. [basic.life]p5
> ("Before the lifetime of an object has started ... The program has undefined
> behavior if ... the pointer is used to access a non-static data member")
>
> The assignment to 'i' might start the lifetime of an 'int' subobject, but
> that's not enough to make the use of 'pb->i' defined if no 'B's lifetime has
> started.
>
>>
>> B* pb = (B*)p;
>> pb->i = 0;
>
>
> The lifetime of the B *ends* when its storage is re-used for the 'short'
> ([basic.life]p1 "The lifetime of an object of type T ends when ... the
> storage which the object occupies is reused"), as Daveed said. This happens
> some time after the access in the previous line, and the assignment two
> lines down.
>
>>
>> short* ps = (short*)p;
>> *ps = 0;
>>
>> This cannot possibly be construed as starting the lifetime of a B object
>> and a short object, else they would have the same address, which is illegal.
>> Am I missing something?
>
>
> Both a B object and a short object have their lifetimes started in your code
> snippet, but the lifetimes don't overlap.
>
> Confusingly, the start of these lifetimes is *not* called out in any
> particular line of code; it's implied by them. In particular, the casts
> don't have any lifetime effects (contra the straw man at
> http://www.open-std.org/pipermail/ub/2014-January/000406.html). The code
> would be just as defined (or undefined) written as:
>
> void *p = malloc(sizeof(B));
>
> B* pb = (B*)p;
> short* ps = (short*)p;
> pb->i = 0;
>
> *ps = 0;
>
>
> As Matt alluded to in
> http://www.open-std.org/pipermail/ub/2014-January/000456.html, it might be
> possible to say that all lifetime effects are called out in explicit
> expressions without breaking C compatibility, *if* we instead say that
> accessing the members of objects with trivial constructors can be done
> outside of the lifetime of such objects. I have no idea whether that would
> be better or worse than saying that lifetime effects can be implied.
>
>
> Jeffrey
>
>
> _______________________________________________
> ub mailing list
> ***@isocpp.open-std.org<mailto:***@isocpp.open-std.org>
> http://www.open-std.org/mailman/listinfo/ub
>
Jeffrey Yasskin
2014-01-18 02:01:13 UTC
Permalink
Great! That all sounds coherent to me. I'm still concerned because it
breaks compatibility with C, but you mentioned a separate mode, which
could serve as a migration path.

Thanks,
Jeffrey

On Fri, Jan 17, 2014 at 5:12 PM, Herb Sutter <***@microsoft.com> wrote:
> For the first example: I’d be inclined to view it as type-unsafe because of
> the cast from void* to B*, not because of the malloc.
>
>
>
> For the rest: At quick glance, the my_malloc/my_free internals look type
> safe to me, so the issue isn’t wrapping malloc/free per se. Rather, it’s the
> calling code that’s unsafe for casting from void* to B*, which is unchanged
> in the optimized version of the code.
>
>
>
> So IMO the three examples are all the same because they all contain the
> unsafe cast from void* to B*, just moving it around a little.
>
>
>
>
>
> Moving to a question you didn’t ask: What if my_malloc/my_free returned/took
> not void*, but my_class* (so a class-specific allocator, implemented using
> malloc/free)? That is:
>
>
>
> std::map<size_t, std::stack<my_class*>> size_classes = {{16, {}}, {32, {}},
> ...};
>
> my_class* my_malloc(size_t size) {
>
>
> auto size_class = size_classes.lower_bound(size);
> assert(size_class != size_classes.end());
> if (size_class->second.empty())
> return (my_class*)malloc(size_class->first);
>
> void* result = size_class->second.top();
> size_class->second.pop();
> return result;
> }
>
> void my_free(size_t size, my_class* block) {
> size_classes.lower_bound(size)->second.push(block);
> }
>
> Then this would require some decoration around the (single) cast to
> my_class*, perhaps:
>
>
>
> ...
>
> if (size_class->second.empty()) {
> my_class* ret = nullptr;
>
> extern “c-style” {
>
> ret = (my_class*)malloc(size_class->first);
>
> }
>
> return ret;
>
> }
>
> …
>
> And this doesn’t surprise me because I would expect the internals of an
> allocator to be a classic example of code that resorts to the explicit
> type-unsafe escape hatch (“extern “C-style” { }” block around or whatever).
>
>
>
> Herb
>
>
>
>
>
> From: ub-***@open-std.org [mailto:ub-***@open-std.org] On Behalf Of
> Jeffrey Yasskin
> Sent: Friday, January 17, 2014 4:21 PM
>
>
> To: WG21 UB study group
> Subject: Re: [ub] type punning through congruent base class?
>
>
>
> I'd like to complicate things further, in response to the idea that the
> snippet involving 'B' and 'short' is type-punning and that we should
> consider a type-safe mode. :)
>
> First, consider whether the following code, intended to be completely normal
> C code:
>
> B* pb = (B*)malloc(sizeof(B));
> pb->i = 0;
> free(pb);
> short* ps = (short*)malloc(sizeof(short));
> *ps = 0;
> free(ps);
>
> looks like code that should be valid in this type-safe mode. If you'd like
> to ban it, the rest of this post won't have anything interesting for you.
>
> Now imagine I write a library (please forgive compile and logic errors, and
> typos):
>
> std::map<size_t, std::stack<void*>> size_classes = {{16, {}}, {32, {}},
> ...};
>
> void* my_malloc(size_t size) {
> auto size_class = size_classes.lower_bound(size);
> assert(size_class != size_classes.end());
> if (size_class->second.empty())
> return malloc(size_class->first);
> void* result = size_class->second.top();
> size_class->second.pop();
> return result;
> }
>
> void my_free(size_t size, void* block) {
> size_classes.lower_bound(size)->second.push(block);
> }
>
> Then I use it like:
>
> B* pb = (B*)my_malloc(sizeof(B));
> pb->i = 0;
> my_free(sizeof(B), pb);
> short* ps = (short*)my_malloc(sizeof(short));
> *ps = 0;
> my_free(sizeof(short), ps);
>
> Is this worse than the above malloc/free-based code? That is, can users
> write wrappers around malloc and free?
>
> But the compiler can inline the my_malloc use down to:
>
> void *p = malloc(16); // Probably 16.
> B* pb = (B*)p;
> pb->i = 0;
> short* ps = (short*)pb;
> *ps = 0;
>
> using knowledge of the behavior of std::map and std::stack, which is nearly
> identical to the "type-punning" code. So how do we make the type-punning
> invalid without breaking standard malloc-based code or user-written
> libraries?
>
> On Fri, Jan 17, 2014 at 2:56 PM, Herb Sutter <***@microsoft.com> wrote:
>>> Note that this post from Herb arrived after
>>> http://www.open-std.org/pipermail/ub/2014-January/000418.html but was
>>> sent
>>> before, so the thread got a little mixed up.
>>
>> Yes, I've been trying to reply less on this thread until that sync'ed back
>> up. :)
>>
>>
Loading...