The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

C++: pure virtual function called

I've run into an interesting problem, and am curious if anyone else out there has run into this.  I've googled around a bit, and have been unable to find a confirmation of what I think the behavior is.

I have a bunch of interfaces defined as pure virtual ("=0") -- the methods in these interfaces are being called asynchronously on another thread in response to certain events. The objects that handle these events inherit from the interface classes, and implement the pure virtual functions.  So far, so good.

What seems to be happening is that the vtable entries for the pure virtual functions are being overwritten (with the address of __purecall) in the object's dtor.  What is odd, is that this appears to be happening on *entry to* the dtor, as opposed to on *exit from* the dtor.

The net effect is that an attempt to call any of the pure virtual functions in the dtor leads to a crash.  The only solution I've come up with is to make the methods "plain old" virtual, and provide a default implementation for them which does nothing.

Has anyone wresteled with this problem before?  TIA...
BillT Send private email
Monday, January 22, 2007
 
 
care to post your code?
rbny
Monday, January 22, 2007
 
 
Let's see:

We have a 'pure virtual' parent class.

I'll assume we have multiple inheritors of the parent class, each of which has its own instantiation of the 'pure virtual' methods of the parent class.  And perhaps different (or additional) properties and methods in each of the inheritor classes.

I believe this happens should you have a pointer to child class 'b' cast to a pointer to child class 'a'.  Should this happen, then the 'wrong' vtable would be used to look up methods -- giving you the call you're seeing.

Now, assigning a default method body to the parent class -- even if it's an 'error reporting' method -- can at least give you more detail as to what is going wrong.  And that can work even if my theory above is wrong.
AllanL5
Monday, January 22, 2007
 
 
> The only solution I've come up with is to make the methods "plain old" virtual, and provide a default implementation for them which does nothing.

I think you can make them pure *and* define an implementation:

class Foo
{
 virtual void foo() = 0;
};

void Foo::foo() {}

> What is odd, is that this appears to be happening on *entry to* the dtor, as opposed to on *exit from* the dtor.

I don't see that as odd: on entry to the dtor, the derived class has been destroyed, so now the vtable should point to the methods defined in *this* class (which, you say, are pure).

> Has anyone wresteled with this problem before?

Avoid calling any virtual functions in the dtor ... after all, what would be the point? To avoid it in general, virtual functions that you'd like to call from the dtor can delegate to non-virtual implementation functions, e.g.:

class Foo
{
public:
  virtual ~Foo() { implementCleanUp(); }
  virtual void cleanUp() { implementCleanUp(); }
protected:
  void implementCleanUp() { ... stuff happens here ...; }
};
Christopher Wells Send private email
Monday, January 22, 2007
 
 
See this. The question appears in my book as well (wallstreetprogramming.com), but it's also available on 96bosses.com -- third one from the top, I believe:

http://www.96bosses.com/content/view/23/33/

- Cheers
Andrey Butov Send private email
Monday, January 22, 2007
 
 
Thanks for the quick replies:

"I think you can make them pure *and* define an implementation" -- this is true, but it doesn't fix the problem, the vtable still gets overwritten.

"I don't see that as odd: on entry to the dtor, the derived class has been destroyed, so now the vtable should point to the methods defined in *this* class (which, you say, are pure)" -- it is the vtable for "this" that's being overwritten on entry (not the derived class) -- that's what seems odd.

"See this. The question appears in my book as well" -- I'd come across your page in my googling, but it doesn't answer the question: "Is this documented behavior, and if so, where?" I assume it is, since it is consistent with both VC++ (2003) and gcc (3.4).
BillT Send private email
Monday, January 22, 2007
 
 
I think the key question is whether you are trying to call the pure virtual functions from the dtor of the base abstract class (where those functions don't exist) or the dtor of the derived class.
onanon
Monday, January 22, 2007
 
 
The pure virtual methods are being called from another thread, on the derived class, while the derived class is being destroyed. 

There is synchronization code to prevent the dtor from running to completion until the callback function returns -- this is not an issue.

The problem is that the vtable entries for the pure virtual method "disappear" on entry to the derived class's dtor.

I don't understand why it would matter if the methods were being called from a base class, since destruction happens from most-derived to least-derived (opposite of construction)?
BillT Send private email
Monday, January 22, 2007
 
 
> it is the vtable for "this" that's being overwritten on entry (not the derived class) -- that's what seems odd.

Actually the vtables themselves are const (not overwritten): there's once instance of each vtable per class (not per object). What's happening is that on entry to the dtor, the this->vptr is being changed to point to the vtable for this class (whereas previously it pointed to the vtable of the derived class) ... which isn't odd because, now that the derived part is destroyed, this instance is no longer a derived type (it's now an instance of the base type only).

> The pure virtual methods are being called from another thread, on the derived class, while the derived class is being destroyed.

You need a lock or other mechanism to prevent that happening: don't even begin to call the dtor while one of the other virtual methods is being called (and don't begin to call another virtual method while the dtor is being called).
Christopher Wells Send private email
Monday, January 22, 2007
 
 
Chris:

I agree with everything you say, except wrt "on entry to the dtor, the this->vptr is being changed to point to the vtable for this class" -- it actually appears that on entry the vptr is being changed to point to the base of "this", (the class being destroyed is the one that declares and defines the pure virtual methods, not a derived class).

There's already a synchronization mechanism to ensure that the dtor does not complete until all callbacks have completed -- this ensures that member variables are not destroyed while a callback is executing.  I expected the same behavior for the vptr (i.e., that it would be replaced on exit, not on entry to the dtor), but this is clearly not the case.

I can't find this behavior documented anywhere, although that may not be surprising since C++ behavior in the presence of mult. threads is not defined in any standards (although there is a movement to extend the C++ standard to cover multi-threading -- e.g., http://www.hpl.hp.com/research/linux/atomic_ops/index.php4  , http://www.hpl.hp.com/personal/Hans_Boehm/c++mm/boehm-intel.pdf
BillT Send private email
Monday, January 22, 2007
 
 
BillT Send private email
Monday, January 22, 2007
 
 
You probably already checked this but, you are not calling any virtual methods from the constructor, are you? That has been where I have seen this problem occur.
A. Nonymous
Monday, January 22, 2007
 
 
The rolling back of the vtable is well-documented and I'm sure can be found in the standard though I don't have a reference at the moment.  I seem to remember Bjarne talking about this in one of his books. 

It's a common gotcha on constructors when people try to define virtual initializer functions in the base class that are called from the base class constructor and then are surprised to find that their overrides of the initalizer functions in derived classes don't get called because the vtable is still the base's vtable.  Destruction is the mirror image of this same process.  By the time it gets to the base class destructor, the derived class is assumed to be non-existent.  Referring to its members would be undefined behavior.  The ability to call a derived class's virtual functions from a base class constructor or destructor would be dangerous since there's a good chance that you'd try to access member variables defined by the derived class that have either not been initialized yet or have been unitialized by the time your override is called.
SomeBody Send private email
Monday, January 22, 2007
 
 
"The problem is that the vtable entries for the pure virtual method "disappear" on entry to the derived class's dtor.

I don't understand why it would matter if the methods were being called from a base class, since destruction happens from most-derived to least-derived (opposite of construction)?"


As you said: destruction starts with the most derived class and works its way back up to the base.

However, if you have a virtual function that is overridden in a derived class, then it will be called, EVEN AFTER THE DERIVED PART OF THE CLASS INSTANCE HAS BEEN DESTROYED!!!  This is not a good idea, as derived class code may attempt to access data members that have been destroyed (note that base class members haven't been destroyed yet).

Your compiler might be trying to protect you by unwinding the vtable chain ahead of the destruction.  I have seen this behavior in constructors: the base class sets its vtable pointer, then the derived class overwrites it, and then the next layer overwrites that and so on.  The upshot being if you call a virtual function from a constructor then you may not be getting the version you expect.

Work this out on paper if you don't think you see the problem.

Bottom line: don't call virtual functions on an instance being constructed or destroyed.  Ever.  There are design patterns that can do what you want in a cleaner way (e.g. factories, builders).
David Jones Send private email
Monday, January 22, 2007
 
 
"I have seen this behavior in constructors: the base class sets its vtable pointer, then the derived class overwrites it, and then the next layer overwrites that and so on.  The upshot being if you call a virtual function from a constructor then you may not be getting the version you expect."

Calling a virtual function from a constructor is not allowed (*) by C++.

(*) You can do it but you are actually calling the version of the function defined in the current class, which is almost always *not* what you want. Therefore, the following code will generate the OP's error message when class C is instantiated:

class Base
{
  public:
      Base() { moreInitialization(); };

      virtual void moreInitialization() = 0;

};

class C : public Base
{
  public:
      C() : Base() {};

      //  From Base
      virtual void moreInitialization();

};
A. Nonymous
Monday, January 22, 2007
 
 
I hope that OP can post the source code otherwise I am afraid that people are talking about different things. For example, the code:

class Base
{
  public:
      Base() { moreInitialization(); };

      virtual void moreInitialization() = 0;

};

class C : public Base
{
  public:
      C() : Base() {};

      //  From Base
      virtual void moreInitialization();

};

Won't compile at all, unless you do not instantiate C or Base.

It is quite rare that a pure function is called in C++. None of these cases are trivial to contruct.
Burner
Monday, January 22, 2007
 
 
The OP's derived classes have been destroyed by the time the base constructor is called, and he's wondering why calling the pure virtual functions makes it crash? He answered it himself... that's why!
Steve
Monday, January 22, 2007
 
 
Well, I've worked out the problem I was having, and in the interest of sharing that info:

There is an abstract class that defines an interface (call it A), a base class (call it B) that derives from A, and that can be derived from, and a derived class (call it D) that represents what a user would write, and which derives from B.

Assume that B does not define the pure virtual methods declared in A -- for instance, to force the derived class to implement it.

The pure virtual method is a "callback" method -- i.e., it is invoked in response to external events, and on a different thread.

If another thread invokes D's dtor while a callback is in progress, some "funny" things can happen:

- D's dtor executes -- the object is still a "D" at this point.

- B's dtor executes, since it is the next-most derived class.  At this point the compiler replaces the vtable ptr in the object with B's vtable ptr, so the object is now a "B", not a "D" anymore.  Remember that B chose not to define the pure virtual method, in order to force it to be defined in D.

- Another thread invokes the callback function on the object in response to an external event.  At this point you get a "pure virtual error", because B's vtable doesn't have an entry for the callback method -- B's dtor effectively changed the object from a "D" to a "B".

One suggestion was to use a lock to control access to the dtor and callback function. I don't think this works, however, unless the lock is acquired in the most-derived class ("D") -- by the time the object morphs into a "B" (at entry to B's dtor) it is already too late.  If you're designing a class library (as we are), it's also not necessarily reasonable to expect users of the class to remember to lock the object in their dtor.

The only reasonable solution, it would seem, is to not define the callback method as pure virtual, but as a "plain old" virtual method, with a default (no-op) implementation.  When the callback function is invoked in response to the external event, at least there is a vtable entry for the compiler to call.  This assumes, of course, that the callback method can be skipped -- not an unreasonable assumption for an object that is halfway deleted already anyway (at least in some scenarios).

This also assumes that the replacement by the compiler of vtable pointers in the object itself is an atomic operation -- otherwise, things get really ugly.

Thanks to all for their time and help.  Any further comments/suggestions?
BillT Send private email
Monday, January 22, 2007
 
 
> There's already a synchronization mechanism to ensure that the dtor does not complete until all callbacks have completed -- this ensures that member variables are not destroyed while a callback is executing.

Does the mechanism prevent a callback from executing a partially-destroyed D? It doesn't sound good.

> I agree with everything you say, except wrt "on entry to the dtor, the this->vptr is being changed to point to the vtable for this class" -- it actually appears that on entry the vptr is being changed to point to the base of "this", (the class being destroyed is the one that declares and defines the pure virtual methods, not a derived class).

Using MSVC I find that the in the dtor the vptr is set to this::vtable, not to base::vtable ... the following code:

#include "stdafx.h"
#include <iostream>
using namespace std;

class A
{
public:
    virtual ~A() { cout << "~A calls "; foo(); }
    virtual void foo() { cout << "A::foo" << "\r\n"; }
};

class B : public A
{
public:
    virtual ~B() { cout << "~B calls "; foo(); }
    virtual void foo() { cout << "B::foo" << "\r\n"; }
};

class C : public B
{
public:
    virtual ~C() { cout << "~C calls "; foo(); }
    virtual void foo() { cout << "C::foo" << "\r\n"; }
};

int main(int argc, char* argv[])
{
    A* a = new C();
    delete a;
    return 0;
}

produces the following output:

~C calls C::foo
~B calls B::foo
~A calls A::foo

If I change A::foo to pure then I can't even link:

error LNK2019: unresolved external symbol "public: virtual void __thiscall A::foo(void)" (?foo@A@@UAEXXZ) referenced in function "public: virtual __thiscall A::~A(void)" (??1A@@UAE@XZ)

> B's dtor effectively changed the object from a "D" to a "B"

Pretty well, yes: when D's dtor finishes and B's dtor starts, this *is* now a B and is no longer a D (so I think your compiler is right)

> One suggestion was to use a lock to control access to the dtor and callback function. I don't think this works, however, unless the lock is acquired in the most-derived class ("D") -- by the time the object morphs into a "B" (at entry to B's dtor) it is already too late.

I suggested that the lock be acquired *before* calling a virtual method or dtor ... I think that the lock must live outside the object, and you should access it before you call any virtual function (because oherwise, if it's inside the object and you call it via an object method, then after you begin to call the object method to acquire the lock the object might be destroyed by another thread, which releases the memory which contains the reference to the lock which you're trying to acquire).

So to have the lock outside the object, for example, call the virtual function through a proxy object which contains the lock, like this ..

class ICallback
{
public:
    virtual ~ICallback() = 0;
    virtual void hello() = 0;
};

class CallbackBase : public ICallback
{
    //whatever you wanted to do in the abstract base class
};

class CallbackImplementation : public CallbackBase
{
public:
    virtual void hello() { cout << "Hello" << "\r\n"; }
};

class SafelyLockedCallbackProxy
{
private:
    ICallback* m_ptr;
    SomeTypeOfLock m_lock;
public:
    SafelyLockedCallbackProxy(ICallback* ptr)
        : m_ptr(ptr)
    {}
public:
    void hello()
    {
        m_lock.acquire(); //get lock before calling virtual function
        if (m_ptr)
            m_ptr->hello();
        //else do nothing
        m_lock.release();
    }
    void destroy()
    {
        m_lock.acquire(); //get lock before calling virtual function
        delete m_ptr;
        m_ptr = 0;
        m_lock.release();
    }
};

Or, access the callback through a thread-safe collection class, like this:

class SafeCallbackCollection
{
    std::set<ICallback*> m_callbacks;
    SomeTypeOfMultiReaderSingleWriterLock m_lock;
public:
    void hello(ICallback* callback)
    {
        m_lock.acquireReadLock();
        if (m_callbacks.find(callback) != m_callbacks.end())
        {
            //this callback instance still exists
            callback->hello();
        }
        m_lock.releaseReadLock();
    }
    void destroy(ICallback* callback)
    {
        m_lock.acquireWriteLock();
        m_callbacks.erase(callback);
        delete callback;
        m_lock.releaseWriteLock();
    }
    void create(ICallback* callback)
    {
        m_lock.acquireWriteLock();
        m_callbacks.insert(callback);
        m_lock.releaseWriteLock();
    }
};
Christopher Wells Send private email
Monday, January 22, 2007
 
 
>> This assumes, of course, that the callback method can be skipped -- not an unreasonable assumption for an object that is halfway deleted already anyway (at least in some scenarios).

Uhh, why is the callback method still being called while the object is being deleted?  Your problem isn't the virtualness of the callback method but the fact that it's being called at all during destruction.  What do you do about the callback after destruction is complete?  Hope the memory still has read-access and that the pointer hasn't changed to something bad?  The real solution to your problem is to recognize that any call to an instance method from outside the class -- virtual, pure virtual, or otherwise -- is invalid once the destructor has been entered and to simply not call it once the object has begun destruction.
SomeBody Send private email
Monday, January 22, 2007
 
 
"Uhh, why is the callback method still being called while the object is being deleted?"

I would guess it would be a pattern something like;

destructor()
{
  request other thread holding resources for me to delete them, passing "callback" for notification of completion.
  wait on event.
}

virtual callback()
{
  signal event.
  // possibly some other stuff.
}

To ensure that the resources allocated somewhere else by something else get nuked properly.

It's possible that you have a race condition after the signalling; after the event has been waved the destruction thread may run before the signalling thread gets to leave that method. You might not have put some code in there, but the compiler might have done.
Katie Lucas
Tuesday, January 23, 2007
 
 
virtual methods of a derived class can not be called from a base class' destructor.
Achilleas Margaritis Send private email
Tuesday, January 23, 2007
 
 
"Using MSVC I find that the in the dtor the vptr is set to this::vtable, not to base::vtable": that's correct, and is the part that was puzzling at first.

On entry to the dtor, the vtable ptr is set to point to the vtable for that object -- in ~D it is set to D's vtable, and on entry to ~B it is set to B's vtable, at which point the object is no longer a D, and any virtual functions defined in D are no longer accessible.

The bit that was confusing at first is that D still exists -- for instance, its member variables can be examined -- since ~D has not yet returned (at which point the object does cease to exist).  After a bit of head-scratching and reading, though, it makes sense.

FWIW, I found the following esp. helpful:

http://blogs.msdn.com/oldnewthing/archive/2004/04/28/122037.aspx
http://www.parashift.com/c++-faq-lite/dtors.html
http://www.parashift.com/c++-faq-lite/multiple-inheritance.html

Thanks again to all.
BillT Send private email
Tuesday, January 23, 2007
 
 
> This also assumes that the replacement by the compiler of vtable pointers in the object itself is an atomic operation -- otherwise, things get really ugly.

By the way, beware multi-threading and don't try to implement anything too clever, because you can't rely on mere testing to uncover any bugs in your implementation: instead, you need a design whose correctness you can verify by inspection.
Christopher Wells Send private email
Tuesday, January 23, 2007
 
 
why does someone in another thread still have a ptr to the object if you are deleting it?

Had a vaguely similar situation with a socket interface object with a threadproc that would not die
while(bRun)
{
 select(100ms...
 do stuff
}
bStopped = true

waited for bStopped 10 sec in d'tor and it would not die.
finally set bRun to false and had part of a main loop check for bStopped objects and delete them

could you:
use shared_ptr to point to object
have a valid flag in object
when you want to delete, set valid to false and
delete shared_ptr
any call with not valid returns error and caller
gets rid of its shared_ptr
when no more references, object is actually deleted
expr
Tuesday, January 23, 2007
 
 
>> Any further comments/suggestions?

You need thread-safety here, pretty obviously.

Unless I'm misunderstanding, it also sounds like you're counting on "atomicity" of aspects of the implementation of the C++ vtable manipulation and memory management. No way in hell I would do so.

I tend to think that when someone is straining really hard to understand the exact mechanism of some topic like class unwinding during destruction (for instance), there is a basic problem with the design.

There should be absolutely no way that an out-of-thread call to a class's members can occur during construction or destruction. I think that's what you described. That is *not* good. You need to block this from happening. CRITICAL_SECTION, if nothing else.

Mainly I agree with this observation:

>> why does someone in another thread still have a ptr to the object if you are deleting it?

It sounds like you need to revisit the top-down design.
Bored Bystander Send private email
Tuesday, January 23, 2007
 
 
"The objects that handle these events inherit from the interface classes, and implement the pure virtual functions.  So far, so good."

"The net effect is that an attempt to call any of the pure virtual functions in the dtor leads to a crash.  The only solution I've come up with is to make the methods "plain old" virtual, and provide a default implementation for them which does nothing.
"

An attempt to call a virtual function, pure or otherwise, in a destructor of a base class is doomed to fail. Simply because, as others have stated, once in the destructor of the base class, the derived class no longer exists. So although you think you are calling an overloaded function, you are in fact calling the function of the base class, which is, as you yourself defined it, pure virtual.

Your "solution" to make it virtual and provide a default implementation will not help you. You might as well keep it pure virtual and provide an implemenation. Providing an implementation simply keeps your program from crashing, because there is always an implementation to call. Making it pure virtual simply forces derived classes to provide an implementation of their own. But it will never be called.

To summarise: In the destructor of a base class, only the members of the base class itself will be called. Never will overloaded members be called. Never. They do not exist anymore.

Some goes for constructors by the way, only overloaded members don't exist YET, instead of ANYMORE.

So if you want to define template behaviour in a constructor of destructor, as you seem to want to be doing, forget about it. It can't be done (for good reasons). Look up virtual constructor or destructor instead to find a proper solution.
Practical Geezer
Wednesday, January 24, 2007
 
 
I may be late to the party, but I would first isoltate whether this a lifecycle issue or not.  There are two separate approaches depending on the source of the proble.

If it is a lifecycle issue try one (or more) of the following:
1) Carefully review object lifecycles in your design and change to ensure that the object is created and delete in a common place; this often means moving the creation and deletion to a higher level in the code.
2) Try cloning an object instead of copying it.
3) Try wrapping an object in a smart pointer; the smart pointer wrapper will need to be thread-safe if you are doing threading.
4) Try implementing an instance count, incrementing in the constructor and decementing in the destructor.  I am not sure about the thread safety of this approach and this usually leads to greater complexity later, but if you need a really quick fix, this might be appropriate as a patch (go back and put a real fix in later).

If it is not a lifecycle issue, another problem I have seen is calling a child class virtual function from a base class destructor.  It is okay to call a child class virtual function from its own destructor.

This is okay:
class Child : Base
{
 virtual ~Child(void) { Close(); };
 virtual void Close(void) { /* whatever */ };
};

class : Base
{
 virtual ~Base(void) {};
 virtual void Close(void) = 0;
};

This will fail:

class Child : Base
{
 virtual ~Child(void) {};
 virtual void Close(void) { /* whatever */ };
};

class Base
{
 virtual ~Base(void) { Child(); };
 virtual void Close(void) = 0;
};

Remember, destructors (and constructors) are different than other functions.  All of the destructors in the chain get called, the destructors are not overloaded and replaced.
Wayne M.
Wednesday, January 24, 2007
 
 
Re: "Your "solution" to make it virtual and provide a default implementation will not help you. You might as well keep it pure virtual and provide an implemenation. Providing an implementation simply keeps your program from crashing, because there is always an implementation to call. Making it pure virtual simply forces derived classes to provide an implementation of their own. But it will never be called."

This doesn't work -- while you can provide an implementation for a pure virtual function, it does not get placed in the vtable -- the only way to call it is to explicitly qualify it, e.g., A::pure_method() (using the original example).

So (again using the original example), the only way to resolve this particular problem is to provide a non-pure virtual implementation in B.

(This is true at least with VC 7.1 -- haven't bothered to check other compilers).
BillT Send private email
Wednesday, January 24, 2007
 
 
"So (again using the original example), the only way to resolve this particular problem is to provide a non-pure virtual implementation in B."

If B is the derived class, this will NOT resolve the problem.
Once in the destructor of A (the base class), non of the members of B (the derived class) will be called, EVER. B does not exist anymore. All there is in the destructor are the member functions of A itself, even if they are virtual and have been overloaded by B, and those that A may have inherited itself.

Just once more: from the constructor or destructor of a base class, it is impossible to call a derived classes members. The derived class does not exist yet, or anymore.

Unless... the member is static of course.
Practical Geezer
Wednesday, January 24, 2007
 
 
Geezer: 

As per the orig example, A is the abstract class, B is the (concrete) base, and D is the derived class.
BillT Send private email
Wednesday, January 24, 2007
 
 
Oke, scrolling back I think you already knew that when in the  destructor of B members of D may no longer be called and when in the destructor of A, members of B nor D may no longer be called. Whether from the destructor itself, or from other threads.
Practical Geezer
Thursday, January 25, 2007
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz