Document number: N1450=03-0033
Programming Language C++, Library Subgroup
 
Peter Dimov <pdimov@mmltd.net>
Beman Dawes <bdawes@acm.org>
Greg Colvin <greg@colvin.org>
 
March 27, 2003

A Proposal to Add General Purpose Smart Pointers to the Library Technical Report

Revision 1

I. Motivation

Resource management, particularly of dynamically allocated memory, has been a serious concern since the days of the earliest digital computers. Many modern computer languages (Python, Perl, Java) provide built-in facilities for automatically managing dynamically allocated memory. In both the C and C++ languages, leaving dynamic memory management choices up to the programmer is a long-standing design decision.

Use of a pointer-like object called a smart-pointer to manage dynamically allocated memory has been the preferred approach in C++ for many years. Google finds 153,000 references to the term "smart pointer", with the high-ranking listings almost entirely in the context of C++ programming. Leading authors of C++ related books often treat smart pointers in depth. Of the 50 plus C++ libraries available on the Boost web site, the Boost Smart Pointer library is the most commonly visited. Smart pointers are encountered with great frequency in production code. There is clearly great interest and need for quality smart pointers in the C++ community.

The list of possible smart pointer features is extremely broad. The C++ Standard Library currently provides a single smart pointer, std::auto_ptr, which provides specific and focused transfer-of-ownership semantics. Arguably the most common need, however, is for shared-ownership semantics. Because std::auto_ptr does not provide shared-ownership semantics, it cannot even be used with Standard Library containers.

This document proposes a smart-pointer with shared-ownership semantics which meets a very broad spectrum of smart-pointer needs, is based on existing practice, and has many other beneficial characteristics. An implementation is available at http://www.boost.org/libs/smart_ptr.

A. Motivation for a shared-ownership smart pointer

A shared-ownership smart pointer:

B. Motivation for the shared_ptr shared-ownership smart pointer

[Rather than specifying memory management policy via class template parameters, shared_ptr has an alternate constructor with a second argument supplying a destruction policy functor. This is the key to understanding both the design itself and several of the claimed advantages.]

The proposed shared-ownership smart pointer:

C. Motivation for the weak_ptr shared-ownership-observer smart pointer

D. Implementation difficulty

The Boost developers found a shared-ownership smart pointer exceedingly difficult to implement correctly. Others have made the same observation. For example, Scott Meyers [Meyers01] says:

The STL itself contains no reference-counting smart pointer, and writing a good one - one that works correctly all the time - is tricky enough that you don't want to do it unless you have to. I published the code for a reference-counting smart pointer in More Effective C++ in 1996, and despite basing it on established smart pointer implementations and submitting it to extensive pre-publication reviewing by experienced developers, a small parade of valid bug reports has trickled in for years. The number of subtle ways in which reference-counting smart pointers can fail is remarkable.

Example 1

shared_ptr<X> p(new X);

It is very common for even expert implementations to leak the X object when the smart pointer constructor throws an exception.

Example 2

p.reset(new X);

A variation of the above example. The behavior when reset throws should be to delete the X object, and have no other effects, i.e. p must be left unchanged.

Example 3

Taken from Boost's shared_ptr_test.cpp:

struct X
{
    boost::shared_ptr<X> next;
};

void test()
{
    boost::shared_ptr<X> p(new X);
    p->next = boost::shared_ptr<X>(new X);
    p = p->next;
    BOOST_TEST(!p->next);
}

Again, it is surprising how many experts get this wrong.

weak_ptr represents an additional source of difficulty. Its stored pointer can be invalidated literally at any time; even in strictly single threaded and synchronous programs it is possible for the code to call a library that calls back a routine invalidating a weak_ptr. As the value of a pointer referring to deallocated storage is indeterminate (5.3.5/4), the weak_ptr implementation must take care not to access the stored pointer value after the weak_ptr has expired. Furthermore, the weak_ptr interface should protect users from accidentally accessing a pointer to deallocated storage.

II. Impact On the Standard

This proposal is almost a pure library extension. It proposes changes to an existing header, <memory>, but it does not require changes to any standard classes or functions and it does not require changes to any of the standard requirement tables. It does not require any changes in the core language, and it has been implemented in standard C++. The proposal does not depend on any other library extensions.

III. Design Decisions

A. General Principles

1. "As Close to Raw Pointers as Possible, but no Closer"

Ordinary C++ pointers have some interesting properties that make them suitable for a wide range of tasks. It is possible to copy, assign, pass and return by value. It is possible to "destroy" a pointer to an incomplete type. It is possible to refer to any object's address using a pointer to void. Pointers to derived classes are implicitly convertible to pointers to base classes, and it is possible to use static_cast or dynamic_cast to perform the opposite conversion.

The two main problems with ordinary pointers are the need for manual resource management and the possibility of accessing an invalid (dangling) pointer.

The proposed smart pointers retain the strengths of raw pointers, eliminating their weaknesses. shared_ptr and weak_ptr fully support incomplete types, cv-qualified types, and void as a template parameter. They support derived-to-base implicit conversions. shared_ptr supports static and dynamic casts, and will automatically delete (or otherwise release) the object it owns when the last shared_ptr instance is destroyed. weak_ptr eliminates the possibility of accessing a dangling pointer.

2. "Just Do the Right Thing"

The versatility described in the previous paragraphs can lead to some tricky situations. A shared_ptr can be created in a shared library and then destroyed by the application. By using implicit conversions, the last remaining shared_ptr instance to an object may turn out to have a template parameter of void, a base with an inaccessible or nonvirtual destructor, or an incomplete type.

The general principle followed by the proposed design is that in such situations, the implementation should "just do the right thing" and invoke the proper destructor or operator delete. This implies that the information necessary to properly destroy the managed object is fully captured at the point where the smart pointer is constructed.

3. No Extra Parameters

Following the "as close as possible" principle, the proposed smart pointers have a single template parameter, the type of the pointee. Avoiding additional parameters ensures interoperability between libraries from different authors, and also makes shared_ptr easier to use, teach and recommend.

4. The "Shares Ownership with" Equivalence Relation

The proposed text uses "p shares ownership with q" to indicate that p and q have both originated from the same smart pointer, and own the same object. "Shares ownership with" is an equivalence relation. It is reflexive (p shares ownership with p), commutative (if p shares ownership with q, q also shares ownership with p), and transitive (if p shares ownership with q and q shares ownership with r, p shares ownership with r). This relation divides all non-empty smart pointers into equivalence classes. The number of shared_ptr instances in p's equivalence class can be obtained using p.use_count(). The last shared_ptr instance in a given equivalence class is responsible for destroying the owned object.

In a typical counted implementation, two smart pointers share ownership when they share the same reference count. In a linked list implementation, two smart pointers share ownership when they are part of the same list.

5. The "Empty Pointer" Concept

Default-constructed smart pointers, and smart pointer instances that have been reset(), are called empty in the proposed text. It is necessary to introduce this concept to offer the nothrow guarantee for shared_ptr::shared_ptr(), shared_ptr::reset(), weak_ptr::weak_ptr(), and weak_ptr::reset(). p.reset() for smart pointers is often used as an equivalent for delete p for raw pointers, hence the need to guarantee that this operation doesn't throw.

The infrastructure required to keep track of ownership may need to dynamically allocate memory. To ensure that default constructors and reset() do not throw, these operations are allowed to skip this allocation, and produce empty smart pointers that do not track ownership, and do not fully support ownership-related queries such as use_count().

One reasonable implementation of the empty pointer concept is to reuse one or more statically allocated reference counts for default-constructed pointer instances. Under this model, empty pointers do represent one or more equivalence classes under the ownership equivalence relation described in the previous section, and use_count() can return "realistic" values that increase and decrease as additional empty pointers are created or destroyed.

Another approach, used in the Boost implementation, is to not allocate a reference count at all for empty pointers, using a NULL pointer to count as an "empty mark". Under this scheme empty pointers have a constant use_count() (zero in the Boost implementation.)

Having empty pointers always return zero from use_count(), and hence eliminating nearly all of the unspecified behavior, is a good thing from users' point of view. The following example (supplied by Howard Hinnant) illustrates the motivation for this additional requirement:

int main()
{
    try
    {
        shared_ptr<int> sp0;
        weak_ptr<int> wp(sp0);
        shared_ptr<int> sp(wp);
        cout << "Done\n";
    }
    catch (exception const & e)
    {
        cerr << e.what() << '\n';
    }
}

The original version of the proposal conservatively assumed that imposing the requirement that empty pointers have use_count() == 0 on implementations might effectively disallow implementation #1 as reliably detecting an empty pointer can be expensive. It was unspecified whether the code above would print "Done" or "std::bad_weak_ptr".

We have since produced a proof of concept implementation that follows the first model and statisfies the requirement without significant size or performance implications. As a result, this version of the proposal requires use_count() == 0 to hold for all empty pointers, and the code sample will reliably print "std::bad_weak_ptr".

The authors thank Howard Hinnant for his observations and help with preparing the proof of concept implementation.

B. shared_ptr

1. Pointer Constructor

The shared_ptr constructor that takes a raw pointer is templated on the pointer type, in order to capture as much information as possible at the point of construction. This allows, for example, a shared_ptr<void> instance constructed as:

shared_ptr<void> px(new X);

to properly invoke ~X at destruction time, in accordance with the "just do the right thing" principle.

The templated constructor cannot be used to construct a shared_ptr from the literal zero, or another integral constant expression with a value of zero, since the type of the pointer cannot be determined. The shared_ptr equivalent of NULL is obtained from the default constructor. In contexts where it's important to obtain a NULL instance of a pointer type P in a generic way, the construct:

P p = P();

may be used instead of

P p(0);

Implementations are allowed to throw exceptions other than std::bad_alloc since on some platforms they might need to create a mutex to synchronize the access to the ownership infrastructure.

A common concern, dating back to [Colvin94], is that it is very easy to violate the preconditions of smart pointer operations. This proposal attempts to systematically eliminate undefined behavior possibilities as long as the user is manipulating already valid shared_ptr (and weak_ptr) instances, but using the pointer constructor correctly is still up to the user.

One possible alternative is to eliminate this unsafe constructor, and instead provide a family of factory functions of the form:

template<class T> shared_ptr<T> shared_new()
{
    return shared_ptr<T>(new T());
}

template<class T, class A1> shared_ptr<T> shared_new(A1 const & a1)
{
    return shared_ptr<T>(new T(a1));
}

template<class T, class A1, class A2> shared_ptr<T> shared_new(A1 const & a1, A2 const & a2)
{
    return shared_ptr<T>(new T(a1, a2));
}

This approach has not been adopted for the following reasons:

As an alternative, the Boost implementation offers an experimental debug mode [Boost03a] that catches many common errors at runtime, including misuse of the pointer constructor and invoking delete on a pointer managed by shared_ptr.

2. Deleter Constructor

Custom deleters allow a factory function returning a shared_ptr to insulate the shared_ptr user from its memory allocation strategy. Since the deleter is not part of the type, changing the allocation strategy does not break source or binary compatibility, and does not require a client recompilation. For example, a "no-op" deleter is useful when returning a shared_ptr to a statically allocated object. A deleter that invokes the Release() member function enables shared_ptr to hold a pointer to a COM object, and other variations allow a shared_ptr to be used as a wrapper for another smart pointer, easing interoperability. See [Dimov03] for many other examples for the tremendous expressive power gained by the custom deleter support.

In the Boost implementation, supporting custom deleters does not impose additional memory overhead on the shared_ptr itself. See Memory Footprint [E.1]. Earlier shared_ptr versions did not offer the deleter constructor, but in order to meet the "just do the right thing" requirement (III.A.2), the implementation kept a deleter under the hood anyway.

The requirement for the copy constructor of D to not throw comes from the pass by value. If the copy constructor throws, the pointer is leaked. Removing the requirement requires a pass by (const) reference.

The main problem with pass by reference lies in its interaction with rvalues. A const reference may still cause a copy, and will require a const D::operator(). A non-const reference won't bind to an rvalue at all. A good solution to this problem is the rvalue reference proposed in [Hinnant02].

For the Boost implementation, an additional reason to avoid pass by reference has been portability; pass by value handles functions and function references well on a wide range of compilers.

3. weak_ptr Constructor

shared_ptr provides a converting constructor from weak_ptr, with the postcondition that the newly constructed shared_ptr shares ownership with the weak_ptr argument (when non-empty.) In other words, it is possible to copy a shared_ptr using an intermediate weak_ptr, and the copy and the original will share ownership. If this postcondition cannot be met, as will happen if all shared_ptr instances sharing ownership with the weak_ptr argument have been already destroyed, the constructor throws an exception of type bad_weak_ptr.

An alternative way of obtaining a shared_ptr from a weak_ptr is to use weak_ptr's member function lock. It indicates failure by returning an empty shared_ptr.

4. std::auto_ptr Constructor

shared_ptr provides a converting constructor from std::auto_ptr. It is almost equivalent to calling release() on the source auto_ptr and using the raw pointer constructor; the only difference is that this constructor provides the strong exception safety guarantee, and in case an exception is thrown, it will leave the source auto_ptr unchanged. To achieve this, the constructor takes the source by a non-const reference; pass by value would be equivalent to using release().

If the "rvalue reference" proposed in [Hinnant02] is accepted, the constructor can take advantage of it in order to be able to take an auto_ptr rvalue as an argument.

5. Assignment and reset

Assignment and reset are specified in terms of construction and swap, but the intent is to not preclude more efficient implementations that have the same effects, hence the note.

6. operator->

operator-> requires that get() != 0 in order to allow conforming implementations to assert in debug mode.

7. use_count and unique

use_count is provided as a testing and debugging aid rather than for use in production code. It allows postconditions to be specified in a form that can be tested by client code, and it has been proven useful in practice when tracking down bugs in projects with complex ownership structures. Production code should not rely on use_count as it may be inefficient. While all known reasonable shared ownership implementations can support use_count, not all can do so in an efficient manner.

The common use_count() == 1 query has been given its own name for efficiency reasons, since some implementations can provide a constant time unique() but not a constant time use_count().

The return type of use_count is signed to avoid pitfalls such as p.use_count() > -1 evaluating to false [Lakos96].

The long return type of use_count does not imply that implementations should always use long for the reference count. In some cases, size_t or its signed equivalent may be a better candidate for the actual type used underneath.

8. Conversion Operator

To meet the "convertible to bool" requirement, shared_ptr has a conversion operator to an unspecified type (the Boost implementation uses a pointer to member).

std::basic_ios's approach of having a conversion to void* has been deemed unsuitable for a smart pointer. Such a conversion enables far too many undesirable constructs to compile (one striking example is invoking delete with a shared_ptr operand.) When migrating code that uses raw pointers to shared_ptr, it is important for constructs that were valid for the original raw pointer based code, but are errors under the new design, to be caught at compile time.

9. operator<

operator< does not compare the stored pointers. It is implemented in terms of the ownership infrastructure. A typical counted implementation would compare the count pointers (using std::less if appropriate.)

One reason to avoid stored pointer comparisons is consistency with weak_ptr, where the value of the stored pointer may be (or become) indeterminate without affecting the ordering provided by operator<. Another motivation for this operator< behavior is support for the std::map< shared_ptr<X>, V > idiom that associates data with objects managed by shared_ptr (with an important special case being X=void.) [Dimov03]

In most typical scenarios, a std::map with shared_ptr as key would behave as if the stored pointers have been used. The exception is that in some situations involving multiple or virtual inheritance, it is possible for two pointers of the same type to point to different sub-objects within the same object. Therefore, two shared_ptr instances storing such pointers but sharing ownership will be considered equivalent by operator<, but may not compare equal when using operator== (defined as a stored pointer comparison.)

One additional benefit of this operator< specification is that it enables "does p share ownership with q" queries to be expressed as !(p < q) && !(q < p).

10. operator<<

A stream insertion operator is explicitly provided for shared_ptr since the conversion operator may cause shared_ptr to be treated in stream insertion contexts as if it were a bool.

11. Casts

Two cast functions, static_pointer_cast and dynamic_pointer_cast, are provided for shared_ptr.

static_pointer_cast is useful when heterogeneous shared_ptr instances are temporarily stored in a container as shared_ptr<void>, along with a corresponding function pointer or polymorphic function object wrapper [Gregor02] that remembers the original type and is able to use static_pointer_cast to obtain the original shared_ptr.

dynamic_pointer_cast is useful for passing shared_ptr instances through a third-party library, then recognizing, and recovering, the original value, as described in the original dynamic_cast rationale [Stroustrup94].

reinterpret_cast and const_cast equivalents have been omitted since they have never been requested by users (although it's possible to emulate a reinterpret_pointer_cast by using an intermediate shared_ptr<void> and a static_pointer_cast).

The cast operations were originally named shared_static_cast and shared_dynamic_cast in the Boost implementation, but it soon became clear that other smart pointer types can (and should) provide the same functionality, preferably in a generic manner, and the functions were renamed to *_pointer_cast to indicate wider applicability.

12. get_deleter

get_deleter is useful for libraries that create shared_ptr instances for client code, and when the client passes a shared_ptr back, need to determine whether that particular shared_ptr originated from the library, and if so, need to access private data associated with the instance that was stored in the deleter at creation.

One example is a library that implements a predefined shared_ptr-based interface. The library needs to interoperate with an existing infrastructure that uses a different smart pointer for its resource management, and the technique described in [Dimov03] is used to wrap the existing smart pointers with a shared_ptr "shell". When the library receives a shared_ptr from the client, it can extract the original wrapped smart pointer (if there is one) by using get_deleter.

The Boost.Python library [Abrahams02] uses get_deleter to extract the original Python object from a shared_ptr.

C. weak_ptr

1. Default Constructor

Default-constructed weak_ptr instances have a use_count() of zero, and attempts to create a shared_ptr from such an instance will fail with a bad_weak_ptr exception. This behavior supports some use cases where a weak_ptr needs to be "two phase constructed", i.e. first default-constructed, then initialized from a shared_ptr via assignment. If a shared_ptr conversion is attempted before initialization, an exception is the appropriate response. See, for example, the "Obtaining a shared_ptr to this" technique in [Dimov03].

2. Converting Constructor

A converting constructor enables weak_ptr<Y> to be converted to weak_ptr<T> when Y* is convertible to T*. This is a standard "feature" of most smart pointer designs and would not deserve mention except for the fact that in weak_ptr's case it is easy to produce an implementation with a subtle bug. When the source weak_ptr has expired, its stored pointer may refer to deallocated storage, its value may be indeterminate, and the implicit conversion from Y* to T* may exhibit "observable" undefined behavior such as a core dump.

Strictly speaking, this caveat applies to the copy constructor as well, but most existing implementations allow an invalid pointer to be copied (and those that don't can use std::memcpy to copy the pointer storage without accessing its value). When Y inherits from T virtually, however, most existing implementations will fail to convert an invalid pointer of type Y* to T*.

This constructor has been provided to support the useful idiom of using std::map< weak_ptr<void>, X > to non-intrusively associate arbitrary data with existing shared_ptr-managed objects.

3. lock

weak_ptr does not provide the usual smart pointer accessors operator*, operator->, or get. Any attempt to use their return values would be inherently unsafe, as weak_ptr's stored pointer can be invalidated at any time. Instead, weak_ptr provides a member function lock that returns a shared_ptr. The returned shared_ptr keeps the object alive, effectively locking it against deletion.

The idiomatic way to access a weak_ptr is as shown below:

weak_ptr<X> target;

// ...

if(shared_ptr<X> px = target.lock())
{
  // use *px to fire at target
}
else
{
  // target lost (expired), acquire a new one
}

D. enable_shared_from_this

Experienced smart pointer users often need the ability to obtain a shared_ptr to this from within a member function. While the technique described in [Dimov03] solves the problem, it leaves the initialization of the weak_ptr member to the user.

This proposal provides direct support for the idiom via the enable_shared_from_this helper class template. In a typical implementation, the three shared_ptr constructors that create unique pointers automatically detect the enable_shared_from_this base and initialize its internal weak_ptr member.

The default constructor, copy constructor, copy assignment operator, and the destructor of enable_shared_from_this are protected. enable_shared_from_this<X> is only useful as a public base class of X.

E. Size and Performance

1. Memory Footprint

In the Boost implementation, the size of shared_ptr (and weak_ptr) is twice the size of a raw pointer. More precisely, sizeof(shared_ptr<T>) == sizeof(weak_ptr<T>) == sizeof(T*) + sizeof(C*), where C is a structure holding the reference counts and other information. This "twin pointer" layout is typical for most existing reference counted smart pointers.

The count structure C has the following members:

On a typical implementation where a pointer and a long are both four bytes in size and no custom deleters are used, sizeof(C) is 20 in singlethreaded builds, 24 in multithreaded builds on Windows (the implementation uses a custom spinlock-based mutex), and 44 on Linux using the platform's pthread_mutex_t.

It is possible to save four bytes when the deleter is an empty class (most deleters are) by taking advantage of the empty base optimization, for example by using a helper component such as boost::compressed_pair [Boost00]. The Boost implementation does not attempt to optimize away the deleter space in order to minimize dependencies and maximize portability (boost::shared_ptr is a critical dependency for many Boost libraries and it is important for it to work reliably on all supported platforms.)

The mutex presents another target for platform-specific optimizations. On Windows, we have been able to use a simple one-word spinlock instead of a 6 word CRITICAL_SECTION. The portable architecture of the implementation that abstracts the mutex concept in a separate class prevents further optimizations, like reusing the use count word as the spinlock, or using the Windows API InterlockedIncrement, InterlockedDecrement, and InterlockedCompareExchange primitives to attempt to eliminate the mutex altogether. Nevertheless, we believe that such optimizations are possible.

The rest of the members of the count structure are a "necessary evil" as far as size is concerned. The virtual table pointer is required for the polymorphic behavior needed for destroying the original pointer using its original type, and to support dynamic libraries on platforms such as Windows where the application and the dynamic library can have their own incompatible versions of ::operator new and ::operator delete. The weak count supports weak_ptr. The original pointer passed at construction time needs to be remembered for shared_ptr<X> to be able to correctly destroy the object when X is incomplete or void, ~X is inaccessible, or ~X is not virtual.

An earlier, less featured, version of boost::shared_ptr used a single long as C. The transition to the new version having a much larger count structure went painlessly, and we registered no user complaints.

2. Construction Performance

The Boost implementation allocates a reference count structure whenever a shared_ptr instance is constructed from a pointer, the typical case being:

shared_ptr<X> px(new X);

Note that in the common case there are two allocations being performed, an object allocation (new X) and a count allocation.

On a typical desktop machine (Athlon 1.4GHz running Windows XP), 1048576 (220) such statements take about 1.75 seconds, a more than adequate performance considering that typical code rarely needs to repeatedly allocate so many objects.

We have also experimented with using a pool allocator for the count structure. The same test is executed for about 1.15 seconds under the same implementation. However, the additional step of reusing the same allocator for the object allocation as well gives a result of about 0.78 seconds, with the likely conclusion (also supported by [Berger02]) that it would be better to optimize the global ::operator new allocator instead of focusing on the internal allocations performed by shared_ptr.

The source code used for the test is available as part of the Boost implementation.

3. Copy Performance

In multi-threaded builds, the Boost implementation always synchronizes reference count operations to make shared_ptr "as thread safe as an int", consistent with the behavior of most standard libraries. This sometimes raises performance concerns since in some cases users can guarantee that two shared_ptr instances that share ownership will never be accessed simultaneously from different threads.

The configuration described above is able to execute 8388608 (223) push_back operations into a std::vector< shared_ptr<int> > (without calling reserve first) in about 2.63 seconds in a single threaded build, 5.44 seconds in a multi-threaded build. A factor of two increase seems like a lot, but the difference rarely impacts real code, since it never needs to repeatedly make so many shared_ptr copies.

The source code used for the test is available as part of the Boost implementation.

F. Alternatives

1. Policy Based Smart Pointers

A policy-based smart pointer (PBSP) design, popularized by Andrei Alexandrescu [Alexandrescu01], could provide all of the features in the current proposal plus could be user-customizable to offer many other variations on the general smart pointer theme.

As attractive as PBSP might be, the authors of the current proposal believe:

If and when a policy-based smart pointer design matures to the point where it is suitable for standardization, a std::shared_ptr typedef template or equivalent could be provided which delivers the same interface and semantics as the current proposal. The interface and semantics of std::shared_ptr have been refined over the years as the best choice for general shared-ownership smart pointer use, including interlibrary communication. Thus any future PBSP would almost certainly supply these common needs as the default policies. That means that implementing a future version of std::shared_ptr as some form of wrapper around a PBSP would always be an option should a PBSP become part of some future standard.

2. Garbage Collection

The detailed rationale for not providing automatic garbage collection as the default memory management technique for C++ is given by [Stroustrup94a], who states that "Optional garbage collection is, I think, the right approach for C++".

Because the current proposal is for a purely library solution, it is complementary rather than competitive with any future proposal to add optional garbage collection as a built-in feature of the language itself.

The Boost implementation includes an experimental collector [Boost03b] based on the algorithm described in [Christopher84] that can be used to detect and free unreachable objects with cyclic shared_ptr references. We have only limited experience with this feature, and it is not part of this proposal.

IV. Proposed Text

A. Additions to header <memory> synopsis (20.4)

  class bad_weak_ptr;

  template<class T> class shared_ptr;

  template<class T, class U>
    bool operator==(shared_ptr<T> const & a, shared_ptr<U> const & b);

  template<class T, class U>
    bool operator!=(shared_ptr<T> const & a, shared_ptr<U> const & b);

  template<class T, class U>
    bool operator<(shared_ptr<T> const & a, shared_ptr<U> const & b);

  template<class T>
    void swap(shared_ptr<T> & a, shared_ptr<T> & b);

  template<class T, class U>
    shared_ptr<T> static_pointer_cast(shared_ptr<U> const & r);

  template<class T, class U>
    shared_ptr<T> dynamic_pointer_cast(shared_ptr<U> const & r);

  template<class E, class T, class Y>
    basic_ostream<E, T> & operator<< (basic_ostream<E, T> & os, shared_ptr<Y> const & p);

  template<class D, class T>
    D * get_deleter(shared_ptr<T> const & p);

  template<class T> class weak_ptr;

  template<class T, class U>
    bool operator<(weak_ptr<T> const & a, weak_ptr<U> const & b);

  template<class T>
    void swap(weak_ptr<T> & a, weak_ptr<T> & b);

  template<class T> class enable_shared_from_this;

B. Class bad_weak_ptr

    namespace std {
      class bad_weak_ptr: public exception
      {
      public:
        bad_weak_ptr();
      };
    }

An exception of type bad_weak_ptr is thrown by the shared_ptr constructor taking a weak_ptr.

    bad_weak_ptr();

Postconditions: what() returns "std::bad_weak_ptr".

Throws: nothing.

C. Class template shared_ptr

The shared_ptr class template stores a pointer, usually obtained via new. shared_ptr implements semantics of shared ownership; the last remaining owner of the pointer is responsible for destroying the object, or otherwise releasing the resources associated with the stored pointer.

    namespace std {
      template<class T> class shared_ptr {
      public:
        typedef T element_type;

        // constructors
        shared_ptr();
        template<class Y> explicit shared_ptr(Y * p);
        template<class Y, class D> shared_ptr(Y * p, D d);
        shared_ptr(shared_ptr const & r);
        template<class Y> shared_ptr(shared_ptr<Y> const & r);
        template<class Y> explicit shared_ptr(weak_ptr<Y> const & r);
        template<class Y> explicit shared_ptr(auto_ptr<Y> & r);

        // destructor
        ~shared_ptr();

        // assignment
        shared_ptr & operator=(shared_ptr const & r);
        template<class Y> shared_ptr & operator=(shared_ptr<Y> const & r);
        template<class Y> shared_ptr & operator=(auto_ptr<Y> & r);

        // modifiers
        void swap(shared_ptr & r);
        void reset();
        template<class Y> void reset(Y * p);
        template<class Y, class D> void reset(Y * p, D d);

        // observers
        T * get() const;
        T & operator*() const;
        T * operator->() const;
        long use_count() const;
        bool unique() const;
        operator unspecified-bool-type() const;
      };

      // comparison
      template<class T, class U>
        bool operator==(shared_ptr<T> const & a, shared_ptr<U> const & b);

      template<class T, class U>
        bool operator!=(shared_ptr<T> const & a, shared_ptr<U> const & b);

      template<class T, class U>
        bool operator<(shared_ptr<T> const & a, shared_ptr<U> const & b);

      // other operators
      template<class E, class T, class Y>
        basic_ostream<E, T> & operator<< (basic_ostream<E, T> & os, shared_ptr<Y> const & p);

      // specialized algorithms
      template<class T> void swap(shared_ptr<T> & a, shared_ptr<T> & b);

      // casts
      template<class T, class U>
        shared_ptr<T> static_pointer_cast(shared_ptr<U> const & r);

      template<class T, class U>
        shared_ptr<T> dynamic_pointer_cast(shared_ptr<U> const & r);

      // get_deleter
      template<class D, class T>
        D * get_deleter(shared_ptr<T> const & p);
    }

shared_ptr is CopyConstructible, Assignable, and LessThanComparable, allowing its use in standard containers. shared_ptr is convertible to bool, allowing its use in boolean expressions and declarations in conditions.

[Example:

    if(shared_ptr<X> px = dynamic_pointer_cast<X>(py))
    {
      // do something with px
    }

—end example.]

shared_ptr constructors

    shared_ptr();

Effects: Constructs an empty shared_ptr.

Postconditions: use_count() == 0 && get() == 0.

Throws: nothing.

    template<class Y> explicit shared_ptr(Y * p);

Requires: p must be convertible to T *. Y must be a complete type. The expression delete p must be well-formed, must not invoke undefined behavior, and must not throw exceptions.

Effects: Constructs a shared_ptr that owns the pointer p.

Postconditions: use_count() == 1 && get() == p.

Throws: bad_alloc or an implementation-defined exception when a resource other than memory could not be obtained.

Exception safety: If an exception is thrown, delete p is called.

    template<class Y, class D> shared_ptr(Y * p, D d);

Requires: p must be convertible to T *. D must be CopyConstructible. The copy constructor and destructor of D must not throw. The expression d(p) must be well-formed, must not invoke undefined behavior, and must not throw exceptions.

Effects: Constructs a shared_ptr that owns the pointer p and the deleter d.

Postconditions: use_count() == 1 && get() == p.

Throws: bad_alloc or an implementation-defined exception when a resource other than memory could not be obtained.

Exception safety: If an exception is thrown, d(p) is called.

    shared_ptr(shared_ptr const & r);
    template<class Y> shared_ptr(shared_ptr<Y> const & r);

Effects: If r is empty, constructs an empty shared_ptr; otherwise, constructs a shared_ptr that shares ownership with r.

Postconditions: get() == r.get() && use_count() == r.use_count().

Throws: nothing.

    template<class Y> explicit shared_ptr(weak_ptr<Y> const & r);

Effects: Constructs a shared_ptr that shares ownership with r and stores a copy of the pointer stored in r.

Postconditions: use_count() == r.use_count().

Throws: bad_weak_ptr when r.expired().

Exception safety: If an exception is thrown, the constructor has no effect.

    template<class Y> shared_ptr(auto_ptr<Y> & r);

Requires: r.release() must be convertible to T *. Y must be a complete type. The expression delete r.release() must be well-formed, must not invoke undefined behavior, and must not throw exceptions.

Effects: Constructs a shared_ptr that stores and owns r.release().

Postconditions: use_count() == 1.

Throws: bad_alloc or an implementation-defined exception when a resource other than memory could not be obtained.

Exception safety: If an exception is thrown, the constructor has no effect.

shared_ptr destructor

    ~shared_ptr();

Effects:

Throws: nothing.

shared_ptr assignment

    shared_ptr & operator=(shared_ptr const & r);
    template<class Y> shared_ptr & operator=(shared_ptr<Y> const & r);
    template<class Y> shared_ptr & operator=(auto_ptr<Y> & r);

Effects: Equivalent to shared_ptr(r).swap(*this).

Returns: *this.

Notes: The use count updates caused by the temporary object construction and destruction are not considered observable side effects, and the implementation is free to meet the effects (and the implied guarantees) via different means, without creating a temporary. In particular, in the example:

  shared_ptr<int> p(new int);
  shared_ptr<void> q(p);
  p = p;
  q = p;

both assignments may be no-ops.

shared_ptr modifiers

    void swap(shared_ptr & r);

Effects: Exchanges the contents of *this and r.

Throws: nothing.

    void reset();

Effects: Equivalent to shared_ptr().swap(*this).

    template<class Y> void reset(Y * p);

Effects: Equivalent to shared_ptr(p).swap(*this).

    template<class Y, class D> void reset(Y * p, D d);

Effects: Equivalent to shared_ptr(p, d).swap(*this).

shared_ptr observers

    T * get() const;

Returns: the stored pointer.

Throws: nothing.

    T & operator*() const;

Requires: get() != 0.

Returns: *get().

Throws: nothing.

Notes: When T is void, the return type of this member function is unspecified, and an attempt to instantiate it renders the program ill-formed.

    T * operator->() const;

Requires: get() != 0.

Returns: get().

Throws: nothing.

    long use_count() const;

Returns: the number of shared_ptr objects, *this included, that share ownership with *this, or 0 when *this is empty.

Throws: nothing.

Notes: use_count() is not necessarily efficient. Use only for debugging and testing purposes, not for production code.

    bool unique() const;

Returns: use_count() == 1.

Throws: nothing.

Notes: unique() may be faster than use_count(). If you are using unique() to implement copy on write, do not rely on a specific value when get() == 0.

    operator unspecified-bool-type () const;

Returns: an unspecified value that, when used in boolean contexts, is equivalent to get() != 0.

Throws: nothing.

Notes: This conversion operator allows shared_ptr objects to be used in boolean contexts, like if (p && p->valid()) {}. The actual target type is typically a pointer to a member function, avoiding many of the implicit conversion pitfalls.

shared_ptr comparison

    template<class T, class U>
      bool operator==(shared_ptr<T> const & a, shared_ptr<U> const & b);

Returns: a.get() == b.get().

Throws: nothing.

    template<class T, class U>
      bool operator!=(shared_ptr<T> const & a, shared_ptr<U> const & b);

Returns: a.get() != b.get().

Throws: nothing.

    template<class T, class U>
      bool operator<(shared_ptr<T> const & a, shared_ptr<U> const & b);

Returns: an unspecified value such that

Throws: nothing.

Notes: Allows shared_ptr objects to be used as keys in associative containers.

shared_ptr operators

    template<class E, class T, class Y>
      basic_ostream<E, T> & operator<< (basic_ostream<E, T> & os, shared_ptr<Y> const & p);

Effects: os << p.get();.

Returns: os.

shared_ptr specialized algorithms

    template<class T>
      void swap(shared_ptr<T> & a, shared_ptr<T> & b);

Effects: Equivalent to a.swap(b).

Throws: nothing.

shared_ptr casts

    template<class T, class U>
      shared_ptr<T> static_pointer_cast(shared_ptr<U> const & r);

Requires: The expression static_cast<T*>(r.get()) must be well-formed.

Returns: If r is empty, an empty shared_ptr<T>; otherwise, a shared_ptr<T> object that stores static_cast<T*>(r.get()) and shares ownership with r.

Throws: nothing.

Notes: the seemingly equivalent expression

shared_ptr<T>(static_cast<T*>(r.get()))

will eventually result in undefined behavior, attempting to delete the same object twice.

    template<class T, class U>
      shared_ptr<T> dynamic_pointer_cast(shared_ptr<U> const & r);

Requires: The expression dynamic_cast<T*>(r.get()) must be well-formed and its behavior defined.

Returns:

Throws: nothing.

Notes: the seemingly equivalent expression

shared_ptr<T>(dynamic_cast<T*>(r.get()))

will eventually result in undefined behavior, attempting to delete the same object twice.

get_deleter

    template<class D, class T>
      D * get_deleter(shared_ptr<T> const & p);

Returns: If *this owns a deleter d of type cv-unqualified D, returns &d; otherwise returns 0.

Throws: nothing.

D. Class template weak_ptr

The weak_ptr class template stores a "weak reference" to an object that's already managed by a shared_ptr. To access the object, a weak_ptr can be converted to a shared_ptr using the member function lock.

    namespace std {
      template<class T> class weak_ptr {

      public:
        typedef T element_type;

        // constructors
        weak_ptr();
        template<class Y> weak_ptr(shared_ptr<Y> const & r);
        weak_ptr(weak_ptr const & r);
        template<class Y> weak_ptr(weak_ptr<Y> const & r);

        // destructor
        ~weak_ptr();

        // assignment
        weak_ptr & operator=(weak_ptr const & r);
        template<class Y> weak_ptr & operator=(weak_ptr<Y> const & r);
        template<class Y> weak_ptr & operator=(shared_ptr<Y> const & r);

        // modifiers
        void swap(weak_ptr & r);
        void reset();

        // observers
        long use_count() const;
        bool expired() const;
        shared_ptr<T> lock() const;
      };

      // comparison
      template<class T, class U>
        bool operator<(weak_ptr<T> const & a, weak_ptr<U> const & b);

      // specialized algorithms
      template<class T>
        void swap(weak_ptr<T> & a, weak_ptr<T> & b);
    }

weak_ptr is CopyConstructible, Assignable, and LessThanComparable, allowing its use in standard containers.

weak_ptr constructors

    weak_ptr();

Effects: Constructs an empty weak_ptr.

Postconditions: use_count() == 0.

Throws: nothing.

    template<class Y> weak_ptr(shared_ptr<Y> const & r);
    weak_ptr(weak_ptr const & r);
    template<class Y> weak_ptr(weak_ptr<Y> const & r);

Effects: If r is empty, constructs an empty weak_ptr; otherwise, constructs a weak_ptr that shares ownership with r and stores a copy of the pointer stored in r.

Postconditions: use_count() == r.use_count().

Throws: nothing.

weak_ptr destructor

    ~weak_ptr();

Effects: Destroys this weak_ptr but has no effect on the object its stored pointer points to.

Throws: nothing.

weak_ptr assignment

    weak_ptr & operator=(weak_ptr const & r);
    template<class Y> weak_ptr & operator=(weak_ptr<Y> const & r);
    template<class Y> weak_ptr & operator=(shared_ptr<Y> const & r);

Effects: Equivalent to weak_ptr(r).swap(*this).

Throws: nothing.

Notes: The implementation is free to meet the effects (and the implied guarantees) via different means, without creating a temporary.

weak_ptr modifiers

    void swap(weak_ptr & r);

Effects: Exchanges the contents of *this and r.

Throws: nothing.

    void reset();

Effects: Equivalent to weak_ptr().swap(*this).

weak_ptr observers

    long use_count() const;

Returns: 0 if *this is empty; otherwise, the number of shared_ptr instances that share ownership with *this.

Throws: nothing.

Notes: use_count() is not necessarily efficient. Use only for debugging and testing purposes, not for production code.

    bool expired() const;

Returns: use_count() == 0.

Throws: nothing.

Notes: expired() may be faster than use_count().

    shared_ptr<T> lock() const;

Returns: expired()? shared_ptr<T>(): shared_ptr<T>(*this).

Throws: nothing.

weak_ptr comparison

    template<class T, class U>
      bool operator<(weak_ptr<T> const & a, weak_ptr<U> const & b);

Returns: an unspecified value such that

Throws: nothing.

Notes: Allows weak_ptr objects to be used as keys in associative containers.

weak_ptr specialized algorithms

    template<class T>
      void swap(weak_ptr<T> & a, weak_ptr<T> & b)

Effects: Equivalent to a.swap(b).

Throws: nothing.

E. Class template enable_shared_from_this

A class can derive from the enable_shared_from_this class template, passing itself as a template parameter, to inherit the shared_from_this member functions that obtain a shared_ptr instance pointing to this.

[Example:

    struct X: public enable_shared_from_this<X>
    {
    };

    int main()
    {
        shared_ptr<X> p(new X);
        shared_ptr<X> q = p->shared_from_this();
        assert(p == q);
        assert(!(p < q ) && !(q < p)); // p and q must share ownership
    }

—end example.]

    namespace std {
      template<class T> class enable_shared_from_this {
      protected:
        enable_shared_from_this();
        enable_shared_from_this(enable_shared_from_this const &);
        enable_shared_from_this & operator=(enable_shared_from_this const &);
        ~enable_shared_from_this();
      public:
        shared_ptr<T> shared_from_this();
        shared_ptr<T const> shared_from_this() const;
      };
    }
    enable_shared_from_this<T>::enable_shared_from_this();
    enable_shared_from_this<T>::enable_shared_from_this(enable_shared_from_this<T> const &);

Effects: Constructs an enable_shared_from_this<T> instance.

Throws: nothing.

    enable_shared_from_this<T> & enable_shared_from_this<T>::operator=(enable_shared_from_this<T> const &);

Returns: *this.

Throws: nothing.

    enable_shared_from_this<T>::~enable_shared_from_this();

Effects: Destroys *this.

Throws: nothing.

    template<class T> shared_ptr<T> 
      enable_shared_from_this<T>::shared_from_this();
    template<class T> shared_ptr<T const> 
      enable_shared_from_this<T>::shared_from_this() const;

Requires: enable_shared_from_this<T> must be an accessible base class of T. *this must be a subobject of an instance t of type T . There must exist at least one shared_ptr instance p that owns &t.

Returns: A shared_ptr<T> instance r that shares ownership with p.

Postconditions: r.get() == this.

[Note: a typical implementation is shown below:

    template<class T> class enable_shared_from_this
    {
    private:

        weak_ptr<T> __weak_this;

    protected:

        enable_shared_from_this() {}
        enable_shared_from_this(enable_shared_from_this const &) {}
        enable_shared_from_this & operator=(enable_shared_from_this const &) { return *this; }
        ~enable_shared_from_this() {}

    public:

        shared_ptr<T> shared_from_this() { return shared_ptr<T>(__weak_this); }
        shared_ptr<T const> shared_from_this() const { return shared_ptr<T const>(__weak_this); }
    };

The three shared_ptr constructors that create unique pointers should detect the presence of an enable_shared_from_this base and assign the newly created shared_ptr to its __weak_this member. —end note.]

V. Revision History

Revision 1:

VI. References

[Abrahams02] Dave Abrahams et al, Boost.Python library documentation, October 2002. Available online at http://www.boost.org/libs/python/

[Alexandrescu01] Andrei Alexandrescu, Modern C++ Design, Addison Wesley, ISBN 0-201-70431-5.

[Berger02] Emery D. Berger, Benjamin G. Zorn, Kathryn S. McKinley, Reconsidering Custom Memory Allocation, ACM SIGPLAN Notices, Vol. 37, Issue 11, November 2002. Available online at ftp://ftp.cs.utexas.edu/pub/emery/papers/reconsidering-custom.pdf

[Boehm02] Hans-J. Boehm, Destructors, Finalizers, and Synchronization, HP Labs Technical Report HPL-2002-335, December 2002. Available online at http://www.hpl.hp.com/techreports/2002/HPL-2002-335.html

[Boost00] Boost compressed_pair utility component documentation. Available online at http://www.boost.org/libs/utility/compressed_pair.htm

[Boost03a] Boost Smart Pointer Library, source file sp_debug_hooks.cpp. Available online at http://www.boost.org/libs/smart_ptr/src/sp_debug_hooks.cpp

[Boost03b] Boost Smart Pointer Library, source file sp_collector.cpp. Available online at http://www.boost.org/libs/smart_ptr/src/sp_collector.cpp

[Christopher84] Thomas W. Christopher, Reference Count Garbage Collection, Software - Practice and Experience, Vol. 14(6), pp. 503-507, June 1984.

[Colvin94] Gregory Colvin, Exception Safe Smart Pointers, C++ committee document N0555=94-168, July 1994. Available online at http://std.dkuug.dk/jtc1/sc22/wg21/docs/papers/1994/N0555.pdf

[Dimov02] Peter Dimov, Howard E. Hinnant, Dave Abrahams, The Forwarding Problem: Arguments, C++ committee document N1385=02-0043, September 2002. Available online at http://std.dkuug.dk/jtc1/sc22/wg21/docs/papers/2002/n1385.htm

[Dimov03] Peter Dimov, Smart Pointer Programming Techniques, March 2003. Available online at http://www.boost.org/libs/smart_ptr/sp_techniques.html

[Ellis94] John R. Ellis and David L. Detlefs, Safe, Efficient Garbage Collection for C++, Usenix Proceedings, February 1994. This paper includes an extensive discussion of weak pointers and an extensive bibliography. Available online at http://www.usenix.org/publications/library/proceedings/c++94/full_papers/ellis.a

[Google03] Searching www.google.com for "reference counted smart pointer" yields 2,400 entries. Anyone doubting that shared-ownership smart pointers have been reinvented a large number of times should read a random selection of these entries.

[Gregor02] Douglas Gregor, A Proposal to add a Polymorphic Function Object Wrapper to the Standard Library, C++ committee document N1402=02-0060, October 2002. Available online at http://std.dkuug.dk/jtc1/sc22/wg21/docs/papers/2002/n1402.htm

[Hinnant02] Howard E. Hinnant, Peter Dimov, and Dave Abrahams, A Proposal to Add Move Semantics Support to the C++ Language, C++ committee document N1377=02-0035, September 2002. Available online at http://std.dkuug.dk/jtc1/sc22/wg21/docs/papers/2002/n1377.htm

[Lakos96] John Lakos, Large-Scale C++ Software Design, section 9.2.2, page 637, Addison-Wesley, July 1996, ISBN 0-201-63362-0.

[Meyers01] Scott Meyers, Effective STL, page 39, Addison-Wesley, June 2001, ISBN 0-201-74962-9.

[Stroustrup94] Bjarne Stroustrup, The Design and Evolution of C++, section 14.2.1, page 307, Addison Wesley, ISBN 0-201-54330-3, March 1994.

[Stroustrup94a] Bjarne Stroustrup, The Design and Evolution of C++, section 10.7.1, page 220, Addison Wesley, ISBN 0-201-54330-3, March 1994, with quote confirmed via private email, February, 2003.