/* Filename: fdcompat.cpp
Author: Adam Berger (adam@addaon.com)
Make: clang++ -std=c++11 fdcompat.cpp
Date: 22 June 2012 */
/*******************************************************************************
WHY THIS DOCUMENT EXISTS:
The purpose of this file is to demonstrate the use of rvalue references,
a feature new to C++11. If you prefer prose to code, Thomas Becker has an
excellent write-up at:
What is an rvalue? The answer is complicated, but as a good approximation, an
rvalue is that which is NOT an lvalue. An lvalue reference (prior to C++11, the
only kind of reference available) is that which can appear on the left hand side
of an assignment (if not const); an rvalue is that which cannot. For many
purposes, it's sufficient to think of rvalue references as one of two things:
(1) references to objects that can never be seen again after this use (that is,
temporaries) and (2) references that are explicitly forced to be rvalues.
Rvalue references serve three primary purposes in C++11:
1) They allow definition of movable but non-copyable types.
2) They allow the use of move semantics (avoiding copies) on other types.
3) They allow perfect forwarding of function arguments.
All three of these use cases are in evidence in the C++11 standard library:
1) std::unique_ptr is movable but non-copyable.
2) std::string provides an efficient move constructor and assignment operator.
3) std::call_once uses forwarding to efficiently call a user-supplied function.
This file will develop a pair of classes designed to allow legacy code using
the POSIX-style file operations (open(2), read(2), close(2), etc.) to achieve
some level of exception-safety with minimal changes to existing code. The author
is NOT proposing that this is a good idea, nor that this should be incorporated
into your code; it is merely a strawman upon which we can hang the use of a few
new features.
C++11 is a new standard, and compiler support is limited. This code is tested
with clang++ 3.1. It does not work using g++ 4.2.1, but works fully with g++
4.8.0. Because it is tested on only a few compilers, it is possible (perhaps
likely) that there a non-standard-compliant abuses of C++11 features here, that
will not work on future, more compliant compilers. If you recognize any such
usage, please let me know.
THE PROJECT:
We would like to allow existing code using POSIX-style file operations to be
made exception safe, with minimal changes to existing code. That is, we would
like code such as the following snippet:
int fd = open(path, flags);
unsigned char *buffer = malloc(NBYTES);
read(fd, buffer, NBYTES);
//perform a potentially throwing operation
free(buffer);
close(fd);
to guarantee that the file is closed and the buffer deallocated regardless of
whether the intermediate operations throw exceptions or not.
While this can be expressed using a try/catch block, the idiomatic C++ approach
is to use RAII ("resource acquisition is initialization"). In other words, we
will define types that wrap the resources of concern (file descriptors and byte
buffers), and have the constructors and destructors of those types manage the
lifetime of the associated resource. This leverages the language's guarantee
that destructors are called deterministically when a variable goes out of scope,
whether the scope was exited normally or abnormally (via an exception). We thus
allow the code above to be minimally modified to:
FileDescriptor fd(path, flags);
Buffer<NBYTES> buffer;
read(fd, buffer, NBYTES); //this line is unchanged!
//perform a potentially throwing operation
free(buffer); //optional, will happen automatically
close(fd); //optional, will happen automatically
The chosen resources have some properties, though, that make standard RAII use
interesting. For the purposes of demonstration, we exaggerate this properties
slightly. In particular, the dup(2) system call and its siblings are assumed to
NOT EXIST. Therefore, file descriptors can be created only via open(2), and each
call to open(2) must have exactly one paired call to close(2); file descriptors
are not copyable. Buffers are copyable, but such a copy is expensive for large
buffers, so we will provide an efficient move constructor.
To the code:
*******************************************************************************/
#include <algorithm> /* For std::swap */
#include <cstdlib> /* For malloc(3) and free(3) */
#include <cstring> /* For memcpy(3) */
#include <fcntl.h> /* For open(2) and O_RDONLY */
#include <unistd.h> /* For close(2) */
#include <sys/errno.h> /* For EBADF */
/* We will have interesting functions print their name when invoked to aid in
understanding program flow. Note that clang++ 3.1 prints somewhat misleading
function names for template functions with rvalue arguments, as it does not
reflect reference collapsing. g++ 4.8.0 is a bit clearer. */
#include <cstdio>
#define TRACE do { printf("%s\n", __PRETTY_FUNCTION__); } while(0)
/* We will wrap a POSIX file descriptor with a FileDescriptor type. This will
be a movable, non-copyable wrapper. The underlying file descriptor will be
closed exactly once, when the final FileDescriptor wrapping it goes out of
scope. To see what we mean by the "final FileDescriptor" of a non-copyable
type, consider this code:
FileDescriptor open_log() { return FileDescriptor("log.txt", O_RDONLY); }
int main() {
FileDescriptor log_fd = open_log();
//.....
}
Although the temporary FileDescriptor created in open_log goes out of scope
at the end of the call to that function, its underlying responsibility is
moved to the FileDescriptor log_fd; so close(2) will be called only when
log_fd goes out of scope (although the temporary FileDescriptor's destructor
will have already been called).
Note that returning a file descriptor by value here requires that it is
either move or copy constructible. Prior to C++11 and the availability of
movable, non-copyable types, we would have to either allow FileDescriptors
to be copied (causing semantic concerns over which copy is responsible for
calling close(2)), or return a non-value type, such as a pointer to a heap-
allocated wrapper (which would then need to be disposed of manually). */
class FileDescriptor {
/* This is the underlying file descriptor that we're wrapping. */
int fd;
/* To close a file descriptor, we call close(2) if it is valid. We need a
way to represent invalid file descriptors for two reasons: (1) POSIX
itself uses them (for example, open(2) returns -1 on failure), and (2)
when a FileDescriptor transfers responsibility for its underlying file
descriptor via a move, it must know now to call close(2) on its own
destruction. */
int close() {
int retval = fd >= 0 ? ::close(fd) : EBADF;
fd = -1;
return retval;
}
public:
/* These two lines are a little bit tricky. You may want to skip them for
now, and come back later, especially because they don't have anything to
do with rvalue references but are instead just a way of addressing a
quirk in how we bridge to legacy code.
Because FileDescriptors support RAII, calling close explicitly on a file
descriptor is unnecessary. However, legacy code is likely to do so
anyway, and managing file descriptor lifetimes explicitly is sometimes
valuable. Consider the following code:
int fd = open("log.txt", O_RDONLY);
//....
close(fd);
We would like to be able to migrate this to:
FileDescriptor fd("log.txt", O_RDONLY);
//....
close(fd);
As will be shown shortly, we allow calls such as read(2) to operate on
a FileDescriptor by allowing an implicit cast to an int (the underlying
type of POSIX file descriptors) via a cast operator. This is dangerous
and not really a good idea, but it does allow minimal code change, as a
FileDescriptor can be used anywhere in the code an int file descriptor
was previously used. But the explicit call to close(2), then, is very
dangerous; if close(fd) called close(2), the underlying file descriptor
would be closed but the FileDescriptor would call close(2) again during
deconstruction.
This friend function, then, routes the call to close(fd) via fd.close(),
which both closes the underlying file descriptor and notifies the wrapper
that the close has been performed. This does NOT fix every case. In the
simplest case, the following is broken:
FileDescriptor fd("log.txt", O_RDONLY);
//....
int fd_copy = fd; //Uses operator(int)
close(fd_copy); //Closes the underlying file descriptor only
There are further issues with C++ function lookup which I won't discuss
at depth here; argument-dependent-lookup is a topic of its own.
Finally, why do we need both rvalue reference and lvalue reference
implementations of close()? close(fd) will, as expected, call the lvalue
version, but close(FileDescriptor("log.txt", O_RDONLY)), while silly,
will not; both versions are needed to hide close(2) in as many cases as
possible. */
friend int close(FileDescriptor & fd) { return fd.close(); }
friend int close(FileDescriptor && fd) { return fd.close(); }
/* We allow the creation of a FileDescriptor that does not manage an open
file descriptor; we mark the underlying resource with a value that
FileDescriptor.close() will recognize, as we cannot close this underlying
file. */
FileDescriptor() : fd(-1) { TRACE; }
/* open(2) is a variadic function. While it only really has two forms:
open(const char *, int) and open(const char *, int, int)
we will treat it for the purposes of demonstration as a fully variadic
function. We would like to allow a FileDescriptor to be created with
any valid set of arguments to open(2).
This is a use case for variadic templates. It's also a use case for
perfect forwarding. Consider:
FileDescriptor fd("log.txt", O_RDWR | O_CREAT, 0777);
This will construct the underlying file descriptor with the result of
the call open("log.txt", O_RDWR | O_CREAT, 0777). A naïve implementation
might look like this:
template <typename... Args>
FileDescriptor(Args ... args) :
fd(open(args...))
{}
However, this passes all arguments to the constructor by value, causing
an unnecessary copy. In this specific case, the types are trivially and
cheaply copyable, so this isn't an issue; but in the general case it can
be a concern. The next attempt might be:
template <typename... Args>
FileDescriptor(Args & ... args) :
fd(open(args...))
{}
This passes the arguments by lvalue reference, but disallows the use of
non-lvalues, such as the constant flags in the example above. And marking
args as const is also not viable, as open(2) might (in the general case)
require non-const values.
The solution is perfect forwarding via rvalue references. We start with
the intuitive:
template <typename... Args>
FileDescriptor(Args && ... args) :
fd(open(args...))
{}
This will compile, but not quite do what is expected. To understand, we
must explore a new template argument deduction rule in C++11. If a
template function takes an argument of type T && for some template
parameter T, when called with an argument of type U, T will resolve to
U & if the argument is an lvalue, and U otherwise. Prior to C++11, the
type would resolve in all cases to U. If we call
FileDescriptor((const char *) lvalue, (int) rvalue)
type inference generates a call to
FileDescriptor<const char * &, int>
which has the following generated implementation:
FileDescriptor(const char * & && arg1, int && arg2) :
fd(open(arg1, arg2))
{}
C++11 provides reference collapsing rules which define constructs such as
const char * & &&; T && && collapses to T &&; and T & &&, T && &, and
T & & all collapse to T &. So our implementation is actually:
FileDescriptor(const char * & arg1, int && arg2) :
fd(open(arg1, arg2))
{}
So arg1 is passed by lvalue reference, which is perfectly forwarded and
legal (since we passed in an lvalue to begin with). Arg2 is passed by
rvalue reference, which is and legal (since we passed in an rvalue to
begin with), but not perfectly forwarded. Consider the case where open(2)
had an implementation that took its second argument by rvalue reference.
We're running afoul of a quirk of C++11: something declared as an rvalue
(arg2 in our case) is an lvalue if it has a name, or an rvalue otherwise.
That is, if the following two implementations of open(2) exist:
int open(const char *, int const &)
int open(const char *, int &&)
the code above will call the first implementation, because arg2 itself
is an lvalue. We need to "discard the name" in order to achieve perfect
forwarding in all cases. The canonical way to do this is using
std::forward<Args>(args)...
which is exactly equivalent to the static cast used below. Recall that
because of reference collapsing, this cast will cast lvalue references
to lvalue references (a no-op), and named rvalue references (which are
lvalues) to unnamed rvalue references (which are rvalues). */
template <typename... Args>
FileDescriptor(Args && ... args) :
fd(open(static_cast<Args &&>(args)...))
{ TRACE; }
/* Because we have defined an explicit constructor, no automatic copy
constructor will be created. This is good, because we don't want this
type to be copyable. However, our template constructor above happens to
match the definition of a copy constructor. This is not an error, and
will not cause failures; but it will cause attempts to copy a
FileDescriptor to give a less-than-useful error message. By explicitly
marking the copy constructors as deleted, we allow the compiler to be
more helpful. Try the following line with and without these functions:
FileDescriptor fd1("log.txt", O_RDONLY);
FileDescriptor fd2(fd1); //Illegal -- non-copyable type
The second line is illegal either way, but messaging should improve. */
FileDescriptor(FileDescriptor &) = delete;
FileDescriptor(FileDescriptor const &) = delete;
/* Here we define our move constructor. This constructor is called when the
new FileDescriptor is taking over ownership of the underlying resource.
We thus need to accomplish two things: (1) set our underlying file
descriptor to the resource we now own and (2) modify the rvalue so that
it will not close the resource when it gets deconstructed.
The static cast in the initialization of fd has no effect in this case,
but is necessary in the general case. Suppose that, instead of being a
primitive int, the underlying resource was represented by a complex type
that itself had both copy and move constructors. Because we know that
rvalue.fd is being overwritten immediately, we want to move it, rather
than copy it, into fd. By casting it to an unnamed rvalue reference, we
achieve this goal; std::move(rvalue.fd) is the standard way of expressing
this.
We mark this function using the new-to-C++11 noexcept keyword. The
noexcept feature is a bit unripe and not necessarily recommended for wide
use, but the following rule of thumb seems reasonable: Move constructors
and move assignment operators should never throw exceptions (often
because they are implemented in terms of the noexcept std::swap), and
should be marked as noexcept. In certain cases, this is required to allow
standard containers to use move semantics instead of copy semantics. */
FileDescriptor(FileDescriptor && rvalue) noexcept :
fd(static_cast<int &&>(rvalue.fd))
{
TRACE;
rvalue.fd = -1;
}
/* The move assignment operator has a straightforward and canonical
implementation. It can be marked noexcept, because std::swap(int&, int&)
is noexcept. After this assignment operator is called, the rvalue will
be deconstructed, which will close the underlying file descriptor that
previously belonged to this FileDescriptor; this FileDescriptor is now
responsible for closing the underlying file descriptor that was assigned
to it. */
FileDescriptor & operator=(FileDescriptor && rvalue) noexcept {
TRACE;
std::swap(fd, rvalue.fd);
return *this;
}
/* When we are deconstructed, we close our underlying resource if it is
valid (the close() member function is responsible for validity checks. */
~FileDescriptor() {
TRACE;
close();
}
/* To allow FileDescriptors to be used as a drop-in replacement for legacy
int file descriptors, we supply a cast operator. This is generally not
recommended for production code, but it makes this example simpler. */
operator int() const {
return fd;
}
};
/* We will wrap a fixed-size byte buffer with a Buffer template type. This is
a copyable type, but copying a buffer takes time proportional to the length
of the buffer, which can be substantial. To avoid this cost when possible,
Buffer is also a movable type; moving a Buffer is a fast, constant-time
operation. Note that to make Buffers movable, we must hold the underlying
allocated memory by pointer, rather than inline. This additional level of
indirection is a cost we incur to allow the move optimization; whether it is
a good trade-off will be application specific. */
template <int N>
class Buffer {
/* This is the underlying memory region that we're wrapping. */
void * buf;
/* Just as with FileDescriptor, we need a way to deconstruct the underlying
resource if it is valid. In the case of a memory region, we name this
deconstruction free(). The nullptr keyword is new to C++11, and serves
the same role as NULL or 0 in earlier C++, but avoids a few catches with
function dispatch on overloaded functions called with a null pointer. */
void free() {
if(buf) ::free(buf);
buf = nullptr;
}
public:
/* With FileDescriptor, we made some effort to make close(fd) notify the
wrapping type that its underlying resource had been closed; we do the
same here so free(buf) updates the underlying Buffer when possible. */
friend void free(Buffer<N> & buf) { buf.free(); }
friend void free(Buffer<N> && buf) { buf.free(); }
/* A buffer is constructed with a valid buffer of the appropriate size. The
use of malloc()/free() to allocate untyped buffers for use with legacy C
APIs (rather than using new[]/delete[]) is personal preference; either is
acceptable as long as usage is consistent. */
Buffer() : buf(malloc(N)) { TRACE; }
/* Because the Buffer type is copyable, we must provide a copy constructor.
This constructor does two things: (1) it allocates space for the new
underlying buffer and (2) fills in that buffer with the copied data.
Because we just allocated the destination, we know it does not alias the
source, and can use memcpy() rather than memmove(). */
Buffer(Buffer const & lvalue) : buf(malloc(N)) {
TRACE;
memcpy(buf, lvalue.buf, N);
}
/* Because the Buffer type is movable, we must provide a move constructor.
This should look extremely familiar from the FileDescriptor case. */
Buffer(Buffer && rvalue) noexcept : buf(rvalue.buf) {
TRACE;
rvalue.buf = nullptr;
}
/* To complete the implementation of a copyable, movable type, we need to
support copy and move assignment operators. We can do so explicitly, but
this causes a fair amount of repeated boilerplate code. The alternative
is to take advantage of the copy-and-swap idiom and provide an assignment
operator that takes its argument by value. Bradley Grainger provides a
good explanation of this idiom at:
good explanation of this idiom at:
To understand how this works, consider two cases. In the first case, the
assignment operator is called with an lvalue (buf1 = buf2). To achieve
pass-by-value, the compiler creates a copy of buf2. Because buf2 is an
lvalue, it does this using the copy constructor. The argument in the
assignment operator thus has a new copy of the underlying resource,
created using the copy constructor; we can then move that copied resource
out of the copied argument. The copied argument will be deconstructed
when it goes out of scope at the end of the assignment operator, and will
thus free our previously-owned resource. Compared to an implementation of
operator=(Buffer const &), this costs one additional swap, which may be
optimized out and is in any case quite cheap.
In the second case, the assignment operator is called with an rvalue. To
achieve pass-by-value, the compiler creates a Buffer to pass in, but it
does so using the move constructor. The newly-created argument thus has
an underlying resource that is cheaply swapped from the rvalue; we simply
swap it again into this Buffer. Again, there is a trivial (or zero) cost
of an additional swap. */
Buffer & operator=(Buffer other) {
TRACE;
std::swap(buf, other.buf);
return *this;
}
/* When a buffer is destructed, it must free its underlying resource, if
valid; as with FileDescriptor, we have a helper method responsible for
the validity checks. */
~Buffer() {
TRACE;
free();
}
/* To allow Buffers to be used as a drop-in replacement for legacy untyped
buffers, we provide a cast operator. As always, cast operators are a bit
sketchy and should be considered carefully. */
operator void *() {
return buf;
}
/* In the case of FileDescriptor, we returned a copy of the underlying
resource handle, so the cast operator was const. However, for a Buffer,
because we return the raw pointer the underlying resource can be modified
through the cast operator. As such, we need to provide a const overload;
in that way, a const Buffer can be used as a const void *, but not as a
(mutable) void *. */
operator void const *() const {
return buf;
}
};
/************************ TEST CODE *******************************************/
/* The best way to understand how rvalue references work is to step through code
that exercises our implementation in a debugger, paying special attention to
exactly which overloaded method is called when. */
int main() {
{ /* Look at FileDescriptor construction and reference collapsing. */
/* Calls FileDescriptor(const char * &, int &&) */
FileDescriptor a("log.txt", O_RDONLY);
/* Calls FileDescriptor(const char * &, int &, int &&) */
int x = O_RDWR | O_CREAT;
FileDescriptor("log.txt", x, 0777);
/* This temporary FileDescriptor is immediately destroyed. */
/* Calls the move constructor, constructing b with a's underlying
resource. After this line, a no longer refers to a valid resource --
this is the danger of name-discarding casts. A construction or move
without the cast would be illegal, since FileDescriptors are not
copyable. */
FileDescriptor b = static_cast<FileDescriptor&&>(a);
/* If we exited the scope now, we would first destruct b, then destruct
a (opposite order of construction). However, we can explicitly close
a first. Note that this turns out to be a no-op, because the resource
associated with a was already closed by the temporary FileDescriptor
to which it was moved. */
close(a);
}
{ /* Look at Buffer copying and moving. */
typedef Buffer<8> B8; /* Use a single buffer size for simplicity. */
B8 a; /* Normal zero-argument constructor. */
B8 b(a); /* Copy constructor. */
/* Copy constructor called to create by-value argument to assignment
operator; the memory region that a originally owned is freed when
the temporary goes out of scope. */
a = b;
/* Move constructor, a is now invalid. */
B8 c(static_cast<B8 &&>(a));
/* Move constructor called to create by-value argument to assignment
operator; a now owns a valid memory region again, and c owns the
invalid one. */
a = static_cast<B8 &&>(c);
}
return 0;
}
