C++ Coroutines: Understanding the Compiler Transform (2023)

• Introduction
• Setting the Scene
• Defining the task type
• Step 1: Determining the promise type
• Step 2: Creating the coroutine state
• Step 3: Call get_return_object()
• Step 4: The initial-suspend point
• Step 5: Recording the suspend-point
• Step 6: Implementing coroutine_handle::resume() and coroutine_handle::destroy()
• Step 7: Implementing coroutine_handle<Promise>::promise() and from_promise()
• Step 8: The beginnings of the coroutine body
• Step 9: Lowering the co_await expression
• Step 10: Implementing unhandled_exception()
• Step 11: Implementing co_return
• Step 12: Implementing final_suspend()
• Step 13: Implementing symmetric-transfer and the noop-coroutine
• One last thing
• Tying it all together

Previous blogs in the series on “Understanding C++ Coroutines” talked about the differentkinds of transforms the compiler performs on a coroutine and its co_await, co_yieldand co_return expressions. These posts described how each expression was lowered bythe compiler to calls to various customisation points/methods on user-defined types.

1. Coroutine Theory
2. C++ Coroutines: Understanding operator co_await
3. C++ Coroutines: Understanding the promise type
4. C++ Coroutines: Understanding Symmetric Transfer

However, there was one part of these descriptions that may have left you unsatisfied.The all hand-waved over the concept of a “suspend-point” and said something vaguelike “the coroutine suspends here” and “the coroutine resumes here” but didn’t reallygo into detail about what that actually means or how it might be implemented by thecompiler.

In this post I am going to go a bit deeper to show how all the concepts fromthe previous posts come together. I’ll show what happens when a coroutine reachesa suspend-point by walking through the lowering of a coroutine into equivalentnon-coroutine, imperative C++ code.

Note that I am not going to describe exactly how a particular compiler lowers coroutinesinto machine code (compilers have extra tricks up their sleeves here), but rather just onepossible lowering of coroutines into portable C++ code.

Warning: This is going to be a fairly deep dive!

For starters, let’s assume we have a basic task type that acts as both an awaitableand a coroutine return-type. For the sake of simplicity, let’s assume that this coroutinetype allows producing a result of type int asynchronously.

In this post we are going to walk through how to lower the following coroutine functioninto C++ code that does not contain any of the coroutine keywords co_await, co_returnso that we can better understand what this means.

// Forward declaration of some other function. Its implementation is not relevant.task f(int x);// A simple coroutine that we are going to translate to non-C++ codetask g(int x) { int fx = co_await f(x); co_return fx * fx;}

To begin, let us first declare the task class that we will be working with.

For the purposes of understanding how the coroutine is lowered, we do not need toknow the definitions of the methods for this type. The lowering will just be insertingcalls to them.

The definitions of these methods are not complicated, and I will leave them as anexercise for the reader as practice for understanding the previous posts.

class task {public: struct awaiter; class promise_type { public: promise_type() noexcept; ~promise_type(); struct final_awaiter { bool await_ready() noexcept; std::coroutine_handle<> await_suspend( std::coroutine_handle<promise_type> h) noexcept; void await_resume() noexcept; }; task get_return_object() noexcept; std::suspend_always initial_suspend() noexcept; final_awaiter final_suspend() noexcept; void unhandled_exception() noexcept; void return_value(int result) noexcept; private: friend task::awaiter; std::coroutine_handle<> continuation_; std::variant<std::monostate, int, std::exception_ptr> result_; }; task(task&& t) noexcept; ~task(); task& operator=(task&& t) noexcept; struct awaiter { explicit awaiter(std::coroutine_handle<promise_type> h) noexcept; bool await_ready() noexcept; std::coroutine_handle<promise_type> await_suspend( std::coroutine_handle<> h) noexcept; int await_resume(); private: std::coroutine_handle<promise_type> coro_; }; awaiter operator co_await() && noexcept;private: explicit task(std::coroutine_handle<promise_type> h) noexcept; std::coroutine_handle<promise_type> coro_;};

The structure of this task type should be familiar to those that have read theC++ Coroutines: Understanding Symmetric Transfer post.

task g(int x) { int fx = co_await f(x); co_return fx * fx;}

When the compiler sees that this function contains one of the three coroutine keywords(co_await, co_yield or co_return) it starts the coroutine transformation process.

The first step here is determining the promise_type to use for this coroutine.

This is determined by substituting the return-type and argument-types of the signature astemplate arguments to the std::coroutine_traits type.

e.g. For our function, g, which has return type task and a single argument of type int,the compiler will look this up using std::coroutine_traits<task, int>::promise_type.

Let’s define an alias so we can refer to this type later:

using __g_promise_t = std::coroutine_traits<task, int>::promise_type;

Note: I am using leading double-underscore here to indicate symbols internal to thecompiler that the compiler generates. Such symbols are reserved by the implementationand should not be used in your own code.

Now, as we have not specialised std::coroutine_traits this will instantiate the primary templatewhich just defines the nested promise_type as an alias of the nested promise_type name of thereturn-type. i.e. this should resolve to the type task::promise_type in our case.

A coroutine function needs to preserve the state of the coroutine, parameters and local variableswhen it suspends so that they remain available when the coroutine is later resumed.

This state, in C++ standardese, is called the coroutine state and is typically heap allocated.

Let’s start by defining a struct for the coroutine-state for the coroutine, g.

We don’t know what the contents of this type are going to be yet, so let’s just leave it empty for now.

struct __g_state { // to be filled out};

The coroutine state contains a number of different things:

• The promise object
• Copies of any function parameters
• Information about the suspend-point that the coroutine is currently suspended at and how to resume/destroy it
• Storage for any local variables / temporaries whose lifetimes span a suspend-point

Let’s start by adding storage for the promise object and parameter copies.

struct __g_state { int x; __g_promise_t __promise; // to be filled out};

Next we should add a constructor to initialise these data-members.

Recall that the compiler will first attempt to call the promise constructor with lvalue-references to the parameter copies,if that call is valid, otherwise fall back to calling the default constructor of the promise type.

Let’s create a simple helper to assist with this:

template<typename Promise, typename... Params>Promise construct_promise([[maybe_unused]] Params&... params) { if constexpr (std::constructible_from<Promise, Params&...>) { return Promise(params...); } else { return Promise(); }}

Thus the coroutine-state constructor might look something like this:

struct __g_state { __g_state(int&& x) : x(static_cast<int&&>(x)) , __promise(construct_promise<__g_promise_t>(x)) {} int x; __g_promise_t __promise; // to be filled out};

Now that we have the beginnings of a type to represent the coroutine-state, let’s also start tostub out the beginnings of the lowered implementation of g() by having it heap-allocatean instance of the __g_state type, passing the function parameters so they can be copied/moved into the coroutine-state.

Some terminology - I use the term “ramp function” to refer to the part of the coroutineimplementation containing the logic that initialises the coroutine state and gets it readyto start executing the coroutine.i.e. it is like an on-ramp for entering execution of the coroutine body.

task g(int x) { auto* state = new __g_state(static_cast<int&&>(x)); // ... implement rest of the ramp function}

Note that our promise-type does not define its own custom operator new overloads,and so we are just calling global ::operator new here.

If the promise type did define a custom operator new then we’d call that instead of theglobal ::operator new. We would first check whether operator new was callable with theargument list (size, paramLvalues...) and if so call it with that argument list. Otherwise,we’d call it with just the (size) argument list. The ability for the operator new to getaccess to the parameter list of the coroutine function is sometimes called “parameter preview”and is useful in cases where you want to use an allocator passed as a parameter to allocatestorage for the coroutine-state.

If the compiler found any definition of __g_promise_t::operator new then we’d lower to thefollowing logic instead:

template<typename Promise, typename... Args>void* __promise_allocate(std::size_t size, [[maybe_unused]] Args&... args) { if constexpr (requires { Promise::operator new(size, args...); }) { return Promise::operator new(size, args...); } else { return Promise::operator new(size); }}task g(int x) { void* state_mem = __promise_allocate<__g_promise_t>(sizeof(__g_state), x); __g_state* state; try { state = ::new (state_mem) __g_state(static_cast<int&&>(x)); } catch (...) { __g_promise_t::operator delete(state_mem); throw; } // ... implement rest of the ramp function}

Also, this promise-type does not define the get_return_object_on_allocation_failure() staticmember function. If this function is defined on the promise-type then the allocation here wouldinstead use the std::nothrow_t form of operator new and upon returning nullptr would thenreturn __g_promise_t::get_return_object_on_allocation_failure();.

i.e. it would look something like this instead:

task g(int x) { auto* state = ::new (std::nothrow) __g_state(static_cast<int&&>(x)); if (state == nullptr) { return __g_promise_t::get_return_object_on_allocation_failure(); } // ... implement rest of the ramp function}

For simplicity for the rest of the example, we’ll just use the simplest form that calls theglobal ::operator new memory allocation function.

The next thing the ramp function does is to call the get_return_object() method on the promiseobject to obtain the return-value of the ramp function.

The return value is stored as a local variable and is returned at the end of the ramp function(after the other steps have been completed).

task g(int x) { auto* state = new __g_state(static_cast<int&&>(x)); decltype(auto) return_value = state->__promise.get_return_object(); // ... implement rest of ramp function return return_value;}

However, now it’s possible that the call to get_return_object() might throw, and in whichcase we want to free the allocated coroutine state. So for good measure, let’s give ownershipof the state to a std::unique_ptr so that it’s freed in case a subsequent operation throwsan exception:

task g(int x) { std::unique_ptr<__g_state> state(new __g_state(static_cast<int&&>(x))); decltype(auto) return_value = state->__promise.get_return_object(); // ... implement rest of ramp function return return_value;}

The next thing the ramp function does after calling get_return_object() is to start executingthe body of the coroutine, and the first thing to execute in the body of the coroutine is theinitial suspend-point. i.e. we evaluate co_await promise.initial_suspend().

Now, ideally we’d just treat the coroutine as initially suspended and then just implement thelaunching of the coroutine as a resumption of the initially suspended coroutine. However, thespecification of the initial-suspend point has a few quirks with regards to how it handlesexceptions and the lifetime of the coroutine state. This was a late tweak to the semanticsof the initial-suspend point just before C++20 was released to fix some perceived issues here.

Within the evaluation of the initial-suspend-point, if an exception is thrown either from:

• the call to initial_suspend(),
• the call to operator co_await() on the returned awaitable (if one is defined),
• the call to await_ready() on the awaiter, or
• the call to await_suspend() on the awaiter

Then the exception propagates back to the caller of the ramp function and the coroutine state isautomatically destroyed.

If an exception is thrown either from:

• the call to await_resume(),
• the destructor of the object returned from operator co_await() (if applicable), or
• the destructor of the object returned from initial_suspend()

Then this exception is caught by the coroutine body and promise.unhandled_exception() is called.

This means we need to be a bit careful how we handle transforming this part, as some parts willneed to live in the ramp function and other parts in the coroutine body.

Also, since the objects returned from initial_suspend() and (optionally) operator co_await()will have lifetimes that span a suspend-point (they are created before the point at which thecoroutine suspends and are destroyed after it resumes) the storage for those objects will need to beplaced in the coroutine state.

In our particular case, the type returned from initial_suspend() is std::suspend_always, whichhappens to be an empty, trivially constructible type. However, logically we still need to store aninstance of this type in the coroutine state, so we’ll add storage for it anyway just to show howthis works.

This object will only be constructed at the point that we call initial_suspend(), so we need toadd a data-member of a certain type that that allows us to explicitly control its lifetime.

To support this, let’s first define a helper class, manual_lifetime that is trivally constructibleand trivially destructible but that lets us explicitly construct/destruct the value stored therewhen we need to.

template<typename T>struct manual_lifetime { manual_lifetime() noexcept = default; ~manual_lifetime() = default; // Not copyable/movable manual_lifetime(const manual_lifetime&) = delete; manual_lifetime(manual_lifetime&&) = delete; manual_lifetime& operator=(const manual_lifetime&) = delete; manual_lifetime& operator=(manual_lifetime&&) = delete; template<typename Factory> requires std::invocable<Factory&> && std::same_as<std::invoke_result_t<Factory&>, T> T& construct_from(Factory factory) noexcept(std::is_nothrow_invocable_v<Factory&>) { return *::new (static_cast<void*>(&storage)) T(factory()); } void destroy() noexcept(std::is_nothrow_destructible_v<T>) { std::destroy_at(std::launder(reinterpret_cast<T*>(&storage))); } T& get() & noexcept { return *std::launder(reinterpret_cast<T*>(&storage)); }private: alignas(T) std::byte storage[sizeof(T)];};

Note that the construct_from() method is designed to take a lambda here rather thantaking the constructor arguments. This allows us to make use of the guaranteed copy-elisionwhen initialising a variable with the result of a function-call to construct the objectin-place. If it were instead to take the constructor arguments then we’d end up callingan extra move-constructor unnecessarily.

Now we can declare a data-member for the temporary returned by promise.initial_suspend()using this manual_lifetime structure.

struct __g_state { __g_state(int&& x); int x; __g_promise_t __promise; manual_lifetime<std::suspend_always> __tmp1; // to be filled out};

The std::suspend_always type does not have an operator co_await() so we do not need toreserve storage for an extra temporary for the result of that call here.

Once we’ve constructed this object by calling intial_suspend(), we then need to call thetrio of methods to implement the co_await expression: await_ready(), await_suspend()and await_resume().

When invoking await_suspend() we need to pass it a handle to the current coroutine.For now we can just call std::coroutine_handle<__g_promise_t>::from_promise() and passa reference to that promise. We’ll look at the internals of what this does a little later.

Also, the result of the call to .await_suspend(handle) has type void and so we do notneed to consider whether to resume this coroutine or another coroutine after callingawait_suspend() like we do for the bool and coroutine_handle-returning flavours.

Finally, as all of the method invocations on the std::suspend_always awaiter are declarednoexcept, we don’t need to worry about exceptions. If they were potentially throwing thenwe’d need to add extra code to make sure that the temporary std::suspend_always objectwas destroyed before the exception propagated out of the ramp function.

Once we get to the point where await_suspend() has returned successfully orwhere we are about to start executing the coroutine body we enter the phase where weno longer need to automatically destroy the coroutine-state if an exception is thrown.So we can call release() on the std::unique_ptr owning the coroutine state toprevent it from being destroyed when we return from the function.

So now we can implement the first part of the initial-suspend expression as follows:

task g(int x) { std::unique_ptr<__g_state> state(new __g_state(static_cast<int&&>(x))); decltype(auto) return_value = state->__promise.get_return_object(); state->__tmp1.construct_from([&]() -> decltype(auto) { return state->__promise.initial_suspend(); }); if (!state->__tmp1.get().await_ready()) { // // ... suspend-coroutine here // state->__tmp1.get().await_suspend( std::coroutine_handle<__g_promise_t>::from_promise(state->__promise)); state.release(); // fall through to return statement below. } else { // Coroutine did not suspend. state.release(); // // ... start executing the coroutine body // } return __return_val;}

The call to await_resume() and the destructor of __tmp1 will appear in the coroutinebody and so they do not appear in the ramp function.

We now have a (mostly) functional evaluation of the initial-suspend point, but we stillhave a couple of TODO’s in the code for this ramp function. To be able to resolve these wewill first need to take a detour to look at the strategy for suspending a coroutine and laterresuming it.

When a coroutine suspends, it needs to make sure it resumes at the same point in thecontrol flow that it suspended at.

It also needs to keep track of which objects with automatic-storage duration are aliveat each suspend-point so that it knows what needs to be destroyed if the coroutine isdestroyed instead of being resumed.

One way to implement this is to assign each suspend-point in the coroutine a unique numberand then store this in an integer data-member of the coroutine state.

Then whenever a coroutine suspends, it writes the number of the suspend-point at which it issuspending to the coroutine state, and when it is resumed/destroyed we then inspectthis integer to see which suspend point it was suspended at.

Note that this is not the only way of storing the suspend-point in the coroutine state,however all 3 major compilers (MSVC, Clang, GCC) use this approach as the time this postwas authored (c. 2022).Another potential solution is to use separate resume/destroy function-pointers for eachsuspend-point, although we will not be exploring this strategy in this post.

So let’s extend our coroutine-state with an integer data-member to store the suspend-pointindex and initialise it to zero (we’ll always use this as the value for the initial-suspendpoint).

struct __g_state { __g_state(int&& x); int x; __g_promise_t __promise; int __suspend_point = 0; // <-- add the suspend-point index manual_lifetime<std::suspend_always> __tmp1; // to be filled out};

When a coroutine is resumed by calling coroutine_handle::resume() we need this to end upinvoking some function that implements the rest of the body of the suspended coroutine.The invoked body function can then look up the suspend-point index and jump to the appropriatepoint in the control-flow.

We also need to implement the coroutine_handle::destroy() function so that it invokesthe appropriate logic to destroy any in-scope objects at the current suspend-point andwe need to implement coroutine_handle::done() to query whether the current suspend-pointis a final-suspend-point.

The interface of the coroutine_handle methods does not know about the concrete coroutinestate type - the coroutine_handle<void> type can point to any coroutine instance.This means we need to implement them in a way that type-erases the coroutine state type.

We can do this by storing function-pointers to the resume/destroy functions for that coroutinetype and having coroutine_handle::resume/destroy() invoke those function-pointers.

The coroutine_handle type also needs to be able to be converted to/from a void* using thecoroutine_handle::address() and coroutine_handle::from_address() methods.

Furthermore, the coroutine can be resumed/destroyed from any handle to that coroutine -not just the handle that was passed to the most recent await_suspend() call.

These requirements lead us to define the coroutine_handle type so that it only contains apointer to the coroutine-state and that we store the resume/destroy function pointers asdata-members of the coroutine state, rather than, say, storing the resume/destroy functionpointers in the coroutine_handle.

Also, since we need the coroutine_handle to be able to point to an arbitrary coroutine-stateobject we need the layout of the function-pointer data-members to be consistent across allcoroutine-state types.

One straight forward way of doing this is having each coroutine-state type inherit from somebase-class that contains these data-members.

e.g. We can define the following type as the base-class for all coroutine-state types

struct __coroutine_state { using __resume_fn = void(__coroutine_state*); using __destroy_fn = void(__coroutine_state*); __resume_fn* __resume; __destroy_fn* __destroy;};

Then the coroutine_handle::resume() method can simply call __resume(), passing apointer to the __coroutine_state object.Similarly, we can do this for the coroutine_handle::destroy() method and the __destroy function-pointer.

For the coroutine_handle::done() method, we choose to treat a null __resume function pointeras an indication that we are at a final-suspend-point. This is convenient since the final suspendpoint does not support resume(), only destroy(). If someone tries to call resume() on acoroutine suspended at the final-suspend-point (which has undefined-behaviour) then they end upcalling a null function pointer which should fail pretty quickly and point out their error.

Given this, we can implement the coroutine_handle<void> type as follows:

namespace std{ template<typename Promise = void> class coroutine_handle; template<> class coroutine_handle<void> { public: coroutine_handle() noexcept = default; coroutine_handle(const coroutine_handle&) noexcept = default; coroutine_handle& operator=(const coroutine_handle&) noexcept = default; void* address() const { return static_cast<void*>(state_); } static coroutine_handle from_address(void* ptr) { coroutine_handle h; h.state_ = static_cast<__coroutine_state*>(ptr); return h; } explicit operator bool() noexcept { return state_ != nullptr; } friend bool operator==(coroutine_handle a, coroutine_handle b) noexcept { return a.state_ == b.state_; } void resume() const { state_->__resume(state_); } void destroy() const { state_->__destroy(state_); } bool done() const { return state_->__resume == nullptr; } private: __coroutine_state* state_ = nullptr; };}

For the more general coroutine_handle<Promise> specialisation, most of the implementationscan just reuse the coroutine_handle<void> implementations. However, we also need to be ableto get access to the promise object of the coroutine-state, returned from the promise() method,and also construct a coroutine_handle from a reference to the promise-object.

However, again we cannot simply point to the concrete coroutine state type since thecoroutine_handle<Promise> type must be able to refer to any coroutine-state whosepromise-type is Promise.

We need to define a new coroutine-state base-class that inherits from __coroutine_stateand which contains the promise object so we can then define all coroutine-state types that usea particular promise-type to inherit from this base-class.

template<typename Promise>struct __coroutine_state_with_promise : __coroutine_state { __coroutine_state_with_promise() noexcept {} ~__coroutine_state_with_promise() {} union { Promise __promise; };};

You might be wondering why we declare the __promise member inside an anonymous union here…

The reason for this is that the derived class created for a particular coroutine functioncontains the definition for the argument-copy data-members. Data members from derived classesare by default initialised after data-members of any base-classes, so declaring the promiseobject as a normal data-member would mean that the promise object was constructed before theargument-copy data-members.

However, we need the constructor of the promise to be called after the constructor of theargument-copies - references to the argument-copies might need to be passed to the promiseconstructor.

So we reserve storage for the promise object in this base-class so that it has a consistentoffset from the start of the coroutine-state, but leave the derived class responsible forcalling the constructor/destructor at the appropriate point after the argument-copies havebeen initialised. Declaring the __promise as a union-member provides this control.

Let’s update the __g_state class to now inherit from this new base-class.

struct __g_state : __coroutine_state_with_promise<__g_promise_t> { __g_state(int&& __x) : x(static_cast<int&&>(__x)) { // Use placement-new to initialise the promise object in the base-class ::new ((void*)std::addressof(this->__promise)) __g_promise_t(construct_promise<__g_promise_t>(x)); } ~__g_state() { // Also need to manually call the promise destructor before the // argument objects are destroyed. this->__promise.~__g_promise_t(); } int __suspend_point = 0; int x; manual_lifetime<std::suspend_always> __tmp1; // to be filled out};

Now that we have defined the promise-base-class we can now implement the std::coroutine_handle<Promise> class template.

Most of the implementation should be largely identical to the equivalent methods in coroutine_handle<void>except with a __coroutine_state_with_promise<Promise> pointer instead of __coroutine_state pointer.

The only new part is the addition of the promise() and from_promise() functions.

• The promise() method is straight-forward - it just returns a reference to the __promise member of the coroutine-state.
• The from_promise() method requires us to calculate the address of the coroutine-state from theaddress of the promise object. We can do this by just subtracting the offset of the __promise memberfrom the address of the promise object.

Implementation of coroutine_handle<Promise>:

namespace std{ template<typename Promise> class coroutine_handle { using state_t = __coroutine_state_with_promise<Promise>; public: coroutine_handle() noexcept = default; coroutine_handle(const coroutine_handle&) noexcept = default; coroutine_handle& operator=(const coroutine_handle&) noexcept = default; operator coroutine_handle<void>() const noexcept { return coroutine_handle<void>::from_address(address()); } explicit operator bool() const noexcept { return state_ != nullptr; } friend bool operator==(coroutine_handle a, coroutine_handle b) noexcept { return a.state_ == b.state_; } void* address() const { return static_cast<void*>(static_cast<__coroutine_state*>(state_)); } static coroutine_handle from_address(void* ptr) { coroutine_handle h; h.state_ = static_cast<state_t*>(static_cast<__coroutine_state*>(ptr)); return h; } Promise& promise() const { return state_->__promise; } static coroutine_handle from_promise(Promise& promise) { coroutine_handle h; // We know the address of the __promise member, so calculate the // address of the coroutine-state by subtracting the offset of // the __promise field from this address. h.state_ = reinterpret_cast<state_t*>( reinterpret_cast<unsigned char*>(std::addressof(promise)) - offsetof(state_t, __promise)); return h; } // Define these in terms of their `coroutine_handle<void>` implementations void resume() const { static_cast<coroutine_handle<void>>(*this).resume(); } void destroy() const { static_cast<coroutine_handle<void>>(*this).destroy(); } bool done() const { return static_cast<coroutine_handle<void>>(*this).done(); } private: state_t* state_; };}

Now that we have defined the mechanism by which coroutines are resumed, we can nowreturn to our “ramp” function and update it to initialise the new function-pointerdata-members we’ve added to the coroutine-state.

Let’s now forward-declare resume/destroy functions of the right signature andupdate the __g_state constructor to initialise the coroutine-stateso that the resume/destroy function-pointers point at them:

void __g_resume(__coroutine_state* s);void __g_destroy(__coroutine_state* s);struct __g_state : __coroutine_state_with_promise<__g_promise_t> { __g_state(int&& __x) : x(static_cast<int&&>(__x)) { // Initialise the function-pointers used by coroutine_handle methods. this->__resume = &__g_resume; this->__destroy = &__g_destroy; // Use placement-new to initialise the promise object in the base-class ::new ((void*)std::addressof(this->__promise)) __g_promise_t(construct_promise<__g_promise_t>(x)); } // ... rest omitted for brevity};task g(int x) { std::unique_ptr<__g_state> state(new __g_state(static_cast<int&&>(x))); decltype(auto) return_value = state->__promise.get_return_object(); state->__tmp1.construct_from([&]() -> decltype(auto) { return state->__promise.initial_suspend(); }); if (!state->__tmp1.get().await_ready()) { state->__tmp1.get().await_suspend( std::coroutine_handle<__g_promise_t>::from_promise(state->__promise)); state.release(); // fall through to return statement below. } else { // Coroutine did not suspend. Start executing the body immediately. __g_resume(state.release()); } return return_value;}

This now completes the ramp function and we can now focus on the resume/destroy functions for g().

Let’s start by completing the lowering of the initial-suspend expression.

When __g_resume() is called and the __suspend_point index is 0 then we need it toresume by calling await_resume() on __tmp1 and then calling the destructor of __tmp1.

void __g_resume(__coroutine_state* s) { // We know that 's' points to a __g_state. auto* state = static_cast<__g_state*>(s); // Generate a jump-table to jump to the correct place in the code based // on the value of the suspend-point index. switch (state->__suspend_point) { case 0: goto suspend_point_0; default: std::unreachable(); }suspend_point_0: state->__tmp1.get().await_resume(); state->__tmp1.destroy(); // TODO: Implement rest of coroutine body. // // int fx = co_await f(x); // co_return fx * fx;}

And when __g_destroy() is called and the __suspend_point index is 0 then we need itto just destroy __tmp1 before then destroying and freeing the coroutine-state.

void __g_destroy(__coroutine_state* s) { auto* state = static_cast<__g_state*>(s); switch (state->__suspend_point) { case 0: goto suspend_point_0; default: std::unreachable(); }suspend_point_0: state->__tmp1.destroy(); goto destroy_state; // TODO: Add extra logic for other suspend-points here.destroy_state: delete state;}

Next, let’s take a look at lowering the co_await f(x) expression.

First we need to evaluate f(x) which returns a temporary task object.

As the temporary task is not destroyed until the semicolon at the end of the statement andthe statement contains a co_await expression, the lifetime of the task therefore spans asuspend-point and so it must be stored in the coroutine-state.

When the co_await expression is then evaluated on this temporary task, we need to callthe operator co_await() method which returns a temporary awaiter object. The lifetime ofthis object also spans the suspend-point and so must be stored in the coroutine-state.

Let’s add the necessary members to the __g_state type:

struct __g_state : __coroutine_state_with_promise<__g_promise_t> { __g_state(int&& __x); ~__g_state(); int __suspend_point = 0; int x; manual_lifetime<std::suspend_always> __tmp1; manual_lifetime<task> __tmp2; manual_lifetime<task::awaiter> __tmp3;};

Then we can update the __g_resume() function to initialise these temporaries and then evaluatethe 3 await_ready, await_suspend and await_resume calls that comprise the rest of theco_await expression.

Note that the task::awaiter::await_suspend() method returns a coroutine-handle so we need togenerate code that resumes the returned handle.

We also need to update the suspend-point index before calling await_suspend() (we’ll use theindex 1 for this suspend-point) and then add an extra entry to the jump-table to ensure thatwe resume back at the right spot.

void __g_resume(__coroutine_state* s) { // We know that 's' points to a __g_state. auto* state = static_cast<__g_state*>(s); // Generate a jump-table to jump to the correct place in the code based // on the value of the suspend-point index. switch (state->__suspend_point) { case 0: goto suspend_point_0; case 1: goto suspend_point_1; // <-- add new jump-table entry default: std::unreachable(); }suspend_point_0: state->__tmp1.get().await_resume(); state->__tmp1.destroy(); // int fx = co_await f(x); state->__tmp2.construct_from([&] { return f(state->x); }); state->__tmp3.construct_from([&] { return static_cast<task&&>(state->__tmp2.get()).operator co_await(); }); if (!state->__tmp3.get().await_ready()) { // mark the suspend-point state->__suspend_point = 1; auto h = state->__tmp3.get().await_suspend( std::coroutine_handle<__g_promise_t>::from_promise(state->__promise)); // Resume the returned coroutine-handle before returning. h.resume(); return; }suspend_point_1: int fx = state->__tmp3.get().await_resume(); state->__tmp3.destroy(); state->__tmp2.destroy(); // TODO: Implement // co_return fx * fx;}

Note that the int fx local variable has a lifetime that does not span a suspend-point andso it does not need to be stored in the coroutine-state. We can just store it as a normallocal variable in the __g_resume function.

We also need to add the necessary entry to the __g_destroy() function to handle whenthe coroutine is destroyed at this suspend-point.

void __g_destroy(__coroutine_state* s) { auto* state = static_cast<__g_state*>(s); switch (state->__suspend_point) { case 0: goto suspend_point_0; case 1: goto suspend_point_1; // <-- add new jump-table entry default: std::unreachable(); }suspend_point_0: state->__tmp1.destroy(); goto destroy_state;suspend_point_1: state->__tmp3.destroy(); state->__tmp2.destroy(); goto destroy_state; // TODO: Add extra logic for other suspend-points here.destroy_state: delete state;}

So now we have finished implementing the statement:

int fx = co_await f(x);

However, the function f(x) is not marked noexcept and so it can potentially throw an exception.Also, the awaiter::await_resume() method is also not marked noexcept and can also potentiallythrow an exception.

When an exception is thrown from a coroutine-body the compiler generates code to catch the exceptionand then invoke promise.unhandled_exception() to give the promise an opportunity to do somethingwith the exception. Let’s look at implementing this aspect next.

The specification for coroutine definitions [dcl.fct.def.coroutine]says that the coroutine behaves as if its function-body were replaced by:

{ promise-type promise promise-constructor-arguments ; try { co_await promise.initial_suspend() ; function-body } catch ( ... ) { if (!initial-await-resume-called) throw ; promise.unhandled_exception() ; }final-suspend : co_await promise.final_suspend() ;}

We have already handled the initial-await_resume-called branch separately in the ramp function,so we don’t need to worry about that here.

Let’s adjust the __g_resume() function to insert the try/catch block around the body.

Note that we need to be careful to put the switch that jumps to the right place insidethe try-block as we are not allowed to enter a try-block using a goto.

Also, we need to be careful to call .resume() on the coroutine handle returned fromawait_suspend() outside of the try/catch block. If an exception is thrown from thecall .resume() on the returned coroutine then it should not be caught by the currentcoroutine, but should instead propagate out of the call to resume() that resumedthis coroutine. So we stash the coroutine-handle in a variable declared at the topof the function and then goto a point outside of the try/catch and execute the callto .resume() there.

void __g_resume(__coroutine_state* s) { auto* state = static_cast<__g_state*>(s); std::coroutine_handle<void> coro_to_resume; try { switch (state->__suspend_point) { case 0: goto suspend_point_0; case 1: goto suspend_point_1; // <-- add new jump-table entry default: std::unreachable(); }suspend_point_0: state->__tmp1.get().await_resume(); state->__tmp1.destroy(); // int fx = co_await f(x); state->__tmp2.construct_from([&] { return f(state->x); }); state->__tmp3.construct_from([&] { return static_cast<task&&>(state->__tmp2.get()).operator co_await(); }); if (!state->__tmp3.get().await_ready()) { state->__suspend_point = 1; coro_to_resume = state->__tmp3.get().await_suspend( std::coroutine_handle<__g_promise_t>::from_promise(state->__promise)); goto resume_coro; }suspend_point_1: int fx = state->__tmp3.get().await_resume(); state->__tmp3.destroy(); state->__tmp2.destroy(); // TODO: Implement // co_return fx * fx; } catch (...) { state->__promise.unhandled_exception(); goto final_suspend; }final_suspend: // TODO: Implement // co_await promise.final_suspend();resume_coro: coro_to_resume.resume(); return;}

There is a bug in the above code, however. In the case that the __tmp3.get().await_resume() callexits with an exception, we would fail to call the destructors of __tmp3 and __tmp2 beforecatching the exception.

Note that we cannot simply catch the exception, call the destructors and rethrow the exceptionhere as this would change the behaviour of those destructors if they were to callstd::unhandled_exceptions() since the exception would be “handled”. However if thedestructor calls this during exception unwind, then call to std:::unhandled_exceptions()should return non-zero.

We can instead define an RAII helper class to ensure that the destructors get called on scopeexit in the case an exception is thrown.

template<typename T>struct destructor_guard { explicit destructor_guard(manual_lifetime<T>& obj) noexcept : ptr_(std::addressof(obj)) {} // non-movable destructor_guard(destructor_guard&&) = delete; destructor_guard& operator=(destructor_guard&&) = delete; ~destructor_guard() noexcept(std::is_nothrow_destructible_v<T>) { if (ptr_ != nullptr) { ptr_->destroy(); } } void cancel() noexcept { ptr_ = nullptr; }private: manual_lifetime<T>* ptr_;};// Partial specialisation for types that don't need their destructors called.template<typename T> requires std::is_trivially_destructible_v<T>struct destructor_guard<T> { explicit destructor_guard(manual_lifetime<T>&) noexcept {} void cancel() noexcept {}};// Class-template argument deduction to simplify usagetemplate<typename T>destructor_guard(manual_lifetime<T>& obj) -> destructor_guard<T>;

Using this utility, we can now use this type to ensure that variables stored in the coroutine-stateare destroyed when an exception is thrown.

Let’s also use this class to call the destructors of the existing varibles so that it also callstheir destructors when they naturally go out of scope.

void __g_resume(__coroutine_state* s) { auto* state = static_cast<__g_state*>(s); std::coroutine_handle<void> coro_to_resume; try { switch (state->__suspend_point) { case 0: goto suspend_point_0; case 1: goto suspend_point_1; // <-- add new jump-table entry default: std::unreachable(); }suspend_point_0: { destructor_guard tmp1_dtor{state->__tmp1}; state->__tmp1.get().await_resume(); } // int fx = co_await f(x); { state->__tmp2.construct_from([&] { return f(state->x); }); destructor_guard tmp2_dtor{state->__tmp2}; state->__tmp3.construct_from([&] { return static_cast<task&&>(state->__tmp2.get()).operator co_await(); }); destructor_guard tmp3_dtor{state->__tmp3}; if (!state->__tmp3.get().await_ready()) { state->__suspend_point = 1; coro_to_resume = state->__tmp3.get().await_suspend( std::coroutine_handle<__g_promise_t>::from_promise(state->__promise)); // A coroutine suspends without exiting scopes. // So cancel the destructor-guards. tmp3_dtor.cancel(); tmp2_dtor.cancel(); goto resume_coro; } // Don't exit the scope here. // // We can't 'goto' a label that enters the scope of a variable with a // non-trivial destructor. So we have to exit the scope of the destructor // guards here without calling the destructors and then recreate them after // the `suspend_point_1` label. tmp3_dtor.cancel(); tmp2_dtor.cancel(); }suspend_point_1: int fx = [&]() -> decltype(auto) { destructor_guard tmp2_dtor{state->__tmp2}; destructor_guard tmp3_dtor{state->__tmp3}; return state->__tmp3.get().await_resume(); }(); // TODO: Implement // co_return fx * fx; } catch (...) { state->__promise.unhandled_exception(); goto final_suspend; }final_suspend: // TODO: Implement // co_await promise.final_suspend();resume_coro: coro_to_resume.resume(); return;}

Now our coroutine body will now destroy local variables correctly in the presence of any exceptions andwill correctly call promise.unhandled_exception() if those exceptions propagate out of the coroutinebody.

It’s worth noting here that there can also be special handling needed for the case where thepromise.unhandled_exception() method itself exits with an exception (e.g. if it rethrowsthe current exception).

In this case, the coroutine would need to catch the exception, mark the coroutine as suspendedat a final-suspend-point, and then rethrow the exception.

For example: The __g_resume() function’s catch-block would need to look like this:

try { // ...} catch (...) { try { state->__promise.unhandled_exception(); } catch (...) { state->__suspend_point = 2; state->__resume = nullptr; // mark as final-suspend-point throw; }}

and we’d need to add an extra entry to the __g_destroy function’s jump table:

switch (state->__suspend_point) {case 0: goto suspend_point_0;case 1: goto suspend_point_1;case 2: goto destroy_state; // no variables in scope that need to be destroyed // just destroy the coroutine-state object.}

Note that in this case, the final-suspend-point is not necessarily the same suspend-point as thefinal-suspend-point as the co_await promise.final_suspend() suspend-point.

This is because the promise.final_suspend() suspend-point will often have some extra temporaryobjects related to the co_await expression which need to be destroyed when coroutine_handle::destroy()is called. Whereas, if promise.unhandled_exception() exits with an exception then those temporaryobjects will not exist and so won’t need to be destroyed by coroutine_handle::destroy().

The next step is to implement the co_return fx * fx; statement.

This is relatively straight-forward compared to some of the previous steps.

The co_return <expr> statement gets mapped to:

promise.return_value(<expr>);goto final-suspend-point;

So we can simply replace the TODO comment with:

state->__promise.return_value(fx * fx);goto final_suspend;

Easy.

The final TODO in the code is now to implement the co_await promise.final_suspend() statement.

The final_suspend() method returns a temporary task::promise_type::final_awaiter type, which willneed to be stored in the coroutine-state and destroyed in __g_destroy.

This type does not have its own operator co_await(), so we don’t need an additional temporaryobject for the result of that call.

Like the task::awaiter type, this also uses the coroutine-handle-returning form ofawait_suspend(). So we need to ensure that we call resume() on the returned handle.

If the coroutine does not suspend at the final-suspend-point then the coroutine-state is implicitlydestroyed. So we need to delete the state object if execution reaches the end of the coroutine.

Also, as all of the final-suspend logic is required to be noexcept, we don’t need to worry aboutexceptions being thrown from any of the sub-expressions here.

Let’s first add the data-member to the __g_state type.

struct __g_state : __coroutine_state_with_promise<__g_promise_t> { __g_state(int&& __x); ~__g_state(); int __suspend_point = 0; int x; manual_lifetime<std::suspend_always> __tmp1; manual_lifetime<task> __tmp2; manual_lifetime<task::awaiter> __tmp3; manual_lifetime<task::promise_type::final_awaiter> __tmp4; // <---};

Then we can implement the body of the final-suspend expression as follows:

final_suspend: // co_await promise.final_suspend { state->__tmp4.construct_from([&]() noexcept { return state->__promise.final_suspend(); }); destructor_guard tmp4_dtor{state->__tmp4}; if (!state->__tmp4.get().await_ready()) { state->__suspend_point = 2; state->__resume = nullptr; // mark as final suspend-point coro_to_resume = state->__tmp4.get().await_suspend( std::coroutine_handle<__g_promise_t>::from_promise(state->__promise)); tmp4_dtor.cancel(); goto resume_coro; } state->__tmp4.get().await_resume(); } // Destroy coroutine-state if execution flows off end of coroutine delete state; return;

And now we also need to update the __g_destroy function to handle this new suspend-point.

void __g_destroy(__coroutine_state* state) { auto* state = static_cast<__g_state*>(s); switch (state->__suspend_point) { case 0: goto suspend_point_0; case 1: goto suspend_point_1; case 2: goto suspend_point_2; default: std::unreachable(); }suspend_point_0: state->__tmp1.destroy(); goto destroy_state;suspend_point_1: state->__tmp3.destroy(); state->__tmp2.destroy(); goto destroy_state;suspend_point_2: state->__tmp4.destroy(); goto destroy_state;destroy_state: delete state;}

We now have a fully functional lowering of the g() coroutine function.

We’re done! That’s it!

Or is it….

It turns out there is actually a problem with the way we have implemented our __g_resume() function above.

The problems with this were discussed in more detail in the previous blog post so if youwant to understand the problem more deeply please take a look at the postC++ Coroutines: Understanding Symmetric Transfer.

The specification for [expr.await] gives a little hint abouthow we should be handling the coroutine-handle-returning flavour of await_suspend:

If the type of await-suspend is std​::​coroutine_­handle<Z>, await-suspend.resume() is evaluated.

[Note 1: This resumes the coroutine referred to by the result of await-suspend.Any number of coroutines can be successively resumed in this fashion, eventually returning controlflow to the current coroutine caller or resumer ([dcl.fct.def.coroutine]). —- end note]

The note there, while non-normative and thus non-binding, is strongly encouraging compilers to implement this in such a way that it performs a tail-call to resume the next coroutine ratherthan resuming the next coroutine recursively. This is because resuming the next coroutinerecursively can easily lead to unbounded stack growth if coroutines resume each other in a loop.

The problem is that we are calling .resume() on the next coroutine from within the body ofthe __g_resume() function and then returning, so the stack space used by the __g_resume()frame is not freed until after the next coroutine suspends and returns.

Compilers are able to do this by implementing the resumption of the next coroutine as atail-call. In this way, the compiler generates code that first pops the the current stackframe, preserving the return-address, and then executes a jmp to the next coroutine’sresume-function.

As we don’t have a mechanism in C++ to specify that a function-call in the tail-positionshould be a tail-call we will need to instead actually return from the resume-function sothat its stack-space can be freed, and then have the caller resume the next coroutine.

As the next coroutine may also need to resume another coroutine when it suspends, and thismay happen indefinitely, the caller will need to resume the coroutines in a loop.

Such a loop is typically called a “trampoline loop” as we return back to the loop from onecoroutine and then “bounce” off the loop back into the next coroutine.

If we modify the signature of the resume-function to return a pointer to the next coroutine’scoroutine-state instead of returning void, then the coroutine_handle::resume() function canthen just immediately call the __resume() function-pointer for the next coroutine to resumeit.

Let’s change the signature of the __resume_fn for a __coroutine_state:

struct __coroutine_state { using __resume_fn = __coroutine_state* (__coroutine_state*); using __destroy_fn = void (__coroutine_state*); __resume_fn* __resume; __destroy_fn* __destroy;};

Then we can write the coroutine_handle::resume() function something like this:

void std::coroutine_handle<void>::resume() const { __coroutine_state* s = state_; do { s = s->__resume(s); } while (/* some condition */);}

The next question then becomes: “What should the condition be?”

This is where the std::noop_coroutine() helper comes into the picture.

The std::noop_coroutine() is a factory function that returns a special coroutinehandle that has a no-op resume() and destroy() method. If a coroutine suspendsand returns the noop-coroutine-handle from the await_suspend() method then thisindicates that there is no more coroutine to resume and that the invocation ofcoroutien_handle::resume() that resumed this coroutine should return to its caller.

So we need to implement std::noop_coroutine() and the condition in coroutine_handle::resume()so that the condition returns false and the loop exits when the __coroutine_state pointerpoints to the noop-coroutine-state.

One strategy we can use here is to define a static instance of __coroutine_state that isdesignated as the noop-coroutine-state. The std::noop_coroutine() function can return acoroutine-handle that points to this object, and we can compare the __coroutine_statepointer to the address of that object to see if a particular coroutine handle is thenoop-coroutine.

First let’s define this special noop-coroutine-state object:

struct __coroutine_state { using __resume_fn = __coroutine_state* (__coroutine_state*); using __destroy_fn = void (__coroutine_state*); __resume_fn* __resume; __destroy_fn* __destroy; static __coroutine_state* __noop_resume(__coroutine_state* state) noexcept { return state; } static void __noop_destroy(__coroutine_state*) noexcept {} static const __coroutine_state __noop_coroutine;};inline const __coroutine_state __coroutine_state::__noop_coroutine{ &__coroutine_state::__noop_resume, &__coroutine_state::__noop_destroy};

Then we can implement the std::coroutine_handle<noop_coroutine_promise> specialisation.

namespace std{ struct noop_coroutine_promise {}; using noop_coroutine_handle = coroutine_handle<noop_coroutine_promise>; noop_coroutine_handle noop_coroutine() noexcept; template<> class coroutine_handle<noop_coroutine_promise> { public: constexpr coroutine_handle(const coroutine_handle&) noexcept = default; constexpr coroutine_handle& operator=(const coroutine_handle&) noexcept = default; constexpr explicit operator bool() noexcept { return true; } constexpr friend bool operator==(coroutine_handle, coroutine_handle) noexcept { return true; } operator coroutine_handle<void>() const noexcept { return coroutine_handle<void>::from_address(address()); } noop_coroutine_promise& promise() const noexcept { static noop_coroutine_promise promise; return promise; } constexpr void resume() const noexcept {} constexpr void destroy() const noexcept {} constexpr bool done() const noexcept { return false; } constexpr void* address() const noexcept { return const_cast<__coroutine_state*>(&__coroutine_state::__noop_coroutine); } private: constexpr coroutine_handle() noexcept = default; friend noop_coroutine_handle noop_coroutine() noexcept { return {}; } };}

And we can update coroutine_handle::resume() to exit when the noop-coroutine-stateis returned.

void std::coroutine_handle<void>::resume() const { __coroutine_state* s = state_; do { s = s->__resume(s); } while (s != &__coroutine_state::__noop_coroutine);}

And finally, we can update our __g_resume() function to now return the __coroutine_state*.

This just involves updating the signature and replacing:

coro_to_resume = ...;goto resume_coro;

with

auto h = ...;return static_cast<__coroutine_state*>(h.address());

and then at the very end of the function (after the delete state; statement) adding

return static_cast<__coroutine_state*>(std::noop_coroutine().address());

Those with a keen eye may have noticed that the coroutine-state type __g_state is actuallylarger than it needs to be.

The data-members for the 4 temporary values each reserve storage for their respective values.However, the lifetimes of some of the temporary values do not overlap and so in theory we cansave space in the coroutine-state by reusing the storage of an object for the next object afterits lifetime has ended.

To be able to take advantage of this we can instead define the data-members in an anonymousunion where appropriate.

Looking at the lifetimes of the temporary varaibles we have:

• __tmp1 - exists only within co_await promise.initial_suspend(); statement
• __tmp2 - exists only within int fx = co_await f(x); statement
• __tmp3 - exists only within int fx = co_await f(x); statement - nested inside lifetime of __tmp2
• __tmp4 - exists only within co_await promise.final_suspend(); statement

Since lifetimes of __tmp2 and __tmp3 overlap we must place them in a struct togetheras they both need to exist at the same time.

However, the __tmp1 and __tmp4 members do not have lifetimes that overlap and so they canbe placed together in an anonymous union.

Thus we can change our data-member definition to:

struct __g_state : __coroutine_state_with_promise<__g_promise_t> { __g_state(int&& x); ~__g_state(); int __suspend_point = 0; int x; struct __scope1 { manual_lifetime<task> __tmp2; manual_lifetime<task::awaiter> __tmp3; }; union { manual_lifetime<std::suspend_always> __tmp1; __scope1 __s1; manual_lifetime<task::promise_type::final_awaiter> __tmp4; };};

Then, because the __tmp2 and __tmp3 variables are now nested inside the __s1 object,we need to update references to them to now be e.g. state->__s1.tmp2. But otherwise therest of the code stays the same.

This should save an additional 16 bytes of the coroutine-state size as we no longer needextra storage + padding for the __tmp1 and __tmp4 data-members - which would otherwisebe padded to the size of a pointer, despite being empty types.

Ok, so the final code we have generated for the coroutine function:

task g(int x) { int fx = co_await f(x); co_return fx * fx;}

is the following:

/////// The coroutine promise-typeusing __g_promise_t = std::coroutine_traits<task, int>::promise_type;__coroutine_state* __g_resume(__coroutine_state* s);void __g_destroy(__coroutine_state* s);/////// The coroutine-state definitionstruct __g_state : __coroutine_state_with_promise<__g_promise_t> { __g_state(int&& x) : x(static_cast<int&&>(x)) { // Initialise the function-pointers used by coroutine_handle methods. this->__resume = &__g_resume; this->__destroy = &__g_destroy; // Use placement-new to initialise the promise object in the base-class // after we've initialised the argument copies. ::new ((void*)std::addressof(this->__promise)) __g_promise_t(construct_promise<__g_promise_t>(this->x)); } ~__g_state() { this->__promise.~__g_promise_t(); } int __suspend_point = 0; // Argument copies int x; // Local variables/temporaries struct __scope1 { manual_lifetime<task> __tmp2; manual_lifetime<task::awaiter> __tmp3; }; union { manual_lifetime<std::suspend_always> __tmp1; __scope1 __s1; manual_lifetime<task::promise_type::final_awaiter> __tmp4; };};/////// The "ramp" functiontask g(int x) { std::unique_ptr<__g_state> state(new __g_state(static_cast<int&&>(x))); decltype(auto) return_value = state->__promise.get_return_object(); state->__tmp1.construct_from([&]() -> decltype(auto) { return state->__promise.initial_suspend(); }); if (!state->__tmp1.get().await_ready()) { state->__tmp1.get().await_suspend( std::coroutine_handle<__g_promise_t>::from_promise(state->__promise)); state.release(); // fall through to return statement below. } else { // Coroutine did not suspend. Start executing the body immediately. __g_resume(state.release()); } return return_value;}/////// The "resume" function__coroutine_state* __g_resume(__coroutine_state* s) { auto* state = static_cast<__g_state*>(s); try { switch (state->__suspend_point) { case 0: goto suspend_point_0; case 1: goto suspend_point_1; // <-- add new jump-table entry default: std::unreachable(); }suspend_point_0: { destructor_guard tmp1_dtor{state->__tmp1}; state->__tmp1.get().await_resume(); } // int fx = co_await f(x); { state->__s1.__tmp2.construct_from([&] { return f(state->x); }); destructor_guard tmp2_dtor{state->__s1.__tmp2}; state->__s1.__tmp3.construct_from([&] { return static_cast<task&&>(state->__s1.__tmp2.get()).operator co_await(); }); destructor_guard tmp3_dtor{state->__s1.__tmp3}; if (!state->__s1.__tmp3.get().await_ready()) { state->__suspend_point = 1; auto h = state->__s1.__tmp3.get().await_suspend( std::coroutine_handle<__g_promise_t>::from_promise(state->__promise)); // A coroutine suspends without exiting scopes. // So cancel the destructor-guards. tmp3_dtor.cancel(); tmp2_dtor.cancel(); return static_cast<__coroutine_state*>(h.address()); } // Don't exit the scope here. // We can't 'goto' a label that enters the scope of a variable with a // non-trivial destructor. So we have to exit the scope of the destructor // guards here without calling the destructors and then recreate them after // the `suspend_point_1` label. tmp3_dtor.cancel(); tmp2_dtor.cancel(); }suspend_point_1: int fx = [&]() -> decltype(auto) { destructor_guard tmp2_dtor{state->__s1.__tmp2}; destructor_guard tmp3_dtor{state->__s1.__tmp3}; return state->__s1.__tmp3.get().await_resume(); }(); // co_return fx * fx; state->__promise.return_value(fx * fx); goto final_suspend; } catch (...) { state->__promise.unhandled_exception(); goto final_suspend; }final_suspend: // co_await promise.final_suspend { state->__tmp4.construct_from([&]() noexcept { return state->__promise.final_suspend(); }); destructor_guard tmp4_dtor{state->__tmp4}; if (!state->__tmp4.get().await_ready()) { state->__suspend_point = 2; state->__resume = nullptr; // mark as final suspend-point auto h = state->__tmp4.get().await_suspend( std::coroutine_handle<__g_promise_t>::from_promise(state->__promise)); tmp4_dtor.cancel(); return static_cast<__coroutine_state*>(h.address()); } state->__tmp4.get().await_resume(); } // Destroy coroutine-state if execution flows off end of coroutine delete state; return static_cast<__coroutine_state*>(std::noop_coroutine().address());}/////// The "destroy" functionvoid __g_destroy(__coroutine_state* s) { auto* state = static_cast<__g_state*>(s); switch (state->__suspend_point) { case 0: goto suspend_point_0; case 1: goto suspend_point_1; case 2: goto suspend_point_2; default: std::unreachable(); }suspend_point_0: state->__tmp1.destroy(); goto destroy_state;suspend_point_1: state->__s1.__tmp3.destroy(); state->__s1.__tmp2.destroy(); goto destroy_state;suspend_point_2: state->__tmp4.destroy(); goto destroy_state;destroy_state: delete state;}

For a fully compilable version of the final code, see:https://godbolt.org/z/xaj3Yxabn

This concludes the 5-part series on understanding the mechanics of C++ coroutines.

This is probably more information than you ever wanted to know about coroutines, buthopefully it helps you to understand what’s going on under the hood and demystifiesthem just a bit.

Thanks for making it through to the end!

Until next time, Lewis.