All posts by botondballo

About botondballo

I am a recent graduate of the University of Toronto, with a degree in Computer Science. I currently work for Mozilla as a software engineer.

Trip Report: C++ Standards Meeting in Cologne, July 2019

Summary / TL;DR (new developments since last meeting in bold)

Project What’s in it? Status
C++20 See below On track
Library Fundamentals TS v3 See below Under development
Concepts Constrained templates In C++20
Parallelism TS v2 Task blocks, library vector types and algorithms, and more Published!
Executors Abstraction for where/how code runs in a concurrent context Targeting C++23
Concurrency TS v2 See below Under active development
Networking TS Sockets library based on Boost.ASIO Published! Not in C++20.
Ranges Range-based algorithms and views In C++20
Coroutines Resumable functions (generators, tasks, etc.) In C++20
Modules A component system to supersede the textual header file inclusion model In C++20
Numerics TS Various numerical facilities Under active development
C++ Ecosystem TR Guidance for build systems and other tools for dealing with Modules Under active development
Contracts Preconditions, postconditions, and assertions Pulled from C++20, now targeting C++23
Pattern matching A match-like facility for C++ Under active development, targeting C++23
Reflection TS Static code reflection mechanisms Publication imminent
Reflection v2 A value-based constexpr formulation of the Reflection TS facilities Under active development, targeting C++23
Metaclasses Next-generation reflection facilities Early development

A few links in this blog post may not resolve until the committee’s post-meeting mailing is published (expected within a few days of August 5, 2019). If you encounter such a link, please check back in a few days.

Introduction

Last week I attended a meeting of the ISO C++ Standards Committee (also known as WG21) in Cologne, Germany. This was the second committee meeting in 2019; you can find my reports on preceding meetings here (February 2019, Kona) and here (November 2018, San Diego), and previous ones linked from those. These reports, particularly the Kona one, provide useful context for this post.

This week the committee reached a very important milestone in the C++20 publication schedule: we approved the C++20 Committee Draft (CD), a feature-complete draft of the C++20 standard which includes wording for all of the new features we plan to ship in C++20.

The next step procedurally is to send out the C++20 CD to national standards bodies for a formal ISO ballot, where they have the opportunity to comment on it. The ballot period is a few months, and the results will be in by the next meeting, which will be in November in Belfast, Northern Ireland. We will then spend that meeting and the next one addressing the comments, and then publishing a revised draft standard. Importantly, as this is a feature-complete draft, new features cannot be added in response to comments; only bugfixes to existing features can be made, and in rare cases where a serious problem is discovered, a feature can be removed.

Attendance at this meeting once again broke previous records, with over 200 people present for the first time ever. It was observed that one of the likely reasons for the continued upward trend in attendance is the proliferation of domain-specific study groups such as SG 14 (Games and Low-Latency Programming) and SG 19 (Machine Learning) which is attracting new experts from those fields.

Note that the committe now tracks its proposals in GitHub. If you’re interested in the status of a proposal, you can find its issue on GitHub by searching for its title or paper number, and see its status — such as which subgroups it has been reviewed by and what the outcome of the reviews were — there.

C++20

Here are the new changes voted into C++20 Working Draft at this meeting. For a list of changes voted in at previous meetings, see my Kona report. (As a quick refresher, major features voted in at previous meetings include modules, coroutines, default comparisons (<=>), concepts, and ranges.)

Technical Specifications

In addition to the C++ International Standard (IS), the committee publishes Technical Specifications (TS) which can be thought of experimental “feature branches”, where provisional specifications for new language or library features are published and the C++ community is invited to try them out and provide feedback before final standardization.

At this meeting, the focus was on the C++20 CD, and not so much on TSes. In particular, there was no discussion of merging TSes into the C++ IS, because the deadline for doing so for C++20 was the last meeting (where Modules and Coroutines were merged, joining the ranks of Concepts which was merged a few meetings prior), and it’s too early to be discussing mergers into C++23.

Nonetheless, the committee does have a few TSes in progress, and I’ll mention their status:

Reflection TS

The Reflection TS was approved for publication at the last meeting. The publication process for this TS is a little more involved than usual: due to the dependency on the Concepts TS, the Reflection TS needs to be rebased on top of C++14 (the Concepts TS’ base document) for publication. As a result, the official publication has not happened yet, but it’s imminent.

As mentioned before, the facilities in the Reflection TS are not planned to be merged into the IS in their current form. Rather, a formulation based on constexpr values (rather than types) is being worked on. This is a work in progress, but recent developments have been encouraging (see the SG7 (Reflection) section) and I’m hopeful about them making C++23.

Library Fundamentals TS v3

This third iteration (v3) of the Library Fundamentals TS continues to be open for new features. It hasn’t received much attention in recent meetings, as the focus has been on libraries targeted at C++20, but I expect it will continue to pick up material in the coming meetings.

Concurrency TS v2

A concrete plan for Concurrency TS v2 is starting to take shape.

The following features are planned to definitely be included:

The following additional features might tag along if they’re ready in time:

I don’t think there’s a timeline for publication yet; it’s more “when the features in the first list are ready”.

Networking TS

As mentioned before, the Networking TS did not make C++20. As it’s now targeting C++23, we’ll likely see some proposal for design changes between now and its merger into C++23.

One such potential proposal is one that would see the Networking TS support TLS out of the box. JF Bastien from Apple has been trying to get browser implementers on board with such a proposal, which might materialize for the upcoming Belfast meeting.

Evolution Working Group

As usual, I spent most of the week in EWG. Here I will list the papers that EWG reviewed, categorized by topic, and also indicate whether each proposal was approved, had further work on it encouraged, or rejected. Approved proposals are targeting C++20 unless otherwise mentioned; “further work” proposals are not.

Concepts

  • (Approved) Rename concepts to standard_case for C++20, while we still can. Concepts have been part of the C++ literature long before the C++20 language feature that allows them to be expressed in code; for example, they are discussed in Stepanov’s Elements of Programming, and existing versions of the IS document describe the notional concepts that form the requirements for various algorithms. In this literature, concepts are conventionally named in PascalCase. As a result, the actual language-feature concepts added to the standard library in C++20 were named in PascalCase as well. However, it was observed that essentially every other name in the standard library uses snake_case, and remaining consistent with that might be more important than respecting naming conventions from non-code literature. This was contentious, for various reasons: (1) it was late in the cycle to make this change; (2) a pure mechanical rename resulted in some conflicts with existing names, necessitating additional changes that went beyond case; and (3) some people liked the visual distinction that PascalCase conferred onto concept names. Nonetheless, EWG approved the change.
  • (Approved) On the non-uniform semantics of return-type-requirements. This proposal axes concept requirements of the form expression -> Type, because their semantics are not consistent with trailing return types which share the same syntax.
  • (Approved) Using unconstrained template template parameters with constrained templates. This paper allows unconstrained template template parameters to match constrained templates; without this change, it would have been impossible to write a template template parameter that matches any template regardless of constraints, which is an important use case.

Contracts

Contracts were easily the most contentious and most heavily discussed topic of the week. In the weeks leading up the meeting, there were probably 500+ emails on the committee mailing lists about them.

The crux of the problem is that contracts can have a range of associated behaviours / semantics: whether they are checked, what happens if they are checked and fail, whether the compiler can assume them to be true in various scenarios, etc. The different behaviours lend themselves to different use cases, different programming models, different domains, and different stages of the software lifecycle. Given the diversity of all of the above represented at the committee, people are having a really hard time agreeing on what set of possible behaviours the standard should allow for, what the defaults should be, and what mechanisms should be available to control the behaviour in case you want something other than the defaults.

A prominent source of disagreement is around the possibility for contracts to introduce undefined behaviour (UB) if we allow compilers to assume their truth, particularly in cases where they are not checked, or where control flow is allowed to continue past a contract failure.

Contracts were voted into the C++20 working draft in June 2018; the design that was voted in was referred to as the “staus quo design” during this week’s discussions (since being in the working draft made it the status quo). In a nutshell, in the status quo design, the programmer could annotate contracts as having one of three levels — default, audit, or axiom — and the contract levels were mapped to behaviours using two global switches (controlled via an implementation-defined mechanism, such as a compiler flag): a “build level” and a “continuation mode”.

The status quo design clearly had consensus at the time it was voted in, but since then that consensus had begun to increasingly break down, leading to a slew of Contracts-related proposals submitted for the previous meeting and this one.

I’ll summarize the discussions that took place this week, but as mentioned above, the final outcome was that Contracts was removed from C++20 and is now targeting C++23.

EWG discussed Contracts on two occasions during the week, Monday and Wednesday. On Monday, we started with a scoping discussion, where we went through the list of proposals, and decided which of them we were even willing to discuss. Note that, as per the committee’s schedule for C++20, the deadline for making design changes to a C++20 feature had passed, and EWG was only supposed to consider bugfixes to the existing design, though as always that’s a blurry line.

Anyways, the following proposals were rejected with minimal discussion on the basis of being a design change:

That left the following proposals to be considered. I list them here in the order of discussion. Please note that the “approvals” from this discussion were effectively overturned by the subsequent removal of Contracts from C++20.

  • (Rejected) What to do about contracts? This proposed two alternative minimal changes to the status quo design, with the primary aim of addressing the UB concerns, but neither had consensus. (Another paper was essentially a repeat of one of the alternatives and was not polled separately.)
  • (Rejected) Axioms should be assumable. This had a different aim (allowing the compiler to assume contracts in more cases, not less) and also did not have consensus.
  • (Approved) Minimizing contracts. This was a two-part proposal. The first part removed the three existing contract levels (default, audit, and axiom), as well as the build level and continuation mode, and made the way the behaviour of a contract checking statement is determined completely implementation-defined. The second part essentially layered on top the “Contracts that work” proposal, which introduces literal semantics: rather than annotating contracts with “levels” which are somehow mapped onto behaviours, contracts are annotated with their desired behaviour directly; if the programmer wants different behaviours in different build modes, they can arrange for that themselves, using e.g. macros that expand to different semantics in different build modes. EWG approved both parts, which was somewhat surprising because “Contracts that work” was previously voted as not even being in scope for discussion. I think the sentiment was that, while this is a design change, it has more consensus than the status quo, and so it’s worth trying to sneak it in even though we’re past the design change deadline. Notably, while this proposal did pass, it was far from unanimous, and the dissenting minority was very vocal about their opposition, which ultimately led to the topic being revisited and Contracts being axed from C++20 on Wednesday.
  • (Approved) The “default” contract build-level and continuation-mode should be implementation-defined. This was also approved, which is also somewhat suprising given that it was mooted by the previous proposal. Hey, we’re not always a completely rational bunch!

To sum up what happened on Monday: EWG made a design change to Contracts, and that design change had consensus among the people in the room at the time. Unfortunately, subsequent discussions with people not in the room, including heads of delegations from national standards bodies, made it clear that the design change was very unlikely to have the consensus of the committee at large in plenary session, largely for timing reasons (i.e. it being too late in the schedule to make such a nontrivial design change).

As people were unhappy with the status quo, but there wasn’t consensus for a design change either, that left removing contracts from C++20 and continuing to work on it in the C++23 cycle. A proposal to do so was drafted and discussed in EWG on Wednesday, with a larger group of people in attendance this time, and ultimately garnered consensus.

To help organize further work on Contracts in the C++23 timeframe, a new Study Group, SG 21 (Contracts) was formed, which would incubate and refine an updated proposal before it comes back to EWG. It’s too early to say what the shape of that proposal might be.

I personally like literal semantics, though I agree it probably wouldn’t have been prudent to make a significant change like that for C++20. I would welcome a future proposal from SG 21 that includes literal semantics.

Modules

A notable procedural development in the area of Modules, is that the Modules Study Group (SG 2) was resurrected at the last meeting, and met during this meeting to look at all Modules-related proposals and make recommendations about them. EWG then looked at the ones SG 2 recommended for approval for C++20:

  • (Approved) Mitigating minor Modules maladies. EWG affirmed SG2’s recommendation to accept the first and third parts (concerning typedef names and default arguments, respectively) for C++20.
  • (Approved) Relaxing redefinition restrictions for re-exportation robustness. This proposal makes “include translation” (the automatic translation of some #include directives into module imports) optional, because it is problematic for some use cases, and solves the problems that motivated mandatory include translation in another way. (Aside: Richard Smith, the esteemed author of this paper and the previous one, clearly has too much time on his hands if he can come up with alliterations like this for paper titles. We should give him some more work to do. Perhaps we could ask him to be the editor of the C++ IS document? Oh, we already did that… Something else then. Finish implementing Concepts in clang perhaps? <wink>)
  • (Approved) Standard library header units for C++20. This allows users to consume C++ standard library headers (but not headers inherited from C like <cmath>) using import rather than #include (without imposing any requirements (yet) that their contents actually be modularized). It also reserves module names whose first component is std, or std followed by a number, for use by the standard library.
  • (Approved) Recognizing header unit imports requires full preprocessing. This tweaks the context sensitivity rules for the import keyword in such a way that tools can quickly scan source files and gather their module dependencies without having to do too much processing (and in particular without having to do a full run of the preprocessor).

There were also some Modules-related proposals that SG2 looked at and decided not to advance for C++20, but instead continue iterating for C++23:

  • (Further work) The inline keyword is not in line with the design of modules. This proposal will be revised before EWG looks at it.
  • (Further work) ABI isolation for member functions. EWG did look at this, towards the end of the week when it ran out of C++20 material. The idea here is that people like to define class methods inline for brevity (to avoid repeating the function header in an out-of-line definition), but the effects this has on linkage are sometimes undesirable. In module interfaces in particular, the recently adopted rule changes concerning internal linkage mean that users can run into hard-to-understand errors as a result of giving methods internal linkage. The proposal therefore aims to dissociate whether a method is defined inline or out of line, from semantic effects on linkage (which could still be achieved by using the inline keyword explicitly). Reactions were somewhat mixed, with some concerns about impacts on compile-time and runtime performance. Some felt that if we do this at all, we should do it in C++20, so our guidance to authors of modular code can be consistent from the get-go; while it seems to be too late to make this change in C++20 itself, the idea of a possible future C++20 Defect Report was raised.

Finally, EWG favourably reviewed at the Tooling Study Group’s plans for a C++ Ecosystem Technical Report. One suggestion made was to give the TR a more narrowly scoped name to reflect its focus on Modules-related tooling (lest people are misled into expecting that it addresses every “C++ Ecosystem” concern).

Coroutines

EWG considered several proposed improvements to coroutines. All of them were rejected for C++20 due to being too big of a change at this late stage.

Coroutines will undoubtedly see improvements in the C++23 timeframe, including possibly having some of the above topics revisited, but of course we’ll now be limited to making changes that are backwards-compatible with the current design.

constexpr

  • (Approved) Enabling constexpr intrinsics by permitting unevaluated inline assembly in constexpr functions. With std::is_constant_evaluated(), you can already give an operation different implementations for runtime and compile-time evaluation. This proposal just allows the runtime implementations of such functions to use inline assembly.
  • (Approved) A tweak to constinit: EWG was asked to clarify the intended rules for non-initializing declarations. The Core Working Group’s recommendation — that a non-initializing declaration of a variable be permitted to contain constinit, and if it does, the initializing declaration must be constinit as well — was accepted.

Comparisons

  • (Approved) Spaceship needs a tune-up. This fixes some relatively minor fallout from recent spaceship-related bugfixes.
  • (Rejected) The spaceship needs to be grounded: pull spaceship from C++20. Concerns about the fact that we keep finding edge cases where we need to tweak spaceship’s behaviour, and that the rules have become rather complicated as a result of successive bug fixes, prompted this proposal to remove spaceship from C++20. EWG disagreed, feeling that the value this feature delivers for common use cases outweighs the downside of having complex rules to deal with uncommon edge cases.

Lightweight Exceptions

In one of the meeting’s more exciting developments, Herb Sutter’s lightweight exceptions proposal (affectionately dubbed “Herbceptions” in casual conversation) was finally discussed in EWG. I view this proposal as being particularly important, because it aims to heal the current fracture of the C++ user community into those who use exceptions and those who do not.

The proposal has four largely independent parts:

  • The first and arguably most interesting part (section 4.1 in the paper) provides a lightweight exception handling mechanism that avoids the overhead that today’s dynamic exceptions have, namely that of dynamic memory allocation and runtime type information (RTTI). The new mechanism is opt-in on a per-function basis, and designed to allow a codebase to transition incrementally from the old style of exceptions to the new one.
  • The next two parts have to do with using exceptions in fewer scenarios:
    • The second part (section 4.2) is about transitioning the standard library to handle logic errors not via exceptions like std::logic_error, but rather via a contract violation.
    • The third part (section 4.3) is about handling allocation failure via termination rather than an exception. Earlier versions of the proposal were more aggressive on this front, and aimed to make functions that today only throw exceptions related to allocation failure noexcept. However, that’s unlikely to fly, as there are good use cases for recovering from allocation failure, so more recent versions leave the choice of behaviour up to the allocator, and aim to make such functions conditionally noexcept.
  • The fourth part (section 4.5), made more realistic by the previous two, aims to make the remaining uses of exceptions more visible by allowing expressions that propagate exceptions to be annotated with the try keyword (there being prior art for this sort of thing in Swift and Rust). Of course, unlike Rust, use of the annotation would have to be optional for backwards compatibility, though one can envision enforcing its use locally in a codebase (or part of a codebase) via static analysis.

As can be expected from such an ambitious proposal, this prompted a lot of discussion in EWG. A brief summary of the outcome for each part:

  1. There was a lot of discussion both about how performant we can make the proposed lightweight exceptions, and about the ergonomics of the two mechanisms coexisting in the same program. (For the latter, a particular point of contention was that functions that opt into the new exceptions require a modified calling convention, which necessitates encoding the exception mode into the function type (for e.g. correct calling via function pointers), which fractures the type system). EWG cautiously encouraged further exploration, with the understanding that further experiments and especially implementation experience are needed to be able to provide more informed directional guidance.
  2. Will be discussed jointly by Evolution and Library Evolution in the future.
  3. EWG was somewhat skeptical about this one. In particular, the feeling in the room was that, while Library Evolution may allow writing allocators that don’t throw and library APIs may be revised to take advantage of this and make some functions conditionally noexcept, there was no consensus to move in the direction of making the default allocator non-throwing.
  4. EWG was not a fan of this one. The feeling was that the annotations would have limited utility unless they’re required, and we can’t realistically ever make them required.

I expect the proposal will return in revised form (and this will likely repeat for several iterations). The road towards achieving consensus on a significant change like this is a long one!

I’ll mention one interesting comment that was made during the proposal’s presentation: it was observed that since we need to revise the calling convention as part of this proposal anyways, perhaps we could take the opportunity to make other improvements to it as well, such as allowing small objects to be passed in registers, the lack of which is a pretty unfortunate performance problem today (certainly one we’ve run into at Mozilla multiple times). That seems intriguing.

Other new features

  • (Approved*) Changes to expansion statements. EWG previously approved a “for ...” construct which could be used to iterate at compile time over tuple-like objects and parameter packs. Prior to this meeting, it was discovered that the parameter pack formulation has an ambiguity problem. We couldn’t find a fix in time, so the support for parameter packs was dropped, leaving only tuple-like objects. However, “for ...” no longer seemed like an appropriate syntax if parameter packs are not supported, so the syntax was changed to “template for“. Unfortunately, while EWG approved “template for“, the Core Working Group ran out of time to review its wording, so (*) the feature didn’t make C++20. It will likely be revisited for C++23, possibly including ways to resolve the parameter pack issue.
  • (Further work) Pattern matching. EWG looked at a revised version of this proposal which features a refined pattern syntax among other improvements. The review was generally favourable, and the proposal, which is targeting C++23, is getting close to the stage where standard wording can be written and implementation experience gathered.

Bug / Consistency Fixes

(Disclaimer: don’t read too much into the categorization here. One person’s bug fix is another’s feature.)

For C++20:

For C++23:

  • (Approved) Size feedback in operator new. This allows operator new to communicate to its caller how many bytes it actually allocated, which can sometimes be larger than the requested amount.
  • (Approved) A type trait to detect scoped enumerations. This adds a type trait to tell apart enum classes from plan enums, which is not necessarily possible to do in pure library code.
  • (Approved in part) Literal suffixes for size_t and ptrdiff_t. The suffixes uz for size_t and z for ssize_t were approved. The suffixes t for ptrdiff_t and ut for a corresponding unsigned type had no consensus.
  • (Further work) Callsite based inlining hints: [[always_inline]] and [[never_inline]]. EWG was generally supportive, but requested the author provide additional motivation, and also clarify if they are orders to the compiler (usable in cases where inlining or not actually has a semantic effect), or just strong optimization hints.
  • (Further work) Defaultable default constructors and destructors for all unions. The motivation here is to allow having unions which are trivial but have nontrivial members. EWG felt this was a valid usecase, but the formulation in the paper erased important safeguards, and requested a different formulation.
  • (Further work) Name lookup should “find the first thing of that name”. EWG liked the proposed simplification, but requested that research be done to quantify the scope of potential breakage, as well as archaeology to better understand the motivation for the current rule (which no one in the room could recall.)

Proposals Not Discussed

As usual, there were papers EWG did not get to discussing at this meeting; see the committee website for a complete list. At the next meeting, after addressing any national body comments on the C++20 CD which are Evolutionary in nature, EWG expects to spend the majority of the meeting reviewing C++23-track proposals.

Evolution Working Group Incubator

Evolution Incubator, which acts as a filter for new proposals incoming to EWG, met for two days, and reviewed numerous proposals, approving the following ones to advance to EWG at the next meeting:

Other Working Groups

Library Groups

Having sat in the Evolution group, I haven’t been able to follow the Library groups in any amount of detail, but I’ll call out some of the library proposals that have gained design approval at this meeting:

Note that the above is all C++23 material; I listed library proposals which made C++20 at this meeting above.

There are also efforts in place to consolidate general design guidance that the Library Evolution group would like to apply to all proposals into a policy paper.

While still at the Incubator stage, I’d like to call attention to web_view, a proposal for embedding a view powered by a web browser engine into a C++ application, for the purpose of allowing C++ applications to leverage the wealth of web technologies for purposes such as graphical output, interaction, and so on. As mentioned in previous reports, I gathered feedback about this proposal from Mozilla engineers, and conveyed this feedback (which was a general discouragement for adding this type of facility to C++) both at previous meetings and this one. However, this was very much a minority view, and as a whole the groups which looked at this proposal (which included SG13 (I/O) and Library Evolution Incubator) largely viewed it favourably, as a promising way of allow C++ applications to do things like graphical output without having to standardize a graphics API ourselves, as previously attempted.

Study Groups

SG 1 (Concurrency)

SG 1 has a busy week, approving numerous proposals that made it into C++20 (listed above), as well as reviewing material targeted for the Concurrency TS v2 (whose outline I gave above).

Another notable topic for SG 1 was Executors, where a consensus design was reviewed and approved. Error handling remains a contentious issue; out of two different proposed mechanics, the first one seems to have the greater consensus.

Progress was also made on memory model issues, aided by the presence of several memory model experts who are not regular attendees. It seems the group may have an approach for resolving the “out of thin air” (OOTA) problem (see relevant papers); according to SG 1 chair Olivier Giroux, this is the most optimistic the group has been about the OOTA problem in ~20 years!

SG 7 (Compile-Time Programming)

The Compile-Time Programming Study Group (SG 7) met for half a day to discuss two main topics.

First on the agenda was introspection. As mentioned in previous reports, the committee put out a Reflection TS containing compile-time introspection facilities, but has since agreed that in the C++ IS, we’d like facilities with comparable expressive power but a different formulation (constexpr value-based metaprogramming rather than template metaprogramming). Up until recently, the nature of the new formulation was in dispute, with some favouring a monotype approach and others a richer type hierarchy. I’m pleased to report that at this meeting, a compromise approach was presented and favourably reviewed. With this newfound consensus, SG 7 is optimistic about being able to get these facilities into C++23. The compromise proposal does require a new language feature, parameter constraints, which will be presented to EWG at the next meeting.

(SG 7 also looked at a paper asking to revisit some of the previous design choices made regarding parameter names and access control in reflection. The group reaffirmed its previous decisions in these areas.)

The second main topic was reification, which can be thought of as the “next generation” of compile-time programming facilities, where you can not only introspect code at compile time, but perform processing on its representation and generate (“reify”) new code. A popular proposal in this area is Herb Sutter’s metaclasses, which allow you to “decorate” classes with metaprograms that transform the class definition in interesting ways. Metaclasses is intended to be built on a suite of underlying facilties such as code injection; there is now a concrete proposal for what those facilities could look like, and how metaclasses could be built on top of them. SG 7 looked at an overview of this proposal, although there wasn’t time for an in-depth design review at this stage.

SG 15 (Tooling)

The Tooling Study Group (SG 15) met for a full day, focusing on issues related to tooling around modules, and in particular proposals targeting the C++ Modules Ecosystem Technical Report mentioned above.

I couldn’t be in the room for this session as it ran concurrently with Reflection and then Herbceptions in EWG, but my understanding is that the main outcomes were:

  • The Ecosystem TR should contain guidelines for module naming conventions. There was no consensus to include conventions for other things such as project structure, file names, or namespace names.
  • The Ecosystem TR should recommend that implementations provide a way to implicitly build modules (that is, to be able to build them even in the absence of separate metadata specifying what modules are to be built and how), without requiring a particular project layout or file naming scheme. It was observed that implementing this in a performant way will likely require fast dependency scanning tools to extract module dependencies from source files. Such tools are actively being worked on (see e.g. clang-scan-deps), and the committee has made efforts to make them tractable (see e.g. the tweak to the context-sensitivity rules for import which EWG approved this week).

A proposal for a file format for describing dependencies of source files was also reviewed, and will continue to be iterated on.

One observation that was made during informal discussion was that SG 15’s recent focus on modules-related tooling has meant less time available for other topics such as package management. It remains to be seen if this is a temporary state of affairs, or if we could use two different study groups working in parallel.

Other Study Groups

Other Study Groups that met at this meeting include:

  • SG 2 (Modules), covered in the Modules section above.
  • SG 6 (Numerics) reviewed a dozen or so proposals, related to topics such as fixed-point numbers, type interactions, limits and overflow, rational numbers, and extended floating-point types. There was also a joint session with SG 14 (Games & Low Latency) and SG 19 (Machine Learning) to discuss linear algebra libraries and multi-dimensional data structures.
  • SG 12 (Undefined and Unspecified Behaviour). Topics discussed include pointer provenance, the C++ memory object model, and various other low-level topics. There was also the usual joint session with WG23 – Software Vulnerabilities; there is now a document describing the two groups’ relationship.
  • SG 13 (I/O), which reviewed proposals related to audio (proposal, feedback paper), web_view, 2D graphics (which continues to be iterated on in the hopes of a revised version gaining consensus), as well as few proposals related to callbacks which are relevant to the design of I/O facilities.
  • SG 14 (Games & Low Latency), whose focus at this meeting was on linear algebra proposals considered in joint session with SG 19
  • SG 16 (Unicode). Topics discussed include guidelines for where we want to impose requirements regarding character encodings, and filenames and the complexities they involve. The group also provided consults for relevant parts of other groups’ papers.
  • SG 19 (Machine Learning). In addition to linear algebra, the group considered proposals for adding statistical mathematical functions to C++ (simple stuff like mean, median, and standard deviation — somewhat surprising we don’t have them already!), as well as graph data structures.
  • SG 20 (Education), whose focus was on iterating on a document setting out proposed educational guidelines.

In addition, as mentioned, a new Contracts Study Group (SG 21) was formed at this meeting; I expect it will have its inaugural meeting in Belfast.

Most Study Groups hold regular teleconferences in between meetings, which is a great low-barrier-to-entry way to get involved. Check out their mailing lists here or here for telecon scheduling information.

Next Meeting

The next meeting of the Committee will be in Belfast, Northern Ireland, the week of November 4th, 2019.

Conclusion

My highlights for this meeting included:

  • Keeping the C++ release train schedule on track by approving the C++20 Committee Draft
  • Forming a Contracts Study Group to craft a high-quality, consensus-bearing Contracts design in C++23
  • Approving constexpr dynamic allocation, including constexpr vector and string for C++20
  • The standard library gaining a modern text formatting facility for C++20
  • Broaching the topic of bringing the -fno-exceptions segment of C++ users back into the fold
  • Breaking record attendance levels as we continue to gain representation of different parts of the community on the committee

Due to the sheer number of proposals, there is a lot I didn’t cover in this post; if you’re curious about a specific proposal that I didn’t mention, please feel free to ask about it in the comments.

Other Trip Reports

Other trip reports about this meeting include Herb Sutter’s, the collaborative Reddit trip report, Timur Doumler’s and Guy Davidson’s — I encourage you to check them out as well!

Trip Report: C++ Standards Meeting in Kona, February 2019

Summary / TL;DR (new developments since last meeting in bold)

Project What’s in it? Status
C++20 See below On track
Library Fundamentals TS v3 See below Under active development
Concepts TS Constrained templates Merged into C++20, including abbreviated function templates!
Parallelism TS v2 Task blocks, library vector types and algorithms, and more Published!
Executors Abstraction for where/how code runs in a concurrent context Not headed for C++20
Concurrency TS v2 See below Under active development
Networking TS Sockets library based on Boost.ASIO Published! Not headed for C++20.
Ranges TS Range-based algorithms and views Merged into C++20!
Coroutines TS Resumable functions, based on Microsoft’s await design Merged into C++20!
Modules v1 A component system to supersede the textual header file inclusion model Published as a TS
Modules v2 Improvements to Modules v1, including a better transition path Merged into C++20!
Numerics TS Various numerical facilities Under active development
Reflection TS Static code reflection mechanisms Approved for publication!
C++ Ecosystem TR Guidance for build systems and other tools for dealing with Modules Early development
Pattern matching A match-like facility for C++ Under active development, targeting C++23

Introduction

A few weeks ago I attended a meeting of the ISO C++ Standards Committee (also known as WG21) in Kona, Hawaii. This was the first committee meeting in 2019; you can find my reports on 2018’s meetings here (November 2018, San Diego), here (June 2018, Rapperswil), and here (March 2018, Jacksonville). These reports, particularly the San Diego one, provide useful context for this post.

This week marked the feature-complete deadline of C++20, so there was a heavy focus on figuring out whether certain large features that hadn’t yet merged into the working draft would make it in. Modules and Coroutines made it; Executors and Networking did not.

Attendance at this meeting wasn’t quite at last meeting’s record-breaking level, but it was still quite substantial. We continued the experiment started at the last meeting of running Evolution Incubator (“EWGI”) and Library Evolution Incubator (“LEWGI”) subgroups to pre-filter / provide high-level directional guidance for proposals targeting the Evolution and Library Evolution groups (EWG and LEWG), respectively.

Another notable procedural development is that the committee started to track proposals in front of the committee in GitHub. If you’re interested in the status of a proposal, you can find its issue on GitHub by searching for its title or paper number, and see its status — such as which subgroups it has been reviewed by and what the outcome of the reviews were — there.

C++20

Here are the new changes voted into C++20 Working Draft at this meeting. For a list of changes voted in at previous meetings, see my San Diego report. (As a quick refresher, major features voted in at previous meetings include default comparisons (<=>), concepts, contracts, and ranges.)

Technical Specifications

In addition to the C++ International Standard (IS), the committee publishes Technical Specifications (TS) which can be thought of experimental “feature branches”, where provisional specifications for new language or library features are published and the C++ community is invited to try them out and provide feedback before final standardization.

At this meeting, the committee iterated on a number of TSes under development.

Reflection TS

The Reflection TS was sent out for its PDTS ballot two meetings ago. As described in previous reports, this is a process where a draft specification is circulated to national standards bodies, who have an opportunity to provide feedback on it. The committee can then make revisions based on the feedback, prior to final publication.

The ballot results (often referred to as “NB comments”, as they are comments from national bodies in response to the ballot) were published between the last meeting and this one, and the TS authors prepared proposed resolutions, which various subgroups reviewed this week. I am pleased to report that the committee addressed all the comments this week, and subsequently voted to publish the TS as amended by the comment resolutions. The final draft is not prepared yet, but I expect it will be in the committee’s next mailing, and will then be transmitted to ISO for official publication.

(I mentioned previously that a procedural snafu necessitated rebasing the TS onto {C++17 + Concepts TS} as it could not reference the not-yet-published C++20 working draft which contains Concepts in their current form. I was slightly mistaken: as the Concepts TS, which was published in 2015, is based on C++14, the Reflection TS actually had to be rebased onto {C++14 + Concepts TS}. Geneva: 1, common sense: 0.)

I wish I could tell you that there is an implementation of the Reflection TS available for experimentation and encourage you to try it out. Unfortunately, to my knowledge there is no such implementation, nor is one imminent. (There is a WIP implementation in a clang branch, but I didn’t get the impression that it’s actively being worked on. I would be delighted to be mistaken on that point.) This state of affairs has led me to reflect (pun intended) on the TS process a bit.

Library Fundamentals TS v3

This third iteration (v3) of the Library Fundamentals TS is under active development, and gained its first new feature at this meeting, a generic scope guard and RAII wrapper. (The remaining contents of the TS working draft are features from v2 which haven’t been merged into the C++ IS yet.)

Merging Technical Specifications into C++20

This meeting was the deadline for merging published TSes into C++20, so naturally a large amount of attention on the outstanding ones such as Modules and Coroutines.

Modules TS

As mentioned in my previous report, Modules gained design approval at the end of the last meeting, in San Diego. This was the culmination of a multi-year effort to reconcile and merge two different approaches to Modules — the design from the Modules TS, which has its roots in Microsoft’s early implementation work, and the Atom proposal which was inspired by Clang Modules — into a unified and cohesive language feature.

I found it interesting to see how the conversation around Modules shifted as the two approaches achieved convergence. For much of the past few years, the discussions and controversies focused on the differences between the two proposals, such as macro support, incremental transition mechanisms, and module organization (preambles and partitions and such).

Now that the compiler implementers have achieved consensus on the language feature, the focus has shifted to parts of the C++ ecosystem outside of the compilers themselves that are affected by Modules — notably, build systems and other tools. The tooling community has a variety of outstanding concerns about Modules, and these concerns dominated the conversation around Modules at this meeting. I talk about this in more detail in the SG15 (Tooling) section below, but my point here is that consensus among compiler implementers does not necessarily imply consensus among the entire C++ community.

All the same, the Core Working Group proceeded with wording review of Modules at full speed, and it was completed in time to hold a plenary vote to merge the feature into C++20. As mentioned, this vote passed, in spite of concerns from the tooling community. That is to say, Modules are now officially in the C++20 working draft!

It’s important to note that this does not mean the committee doesn’t care about the tooling-related concerns, just that it has confidence that the concerns can be addressed between now and the publication of C++20 (or, in the case of issues whose resolution does not require a breaking change to the core language feature, post-C++20).

Coroutines TS

The proponents of the Coroutines TS have been trying to merge it into C++20 for quite some time. Each of the three previous meetings saw an attempt to merge it, with the latter two making it to a plenary vote, only to fail there. The reason it had failed to achieve consensus so far was that there were some concerns abouts its design, and a couple of alternative proposals which attempt to address those concerns (namely, Core Coroutines, which had been under development for a few meetings now, and a new one at this meeting, Symmetric Coroutines).

We are sufficiently late in the cycle that the alternative proposals had no chance of getting into C++20, so the decision the Committee needed to make is, are the improvements these alternatives purport to bring worth delaying the feature until C++23 or later. Thus far, the Committee had been delaying this decision, in the hopes that further development on the alternative proposals would lead to a more informed decision. With this meeting being the deadline for merging a TS into C++20, the opportunities for delay were over, and the decision needed to be made this week.

Knowing that we’re down to the wire, the EWG chair instructed the authors of the various proposals to collaborate on papers exploring the design space, putting the respective proposals into context, and comparing their approaches in detail.

The authors delivered on this request, with commendably thorough analysis papers. I talk about the technical issues a bit below, but the high level takeaways were as follows:

  • Both alternative proposals share an implementation challenge inherent to their attempt to expose the state of a coroutine as a first-class object, that would have signficant language impact. While compiler implementers agreed the proposals are implementable, they estimated the magnitude of the language impact to be sufficiently great that the ability to work out the specification issues and deliver an implementation in the C++23 timeframe was uncertain (that is, going with the alternatives would risk Coroutines being delayed until C++26).
  • Due to the relative novelty of the alternative proposals, as compared to the Coroutines TS which has multiple implementations and deployment experience, meant there was much less certainty about their eventual success, as there may be issues with them yet to be discovered. (As an example, the implementation challenge mentioned above only really came to be understood at this meeting.)
  • At least some of the advantages the alternative proposals would bring to the Coroutines TS could be accomplished via incremental, non-breaking changes post-C++20 (though this would also come with costs, such as greater complexity).

Importantly, all of the authors were more or less in agreement on these points; their differences remained only in the conclusions they drew from them.

This allowed the Committee to make what I believe was a well-informed final decision, which was that merging the Coroutines TS into C++20 gained consensus both in EWG and subsequently in plenary. Notably, it wasn’t “just barely consensus,” either — the ratio of the final vote in plenary was on the order of 10 in favour to 1 against.

Networking TS

The Networking TS did not make C++20, in part due to concerns about its design based on usage experience, and in part because it depends on Executors which also didn’t make it (not even a subset, as was hoped at the last meeting).

Musings on the TS Process

Disclaimer: This section reflects my personal opinions on potentially controversial topics. Caveat lector / feel free to skip / etc.

Recall that using Technical Specifications as a vehicle to allow large proposals to mature before final standardization is a procedural experiment that the Committee embarked on after C++11, and which is still ongoing. I’ve mentioned that opinions on how successful this experiment has been, vary widely within the Committee.

I’ve previously characterized Concepts and Modules as examples of success stories for the TS process, as both features improved significantly between their TS and IS forms.

However, one realization has been on my mind of late: we don’t seem to have a great track record for motivating compiler vendors to implement language features in their TS form. Let’s survey a few examples:

  • Concepts was only implemented in its TS form by GCC. As far as I’m aware, Clang implementation efforts have specifically targeted their C++20 form only.
  • To my knowledge, the Modules TS does not have a complete implementation either; it was partially implemented in MSVC and Clang, but both efforts have since moved on to target the newer, C++20-track formulation.
  • As mentioned above, the Reflection TS does not have a complete implementation, nor is one being actively worked on. Implementation efforts again seem to be focused on the newer, constexpr-based reflection facilities that are targeting C++23.

(If I’m mistaken on any of these points, I apologize in advance; please do point it out in a comment, and I will amend the above list accordingly.)

The Coroutines TS, which has multiple shipping implementations, is a notable exception to the above pattern. Library TS’es such as Networking, Filesystem, Library Fundamentals, and Parallelism also have a good track record of implementation. The fact remains, though, that the majority of our core language TS’es have not managed to inspire complete implementations.

This somewhat calls into question the value of language TS’s as vehicles for gathering use experience: you can’t collect use experience if users don’t have an implementation to use. (By contrast, implementation experience can be gathered from partial implementation efforts, and certainly has been for both Concepts and Modules.)

It also calls into question claims along the lines of “choosing to standardize [Feature X] as a TS first doesn’t mean you [users] have to wait longer to get it; you can just use the TS!” — a claim that I admit to have made myself, multiple times, on this blog.

What are the takeaways from this? Are language TS’es still a good idea? I’m still trying to work that out myself, but I will suggest a couple of takeaways for now:

  • Implementations can move faster than standards. Language TS’es are often snapshots of a rapidly evolving design. By the time a TS is published, its design is often known to have important flaws, and often it’s already been iterated on. Compilers don’t have much of a motivation to polish an implementation of a known-to-be-broken thing, nor users a motivation to use it.
  • Large features take a long time to get right. To take Modules as an example: while the Modules TS didn’t end up being something people can really use in practice, it seems to me that pushing Modules into the C++17 IS would been a mistake as well; given the extent of the feature’s evolution between then and now, locking the design as it stood in ~2016 (the C++17 feature-complete date) into the IS would have resulted in a significantly less baked feature. That suggests to me, that perhaps the choice we gave ourselves back then (“Modules: TS, or C++17?”) was a false choice. Perhaps a better choice would have been to continue iterating on Modules until it’s ready, even if that meant not publishing any spec-like document about Modules in the 2017 timeframe. (Update: see below for a counter-argument.)

Perhaps the actionable suggestion here is to downplay the role of a TS as a way to get a core language feature in front of users early. They do play other roles as well, such as providing a stabilized draft of a feature’s specification to write proposed changes against, and arguably they remain quite useful in that role.

Update: since publishing this, I’ve received private feedback that included suggestions of other advantages of core language TS’es, which I’ve found compelling, and wanted to share:

  • They allow Core wording review of a feature to proceed even while there are outstanding design questions (which can be deferred to post-TS consideration), which can in turn result in important issues being discovered and resolved sooner.
  • They prod implementers by putting them on notice that the feature may be standardized in this form in the absence of feedback. While this may not lead to complete implementations of the TS, it often does lead to partial implementation efforts that generate very valuable feedback.

Continuing with the example of Modules, both of the above considerations were in play, and contributed to the high quality of the feature now headed for C++20.

Evolution Working Group

I spent most of the week in EWG, as usual, although I did elope to some Study Group meetings, and to EWGI for a day.

Here I will list the papers that EWG reviewed, categorized by topic, and also indicate whether each proposal was approved, had further work on it encouraged, or rejected. Approved proposals are targeting C++20 unless otherwise mentioned; “further work” proposals are not, as this meeting was the deadline for EWG approval for C++20.

Contracts

Contracts — which have been added into the C++20 working draft at the last meeting — have been the subject of very extensive mailing list discussions, and what I understand to be fairly heated in-person debates in EWG. I wasn’t in the room for them (I was in EWGI that day), but my understanding is that issues that have come up related to (1) undefined behaviour caused by compilers assuming the truth of contact predicates, and (2) incremental rollout of contracts in a codebase where they may not initially be obeyed at the time of introduction, have led to a plurality of stakeholders to believe that the Contracts feature as currently specified is broken.

To remedy this, three different solutions were proposed. The first two — “Avoiding undefined behaviour in contracts” and “Contracts that work” — attempted to fix the feature in the C++20 timeframe, with different approaches.

The third proposal was to just remove Contracts from C++20.

However, none of these proposals gained EWG consensus, so for now the status quo — a feature believed to be a broken in the working draft — remains.

I expect that updated proposals to resolve this impasse will forthcome at the next meeting, though I cannot predict their direction.

EWG did manage to agree on one thing: to rename the context-sensitive keywords that introduce pre- and post-conditions from expects and ensures (respectively) to pre and post. Another proposed tweak, to allow contract predicates on non-first decarations, failed to gain consensus.

Modules

EWG reviewed a handful of Modules-related proposals:

  • (Approved) Constrained internal linkage for modules. This is a design fix that supersedes two other proposals related to linkage, “Modules: ADL & internal linkage” and “Module partitions are not a panacea”, by making code that would run into the underlying issues ill-formed. Notably, the proposal requires a diagnostic when the rules it introduces are violated; I’m very glad of this, as I find the “ill-formed, no diagnostic required” semantics that pervade the language (particularly those where there is actual implementation divergence on whether or not an error diagnostic is issued) a significant pitfall.
  • (Sent to SG15) Implicit module partition lookup. This aims to address part of the tooling-related concerns around Modules by introducing a standard mechanism by which module names are resolved to file names. EWG felt that it would be more appropriate for this proposal to target the C++ Ecosystem TR rather than the IS, and accordingly forwarded the paper to SG 15 (Tooling).
  • (Further work) Module resource dependency propagation. This is a proposal to allow annotating modular source files with the filenames of resource files (think e.g. an image that the code in the source file needs to use, that is shipped with an application) that they depend on, with the annotation appearing on the module declaration; the idea is that these annotations could inform a build system which could extract them and make them part of its dependency graph. EWG was sympathetic to the objectives but recognized that a proposal like this has specification challenges, as the current standard says very little about aspects of the host environment in which translation (building the program) takes place.

A couple of informational papers were also looked at:

  • Are Modules fast? attempts to characterize the performance impact of Modules via a microbenchmark. The gist of the results is that Modules tend to increase throughput while potentially also increasing latency, depending on the shape of your program’s dependency graph, an observation which is also corroborated by real-world deployment experience.
  • Make me a module describes an experimental implementation of build system support for Modules in GNU make.

Coroutines

Coroutines was probably the most talked-about subject at this meeting. I summarized the procedural developments that led to the Coroutines TS ultimately being merged into C++20, above.

Preceding that consequential plenary vote was an entire day of design discussion in EWG. The papers that informed the high-level directional discussion included:

  • The alternative proposals: Core Coroutines and Symmetric Coroutines.
  • Two papers outlining how something that accomplishes many of the goals of the alternative proposals can be built on top of the Coroutines TS in a backwards compatible fashion (the first of these is the “unified” proposal by Facebook that I mentioned in my last report).
  • Two analysis papers comparing the various approaches in detail, one focused on use cases and the second on language and implementation impact.
  • An experience report about implementing a coroutine TS frontend to an existing tasking library.

I’d say the paper that had the biggest impact on the outcome was the analysis paper about the language and implementation impact. This is what discussed, in detail, what I described above as an “implementation challenge” shared by Core Coroutines and Symmetric Coroutines. The issue here is that both of these proposals aim to expose the coroutine frame — the data structure that stores the coroutine state, including local variables that persist across suspension points — as a first-class object in C++. The reason this is challenging is that first-class C++ objects have certain properties, such as their sizeof being known at constant expression evaluation time, which happens in the compiler front-end; however, the size of a coroutine frame is not known with any reasonable amount of accuracy until after optimizations and other tasks more typically done by the middle- or back-end stages of a compiler. Implementer consensus was that introducing this kind of depedency of the front-end on optimization passes is prohibitive in terms of implementation cost. The paper explores alternatives that involve language changes, such as introducing the notion of “late-sized types” whose size is not available during constant expression evaluation; some of these were deemed to be implementable, but the required language changes would have been extensive and still required a multi-year implementation effort. (The problem space here also has considerable overlap with variable-length arrays, which the committee has not been able to agree on to date.)

This, I believe was the key conclusion that convinced EWG members that if we go with the alternatives, we’re not likely to have Coroutines until the C++26 timeframe, and in light of that choice, to choose having the Coroutines TS now.

EWG also looked at a couple of specific proposed changes to the Coroutines TS, both of which were rejected:

  • (Rejected) The trouble with coroutine_traits. This would have enhanced the ability of a programmer to customize the behaviour of a third-party coroutine type. I think the main reason for rejection was that the proposal involved new syntax, but the specific syntax had not been decided on, and there wasn’t time to hash it out in the C++20 timeframe. The proposal may come back as an enhancement in C++23.
  • (Rejected) Coroutines TS simplifications. There weren’t strong objections to this, but ultimately proceeding with the TS unmodified had the greater consensus as it has implementation experience.

constexpr

The march to make ever more things possible in constexpr continued this week:

  • (Approved) Permitting trivial default initialization in constexpr contexts. “Trivial default initialization” refers to things like int x; at local scope, which leaves x uninitialized. This is currently ill-formed in a constexpr context; this proposal relaxes it so that it’s only ill-formed if you actually try to read the uninitialized value. The interesting use cases here involve arrays, such as the one used to implement a small-vector optimization.
  • (Approved) Adding the constinit keyword. This is a new keyword that can be used on a variable declaration to indicate that the initial value must be computed at compile time, without making the variable const (so that the value can be modified at runtime).
  • (Further work) constexpr structured bindings. This is targeting C++23; EWG didn’t request any specific design changes, but did request implementation experience.
  • (Rejected) An update on “More constexpr containers”. This proposal was previously approved by EWG, and had two parts: first, allowing dynamic allocation during constant evaluation; and second, allowing the results of the dynamic allocation to survive to runtime, at which time they are considered static storage. Recent work on this proposal unearthed an issue with the second part, related to what is mutable and what is constant during the constant evaluation. The authors proposed a solution, but EWG found the solution problematic for various reasons. After lengthy discussion, people agreed that a better solution is desired, but we don’t have time to find one for C++20, and the “promotion to static storage” ability can’t go forward without a solution, so this part of the proposal was yanked and will be looked at again for C++23. (The first part, dynamic allocations without promotion to static storage, remains on track for C++20.)

Comparisons

Pattern matching

  • (Further work) Pattern matching. This is one of the most exciting proposals to look forward to in C++23; it will bring a pattern matching facility comparable to that in Rust and other modern languages, to C++. EWG spent most of an afternoon on it and gave the authors a lot of guidance, including on syntax choices, parseability, readability, and the proposed customization point design.
  • (Rejected) Disallow _ usage in C++20 for pattern matching in C++23. By the same authors as the pattern matching proposal, this paper tried to land-grab the _ identifier in C++20 for future use as a wildcard pattern in C++23 pattern matching. EWG wasn’t on board, due to concerns over existings uses of _ in various libraries, and the availability of other potential symbols. This does mean that the wildcard pattern in C++23 pattern matching will (very likely) have to be spelt some other way than _.

Other new features

  • (Approved) Expansion statements. This is a form of compile-time for loop that can iterate over tuple-like objects, constexpr ranges, and parameter packs. The feature has been revised to address previous EWG feedback and use a single syntax, for...
  • (Approved) using enum. This allows bringing all the enumerators of an enumeration, or just a specific enumerator, into scope such that they can be referenced without typing the enumeration name or enclosing type name. Approved with the modification that it acts like a series of using-declarations.

Bug / Consistency Fixes

(Disclaimer: don’t read too much into the categorization here. One person’s bug fix is another’s feature.)

  • (Approved) char8_t backwards compatibility remediation. This contains a couple of minor, library-based mitigations for the backwards compatibility breakage caused by u8 literals changing from type char to char8_t. Additional library and language-based mitigations were mentioned but not proposed.
  • (Approved) Reference capture of structured bindings. Value capture was already approved at the previous meeting.
  • (Approved) Implicit creation of objects for low-level object manipulation. This is largely standard wording changes to make certain widely-used code patterns, such as using malloc() to allocate a POD object, defined. It also introduces a new “barrier operation” std::bless, a kind of counterpart to std::launder, which facilities writing custom operations that implicitly create objects like malloc(). One point that came up during discussion is that this proposal makes things like implementing a small vector optimization in constexpr possible (recall that things which trigger undefined behaviour at runtime are ill-formed during constant evaluation).
  • (Approved) Deprecating volatile. Despite the provocative title, which is par for the course from our esteemed JF Bastien, this only deprecates uses of volatile which have little to no practical use, such as volatile-qualified member functions.
  • (Approved) Layout-compatibility and pointer-interconvertibility traits. This allows checking at compile time whether certain operations like converting between two unrelated pointer types would be safe.
  • (Approved) [[nodiscard("should have a reason")]]. This extends the ability to annotate an attribute with a reason string, which [[deprecated]] already has, to [[nodiscard]].
  • (Approved) More implicit moves. This extends the compiler’s ability to implicitly move rather than copy an object in some situations where it knows the original is about to go out of scope anyways. (The suggested future extension regarding assignment operators was not encouraged.)
  • (Further work) Ultimate copy elision. This is an ambitious proposal to give compilers license to elide copies in cases not covered by the as-if rule (i.e. cases where the compiler can’t prove the elision isn’t observable; this is typically the case when the copy constructor being invoked isn’t entirely inline). The benefits are clear, but there are concerns that the proposed changes are not sound; more analysis and implementation experience is needed.

Proposals Not Discussed

Notable among proposals that didn’t come up this week is Herb’s static exceptions proposal. As this is a C++23-track proposal, it was deliberately kept out of EWG so far to avoid distracting from C++20, but it is expected to come up at the next meeting in Cologne.

Evolution Working Group Incubator

The EWG Incubator group (EWGI), meeting for the second time since its inception at the last meeting, continued to do a preliminary round of review on EWG-bound proposals.

I wasn’t there for most of the week, but here are the papers the group forwarded to EWG:

Numerous other proposals were asked to return to EWGI with revisions. I’ll call out a few particularly interesting ones:

  • Overload sets as function parameters. Being able to pass around overload sets has been proposed and shot down numerous times before. The novely in this approach is that it’s opt-in at the callee side, not the caller side.
  • Parametric expressions. This is an ambitious proposal that aims to bring a sort of a hygienic macro system to C++.
  • Object relocations in terms of move plus destroy. This aims to solve some common performance issues in the implementation of container types, where for some types of objects, relocating them to newly allocated storage can safely be done via a memcpy rather than invoking a move constructor and destructor, but the infrastructure for identifying such types is not present.
  • Language variants. This would add a core-language sum type, similar to Rust’s enums, to C++, as an alternative to the library-based std::variant.

Other Working Groups

Library Groups

Having sat in the Evolution groups, I haven’t been able to follow the Library groups in any amount of detail, but I’ll call out some of the more notable library proposals that have gained design approval at this meeting:

Study Groups

SG 1 (Concurrency)

C++20-track work reviewed this week included revisions to joining thread, and deprecating volatile.

v1 of the Concurrency TS will be withdrawn; v2 continues to be under active development, with asymmetric fences approved for it this week.

Executors continue to be a hot topic. SG 1 forwaded two papers related to them onward to the Library Evolution Working Group, while three others remain under review. An earlier plan to ship a subset of executors in C++20 had to be scrapped, because LEWG requested the the “property” mechanism it relies on be generalized, but it was too late in the cycle to progress that for C++20. As a result, Executors are now targeting C++23.

Other proposals under active review in SG 1 concern fibers, concurrent associative data structures, memory model issues, volatile_load and volatile_store, customization points for atomic_ref, and thread-local storage.

SG 7 (Compile-Time Programming)

The main topic in SG 7 continues to be deciding on the high-level direction for constexpr-based reflection in the (hopefully) C++23 timeframe. The two proposals on the table are scalable reflection in C++ and constexpr reflexpr; their main point of divergence is whether they use a single type (meta::info) to represent all compile-time reflection metadata objects (also known as reflections), or whether there should be a family / hierarchy (not necessarily inheritance-based) of such types (meta::variable, meta::function, etc.).

SG 7 expressed a preference for the “family of types” approach at the last meeting, however the point continues to be debated as the proposal authors gather more experience. The “single type” approach has been motivated by implementation experience in the EDG and Clang compilers, which has suggested this can achieve better compile-time performance. The “family of types” approach is motivated more by API design considerations, as expressed in the position paper constexpr C++ is not constexpr C.

While a consensus on this point is yet to emerge, a possible (and potentially promising) direction might be to build the “family of types” approach as a layer on top of the “single type” approach, which would be the one implemented using compiler primitives.

SG 7 also reviewed a proposal for a modern version of offsetof, which was forwaded to LEWG.

SG 15 (Tooling)

The Tooling Study Group (SG 15) met for an evening session, primarily to discuss tooling-related concerns around Modules.

As mentioned above, now that Modules has achieved consensus among compiler implemeters, tooling concerns (nicely summarized in this paper) are the remaining significant point of contention.

The concerns fall into two main areas: (1) how build systems interact with Modules, and (2) how non-compiler tools that consume source code (such as static analyzers) can continue to do so in a Modular world. The heart of the issue is that components of the C++ ecosystem that previously needed to rely only on behaviour specified in the C++ standard, and some well-established conventions (e.g. that compilers find included headers using a search path that can be disclosed to tools as well), now in a Modular world need to rely on behaviours that are out of scope of the C++ standard and for which established conventions are yet to emerge (such as how module names are mapped to module interface files, or how translation of imported modules is invoked / performed).

To address these concerns, SG 15 has announced what I view as probably the most exciting development since the group’s inception: that it will aim to produce and publish a C++ Ecosystem Technical Report containing guidance regarding the above-mentioned areas.

A Technical Report (TR) is a type of published ISO document which is not a specification per se, but contains guidance or discussion pertaining to topics covered by other specifications. The committee has previously published a TR in 2006, on C++ performance.

A TR seems like an appropriate vehicle for addressing the tooling-related concerns around Modules. While the committee can’t mandate e.g. how module names should map to file names, by providing guidance about it in the C++ Ecosystem TR, hopefully we can foster the emergence of widely followed conventions and best practices, which can in turn help maintain a high level of interoperability for tools.

The announcement of plans for a C++ Ecosystem TR did not completely assuage tool authors’ concerns; some felt that, while it was a good direction, Modules should be delayed and standardized in tandem with the TR’s publication in the C++23 timeframe. However, this was a minority view, and as mentioned Modules went on to successfully merge into the C++20 working draft at the end of the meeting.

Other Study Groups

Other Study Groups that met at this meeting include:

  • SG 6 (Numerics), which met for about two days and reviewed a dozen or so proposals. Topics discussed included utility functions, floating point types, number representations, and linear algebra (the latter being a hot topic for the committee these days, with a lot of interest from the new SG 19 (Machine Learning) as well).
  • SG 12 (Undefined and Unspecified Behaviour), which met to discuss an informational paper on pointer provenance and a paper about signed integer overflow; the latter was referred to SG 20 as a matter of educating C++ programmers. There was also the now-usual joint session with WG23 – Software Vulnerabilities, where additional sections of the C++ vulnerabilities document were reviewed; there will also be upcoming work on MISRA.
  • SG 13 (Human/Machine Interface), which met for half a day to review a proposal for a standard audio API, which generated a lot of interest. There were no developments related to 2D graphics at this meeting.
  • SG 16 (Unicode). Papers reviewed include compile-time regular expressions, source-code information capture, and charset transcoding, transformation, and transliteration.
  • SG 19 (Machine Learning) had its first meeting this week. This paper provides a good overview of how SG 19 envisions structuring its work. The initial work is understandably focused on fundamentals such as linear algebra primitives. Like many other study groups, SG 19 will hold monthly teleconferences to make progress in between in-person meetings.
  • SG 20 (Education) also had its first in-person meeting. They plan to produce a “standing paper” of educational guidelines (see this proposed early draft). They will also hold monthly telecons.

Next Meetings

The next meeting of the Committee will be in Cologne, Germany, the week of July 15th, 2019.

Conclusion

This was an eventful and productive meeting, and it seems like the progress made at this meeting has been well-received by the user community as well! With Modules and Coroutines joining the ranks of Concepts, Ranges, contracts, default comparisons and much else in the C++20 working draft, C++20 is promising to the most significant language update since C++11.

Due to sheer number of proposals, there is a lot I didn’t cover in this post; if you’re curious about a specific proposal that I didn’t mention, please feel free to ask about it in the comments.

Featured Song: Alone You Breathe

Well, it’s been a while since I wrote one of these 🙂 Not for lack of musical inspiration, mind you — I’ve discovered a number of great bands over the past year that I’ve been itching to share — just been lazy to actually write the posts, I suppose. Going to try and do it more often!

Today’s selection is by Savatage, an American metal band whose members would go on to found the Trans-Siberian Orchestra (whom I’ve featured before).

“Alone You Breathe”, from the 1994 album Handful of Rain is a heartbreakingly sad and beautiful song that Savatage co-founder Jon Oliva wrote about his brother and fellow band member Criss, who had passed away in an accident the previous year.

In a tribute to Criss, the song reuses a stanza of lyrics (quoted below) from “Believe”, another really beautiful Savatage song from their 1991 album Streets: A Rock Opera which Criss wrote.



Featured lyrics:

I am the way, I am the light
I am the dark inside the night
I hear your hopes, I feel your dreams
And in the dark I hear your screams

Full lyrics and some discussion can be found here.

Enjoy!

Trip Report: C++ Standards Meeting in San Diego, November 2018

Summary / TL;DR

<!–

–>

Project What’s in it? Status
C++17 See list Published!
C++20 See below On track
Library Fundamentals TS v3 See below Under active development
Concepts TS Constrained templates Merged into C++20, including (now) abbreviated function templates!
Parallelism TS v2 Task blocks, library vector types and algorithms, and more Published!
Executors Abstraction for where/how code runs in a concurrent context Subset headed for C++20, rest in C++23
Concurrency TS v2 See below Under development. Depends on Executors.
Networking TS Sockets library based on Boost.ASIO Published! Not headed for C++20.
Ranges TS Range-based algorithms and views Merged into C++20!
Coroutines TS Resumable functions, based on Microsoft’s await design Published! C++20 merge uncertain
Modules v1 A component system to supersede the textual header file inclusion model Published as a TS
Modules v2 Improvements to Modules v1, including a better transition path On track to be merged into C++20
Numerics TS Various numerical facilities Under active development
Graphics TS 2D drawing API Future uncertain
Reflection TS Static code reflection mechanisms PDTS ballot underway; publication expected in early 2019

A few links in this blog post may not resolve until the committee’s post-meeting mailing is published (expected any day now). If you encounter such a link, please check back in a few days.

Introduction

A few weeks ago I attended a meeting of the ISO C++ Standards Committee (also known as WG21) in San Diego, California. This was the third committee meeting in 2018; you can find my reports on preceding meetings here (June 2018, Rapperswil) and here (March 2018, Jacksonville), and earlier ones linked from those. These reports, particularly the Rapperswil one, provide useful context for this post.

This meeting broke records (by a significant margin) for both attendance (~180 people) and number of proposals submitted (~270). I think several factors contributed to this. First, the meeting was in California, for the first time in the five years that I’ve been attending meetings, thus making it easier to attend for Bay Area techies who weren’t up for farther travels. Second, we are at the phase of the C++20 cycle where the door is closing for new proposals targeting to C++20, so for people wanting to get features into C++20, it was now or never. Finally, there has been a general trend of growing interest in participation in C++ standardization, and thus attendance has been rising even independently of other factors.

This meeting was heavily focused on C++20. As discussed in the committee’s standardization schedule document, this was the last meeting to hear new proposals targeting C++20, and the last meeting for language features with significant library impact to gain design approval. A secondary focus was on in-flight Technical Specifications, such as Library Fundamentals v3.

To accommodate the unprecedented volume of new proposals, there has also been a procedural change at this meeting. Two new subgroups were formed: Evolution Incubator (“EWGI”) and Library Evolution Incubator (“LEWGI”), which would look at new proposals for language and library changes (respectively) before forwarding them to the Evolution or Library Evolution Working Groups (EWG and LEWG). The main purpose of the incubators is to reduce the workload on the main Evolution groups by pre-filtering proposals that need additional work before being productively reviewed by those groups. A secondary benefit was to allow the attendees to be spread out across more groups, as otherwise EWG and LEWG would have likely exceeded their room capacities.

C++20

Here are the new changes voted into C++20 Working Draft at this meeting. For a list of changes voted in at previous meetings, see my Rapperswil report.

Technical Specifications

In addition to the C++ International Standard (IS), the committee publishes Technical Specifications (TS) which can be thought of experimental “feature branches”, where provisional specifications for new language or library features are published and the C++ community is invited to try them out and provide feedback before final standardization.

At this meeting, the committee iterated on a number of TSes under development.

Reflection TS

The Reflection TS was sent out for its PDTS ballot at the last meeting. As described in previous reports, this is a process where a draft specification is circulated to national standards bodies, who have an opportunity to provide feedback on it. The committee can then make revisions based on the feedback, prior to final publication.

The PDTS ballot is still ongoing, so there wasn’t much to do on this front at this meeting. We expect the ballot results to be ready by the next meeting (February 2019, in Kona), at which time we’ll address the ballot comments and, time permitting, approve the revised TS for publication.

One minor snafu discovered at this meeting is that prior to the PDTS ballot, the Reflection TS, which depends on Concepts, has been rebased onto C++20, to take advantage of C++20 Concepts (previously, it was based on the Concepts TS). Unfortunately, ISO rules don’t allow publishing a TS before its base document is published, which means that to publish the Reflection TS as-is, we’d have to wait to do it concurrently with the C++20 publication in late 2020. We very much don’t want to wait that long, since the purpose of the Reflection TS is to gather feedback from users in preparation for revised Reflection features in C++23, and the earlier we start getting that feedback, the better. So, we’ll have to un-rebase the Reflection TS onto {C++17 + Concepts TS} to be able to publish it in early 2019 as planned. Isn’t red tape fun?

Library Fundamentals TS v3

This third iteration (v3) of the Library Fundamentals TS is open for new features to be added. (The TS working draft currently contains features from v2 which haven’t been merged into the C++ IS yet.) The only changes voted in at this meeting were a rebase and some issue resolutions, but a number of new features are on the way.

Executors

As discussed below, the revised plans for Executors are for a subset of them to target C++20, and the rest C++23. An Executors TS is not planned at this time.

Merging Technical Specifications into C++20

Turning now to Technical Specifications that have already been published, but not yet merged into the IS, the C++ community is eager to see some of these merge into C++20, thereby officially standardizing the features they contain.

Ranges TS

The Ranges TS modernizes and Conceptifies significant parts of the standard library (the parts related to algorithms and iterators), as well as introducing exciting new features such as range views.

After years of hard work developing these features and going through the TS process, the Ranges TS was finally merged into C++20, paving the way for wider adoption of these features.

Concepts TS

The approval of abbreviated function templates for C++20 at this meeting can be thought of as completing the merge of the Concepts TS into C++20: all the major features in the TS have now been merged, with some design modifications inspired by implementer and user feedback.

While the journey took longer than was initially hoped, in my opinion Concepts is a better feature for the design changes made relative to the Concepts TS, and as such this is an example of the TS process working as intended.

Modules TS

Modules remains one of the most highly anticipated features by the C++ user community. This meeting saw really good progress on Modules: a “merged” Modules design, combining aspects of the Modules TS and the alternative Atom proposal, gained design approval for C++20.

This outcome exceeded expectations in that previously, the merged proposal seemed more likely to target a Modules TS v2 or C++23, with a subset possibly targeting C++20; however, thanks in significant part to the special one-off Modules-focused Bellevue meeting in September, good enough progress was made on the merged design that the authors were comfortable proposing putting the entire thing into C++20, which EWG subsequently approved.

As this is a large proposal, wording review by the Core Working Group will take some time, and as such, a plenary vote to merge the reviewed wording into the C++20 working draft won’t take place until the next meeting or the one after; however, as all the major compiler implementers seem to be on board with this design, and there is overwhelming demand for the feature from the user community, I expect smooth sailing for that vote.

In fewer words: Modules is on track for C++20!

Coroutines TS

The Coroutines TS was once again proposed for merger into C++20 at this meeting. This is the third time this proposal was made (the other two times being at the previous two meetings). At the last meeting, the proposal got as far as a plenary vote at the end of the week, which narrowly failed.

The opposition to merging the TS into C++20 comes from the fact that a number of people have concerns about the Coroutines TS design (some of them are summarized in this paper), and an alternative proposal that addresses these concerns (called “Core Coroutines”) is under active development. Unfortunately, Core Coroutines is not sufficiently-baked to make C++20, so going with it would mean delaying Coroutines until C++23. Opinions differ on whether this is a worthwhile tradeoff: the Core Coroutines authors are of the view that C++ will remain a relevant language for 50 years or more, and waiting 3 years to improve a feature’s design is worthwhile; others have made it clear that they want Coroutines yesterday.

After the failure of last meeting’s merger proposal, it was hoped that waiting one more meeting would allow for the Core Coroutines proposal to mature a bit. While we knew it wouldn’t be ready for C++20, we figured the added maturity would allow us to better understand what we would be giving up by merging the Coroutines TS into C++20, and possibly identify changes we could make the Coroutines TS before C++20’s publication that would make incremental improvements inspired by Core Coroutines backwards-compatible, thereby allowing us to make a more informed decision on the C++20 merger.

Core Coroutines did make significant progress since the last meeting: the updated proposal is simpler, more fleshed out, and has a cleaner syntax. The impasse has also inspired efforts, led by Facebook, to combine the two proposals in such a way that would unblock the merger into C++20, and allow for backwards-comaptible improvements achieving many of the goals of Core Coroutines in C++23, but these efforts are at a relatively early stage (a paper describing the combined design in detail was circulated for the first time while the meeting was underway).

Ultimately, waiting a meeting doesn’t seem to have changed many people’s minds, and we saw a replay of what happened in Rapperswil: EWG narrowly passed the merger, and plenary narrowly rejected it; interestingly, the level of consensus in plenary appears to have decreased slightly since Rapperswil.

To keep C++20 on schedule, the final deadline for approving a TS merger is the next meeting, at Kona. The merger will undoubtedly be re-proposed then, and there remains some optimism that further development of Facebook’s combined proposal might allow us to gain the required confidence in a future evolution path to approve the merger for C++20; otherwise, we’re looking at getting Coroutines in C++23.

Networking TS

It’s looking like the Networking TS will not be merged into C++20, in large part due to the concerns presented this paper discussing usage experience. The TS will instead target C++23.

Evolution Working Group

With the increased number of subgroups meeting in parallel, it’s becoming more challenging to follow what goes on in the committee.

I usually sit in EWG for the duration of the meeting, and summarize the design discussions that take place in that group. I will try to do so again, but I did miss some EWG time while sitting in some study group meetings and Evolution Incubator meetings, so expect some reduction in the amount of detail. If you have specific questions that I didn’t cover, feel free to ask in the comments.

This time, I’ll categorize proposals by topic. For your convenience, I still indicate whether each proposal was approved, had further work on it encouraged, or rejected. Proposals are targeting C++20 unless otherwise mentioned.

Concepts

The headline item here is the approval of the compromise design for abbreviated function templates (AFTs). With this syntax, AFTs look like this:

void f(Concept auto x);

This makes both the “I want to write a function template without the template<...> notation” and the “I want to be able to tell syntactically if a function is a template” camps happy (the latter because the auto tells you the parameter has a deduced type, and therefore the function is a template).

You can also use Concept auto as a return type, and as the type of a variable. In each case, the type is deduced, and the deduced type has to satisy the concept. The paper as written would have allowed the return type and variable cases to omit the auto, but this didn’t have consensus and was removed.

Note that you can write just void f(auto x); as well, making functions consistent with lambdas which could already do this.

Finally, as part of this change, a restriction was imposed on the template <Concept T> notation, that T has to be a type. For non-type and template template parameters, constraints can only be specified using a requires-clause. The motivation here is to be able to tell syntactically what type of entity T is.

A few other Concepts-related proposals were looked at:

  • (Further work) How to make terse notation soar with class template argument deduction. The idea here is to combine class template argument deduction (CTAD) and Concepts such that a class template name (e.g. tuple) can be used as a parameter type as if it were a concept (with the concept being, roughly, “this type is a specialization of tuple“). The proposal was generally well-received, but there are some technical details to iron out, and design alternatives to consider (e.g. spelling it tuple<auto...>), so this will be revisited for C++23.
  • (Rejected) A simple proposal for unifying generic and object-oriented programming. This is a more ambitious proposal to try to allow writing code that works with a set of polymorphic types, that looks the same regardless of whether the polymorphism is dynamic (inheritance) or static (concepts). Reception was mixed; some felt this would introduce a new programming model with relatively little benefit.
  • (Rejected) Concept-defined placeholder types. This would have allowed defining a “placeholder type” constained by a concept, and using that type in place of the concept. It didn’t really fit with the AFT design that was approved.
  • (Rejected) Multi-argument constrained parameter. This proposed a whitespace-based syntax for introducing multiple constrained parameters in a template parameter list, e.g. template <EqualityComparableWith T U>. EWG didn’t feel the whitespace syntax was an improvement over other syntaxes that have been rejected, like template <EqualityComparableWith{T, U}>.

EWG ran out of time to review the updated “constraining Concepts overload sets” proposal. However, there was some informal speculation that the chances of this proposal making C++20 have diminished, because the proposal has grown a lot more complex in an attempt to address EWG’s feedback on the previous version, which suggests that feedback had touched on some hard problems that we may not be in a good position to solve at this time.

Modules

As mentioned, perhaps the biggest high-point of this meeting was EWG’s approval of the merged Modules design for C++20. “Merged” here refers to the proposal combining aspects of the Modules TS design, and the alternative Atom proposal. Perhaps most significantly, the design borrows the Atom proposal’s legacy header imports feature, which is intended to better facilitate incremental transition of existing large codebases to Modules.

Several minor modifications to this design and related changes were also proposed:

  • (Approved) Making module a context-sensitive keyword, take two. Following consistent feedback from many segments of the user community that making module a hard keyword would break too much code, a new proposal for making it context-sensitive, this time with simpler disambiguation rules, was approved.
  • (Approved) Single-file modules with the Atom semantic properties rule. This allows module authors to do certain things that previously required separate module partitions in separate files, in one file.
  • (Approved) Module preamble is unnecessarily fragile. This tweaks the rules for where a module file’s “preamble” (the area containing the module declaration and imports) ends, with a view to making the user model simpler.
  • (Approved) Redefinitions in legacy imports. This clarifies some of the rules in scenarios involving legacy header imports.
  • (Further work) Modules and freestanding. This mostly has to do with how to split the standard library into modules, with the relevance to EWG being that we should have a consistent approach for dealing with freestanding implementations in the language and in the library. EWG did not reach a consensus on this topic, mostly because there are a wide variety of freestanding environments with different constraints, and a single subset of the language does not fit all of them.
  • (Further work) Inline module partitions. This is a generalization of “Single-file modules with the Atom semantic properties rule”, which would allow defining an arbitirary number of module partitions “inline” in a single file. EWG encouraged further development of this idea, but for post-C++20.
  • (Rejected) Global module fragment is unnecessary. The global module fragment is one of two mechanisms for transitioning existing code to Modules (the other being legacy header imports). The author of this paper suggested that just legacy header imports may be sufficient, but this was emphatically argued against based on implementation experience at some companies, leading to the proposal’s rejection.
  • (Rejected) Retiring pernicious language constructs in module contexts. This paper suggested that Modules was an opportunity to shed some of the language’s legacy cruft by making certain constructs invalid inside a module (while they would remain valid in non-modular code for backwards compatibility). There wasn’t much enthusiasm for this idea, largely because it’s expected that people will want to be able to freely copy / migrate code from a non-modular context to a modular context and vice versa.

Contracts

  • (Approved) Access control in contract conditions. This was the subject of a very long and drawn-out debate on the committee mailing lists which I won’t attempt to summarize, but the outcome was that pre- and post-conditions on member functions can reference private and protected variables inside the class, even though we think of them as being part of the class’s public interface.
  • (Approved) Contract postconditions and return type deduction. This is a tweak regarding the interaction between postconditions and return type deduction, with the intention to avoid surprising behaviour. Option 3 from the paper had consensus.
  • (Further work) Allowing contract predicates on non-first declarations. EWG was open to this idea, but some implementation issues (such as who emits the code for the contract check) need to be ironed out.
  • (Further work) Undefined behaviour in contract violations. This was another topic that engendered very extensive mailing list discussion. No decision was made this week, but the likely direction is to specify that contracts (except perhaps axioms) do not allow compilers to assume additional things they couldn’t already assume.
  • (Rejected) Contracts updates. Of the three minor changes proposed in this paper, the first was a trivial wording change (which was approved); the second had no consensus; and the third was deemed unimplementable.

constexpr

Continuing with the committee’s concerted effort to make clunkier forms of compile-time programming (such as template metaprogramming) unnecessary, EWG approved further extensions to constexpr:

Coroutines

I mentioned above that EWG narrowly passed the latest version of a proposal to merge the Coroutines TS into C++20, only to have it rejected in a plenary vote.

The technical discussion of this topic centred around an updated version of the competing Core Coroutines proposal, and a paper by Facebook engineers arguing that most of the benefits of Core Coroutines could be achieved through extensions to the Coroutines TS, and we should therefore go ahead with the Coroutines TS in C++20.

An interesting development that emerged mid-meeting is the Facebook folks coming up with a “unified coroutines” proposal that aims to achieve consensus by combining aspects of the two competing proposals. There wasn’t really enough time for the committee to digest this proposal, but we are all hopeful it will help us make an informed final decision (final for C++20, that is) at the next meeting.

Structured Bindings

  • (Approved in part) Extend structured bindings to be more like variable declarations. Structured bindings can now be static, thread_local, or constexpr; in each case, this applies to the entire composite object being destructured. Rules around linkage were also clarified. Capture of bindings by a lambda was deferred for further work.
  • (Further work) Simplify the customization point for structured bindings. EWG wholehearted wants an overhaul of the customization point (the current one just piggybacks on the customization point for tuple-like that we already had in the language), but felt this proposal addressed just one piece of what is a larger puzzle. A more complete proposal may look something like the operator extract from an earlier pattern matching proposal.
  • (Rejected) Structured bindings with explicit types. This was rejected because the use cases will be addressed more comprehensively with pattern matching.

Class Template Argument Deduction (CTAD)

  • (Approved in part) Filling holes in class template argument deduction. CTAD now works with aggregates, alias templates, and inheriting constructors. Making CTAD work with partial template argument lists was rejected because it would be a breaking change in some cases (e.g. consider vector<any>(MyAlloc())).
  • (Rejected) Improving function templates with CTAD. EWG found that this would involve a lot of complexity, since with function templates you don’t just have one template definition as with class templates, but a whole overload set.

Comparisons

Most comparison-related proposals involved early adopters trying out the spaceship operator (<=>) and discovering problems with it.

  • (Approved) I did not order this! Why is it on my bill?, which probably deserves a medal of some sort for most creative paper title. (Explanation: the paper concerns scenarios where you don’t care about ordering your type, only equality-comparing it, you implement a defaulted operator<=> (because that’s “the C++20 way” for all comparison use cases), and you pay a performance penalty that wouldn’t be there with hand-written code to deal with equality comparison only.) A related paper offers a solution, which is along the lines of making == be its own thing and not fall back to using <=>, since that’s where the inefficiency stems from (for types like string, if the lengths are different you can answer “not equal” much faster than if you’d have to answer “less” or “greater than”). A second part of the proposal, where a defaulted <=> would also generate a defaulted ==, so that users can be largely oblivious to this problem and just default one operator (<=>), was more controversial, but was still approved over some objections.
  • (Approved) When do you actually use <=>? The crux of this paper is that we’ve had to invent a library function compare_3way() wrapping <=> and that’s what we want to use most of the time, so we should just give <=> the semantics of that function.
  • (Mooted) weak_equality considered harmful. This proposal has become moot as implementations of == are no longer generated in terms of <=>. (As a result, weak_equality and strong_equality are no longer used and will likely be removed in the future.)
  • (Rejected) Chaining comparisons. Despite previous encouragement, this was now rejected due to concerns about teachability and implementation issues.

Other New Features

  • (Further work) Deducing this. This proposal allows writing member functions where the type of this is deduced, thereby eliminating the need to duplicate implementations for things like const vs. non-const objects, and other sources of pain. There was a fair amount of technical discussion concerning recursive lambdas (which this proposal hopes to enable), name lookup rules, and other semantic details. The authors will return with a revised proposal.
  • (Rejected) Towards a lazy forwarding mechanism for C++. This would allow declaring function parameters to be “lazy”, such that their arguments are evaluated upon their use inside the function (and possibly not at all if there is no use), rather than at the call site; participants pointed out a similarity to Algol’s “call by name” feature. EWG wasn’t categorically opposed to the notion of lazy parameters, but the notion of having them without any call-site syntax (like this paper proposes) was controversial.

Bug / Consistency Fixes

(Disclaimer: don’t read too much into the categorization here. One person’s bug fix is another’s feature.)

  • (Approved) Allow initializing aggregates from a parenthesized list of values. This finally allows things like vector::emplace_back() to work for aggregates.
  • (Approved) Contra CWG DR1778. This has to do with noexcept and explicitly defaulted functions. The first option from the paper was approved.
  • (Approved) Permit conversions to arrays of unknown bound. The motivation cited for this is working in environments where dynamic allocation is not allowed and use of pointers is restricted, and thus passing around variable-length arrays as arrays of unknown bound are the only way to work with dynamically sized data ranges.
  • (Approved) Array size deduction in new-expressions. This is a minor consistency fix that was also approved as a Defect Report against older language versions.
  • (Approved) Nested inline namespaces. This allows using the C++17 nested namespace syntax in cases where one or more of the namespaces are inline. Example: namespace foo::inline bar::baz { } is short for namespace foo { inline namespace bar { namespace baz { }}}. inline is not allowed in the leading position as people might mistakenly think it applies to the innermost namespace.
  • (Further work) Conditionally trivial special member functions. This is a small but important fix for library implementers who would otherwise have to use labour-intensive techniques to meet the triviality requirements set out for standard library types. This was essentially approved, but specification difficulties necessitate one more round of review.
  • (Further work) Ultimate copy elision. This aims to expand the set of scenarios in which the compiler is allowed to elide copies and moves (note: unlike the C++17 “guaranteed copy elision” feature, this is not requiring compilers to elide copies in these new scenarios, just allowing them). EWG liked the general idea but had concerns about the potential for code breakage in some scenarios.
  • (Further work) Adding the [[constinit]] attribute. The motivation here is cases where you want to guarantee that a variable’s initial value is computed at compile time (so no dynamic initialization required), without making the variable const (so that you can assign new values to it at runtime). EWG liked the idea but preferred using a keyword rather than an attribute. An alternative to decorate the initializer rather than the variable had no consensus.
  • (Postponed) short float. This proposal continues to face challenges due to concerns about different implementations using different sizes for it, or even different representations within the same size (number of bits in mantissa vs. exponent). As a result, there was no consensus for moving forward with it for C++20. There remains strong interest in the topic, so I expect it will come back for C++23, possibly under a different name (such as float16_t instead of short float, to specify the size more concretely).
  • (Rejected) Deprecate the addressof operator. This proposes to deprecate the overloading of operator &. EWG didn’t feel that removal was realistic given that we don’t have a good handle on the breadth of usage in the wild, and didn’t want to entertain deprecation without an intention to remove as a follow-up.

Evolution Working Group Incubator

As mentioned above, due to the increased quantity of proposals, an “EWG Incubator” group (EWGI) was also spun up to do a preliminary round of review on some proposals that EWG couldn’t get to this week, in the hope of making them better-baked for their eventual EWG review at a future meeting.

I only attended EWGI for half a day, so I don’t have much to report about the discussions that went on, but I will list the papers the group forwarded to EWG:

There were also a couple of papers EWGI referred for EWG review not necessarily because they’re sufficiently baked, but because they would benefit from evaluation by a larger group:

Numerous other proposals were asked to return to EWGI with revisions. I’ll call out a couple:

  • There were two proposals for pattern matching. The feature had strong support, and the authors were asked to return with a combined proposal.
  • There was another attempt at named arguments (called “labelled parameters” in the proposal). The novelty in this approach was putting the names in the type system, but without actually modifying any semantic rules like overload resolution, by encoding the labels using existing mechanisms in the type system, and then layering a “sugar” syntax on top. EWGI’s feedback was that the attempted abstraction will leak, and we’ll have to end up making deeper modifications to the type system after all, to have a usable feature. Encouragement to return was weak but existent.

Papers not discussed

There were, of course, also papers that neither EWG nor EWGI had the time to look at during this meeting; among them was Herb’s static exceptions proposal, which is widely anticipated, but not targeting C++20.

I’ll also briefly mention the lifetimebound proposal which Mozillians have expressed a particular interest in due to the increased lifetime safety it would bring: the authors feel that Microsoft’s lifetime checker, whose model of operation is now described in a paper is doing an adequate job of satisfying this use case outside of the core language rules (via annotations + a separate static checker). Microsoft’s lifetime checker ships with MSVC, and has a work-in-progress implementation in Clang as well, which can be tried out in Compiler Explorer, and will hopefully be open-sourced soon. See also Roc’s blog post on this subject.

Other Working Groups

Library Groups

Having sat in the Evolution groups, I haven’t been able to follow the Library groups in any amount of detail, but I’ll call out some of the more notable library proposals that have gained design approval at this meeting:

And a few notable proposals which are still undergoing design review, and are being treated with priority:

There are numerous other proposals in both categories above, I’m just calling out a few that seem particularly noteworthy. Please see the committee’s website for a full list of proposals.

Study Groups

SG 1 (Concurrency)

Most of the C++20-track work (jthread, Executors subset, synchronization omnibus paper, memory model fixes) has progressed out of the Concurrency Study Group and is mentioned above.

For Executors, the current plan is to put a subset of the unified proposal (specifically including “one way” and “bulk one way” executors, but not the other kinds) into C++20, and the rest into C++23; a TS is not planned at this time.

Coroutines-related library additions are not being reviewed yet; they need more bake time, and integration with the next revision of Executors.

SG 1 has opinions on freestanding implementations: they feel omitting thread_local from a freestanding implementation is fine, but omitting non-lock-free atomics or thread-safe statics is more controversial.

SG 7 (Compile-Time Programming)

There were two meetings related to compile-time programming this week. The first was an evening session where the committee re-affirmed its preference for constexpr-based metaprogramming as the future of C++ metaprogramming, in preference to template metaprogramming (TMP). (There was some confusion in this regard, as there was a proposal to standardize Boost.Mp11, a TMP library. The feeling at the end of the meeting was that with constexpr metaprogramming just around the corner, it’s probably not the best use of committee time to standardize a TMP library.)

The second was an afternoon meeting of SG 7, where the main agenda item was reviewing two proposals for reflection based on constexpr metaprogramming: constexpr reflexpr, and scalable reflection in C++. The first is by the authors of the Reflection TS, and tries to carry over the Reflection TS facilities to the land of constexpr in a relatively straightforward way. The second is a variation of this approach that reflects experience gained from experimentation by some implementers. Both proposals also go further than the Reflection TS in functionality, by supporting reification, which involves going from meta-objects obtained via reflection back to the entities they represent.

One notable difference between the two proposals is that the first uses meta-objects of different types to represent different kinds of entities (e.g. meta::variable, meta::function, etc.), whereas the second uses just one type (meta::info) for all meta-objects, and requires using operations on them (e.g. is_variable()) to discriminate. The authors of the second proposal claim this is necessary for compile-time performance to be manageable; however, from an interface point of view the group preferred the different-types approach, and some implementers thought the performance issues could be solved. At the same time, there was agreement that while there should be different types, they should not form an inheritance hierarchy, but rather be type-erased by-value wrappers. In addition, the group felt that errors should be visible in the type system; that is, rather than having meta-objects admit an invalid state, reflection operations that can fail should return something like expected<meta::info> instead.

The target ship vehicle for a constexpr-based reflection facility is not set in stone yet, but people are hopeful for C++23.

In addition, SG 7 approved some guidelines for what kinds of library proposals should require SG 7 review.

SG 13 (Human/Machine Interface)

The Human/Machine Interface Study Group (SG 13) deals with proposals for graphics, event handling, and other forms of user interaction.

Its main product so far has been the 2D graphics proposal, which had been making good progress until it lost consensus to move forward at the last meeting. As there was still significant interest in this proposal in many user communities (see e.g. this paper arguing strongly for it), the Convenor asked SG 13 to have another look at it, to see if consensus could somehow be re-attained. There wasn’t extensive technical discussion of the proposal at this meeting, but we did go over some feedback from potential implementers; it was suggested that the author and other interested parties spend some time talking to graphics experts, many of whom are found in the Bay area (though not the ones at Mozilla – our graphics folks are mostly in the Toronto office).

The group also discussed the web_view proposal, which was positioned as an alternative to rolling our own graphics API. As the proposal effectively involves shipping a web platform implementation as part of the C++ standard library, this proposal has a lot of relevance to Mozilla. As such, I solicited feedback on it on Mozilla’s platform mailing list, and the feedback was pretty universally that this is not a good fit for the C++ standard library. I relayed this feedback at this meeting; nonetheless, the group as a whole was in favour of continuing to pursue this proposal. In fact, the group felt this and 2D graphics serve different use cases and should both be pursued in parallel. (Admittedly, there’s some selection bias going on here: people who choose to attend a meeting of SG 13 are naturally likely to be in favour of proposals in this topic area. I’m curious to see how these proposals will fare in front of a larger voting audience.)

There was also some general discussion of other topics in scope for this group. There are plans for bring forward a proposal for an audio API, and there were also ideas thrown around about things like event handling, user input, sensors, and VR.

SG 15 (Tooling)

The Tooling Study Group (SG 15) met for an evening session, and numerous papers concerning a variety of topics were presented.

The most pressing topic was how to integrate Modules with build systems. The problem is nicely summarized in this paper, and proposed solutions range from a separate “module mapper” component to relying on conventions.

The other major topic was general discussion about where to go in the space of dependency and package management. Ideas presented here include a set of APIs to allow components of a package ecosystem to interface with each other without requiring a particular implementation for any one component, and ideas around package specification.

I don’t feel like a lot of decisions were made in this session, and the group as a whole seems to be conflicted about what its role is given that these areas are not in the purview of the C++ standards document itself, but I still think the evening served as a valuable opportunity for pioneers in these areas to exchange areas and build mindshare around the tooling problems facing the C++ community.

Other Study Groups

Other Study Groups that met at this meeting include:

  • SG 6 (Numerics), which met for about a day and a half and reviewed a dozen or so proposals
  • SG 12 (Undefined and Unspecified Behaviour), which met both on its own (largely due discuss Contracts) and in joint session with WG23 – Software Vulnerabilities (where the focus was on vulnerabilities related to control structures)
  • SG 16 (Unicode), for which this was the first in-person meeting. The group approved a set of high-level priorities in addition to reviewing several specific proposals.

Freestanding Implementations

Not a study group, but this didn’t really fit anywhere else: there was an evening session to try to clarify the committee’s approach to freestanding implementations.

Freestanding implementations are, roughly speaking, those which cannot assume the presence of a full complement of operating system services, because they’re e.g. targeting kernel code or other “bare metal” scenarios; such implementations cannot practically make use of all language features, such as exceptions.

The standard currently defines a subset of the library that is intended to be supported on freestanding implementations, but defines no such subset for the language. Attempts to define such a subset tend to be stymied by the fact that different environments have different constraints, so one subset does not fit all.

The session didn’t reach any firm conclusions, but one possible direction is to avoid trying to define subsets, and instead make it easier for target environments to not use features of the language that are not applicable or practical for it.

New Study Groups

Two new Study Groups were announced at this meeting. Quoting their charters from Herb Sutter’s trip report:

SG 19 (Machine Learning):

We feel we can leverage C++’s strengths in generic programming, optimization and acceleration, as well as code portability, for the specific domain of Machine Learning. The aim of SG19 is to address and improve on C++’s ability to support fast iteration, better support for array, matrix, linear algebra, in memory passing of data for computation, scaling, and graphing, as well as optimization for graph programming.

SG 20 (Education):

We feel we have an opportunity to improve the quality of C++ education, to help software developers correctly use our language and ecosystem to write correct, maintainable, and performing software. SG20 aims to create curriculum guidelines for various levels of expertise and application domains, and to stimulate WG21 paper writers to include advise on how to teach the new feature they are proposing to add to the standard.

Next Meetings

The next meeting of the Committee will be in Kona, Hawaii, the week of February 18th, 2019.

Conclusion

C++ standards development continues to progress at an unprecedented pace. My highlights for this meeting included:

  • Modules gaining design approval to go into C++20
  • Abbreviated function templates reaching consensus, to round out Concepts in C++20
  • Ranges being voted into the C++20 working draft
  • Coroutines continuing to progress towards a unified design that can hopefully achieve consensus

With the big-ticket items above, not to mention Contracts, operator spaceship, and many other goodies, C++20 is shaping up to be a very impressive release!

Due to sheer number of proposals, there is a lot I didn’t cover in this post; if you’re curious about a specific proposal that I didn’t mention, please feel free to ask about it in the comments.

Other Trip Reports

In addition to Herb’s, other trip reports about this meeting include Corentin Jabot’s, a collaborative Reddit report, and a podcast focused on Library Evolution by Ashley Hedberg. I encourage you to check them out as well!

Review of the Purism Librem 13

Towards the end of last year, I got a new laptop: the Purism Librem 13. It replaced the Lenovo ThinkPad X250 that I was using previously, which maxed out at 8 GB RAM and was beginning to be unusable for Firefox builds.

This is my first professional laptop that isn’t a ThinkPad; as I’ve now been using it for over half a year, I thought I’d write some brief notes on what my experience with it has been like.

Why Purism?

My main requirement from a work point of view was having at least 16 GB RAM while staying in the same weight category as the X250. There were options meeting those criteria in the ThinkPad line (like the X270 or newer generations of X1 Carbon), so why did I choose Purism?

Purism is a social benefit corporation that aims to make laptops that respect your privacy and freedom — at the hardware and firmware levels in addition to software — while remaining competitive with other productivity laptops in terms of price and specifications.

The freedom-respecting features of the Librem 13 that you don’t typically find in other laptops include:

  • Hardware kill switches for WiFi/Bluetooth and the microphone/camera
  • A open-source bootloader (coreboot)
  • A disabled Intel Management Engine, a component of Intel CPUs that runs proprietary software at (very) elevated privilege levels, which Intel makes very hard to disable or replace
  • An attempt to ship hardware components with open-source firmware, though this is very much a work in progress
  • Tamper evidence via Heads, though this is a newer feature and was not available at the time I purchased my Librem 13.

These are features I’ve long wanted in my computing devices, and it was exciting to see someone producing competitively priced laptops with all the relevant configuration, sourcing of parts, compatibility testing etc. done for you.

Hardware

Material

The Librem’s aluminum chassis looks nicer and feels sturdier than the X250’s plastic one.

Screen

At 13.3″, the Librem’s screen size is a small but noticeable and welcome improvement over the X250’s 12.5″.

The X250 traded off screen size for battery life. It’s the same weight as the 14″ ThinkPad X1 Carbon; the weight savings from a smaller screen size go into extra thickness, which allows for a second battery. I was pleased to see that the Librem, which is the same thickness as the X1 Carbon and only has one battery, has comparable battery life to the X250 (5-6 hours on an average workload).

The Librem’s screen is not a touchscreen. I noticed this because I used the X250’s touchscreen to test touch event support in Firefox, but I don’t think the average user has much use of a touchscreen in a conventional laptop (it’s more useful in 2-in-1 laptops, which Purism also offers, and that does have a touchscreen), so I don’t hold this against Purism.

The maximum swivel angle between the Librem’s keyboard and its screen is 130 degrees, compared to the X250’s almost 180 degrees. I did occasionally use the X250’s greater swivel angle (e.g. when lying on a couch), but I didn’t find its absence in the Librem to be a significant issue.

Touchpad

The one feature of ThinkPad laptops that I miss the most in the Librem, is the TrackPoint, the red button in the middle of the keyboard that allows you to move the cursor without having to move your hand down to the touchpad. I didn’t realize how much I relied on this until I didn’t have it, though I’ve been getting by without it. (I view it as additional motivation for me to use the keyboard more and the cursor less.)

Also missing in the Librem are the buttons above the touchpad for left-, right-, and middle-clicking; you instead have to click by tapping the touchpad with one, two, or three fingers (respectively), which I find more awkward and prone to accidental taps.

Finally, while I haven’t noticed this very much myself (but I tend not to be very discerning in this area), several people who have briefly used my Librem commented that the sensitivity of its touchpad is significantly reduced compared to other touchpads they’re used to.

Keyboard

The Librem’s keys feel better to the press than the X250’s. However, I’ve found you have to hit the keys fairly close to their centre for the press to register; the X250’s keys were more sensitive in this respect (hitting the side of the key would still trigger it), so this took some getting used to.

The keyboard can be backlit (at two different levels of intensity, though I don’t think I’ve ever used the second one). However, the shortcut to activate the backlight (Fn + F10) is significantly harder to find in the dark than the X250’s (Fn + Space).

I’ve also found the Librem’s keys get sweaty more easily, I’m guessing due to different materials.

Layout

The Librem’s keyboard layout differs from the X250’s in several small but important ways. Some of the changes are welcome; others, less so.

Here is a picture of the keyboard to illustrate:

Librem 13 keyboard

  • One thing that I think the Librem’s keyboard gets right that the X250 got wrong, is that the key in the bottom left corner is Ctrl, with Fn being next to it, rather than the other way around. I find this significantly aids muscle memory when moving between the Librem’s keyboard and external / desktop keyboards (which invariably have Ctrl in the bottom left corner). (I know that issues like this can technically be worked around by remapping keys, but it’s nice not to have to.)
  • On the other hand, the biggest deficiency in the Librem’s keyboard is the lack of PageUp, PageDown, Home, and End keys. The X250 had all of these: PageUp and PageDown above the right and left arrow keys, Home and End in the top row. With the Librem, you have to use the arrow keys with the Fn modifier to invoke these operations. My typing style is such that I use these operations fairly heavily, and as such I’ve missed the separate keys a lot.
  • A related minor annoyance is the fact that the rightmost key in the second row from the bottom is not Shift as it usually is, but a second Fn key; that’s also an impedient to muscle memory across different keyboards.
  • Lastly, the key in the top right corner is the power key, not Delete which is what I was used to from the X250.

None of these are necessarily dealbreakers, but they did take some getting used to.

Microphone

Every time I’ve tried the Librem’s microphone so far, the recording quality has been terrible, with large amounts of static obscuring the signal. I haven’t yet had a chance to investigate whether this is a hardware or software issue.

Software

The Librem 13 comes with Purism’s own Linux distribution, PureOS. PureOS is basically a light repack of Debian and GNOME 3, with some common software pre-installed and, in some cases, re-branded.

I got the impression that PureOS and its software doesn’t get much in the way of maintenance. For example, for the re-branded browser that came with PureOS, “PureBrowser”, the latest version available in the PureOS repository at the time I got my Librem was based on Firefox 45 ESR, which had been out of support for some 6 months by that time!

I’m also not a huge fan of GNOME 3. I tolerated this setup for all of about two weeks, and then decided to wipe the PureOS installation and replace it with a plain Debian stable installation, with KDE, my preferred desktop environment. This went without a hitch, indicating that — as far as I can tell — there isn’t anything in the PureOS patches that’s necessary for running on this hardware.

Generally, running Linux on the Librem 13 has been a smooth experience; I haven’t seen much in the way of glitches or compatibility issues. Occasionally, I get something like a crashed power management daemon (shortcuts to increase/decrease brightness stop working), but nothing too serious.

Conclusion

The Purism Librem 13 has largely lived up to my goal of having a lightweight productivity laptop with a decent amount of memory (though I’m sad to say that the Firefox build has continued to get larger and slower over time, and linking is sometimes a struggle even with 16 GB of RAM…) while also going the extra mile to protect my privacy and freedoms. The Librem 13 has a few deficiencies in comparison to the ThinkPad line, but they’re mostly in the category of papercuts. At the end of the day it boils down to whether living with a few small annoyances to benefit from the additional privacy features is the right tradeoff for you. For me, so far, it has been, although I certainly hope the Purism folks take feedback like this into account and improve future iterations of the Librem line.

Trip Report: C++ Standards Meeting in Rapperswil, June 2018

Summary / TL;DR

Project What’s in it? Status
C++17 See list Published!
C++20 See below On track
Library Fundamentals TS v2 source code information capture and various utilities Published! Parts of it merged into C++17
Concepts TS Constrained templates Merged into C++20 with some modifications
Parallelism TS v2 Task blocks, library vector types and algorithms, and more Approved for publication!
Transactional Memory TS Transaction support Published! Not headed towards C++20
Concurrency TS v1 future.then(), latches and barriers, atomic smart pointers Published! Parts of it merged into C++20, more on the way
Executors Abstraction for where/how code runs in a concurrent context Final design being hashed out. Ship vehicle not decided yet.
Concurrency TS v2 See below Under development. Depends on Executors.
Networking TS Sockets library based on Boost.ASIO Published!
Ranges TS Range-based algorithms and views Published! Headed towards C++20
Coroutines TS Resumable functions, based on Microsoft’s await design Published! C++20 merge uncertain
Modules v1 A component system to supersede the textual header file inclusion model Published as a TS
Modules v2 Improvements to Modules v1, including a better transition path Under active development
Numerics TS Various numerical facilities Under active development
Graphics TS 2D drawing API No consensus to move forward
Reflection TS Static code reflection mechanisms Send out for PDTS ballot
Contracts Preconditions, postconditions, and assertions Merged into C++20

A few links in this blog post may not resolve until the committee’s post-meeting mailing is published (expected within a few days of June 25, 2018). If you encounter such a link, please check back in a few days.

Introduction

A couple of weeks ago I attended a meeting of the ISO C++ Standards Committee (also known as WG21) in Rapperswil, Switzerland. This was the second committee meeting in 2018; you can find my reports on preceding meetings here (March 2018, Jacksonville) and here (November 2017, Albuquerque), and earlier ones linked from those. These reports, particularly the Jacksonville one, provide useful context for this post.

At this meeting, the committee was focused full-steam on C++20, including advancing several significant features — such as Ranges, Modules, Coroutines, and Executors — for possible inclusion in C++20, with a secondary focus on in-flight Technical Specifications such as the Parallelism TS v2, and the Reflection TS.

C++20

C++20 continues to be under active development. A number of new changes have been voted into its Working Draft at this meeting, which I list here. For a list of changes voted in at previous meetings, see my Jacksonville report.

Technical Specifications

In addition to the C++ International Standard (IS), the committee publishes Technical Specifications (TS) which can be thought of experimental “feature branches”, where provisional specifications for new language or library features are published and the C++ community is invited to try them out and provide feedback before final standardization.

At this meeting, the committee voted to publish the second version of the Parallelism TS, and to send out the Reflection TS for its PDTS (“Proposed Draft TS”) ballot. Several other TSes remain under development.

Parallelism TS v2

The Parallelism TS v2 was sent out for its PDTS ballot at the last meeting. As described in previous reports, this is a process where a draft specification is circulated to national standards bodies, who have an opportunity to provide feedback on it. The committee can then make revisions based on the feedback, prior to final publication.

The results of the PDTS ballot had arrived just in time for the beginning of this meeting, and the relevant subgroups (primarily the Concurrency Study Group) worked diligently during the meeting to go through the comments and address them. This led to the adoption of several changes into the TS working draft:

The working draft, as modified by these changes, was then approved for publication!

Reflection TS

The Reflection TS, based on the reflexpr static reflection proposal, picked up one new feature, static reflection of functions, and was subsequently sent out for its PDTS ballot! I’m quite excited to see efficient progress on this (in my opinion) very important feature.

Meanwhile, the committee has also been planning ahead for the next generation of reflection and metaprogramming facilities for C++, which will be based on value-based constexpr programming rather than template metaprogramming, allowing users to reap expressiveness and compile-time performance gains. In the list of proposals reviewed by the Evolution Working Group (EWG) below, you’ll see quite a few of them are extensions related to constexpr; that’s largely motivated by this direction.

Concurrency TS v2

The Concurrency TS v2 (no working draft yet), whose notable contents include revamped versions of async() and future::then(), among other things, continues to be blocked on Executors. Efforts at this meeting focused on moving Executors forward.

Library Fundamentals TS v3

The Library Fundementals TS v3 is now “open for business” (has an initial working draft based on the portions of v2 that have not been merged into the IS yet), but no new proposals have been merged to it yet. I expect that to start happening in the coming meetings, as proposals targeting it progress through the Library groups.

Future Technical Specifications

There are (or were, in the case of the Graphics TS) some planned future Technical Specifications that don’t have an official project or working draft at this point:

Graphics

At the last meeting, the Graphics TS, set to contain 2D graphics primitives with an interface inspired by cairo, ran into some controversy. A number of people started to become convinced that, since this was something that professional graphics programmers / game developers were unlikely to use, the large amount of time that a detailed wording review would require was not a good use of committee time.

As a result of these concerns, an evening session was held at this meeting to decide the future of the proposal. A paper arguing we should stay course was presented, as was an alternative proposal for a much lighter-weight “diet” graphics library. After extensive discussion, however, neither the current proposal nor the alternative had consensus to move forward.

As a result – while nothing is ever set in stone and the committee can always change in mind – the Graphics TS is abandoned for the time being.

(That said, I’ve heard rumours that the folks working on the proposal and its reference implementation plan to continue working on it all the same, just not with standardization as the end goal. Rather, they might continue iterating on the library with the goal of distributing it as a third-party library/package of some sort (possibly tying into the committee’s exploration of improving C++’s package management ecosystem).)

Executors

SG 1 (the Concurrency Study Group) achieved design consensus on a unified executors proposal (see the proposal and accompanying design paper) at the last meeting.

At this meeting, another executors proposal was brought forward, and SG 1 has been trying to reconcile it with / absorb it into the unified proposal.

As executors are blocking a number of dependent items, including the Concurrency TS v2 and merging the Networking TS, SG 1 hopes to progress them forward as soon as possible. Some members remain hopeful that it can be merged into C++20 directly, but going with the backup plan of publishing it as a TS is also a possibility (which is why I’m listing it here).

Merging Technical Specifications into C++20

Turning now to Technical Specifications that have already been published, but not yet merged into the IS, the C++ community is eager to see some of these merge into C++20, thereby officially standardizing the features they contain.

Ranges TS

The Ranges TS, which modernizes and Conceptifies significant parts of the standard library (the parts related to algorithms and iterators), has been making really good progress towards merging into C++20.

The first part of the TS, containing foundational Concepts that a large spectrum of future library proposals may want to make use of, has just been merged into the C++20 working draft at this meeting. The second part, the range-based algorithms and utilities themselves, is well on its way: the Library Evolution Working Group has finished ironing out how the range-based facilities will integrate with the existing facilities in the standard library, and forwarded the revised merge proposal for wording review.

Coroutines TS

The Coroutines TS was proposed for merger into C++20 at the last meeting, but ran into pushback from adopters who tried it out and had several concerns with it (which were subsequently responded to, with additional follow-up regarding optimization possibilities).

Said adopters were invited to bring forward a proposal for an alternative / modified design that addressed their concerns, no later than at this meeting, and so they did; their proposal is called Core Coroutines.

Core Coroutines was reviewed by the Evolution Working Group (I summarize the technical discussion below), which encouraged further iteration on this design, but also felt that such iteration should not hold up the proposal to merge the Coroutines TS into C++20. (What’s the point in iterating on one design if another is being merged into the IS draft, you ask? I believe the thinking was that further exploration of the Core Coroutines design could inspire some modifications to the Coroutines TS that could be merged at a later meeting, still before C++20’s publication.)

As a result, the merge of the Coroutines TS came to a plenary vote at the end of the week. However, it did not garner consensus; a significant minority of the committee at large felt that the Core Coroutines design deserved more exploration before enshrining the TS design into the standard. (At least, I assume that was the rationale of those voting against. Regrettably, due to procedural changes, there is very little discussion before plenary votes these days to shed light on why people have the positions they do.)

The window for merging a TS into C++20 remains open for approximately one more meeting. I expect the proponents of the Coroutines TS will try the merge again at the next meeting, while the authors of Core Coroutines will refine their design further. Hopefully, the additional time and refinement will allow us to make a better-informed final decision.

Networking TS

The Networking TS is in a situation where the technical content of the TS itself is in a fairly good shape and ripe for merging into the IS, but its dependency on Executors makes a merger in the C++20 timeframe uncertain.

Ideas have been floated around of coming up with a subset of Executors that would be sufficient for the Networking TS to be based on, and that could get agreement in time for C++20. Multiple proposals on this front are expected at the next meeting.

Modules

Modules is one of the most-anticipated new features in C++. While the Modules TS was published fairly recently, and thus merging it into C++20 is a rather ambitious timeline (especially since there are design changes relative to the TS that we know we want to make), there is a fairly widespread desire to get it into C++20 nonetheless.

I described in my last report that there was a potential path forward to accomplishing this, which involved merging a subset of a revised Modules design into C++20, with the rest of the revised design to follow (likely in the form of a Modules TS v2, and a subsequent merge into C++23).

The challenge with this plan is that we haven’t fully worked out the revised design yet, never mind agreed on a subset of it that’s safe for merging into C++20. (By safe I mean forwards-compatible with the complete design, since we don’t want breaking changes to a feature we put into the IS.)

There was extensive discussion of Modules in the Evolution Working Group, which I summarize below. The procedural outcome was that there was no consensus to move forward with the “subset” plan, but we are moving forward with the revised design at full speed, and some remain hopeful that the entire revised design (or perhaps a larger subset) can still be merged into C++20.

What’s happening with Concepts?

The Concepts TS was merged into the C++20 working draft previously, but excluding certain controversial parts (notably, abbreviated function templates (AFTs)).

As AFTs remain quite popular, the committee has been trying to find an alternative design for them that could get consensus for C++20. Several proposals were heard by EWG at the last meeting, and some refined ones at this meeting. I summarize their discussion below, but in brief, while there is general support for two possible approaches, there still isn’t final agreement on one direction.

The Role of Technical Specifications

We are now about 6 years into the committee’s procedural experiment of using Technical Specifications as a vehicle for gathering feedback based on implementation and use experience prior to standardization of significant features. Opinions differ on how successful this experiment has been so far, with some lauding the TS process as leading to higher-quality, better-baked features, while others feel the process has in some cases just added unnecessary delays.

The committee has recently formed a Direction Group, a small group composed of five senior committee members with extensive experience, which advises the Working Group chairs and the Convenor on matters related to priority and direction. One of the topics the Direction Group has been tasked with giving feedback on is the TS process, and there was evening session at this meeting to relay and discuss this advice.

The Direction Group’s main piece of advice was that while the TS process is still appropriate for sufficiently large features, it’s not to be embarked on lightly; in each case, a specific set of topics / questions on which the committee would like feedback should be articulated, and success criteria for a TS “graduating” and being merged into the IS should be clearly specified at the outset.

Evolution Working Group

I’ll now write in a bit more detail about the technical discussions that took place in the Evolution Working Group, the subgroup that I sat in for the duration of the week.

Unless otherwise indicated, proposals discussed here are targeting C++20. I’ve categorized them into the usual “accepted”, “further work encouraged”, and “rejected” categories:

Accepted proposals:

  • Standard library compatibility promises. EWG looked at this at the last meeting, and asked that it be revised to only list the types of changes the standard library reserves to make; a second list, of code patterns that should be avoided if you want a guarantee of future library updates not breaking your code, was to be removed as it follows from the first list. The revised version was approved and will be published as a Standing Document (pending a plenary vote).
  • A couple of minor tweaks to the contracts proposal:
    • In response to implementer feedback, the always checking level was removed, and the source location reported for precondition violations was made implementation-defined (previously, it had to be a source location in the function’s caller).
    • Virtual functions currently require that overrides repeat the base function’s pre- and postconditions. We can run into trouble in cases where the base function’s pre- or postcondition, interpreted in the context of the derived class, has a different meaning (e.g. because the derived class shadows a base member’s name, or due to covariant return types). Such cases were made undefined behaviour, with the understanding that this is a placeholder for a more principled solution to forthcome at a future meeting.
  • try/catch blocks in constexpr functions. Throwing an exception is still not allowed during constant evaluation, but the try/catch construct itself can be present as long as only the non-throwing codepaths as exercised at compile time.
  • More constexpr containers. EWG previously approved basic support for using dynamic allocation during constant evaluation, with the intention of allowing containers like std::vector to be used in a constexpr context (which is now happening). This is an extension to that, which allows storage that was dynamically allocated at compile time to survive to runtime, in the form of a static (or automatic) storage duration variable.
  • Allowing virtual destructors to be “trivial”. This lifts an unnecessary restriction that prevented some commonly used types like std::error_code from being used at compile time.
  • Immediate functions. These are a stronger form of constexpr functions, spelt constexpr!, which not only can run at compile time, but have to. This is motivated by several use cases, one of them being value-based reflection, where you need to be able to write functions that manipulate information that only exists at compile-time (like handles to compiler data structures used to implement reflection primitives).
  • std::is_constant_evaluated(). This allows you to check whether a constexpr function is being invoked at compile time or at runtime. Again there are numerous use cases for this, but a notable one is related to allowing std::string to be used in a constexpr context. Most implementations of std::string use a “small string optimization” (SSO) where sufficiently small strings are stored inline in the string object rather than in a dynamically allocated block. Unfortunately, SSO cannot be used in a constexpr context because it requires using reinterpret_cast (and in any case, the motivation for SSO is runtime performance), so we need a way to make the SSO conditional on the string being created at runtime.
  • Signed integers are two’s complement. This standardizes existing practice that has been the case for all modern C++ implementations for quite a while.
  • Nested inline namespaces. In C++17, you can shorten namespace foo { namespace bar { namespace baz { to namespace foo::bar::baz {, but there is no way to shorten namespace foo { inline namespace bar { namespace baz {. This proposal allows writing namespace foo::inline bar::baz. The single-name version, namespace inline foo { is also valid, and equivalent to inline namespace foo {.

There were also a few that, after being accepted by EWG, were reviewed by CWG and merged into the C++20 working draft the same week, and thus I already mentioned them in the C++20 section above:


Proposals for which further work is encouraged:

  • Generalizing alias declarations. The idea here is to generalize C++’s alias declarations (using a = b;) so that you can alias not only types, but also other entities like namespaces or functions. EWG was generally favourable to the idea, but felt that aliases for different kinds of entities should use different syntaxes. (Among other considerations, using the same syntax would mean having to reinstate the recently-removed requirement to use typename in front of a dependent type in an alias declaration.) The author will explore alternative syntaxes for non-type aliases and return with a revised proposal.
  • Allow initializing aggregates from a parenthesized list of values. This idea was discussed at the last meeting and EWG was in favour, but people got distracted by the quasi-related topic of aggregates with deleted constructors. There was a suggestion that perhaps the two problems could be addressed by the same proposal, but in fact the issue of deleted constructors inspired independent proposals, and this proposal returned more or less unchanged. EWG liked the idea and initially approved it, but during Core Working Group review it came to light that there are a number of subtle differences in behaviour between constructor initialization and aggregate initialization (e.g. evaluation order of arguments, lifetime extension, narrowing conversions) that need to be addressed. The suggested guidance was to have the behaviour with parentheses match the behaviour of constructor calls, by having the compiler (notionally) synthesize a constructor to call when this notation is used. The proposal will return with these details fleshed out.
  • Extensions to class template argument deduction. This paper proposed seven different extensions to this popular C++17 feature. EWG didn’t make individual decisions on them yet. Rather, the general guidance was to motivate the extensions a bit better, choose a subset of the more important ones to pursue for C++20, perhaps gather some implementation experience, and come back with a revised proposal.
  • Deducing this. The type of the implicit object parameter (the “this” parameter) of a member function can vary in the same ways as the types of other parameters: lvalue vs. rvalue, const vs. non-const. C++ provides ways to overload member functions to capture this variation (trailing const, ref-qualifiers), but sometimes it would be more convenient to just template over the type of the this parameter. This proposal aims to allow that, with a syntax like this:

    template <typename Self>
    R foo(this Self&& self, /* other parameters */);

    EWG agreed with the motivation, but expressed a preference for keeping information related to the implicit object parameter at the end of the function declaration, (where the trailing const and ref-qualifiers are now), leading to a syntax more like this:

    template <typename Self>
    R foo(/* other parameters */) Self&& self

    (the exact syntax remains to be nailed down as the end of a function declaration is a syntactically busy area, and parsing issues have to be worked out).
    EWG also opined that in such a function, you should only be able to access the object via the declared object parameter (self in the above example), and not also using this (as that would lead to confusion in cases where e.g. this has the base type while self has a derived type).
  • constexpr function parameters. The most ambitious constexpr-related proposal brought forward at this meeting, this aimed to allow function parameters to be marked as constexpr, and accordingly act as constant expressions inside the function body (e.g. it would be valid to use the value of one as a non-type template parameter or array bound). It was quickly pointed out that, while the proposal is implementable, it doesn’t fit into the language’s current model of constant evaluation; rather, functions with constexpr parameters would have to be implemented as templates, with a different instantiation for every combination of parameter values. Since this amounts to being a syntactic shorthand for non-type template parameters, EWG suggested that the proposal be reformulated in those terms.
  • Binding returned/initialized objects to the lifetime of parameters. This proposal aims to improve C++’s lifetime safety (and perhaps take one step towards being more like Rust, though that’s a long road) by allowing programmers to mark function parameters with an annotation that tells the compiler that the lifetime of the function’s return value should be “bound” to the lifetime of the parameter (that is, the return value should not outlive the parameter).
    There are several options for the associated semantics if the compiler detects that the lifetime of a return value would, in fact, exceed the lifetime of a parameter:

    • issue a warning
    • issue an error
    • extend the lifetime of the returned object



    In the first case, the annotation could take the form of an attribute (e.g. [[lifetimebound]]). In the second or third case, it would have to be something else, like a context-sensitive keyword (since attributes aren’t supposed to have semantic effects). The proposal authors suggested initially going with the first option in the C++20 timeframe, while leaving the door open for the second or third option later on.
    EWG agreed that mitigating lifetime hazards is an important area of focus, and something we’d like to deliver on in the C++20 timeframe. There was some concern about the proposed annotation being too noisy / viral. People asked whether the annotations could be deduced (not if the function is compiled separately, unless we rely on link-time processing), or if we could just lifetime-extend by default (not without causing undue memory pressure and risking resource exhaustion and deadlocks by not releasing expensive resources or locks in time). The authors will investigate the problem space further, including exploring ways to avoid the attribute being viral, and comparing their approach to Rust’s, and report back.

  • Nameless parameters and unutterable specializations. In some corner cases, the current language rules do not give you a way to express a partial or explicit specialization of a constrained template (because a specialization requires repeating the constraint with the specialized parameter values substituted in, which does not always result in valid syntax). This proposal invents some syntax to allow expressing such specializations. EWG felt the proposed syntax was scary, and suggested coming back with better motivating examples before pursuing the idea further.
  • How to catch an exception_ptr without even trying. This aims to allow getting at the exception inside an exception_ptr without having to throw it (which is expensive). As a side effect, it would also allow handling exception_ptrs in code compiled with -fno-exceptions. EWG felt the idea had merit, even though performance shouldn’t be the guiding principle (since the slowness of throw is technically a quality-of-implementation issue, although implementations seem to have agreed to not optimize it).
  • Allowing class template specializations in associated namespaces. This allows specializing e.g. std::hash for your own type, in your type’s namespace, instead of having to close that namespace, open namespace std, and then reopen your namespace. EWG liked the idea, but the issue of which names — names in your namespace, names in std, or both — would be visible without qualification inside the specialization, was contentious.

Rejected proposals:

  • Define basic_string_view(nullptr). This paper argued that since it’s common to represent empty strings as a const char* with value nullptr, the constructor of string_view which takes a const char* argument should allow a nullptr value and interpret it as an empty string. Another paper convincingly argued that conflating “a zero-sized string” with “not-a-string” does more harm than good, and this proposal was accordingly rejected.
  • Explicit concept expressions. This paper pointed out that if constrained-type-specifiers (the language machinery underlying abbreviated function templates) are added to C++ without some extra per-parameter syntax, certain constructs can become ambiguous (see the paper for an example). The ambiguity involves “concept expressions”, that is, the use of a concept (applied to some arguments) as a boolean expression, such as CopyConstructible<T>, outside of a requires-clause. The authors proposed removing the ambiguity by requiring the keyword requires to introduce a concept expression, as in requires CopyConstructible<T>. EWG felt this was too much syntactic clutter, given that concept expressions are expected to be used in places like static_assert and if constexpr, and given that the ambiguity is, at this point, hypothetical (pending what hapens to AFTs) and there would be options to resolve it if necessary.

Concepts

EWG had another evening session on Concepts at this meeting, to try to resolve the matter of abbreviated function templates (AFTs).

Recall that the main issue here is that, given an AFT written using the Concepts TS syntax, like void sort(Sortable& s);, it’s not clear that this is a template (you need to know that Sortable is a concept, not a type).

The four different proposals in play at the last meeting have been whittled down to two:

  • An updated version of Herb’s in-place syntax proposal, with which the above AFT would be written void sort(Sortable{}& s); or void sort(Sortable{S}& s); (with S in the second form naming the concrete type deduced for this parameter). The proposal also aims to change the constrained-parameter syntax (with which the same function could be written template <Sortable S> void sort(S& s);) to require braces for type parameters, so that you’d instead write template <Sortable{S}> void sort(S& s);. (The motivation for this latter change is to make it so that ConceptName C consistently makes C a value, whether it be a function parameter or a non-type template parameter, while ConceptName{C] consistently makes C a type.)
  • Bjarne’s minimal solution to the concepts syntax problems, which adds a single leading template keyword to announce that an AFT is a template: template void sort(Sortable& s);. (This is visually ambiguous with one of the explicit specialization syntaxes, but the compiler can disambiguate based on name lookup, and programmers can use the other explicit specialization syntax to avoid visual confusion.) This proposal leaves the constrained-parameter syntax alone.

Both proposals allow a reader to tell at a glance that an AFT is a template and not a regular function. At the same time, each proposal has downsides as well. Bjarne’s approach annotates the whole function rather than individual parameters, so in a function with multiple parameters you still don’t know at a glance which parameters are concepts (and so e.g. in a case of a Foo&& parameter, you don’t know if it’s an rvalue reference or a forwarding reference). Herb’s proposal messes with the well-loved constrained-parameter syntax.

After an extensive discussion, it turned out that both proposals had enough support to pass, with each retaining a vocal minority of opponents. Neither proposal was progressed at this time, in the hope that some further analysis or convergence can lead to a stronger consensus at the next meeting, but it’s quite clear that folks want something to be done in this space for C++20, and so I’m fairly optimistic we’ll end up getting one of these solutions (or a compromise / variation).

In addition to the evening session on AFTs, EWG looked at a proposal to alter the way name lookup works inside constrained templates. The original motivation for this was to resolve the AFT impasse by making name lookup inside AFTs work more like name lookup inside non-template functions. However, it became apparent that (1) that alone will not resolve the AFT issue, since name lookup is just one of several differences between template and non-template code; but (2) the suggested modification to name lookup rules may be desirable (not just in AFTs but in all constrained templates) anyways. The main idea behind the new rules is that when performing name lookup for a function call that has a constrained type as an argument, only functions that appear in the concept definition should be found; the motivation is to avoid surprising extra results that might creep in through ADL. EWG was supportive of making a change along these lines for C++20, but some of the details still need to be worked out; among them, whether constraints should be propagated through auto variables and into nested templates for the purpose of applying this rule.

Coroutines

As mentioned above, EWG reviewed a modified Coroutines design called Core Coroutines, that was inspired by various concerns that some early adopters of the Coroutines TS had with its design.

Core Coroutines makes a number of changes to the Coroutines TS design:

  • The most significant change, in my opinion, is that it exposes the “coroutine frame” (the piece of memory that stores the compiler’s transformed representation of the coroutine function, where e.g. stack variables that persist across a suspension point are stored) as a first-class object, thereby allowing the user to control where this memory is stored (and, importantly, whether or not it is dynamically allocated).
  • Syntax changes:
    • To how you define a coroutine. Among other motivations, the changes emphasize that parameters to the coroutine act more like lambda captures than regular function parameters (e.g. for reference parameters, you need to be careful that the referred-to objects persist even after a suspension/resumption).
    • To how you call a coroutine. The new syntax is an operator (the initial proposal being [<-]), to reflect that coroutines can be used for a variety of purposes, not just asynchrony (which is what co_await suggests).
  • A more compact API for defining your own coroutine types, with fewer library customiztion points (basically, instead of specializing numerous library traits that are invoked by compiler-generated code, you overload operator [<-] for your type, with more of the logic going into the definition of that function).

EWG recognized the benefits of these modifications, although there were a variety of opinions as to how compelling they are. At the same time, there were also a few concerns with Core Coroutines:

  • While having the coroutine frame exposed as a first-class object means you are guaranteed no dynamic memory allocations unless you place it on the heap yourself, it still has a compiler-generated type (much like a lambda closure), so passing it across a translation unit boundary requires type erasure (and therefore a dynamic allocation). With the Coroutines TS, the type erasure was more under the compiler’s control, and it was argued that this allows eliding the allocation in more cases.
  • There were concerns about being able to take the sizeof of the coroutine object, as that requires the size being known by the compiler’s front-end, while with the Coroutines TS it’s sufficient for the size to be computed during the optimization phase.
  • While making the customization API smaller, this formulation relies on more new core-language features. In addition to introducing a new overloadable operator, the feature requires tail calls (which could also be useful for the language in general), and lazy function parameters, which have been proposed separately. (The latter is not a hard requirement, but the syntax would be more verbose without them.)

As mentioned, the procedural outcome of the discussion was to encourage further work on the Core Coroutines, while not blocking the merger of the Coroutines TS into C++20 on such work.

While in the end there was no consensus to merge the Coroutines TS into C++20 at this meeting, there remains fairly strong demand for having coroutines in some form in C++20, and I am therefore hopeful that some sort of joint proposal that combines elements of Core Coroutines into the Coroutines TS will surface at the next meeting.

Modules

As of the last meeting, there were two alternative Modules designs before the committee: the recently-published Modules TS, and the alternative proposal from the Clang Modules implementers called Another Take On Modules (“Atom”).

Since the last meeting, the authors of the two proposals have been collaborating to produce a merged proposal that combines elements from both proposals.

The merged proposal accomplishes Atom’s goal of providing a better mechanism for existing codebases to transition to Modules via modularized legacy headers (called legacy header imports in the merged proposal) – basically, existing headers that are not modules, but are treated as-if they were modules by the compiler. It retains the Modules TS mechanism of global module fragments, with some important restrictions, such as only allowing #includes and other preprocessor directives in the global module fragment.

Other aspects of Atom that are part of the the merged proposal include module partitions (a way of breaking up the interface of a module into multiple files), and some changes to export and template instantiation semantics.

EWG reviewed the merged proposal favourably, with a strong consensus for putting these changes into a second iteration of the Modules TS. Design guidance was provided on a few aspects, including tweaks to export behaviour for namespaces, and making export be “inherited”, such that e.g. if the declaration of a structure is exported, then its definition is too by default. (A follow-up proposal is expected for a syntax to explicitly make a structure definition not exported without having to move it into another module partition.) A proposal to make the lexing rules for the names of legacy header units be different from the existing rules for #includes failed to gain consensus.

One notable remaining point of contention about the merged proposal is that module is a hard keyword in it, thereby breaking existing code that uses that word as an identifier. There remains widespread concern about this in multiple user communities, including the graphics community where the name “module” is used in existing published specifications (such as Vulkan). These concerns would be addressed if module were made a context-sensitive keyword instead. There was a proposal to do so at the last meeting, which failed to gain consensus (I suspect because the author focused on various disambiguation edge cases, which scared some EWG members). I expect a fresh proposal will prompt EWG to reconsider this choice at the next meeting.

As mentioned above, there was also a suggestion to take a subset of the merged proposal and put it directly into C++20. The subset included neither legacy header imports nor global module fragments (in any useful form), thereby not providing any meaningful transition mechanism for existing codebases, but it was hoped that it would still be well-received and useful for new codebases. However, there was no consensus to proceed with this subset, because it would have meant having a new set of semantics different from anything that’s implemented today, and that was deemed to be risky.

It’s important to underscore that not proceeding with the “subset” approach does not necessarily mean the committee has given up on having any form of Modules in C++20 (although the chances of that have probably decreased). There remains some hope that the development of the merged proposal might proceed sufficiently quickly that the entire proposal — or at least a larger subset that includes a transition mechanism like legacy header imports — can make it into C++20.

Finally, EWG briefly heard from the authors of a proposal for modular macros, who basically said they are withdrawing their proposal because they are satisfied with Atom’s facility for selectively exporting macros via #export directives, which is being treated as a future extension to the merged proposal.

Papers not discussed

With the continued focus on large proposals that might target C++20 like Modules and Coroutines, EWG has a growing backlog of smaller proposals that haven’t been discussed, in some cases stretching back to two meetings ago (see the the committee mailings for a list). A notable item on the backlog is a proposal by Herb Sutter to bridge the two worlds of C++ users — those who use exceptions and those who not — by extending the exception model in a way that (hopefully) makes it palatable to everyone.

Other Working Groups

Library Groups

Having sat in EWG all week, I can’t report on technical discussions of library proposals, but I’ll mention where some proposals are in the processing queue.

I’ve already listed the library proposals that passed wording review and were voted into the C++20 working draft above.

The following are among the proposals have passed design review and are undergoing (or awaiting) wording review:

The following proposals are still undergoing design review, and are being treated with priority:

The following proposals are also undergoing design review:

As usual, there is a fairly long queue of library proposals that haven’t started design review yet. See the committee’s website for a full list of proposals.

(These lists are incomplete; see the post-meeting mailing when it’s published for complete lists.)

Study Groups

SG 1 (Concurrency)

I’ve already talked about some of the Concurrency Study Group’s work above, related to the Parallelism TS v2, and Executors.

The group has also reviewed some proposals targeting C++20. These are at various stages of the review pipeline:

Proposals before the Library Evolution Working Group include latches and barriers, C atomics in C++, and a joining thread.

Proposals before the Library Working Group include improvements to atomic_flag, efficient concurrent waiting, and fixing atomic initialization.

Proposls before the Core Working Group include revising the C++ memory model. A proposal to weaken release sequences has been put on hold.

SG 7 (Compile-Time Programming)

It was a relatively quiet week for SG 7, with the Reflection TS having undergone and passed wording review, and extensions to constexpr that will unlock the next generation of reflection facilities being handled in EWG. The only major proposal currently on SG 7’s plate is metaclasses, and that did not have an update at this meeting.

That said, SG 7 did meet briefly to discuss two other papers:

  • PFA: A Generic, Extendable and Efficient Solution for Polymorphic Programming. This aims to make value-based polymorphism easier, using an approach similar to type erasure; a parallel was drawn to the Dyno library. SG 7 observed that this could be accomplished with a pure library approach on top of existing reflection facilities and/or metaclasses (and if it can’t, that would signal holes in the reflection facilities that we’d want to fill).
  • Adding support for type-based metaprogramming to the standard library. This aims to standardize template metaprogramming facilities based on Boost.Mp11, a modernized version of Boost.MPL. SG 7 was reluctant to proceed with this, given that it has previously issued guidance for moving in the direction of constexpr value-based metaprogramming rather than template metaprogramming. At the same time, SG 7 recognized the desire for having metaprogramming facilities in the standard, and urged proponents on the constexpr approach to bring forward a library proposal built on that soon.

SG 12 (Undefined and Unspecified Behaviour)

SG 12 met to discuss several topics this week:

  • Reviewed a proposal to allow implicit creation of objects for low-level object manipulation (basically the way malloc() is used), which aims to standardize existing practice that the current standard wording makes undefined behaviour.
  • Reviewed a proposed policy around preserving undefined behaviour, which argues that in some cases, defining behaviour that was previously undefined can be a breaking change in some sense. SG 12 felt that imposing a requirement to preserve undefined behaviour wouldn’t be realistic, but that proposal authors should be encouraged to identify cases where proposals “break” undefined behaviour so that the tradeoffs can be considered.
  • Held a joint meeting with WG 23 (Programming Language Vulnerabilities) to collaborate further on a document describing C++ vulnerabilities. This meeting’s discussion focused on buffer boundary conditions and type conversions between pointers.

SG 15 (Tooling)

The Tooling Study Group (SG 15) held its second meeting during an evening session this week.

The meeting was heavily focused on dependency / package mangement in C++, an area that has been getting an increased amount of attention of late in the C++ community.

SG 15 heard a presentation on package consumption vs. development, whose author showcased the Build2 build / package management system and its abilities. Much of the rest of the evening was spent discussing what requirements various segments of the user community have for such a system.

The relationship between SG 15 and the committee is somewhat unusual; actually standardizing a package management system is beyond the committee’s purview, so the SG serves more as a place for innovators in this area to come together and hash out what will hopefully become a de facto standard, rather than advancing any proposals to change the standards text itself.

It was observed that the heavy focus on package management has been crowding out other areas of focus for SG 15, such as tooling related to static analysis and refactoring; it was suggested that perhaps those topics should be split out into another Study Group. As someone whose primary interest in tooling lies in these latter areas, I would welcome such a move.

Next Meetings

The next full meeting of the Committee will be in San Diego, California, the week of November 8th, 2018.

However, in an effort to work through some of the committee’s accumulated backlog, as well as to try to make a push for getting some features into C++20, three smaller, more targeted meetings have been scheduled before then:

  • A meeting of the Library Working Group in Batavia, Illinois, the week of August 20th, 2018, to work through its backlog of wording review for library proposals.
  • A meeting of the Evolution Working Group in Seattle, Washington, from September 20-21, 2018, to iterate on the merged Modules proposal.
  • A meeting of the Concurrency Study Group (with Library Evolution Working Group attendance also encouraged) in Seattle, Washington, from September 22-23, 2018, to iterate on Executors.

(The last two meetings are timed and located so that CppCon attendees don’t have to make an extra trip for them.)

Conclusion

I think this was an exciting meeting, and am pretty happy with the progress made. Highlights included:

  • The entire Ranges TS being on track to be merged into C++20.
  • C++20 gaining standard facilities for contract programming.
  • Important progress on Modules, with a merged proposal that was very well-received.
  • A pivot towards package management, including as a way to make graphical progamming in C++ more accessible.

Stay tuned for future reports from me!

Other Trip Reports

Some other trip reports about this meeting include Bryce Lelbach’s, Timur Doumler’s, and Guy Davidson’s. I encourage you to check them out as well!

Trip Report: C++ Standards Meeting in Jacksonville, March 2018

Summary / TL;DR

Project What’s in it? Status
C++17 See list Published!
C++20 See below On track
Library Fundamentals TS v2 source code information capture and various utilities Published! Parts of it merged into C++17
Concepts TS Constrained templates Merged into C++20 with some modifications
Parallelism TS v2 Task blocks, library vector types and algorithms, and more Sent out for PDTS ballot
Transactional Memory TS Transaction support Published! Not headed towards C++20
Concurrency TS v1 future.then(), latches and barriers, atomic smart pointers Published! Parts of it merged into C++20, more on the way
Executors Abstraction for where/how code runs in a concurrent context Reached design consensus. Ship vehicle not decided yet.
Concurrency TS v2 See below Under development. Depends on Executors.
Networking TS Sockets library based on Boost.ASIO Publication imminent
Ranges TS Range-based algorithms and views Published!
Coroutines TS Resumable functions, based on Microsoft’s await design Published!
Modules TS A component system to supersede the textual header file inclusion model Voted for publication!
Numerics TS Various numerical facilities Under active development
Graphics TS 2D drawing API Under design review; some controversy
Reflection TS Code introspection and (later) reification mechanisms Initial working draft containing introspection proposal passed wording review
Contracts Preconditions, postconditions, and assertions Proposal under wording review, targeting C++20

A few links in this blog post may not resolve until the committee’s post-meeting mailing is published (expected within a few days of April 2, 2018). If you encounter such a link, please check back in a few days.

Introduction

A couple of weeks ago I attended a meeting of the ISO C++ Standards Committee (also known as WG21) in Jacksonville, Florida. This was the first committee meeting in 2018; you can find my reports on 2017’s meetings here (February 2017, Kona), here (July 2017, Toronto), and here (November 2017, Albuquerque). These reports, particularly the Albuquerque one, provide useful context for this post.

With the final C++17 International Standard (IS) having been officially published, this meeting was focused on C++20, and the various Technical Specifications (TS) we have in flight.

C++17

As mentioned, C++17 has been officially published, around the end of last year. The official published version can be purchased from ISO’s website; a draft whose technical content is identical is available free of charge here.

See here for a list of new language and library features in C++17.

The latest versions of GCC and Clang both have complete support for C++17, modulo bugs. MSVC has significant partial support, but full support is still a work in progress.

C++20

C++20 is under active development. A number of new changes have been voted into its Working Draft at this meeting, which I list here. For a list of changes voted in at previous meetings, see my Toronto and Albuquerque reports.

Technical Specifications

In addition to the C++ International Standard, the committee publishes Technical Specifications (TS) which can be thought of experimental “feature branches”, where provisional specifications for new language or library features are published and the C++ community is invited to try them out and provide feedback before final standardization.

The committee recently published four TSes – Coroutines, Ranges, Networking, and most recently, Modules – and several more are in progress.

Modules TS

The last meeting ended with the Modules TS close to being ready for a publication vote, but not quite there yet, as the Core Working Group (CWG) was still in the process of reviewing resolutions to comments sent in by national standards bodies in response to the PDTS (“Proposed Draft TS”) ballot. Determined not to leave the resolution of the matter to this meeting, CWG met via teleconference on four different occasions in between meetings to finish the review process. Their efforts were successful; in particular, I believe that the issues that I described in my last report as causing serious implementer concerns (e.g. the “views of types” issue) have been resolved. The revised document was voted for publication a few weeks before this meeting (also by teleconference).

That allowed the time during this meeting to be spent discussing design issues that were explicitly deferred until after the TS’s publication. I summarize that technical discussion below.

Parallelism TS v2

The Parallelism TS v2 has picked up one last major feature: data-parallel vector types and operations, also referred to as “SIMD”. With that in place, Parallelism TS was sent out for its PDTS ballot.

Concurrency TS v2

The Concurrency TS v2 (no working draft yet) is continuing to take shape. There’s a helpful paper that summarizes its proposed contents and organization.

A notable component of the Concurrency TS v2 that I didn’t mention in my last report is a revised version of future::then() (the original version appeared in the Concurrency TS v1, but there was consensus against moving forward with it in that form). This, however, depends on Executors, which will be published independently of the Concurrency TS v2, either in C++20 or a TS of its own.

Library Fundamentals TS v3

The Library Fundementals TS is a sort of a grab-bag TS for library proposals that are not large enough to get their own TS (like Networking did), but experimental enough not to go directly into the IS. It’s now on its third iteration, with v1 and significant components of v2 having merged into the IS.

No new features have been voted into v3 yet, but an initial working draft has been prepared, basically by taking v2 and removing the parts of it that have merged into C++17 (including optional and string_view); the resulting draft will be open to accept new proposals at future meetings (I believe mdspan (a multi-dimensional array view) and expected<T> (similar to Rust’s Result<T>) are headed that way).

Reflection TS

After much anticipation, the Reflection TS is now an official project, with its initial working draft based on the latest version of the reflexpr static introspection proposal. I believe the extensions for static reflection of functions are targeting this TS as well.

It’s important to note that the Reflection TS is not the end of the road for reflection in C++; further improvements, including a value-based (as opposed to type-based) interface for reflection, and metaclasses, are being explored (I write more about these below).

Future Technical Specifications

There are some planned future Technical Specifications that don’t have an official project or working draft yet:

Graphics

The proposal for a Graphics TS, set to contain 2D graphics primitives with an interface inspired by cairo, continues to be under discussion in the Library Evolution Working Group (LEWG).

At this meeting, the proposal has encountered some controversy. A library like this is unlikely to be used for high-performance production use cases like games and browsers; the target market is more people teaching and learning C++, and non-performance-intensive GUI applications. Some people consider that to be a poor use of committee time (it was observed that a large proposal like this would tie up the Library Working Group for one or two full meetings’ worth of wording review). On the other hand, the proposal’s authors have been “strung along” by the committee for a couple of years now, and have invested significant time into polishing the proposal to be standards-quality.

The committee plans to hold an evening session at the next meeting to decide the future of the proposal.

Executors

Executors are a important concurrency abstraction for which the committee has been trying to hash out a suitable design for a long time. There is finally consensus on a design (see the proposal and accompanying design paper), and the Concurrency Study Group had been planning to publish it in its own Technical Specification.

Meanwhile, it became apparent that several other proposals depend on executors, including Networking (which isn’t integrated with executors in its TS form, but people would like it to be prior to merging it into the IS), the planned improvements to future, and new execution policies for parallel algorithms. Coroutines doesn’t necessarily have a dependency, but there are still integration opportunities.

As a result, the Concurrency Study Group is eyeing the possibility of getting executors directly into C++20 (instead of going through a TS), to unblock dependent proposals sooner.

Merging Technical Specifications into C++20

After a TS has been published and has garnered enough implementation and use experience that the committee is confident enough to officially standardize its contents, it can be merged into the standard. This happened with e.g. the Filesystems and Parallelism TSes in C++17, and significant parts of the Concepts TS in C++20.

As the committee has a growing list of published-but-not-yet-merged TSes, there was naturally some discussion of which of these would be merged into C++20.

Coroutines TS

The Coroutines TS was proposed for merger into C++20 at this meeting. There was some pushback from adopters who tried it out and brought up several concerns (these concerns were subsequently responded to).

We had a lively discussion about this in the Evolution Working Group (EWG). I summarize the technical points below, but the procedural outcome was that those advocating for significant design changes will have until the next meeting to bring forward a concrete proposal for such changes, or else “forever hold their peace”.

Some felt that such a “deadline” is a bit heavy-handed, and I tend to agree with that. While there certainly needs to be a limit on how long we wait for hypothetical future proposals that improve on a design, the Coroutines TS was just published in November 2017; I don’t think it’s unreasonable to ask that implementers and users be given more than a few months to properly evaluate it and formulate high-quality proposals to improve it if appropriate.

Ranges TS

The Ranges TS modernizes and Conceptifies significant parts of the standard library (the parts related to algorithms and iterators).

Its merge into the IS is planned to happen in two parts: first, the foundational Concepts that a large spectrum of future library proposals may want to make use of, and then the range-based algorithms and utilities themselves. The purpose of the split is to allow the first part to merge into the C++20 working draft as soon as possible, thereby unblocking proposals that wish to use the foundational Concepts.

The first part is targeting C++20 pretty firmly; the second part is still somewhat up in the air, with technical concerns relating to what namespace the new algorithms will go into (there was previously talk of a std2 namespace to serve as a place to house new-and-improved standard library facilities, but that has since been scrapped) and how they will relate to the existing algorithms; however, the authors are still optimistic that the second half can make C++20 as well.

Networking TS

There is a lot of desire to merge the Networking TS into C++20, but the dependence on executors makes that timeline challenging. As a best case scenario, it’s possible that executors go into C++20 fairly soon, and there is time to subsequently merge the Networking TS into C++20 as well. However, that schedule can easily slip to C++23 if the standardization of executors runs into a delay, or if the Concurrency Study Group chooses to go the TS route with executors.

The remaining parts of the Concepts TS

The Concepts TS was merged into the C++20 working draft in Toronto, but without the controversial abbreviated function templates (AFTs) feature (and some related things).

I mentioned that there was still a lot of demand for AFTs, even if there was no consensus for them in their Concepts TS form, and that alternative AFT proposals targeting C++20 would be forthcoming. Several such proposals were brought forward at this meeting; I discuss them below. While there wasn’t final agreement on any of them at this meeting, there was consensus on a direction, and there is relative optimism about being able to get AFTs in some form into C++20.

What about Modules?

The Modules TS was just published a few weeks ago, so talk of merging it into the C++ IS is a bit premature. Nonetheless, it’s a feature that people really want, and soon, and so there was a lot of informal discussion about the possibility of such a merge.

There were numerous proposals for post-TS design changes to Modules brought forward at this meeting; I summarize the EWG discussion below. On the whole, I think the design discussions were quite productive. It certainly helped that the Modules TS is now published, and design concerns could no longer be postponed as “we’ll deal with this post-TS”.

I think it’s too early to speculate about the prospects of getting Modules into C++20, but there seems to be a potential path forward, which I describe below as well.

Evolution Working Group

I’ll now write in a bit more detail about the technical discussions that took place in the Evolution Working Group, the subgroup that I sat in for the duration of the week.

Unless otherwise indicated, proposals discussed here are targeting C++20. I’ve categorized them into the usual “accepted”, “further work encouraged”, and “rejected” categories:

Accepted proposals:

  • A couple of minor tweaks to the Coroutines TS: symmetric coroutine transfer, and parameter preview for coroutine promise constructor.
  • Clarifications about the behaviour of contract checks that modify observable (e.g. global) state. The outcome was that evaluating such a contract check constitutes undefined behaviour.
  • Class types in non-type template parameters. This is a long-desired feature, with an example use case being format strings checked at compile-time, and one of the few remaining gaps in the language where user-defined types don’t have all the powers of built-in types. The feature had been blocked on the issue of how to determine the equivalence of two non-type template parameters of class type (which is needed to be able to establish the equivalence of template specializations). Default comparisons finally provided a way forward here; class types used as non-type template parameters need to have a defaulted operator<=> (as do their members).
  • Static reflection of functions. This is an extension to the reflexpr proposal to allow reflecting over functions. You can’t reflect over an overload set; rather, reflexpr can accept a function call expression as an argument, perform overload resolution (without evaluating the call), and reflect the chosen overload. This is targeting the Reflection TS, not C++20.
  • Standard containers and constexpr. This proposal aims to allow the use of dynamic allocation in a constexpr context, so as to make e.g. std::vector usable by constexpr functions. This is accomplished by allowing destructors to be constexpr, and allowing new-expressions and std::allocator to be used in a constexpr context. (The latter is necessary because something like std::vector, which maintains a partially initialized dynamic allocation, can’t be implemented using new-expressions alone. operator new itself isn’t supported, because it loses information about the type of the allocated storage; std::allocator::allocate(), which preserves such information, needs to be used instead.) The proposal as currently formulated does not allow dynamic allocations to “survive” beyond constant experssion evaluation; there will be a future extension to allow this, where “surviving” allocations will be promoted to static or automatic storage duration as appropriate.
  • char8_t: a type for UTF-8 characters and strings. This is a combined core language + library proposal; the language parts include introducing a new char8_t type, and changing the behaviour of u8 character and and string literals to use that type. The latter changes are breaking, though the expected breakage is fairly slight, especially for u8 character literals which are new in C++17 and not heavily used yet.

    Discussion of this proposal centered around the big-picture plan of how UTF-8 adoption will work, and whether we can’t just work towards char itself implying a UTF-8 encoding. Several people argued that that’s unlikely to happen, due to large amounts of legacy code that don’t treat char as UTF-8, and due to the special role of char as an “aliasing” type (where an array of char is allowed to serve as the underlying storage for objects of other types) which prevents compilers from optimizing uses of char the way they could optimize char8_t (which, importantly, would be a non-aliasing type).

    In the end, EWG gave the green-light to the direction outlined in the paper. (There was a brief discussion of pursuing this as a TS, but there was no consensus for this, in part because people felt that if we’re going to change the meaning of u8 literals, we might as well do it now before the C++17 meaning gets a lot of adoption.)
  • explicit(bool). This allows constructors to be declared as “conditionally explicit”, based on a compile-time condition. This is mostly useful for wrapper types like pair or optional, where we want their constructors to be explicit iff. the constructors of their wrapped types are.
  • Checking for abstract class types. This tweaks the rules regarding when attempted use of an abstract type as a complete object is diagnosed, to avoid situations where a class definition retroactively makes a previously declared function that uses the type ill-formed.

There were also a few that, after being accepted by EWG, were reviewed by CWG and merged into the C++20 working draft the same week, and thus I already mentioned them in the C++20 section above:

Finally, EWG decided to pull the previously-approved proposal to allow string literals in non-type template parameters, because the more general facility to allow class types in non-type template parameters (which was just approved) is a good enough replacement. (This is a change from the last meeting, when it seemed like we would want both.) The main difference is that you now have to wrap your character array into a struct (think fixed_string or similar), and use that as your template parameter type. (The user-defined literal part of P0424 is still going forward, with a corresponding adjustment to the allowed template parameter types.)


Proposals for which further work is encouraged:

  • C++ stability, velocity, and deployment plans. This is a proposal for a Standing Document (SD; a less-official-than-a-standard committee document, typically with procedural rather than technical content) outlining the procedure by which breaking changes can be made to C++. It classifies breaking changes by level of detectability (e.g. statically detectable and causes a compiler error, statically detectable but doesn’t cause a compiler error, not statically detectable), and issues guidance for whether and how changes in each category can be made. EWG encouraged the authors to come back with specific wording for the proposed SD.
  • Standard library compatibility promises. This is another proposal for a Standing Document, outlining what compatibility promises the C++ standard library makes to its users, and what kind of future changes it reserves to make. (As an example, the committee reserves the right to add new overloads to standard library functions. This may break user code that tries to take the address of a standard library function, and we want to make it clear that such breakage is par for the course; if you want a guarantee that your code will compile without modifications in future standards, you can only call standard library functions, not take their address.)
  • LEWG wishlist for EWG. This is a wishlist of core language issues that the Library Evolution Working Group would like to see addressed to solve problems facing library authors and users. Some of the items included reining in overeager ADL (see below for a proposal to do just that), making it easier to avoid lifetime errors, dealing with ABI breakage, and finding alternatives for the remaining use cases of macros. EWG encouraged future proposals in these areas, or discussion papers that advance our understanding of the problem (for example, a survey of macro use cases that don’t have non-macro alternatives).
  • Extending the offsetof macro to allow computing the offset to a member given a pointer-to-member variable (currently it requires being given the member’s name). EWG thought this was a valid use case, but expressed a preference for a different syntax rather than overloading the offsetof macro.
  • Various proposed extensions to the Modules TS, which I talk about below.
  • Towards consistency between <=> and other comparison operators. The background to this proposal is that when the <=> operator was introduced, there were a few cases where the specified behaviour was a departure from the corresponding behaviour for the existing two-way comparison operators. These were cases where we would have liked to change the behaviour for the existing operators, but couldn’t due to backwards compatibility considerations. <=>, however, being new to the language, had no such backwards compatibility considerations, so the authors specified the more-desirable behaviour for it. The downside is that this introduced inconsistencies between <=> and the two-way comparison operators.

    This proposal aims to resolve those inconsistencies, in some cases by changing the behaviour of the two-way operators after all. There were five specific areas of change:

    • Sign safety. Today, -1 < 1u evaluates to false due to sign conversion, which is not the mathematically correct result. -1 <=> 1u, on the other hand, is a compiler error. EWG decided that both should in fact work and give the mathematically correct result (which for -1 < 1u is a breaking change, though in practice it’s likely to fix many more bugs than it introduces), though whether this will happen in C++20, or after a longer transition period, remains to be decided.
    • Enum safety. Today, C++ allows two-way comparisons between enumerators of distinct enumerator types, and between enumerators and floating-point values. Such comparisons with <=> are ill-formed. EWG felt they should be made ill-formed for two-way comparisons as well, though again this may happen by first deprecating them in C++20, and only actually making them ill-formed in a future standard. (Comparisons between enumerators and integer values are common and useful, and will be permitted for all comparison operators.)
    • Array safety. Two-way comparisons between operands of array type will be deprecated.
    • Null safety. This is just a tweak to make <=> between a pointer and nullptr return strong_equality rather than strong_ordering.
    • Function pointer safety. EWG expressed a preference for allowing all comparisons between function pointers, and requiring implementers to impose a total order on them. Some implementers indicated they need to investigate the implementability of this on some architectures and report back.
  • Chaining comparisons. This proposes making chains of comparisons, such as a == b == c or a < b <= c, have their expected mathematical meaning (which is currently expressed in C++ in a more cumbersome way, e.g. a == b && b == c). This is a breaking change, since such expressions currently have a meaning (evaluate the first comparison, use its boolean result as the value for the second comparison, and so on). It’s been proposed before, but EWG was worried about the silent breaking change. Now, the authors have surveyed a large body of open-source code, and found zero instances of such expressions where the intended meaning was the current meaning, but several instances where the intended meaning was the proposed meaning (and which would therefore be silently fixed by this proposal). Importantly, comparison chains are only allowed if the comparisons in the chain are either all =, all < and <=, or all > and >=; other chains like a < b > c are not allowed, unlike e.g in Python. In the original proposal, such “disallowed” chains would have retained their current meaning, but EWG asked that they be made ill-formed instead, to avoid confusion. The proposal also contained a provision to have folds over comparisons (e.g. a < ..., where a is a function parameter pack) expand to a chained comparison, but EWG chose to defer that part of the proposal until more implementation experience can be gathered.
  • Size feedback in operator new. This proposes overloads of operator new that return how much memory was allocated (which may be more than what was asked for), so the caller can make use of the entire allocation. EWG agreed with the use case, but had some concerns about the explosion of operator new overloads (each new variation that’s added doubles the number of overloads; with this proposal, it would be 8), and the complications around having the new overloads return a structure rather than void*, and asked the authors to come back after exploring the design space a bit more.
  • The assume_aligned attribute. The motivation is to allow authors to signal to the compiler that a variable holds a value with a particular alignment at a given point in time, for purposes such as more efficient vectorization. The alignment is a property of the variable’s value at a point in time, not of the variable itself (e.g. you can subsequently increment the pointer and it will no longer have that alignment). EWG liked the idea but felt that the proposed semantics about where the attribute could apply (for example, that it could apply to parameter variables but not local variables) were confusing. Suggested alternatives included a magic library function (which would more clearly apply at the time it’s called), and something you can place into a contract check.
  • Fixing ADL. This is a resurrection of a proposal that’s more than a decade old, to fix argument-dependent lookup (ADL). ADL often irks people because it’s too eager, and often finds overloads in other namespaces that you didn’t intend. This proposal to fix it was originally brought forward in 2005, but was deferred at the time because the committee was behind in shipping C++0x (which became C++11); it finally came back now. It aims to make two changes to ADL:
    • Narrow the rules for what makes a namespace an associated namespace for the purpose of ADL. The current rules are very broad; in particular, it includes not only the namespaces of the arguments of a function call, but the namespaces of the template parameters of the arguments, which is responsible for a lot of unintended matches. The proposal would axe the template parameters rule.
    • Even if a function is found in an associated namespace, only consider it a match if it has a parameter matching the argument that caused the namespace to be associated, in the relevant position.

    This is a scary change, because it has the potential to break a lot of code. EWG’s main feedback was that the authors should try implementing it, and test some large codebases to understand the scope of breakage. There were also some concerns about the how the second change would interact with Concepts (and constrained templates in general). The proposal will come back for further review.

  • A proposed language-level mitigation for Spectre variant 1, which I talk about below.
  • Allow initializing aggregates from a parenthesized list of values. This aims to solve a long-standing issue where e.g. vector::emplace() didn’t work with aggregate types, because the implementation of emplace() would do new T(args...), while aggregates required new T{args...}. A library solution was previously proposed for this, but the library groups were unhappy with it because it felt like a workaround for a language deficiency, and it would have had to be applied everywhere in the library where it was a problem (with vector::emplace() being just one example). This proposal fixes the deficiency at the language level. EWG generally liked the idea, though there was also a suggestion that a related problem with aggregate initialization (deleted constructors not preventing it) be solved at the same time. There was also a suggestion that the proposal only apply in dependent contexts (since in non-dependent contexts, you know what kind of initialization you need to use), but that was shot down.
  • Signed integers are two’s complement. The standard currently allows various representations for signed integers, but two’s complement is the only one used in practice, on all modern architectures; this proposal aims to standardize on that, allowing code to portably rely on the representation (and e.g. benefit from hardware capabilities like an arithmetic right shift). EWG was supportive of the idea, but expressed a preference for touching base with WG14 (the C standards committee) to make sure they’re on board with this change. (The original version of this proposal would also have defined the overflow behavior for signed integers as wrapping; this part was rejected in other subgroups and never made it to EWG.)
  • Not a proposal, but the Core Working Group asked EWG whether non-template functions should be allowed to be constrained (with a requires-clause). There are some use cases for this, such as having multiple implementations of a function conditioned on some compile-time condition (e.g. platform, architecture, etc.). However, this would entail some specification work, as the current rules governing overloading of constrained functions assume they are templates, and don’t easily carry over to non-templates. EWG opted not to allow them until someone writes a paper giving sufficient motivation.

Rejected proposals:

  • Supporting offsetof for all classes. offsetof is currently only guaranteed to work for standard-layout classes, but there are some use cases for it related to memory-mapped IO, serialization, and similar low-level things, that require it to work for some classes that aren’t standard-layout. EWG reiterated the feedback it gave on the previous proposal on this topic: to expand the definition of standard-layout to include the desired types. EWG was disinclined to allow offsetof for all classes, including ones with virtual bases, as proposed in this paper; it was felt that this more general goal could be accomplished with a future reflection-based facility.
  • Structured bindings with polymorphic lambdas. This would have allowed a structured binding declaration (e.g. auto [a, b]) as a function parameter, with the semantics that it binds to a single argument (the composite object), and is decomposed into the named consituents on the callee side. EWG sympathized with the goal, but had a number of concerns including visual ambiguity with array declarators, and encouraging the use of templates (and particularly under-constrained templates, until structured bindings are extended to allow a concept in place of auto) where otherwise you might use a non-template.
  • Structured binding declaration as a condition. This would have allowed a condition like if (auto [a, b] = f()), where the condition evaluates to the composite object returned to f() (assuming that object is already usable as a condition, e.g. by having a conversion operator to bool). EWG felt that the semantics weren’t obvious (in particular, people might think one of the decomposed variables is used as the condition). There were also unanswered questions like, in the case of a composite object that uses get<>() calls to access the decomposed variables, whether those calls happen before or after the call to the conversion operator. It was pointed out that you can already use a structured binding in a condition if you use the “if with initializer” form added in C++17, e.g. if (auto [result, ok] = f(); ok), and this is preferable because it makes clear what the condition is. (Some people even expressed a desire for deprecating the declaration-as-condition form altogether, although there was also opposition to that.)

Spectre

No significant meeting of software engineers in the past few months has gone without discussion of Spectre, and this standards meeting was no exception.

Google brought forward a proposal for a language-level mitigation for variant #1 of Spectre (which, unlike variant #2, has no currently known hardware-level mitigation). The proposal allows programmers to harden specific branches against speculation, like so:


  if [[protect_from_speculation(args...)]] (predicate) {
    // use args
  }

args... here is a comma-separated list of one or more variables that are in scope. The semantics is that, if predicate is false, any speculative execution inside the if block treats each of the args as zero. This protects against the exploit, which involves using side channels to recover information accessed inside (misspeculated execution of) the branch at a location that depends on args.

The described semantics can be implemented in assembly; see this llvm-dev post for a description of the implementation approach.

For performance reasons, the proposed hardening is opt-in (as opposed to “harden all branches this way”, although compilers can certainly offer that as an option for non-performance-critical programs), and only as aggressive as it needs to be (as opposed to “disable speculation entirely for this branch”).

The language-level syntax to opt a branch into the hardening remains to be nailed down; the attribute syntax depicted above is one possibility. One complication is that if statements are not the only language constructs that compile down to branches; there are others, including some subtler ones like virtual function dispatch. The chosen syntax should be flexible enough to allow hardening all relevant constructs.

In terms of standardizing this feature, one roadblock is that the C++ standard defines the behavior of programs in terms of an abstract machine, and the semantics of the proposed hardening concern lower-level notions that cannot be described in such terms. As the committee is unlikely to reinvent the C++ abstract machine to allow reasoning about such things as speculative execution in normative wording, it may end up being the case that the syntax of the language construct is described normatively, while its semantics is described non-normatively.

This proposal will return to EWG in a more concrete form at the next meeting. As portably mitigating Spectre is a rather urgent desire in the C++ community, there was some talk of somehow standardizing this feature “out of band” rather than waiting for C++20, though it wasn’t clear what that might look like.

Concepts

EWG had an evening session to discuss proposals related to Concepts, particularly abbreviated function templates (AFTs).

To recap, AFTs are function templates declared without a template parameter list, with concept names used instead of type names in the signature. An example is void sort(Sortable& s);, which is a shorthand for template <Sortable __S> void sort(__S& s);. Such use of a concept name in place of a type name is called a constrained-type-specifier. In addition to parameter types, the Concepts TS allowed constrained-type-specifiers in return types (where the meaning was “the function’s return type is deduced, but also has to model this concept”), and in variable declarations (where the meaning was “the variable’s type is deduced, as if declared with auto, but also has to model this concept”).

constrained-type-specifiers did not make it into C++20 when the rest of the Concepts TS was merged, mostly because there were concerns that you can’t tell apart an AFT from a non-template function without knowing whether the identifiers that appear in the parameter list name types or concepts.

Four proposals were presented at this evening session, which aimed to get AFTs and/or other forms of constrained-type-specifiers into C++20 in some form.

I’ll also mention that the use of a concept name inside a template parameter list, such as template <Sortable S> (which is itself a shorthand for template <typename S> requires Sortable<S>), is called a constrained-parameter. constrained-parameters have been merged into the C++20 working draft, but some of the proposals wanted to make modifications to them as well, for consistency.

Three of the discussed proposals took the approach of a inventing a new syntax for constrained-type-specifiers (and in some cases constrained-parameters) that wasn’t just an identifier, thus syntactically distinguishing AFTs from non-template functions.

  • Concept-constrained auto proposed the syntax auto<Sortable>. The proposal as written concerned variable declarations only, but one could envision extending this to other uses of constrained-type-specifiers.
  • An adjective syntax for concepts proposed Sortable typename S as an alternative syntax for constrained-parameters, with a possible future extension of Sortable auto x for constrained-type-specifiers. The idea is that the concept name is tacked, like an adjective, onto the beginning of what you’d write without concepts.
  • Concepts in-place syntax proposed Sortable{S} for constrained-parameters, and Sortable{S} s for constrained-type-specifiers (where S would be an additional identifier the declaration introduces, that names the concrete type deduced for the parameter/variable). You could also write Sortable{} s if you didn’t want/need to name the type. One explicit design goal of this proposal was that if, in the future, the committee changes its mind about AFTs needing to be syntactically distinguishable from non-template functions (because we get more comfortable with them, or are happy to rely more on tooling to tell them apart), the empty braces could be dropped altogether, and we’d arrive precisely at the Concepts TS syntax.

An additional idea that was floated, though it didn’t have a paper, was to just use the Concepts TS syntax, but add a single syntactic marker, such as a bare template keyword before the function declaration (as opposed to per-parameter syntactic markers, as in the above proposals).

Of these ideas, Sortable{S} had the strongest support, with “Concepts TS syntax + single syntatic marker” coming a close second. The proponents of these ideas indicated that they will try to collaborate on a revised proposal that can hopefully gain consensus among the entire group.

The fourth paper that was discussed attacked the problem from a different angle: it proposed adopting AFTs into C++20 without any special syntactic marker, but also changing the way name lookup works inside them, to more closely resemble the way name lookup works inside non-template functions. The idea was that, perhaps if the semantics of AFTs are made more similar to non-template functions (name lookup is one of the most prominent semantic differences between template and non-template code), then we don’t need to syntactically distinguish them. The proponents of having a syntactic marker did not find this a convincing argument for adopting AFTs without one, but it was observed that the proposed name lookup change might be interesting to explore independently. At the same time, others pointed out similarities between the proposed name lookup rules and C++0x concepts, and warned that going down this road would lead to C++0x lookup rules (which were found to be unworkable).

(As an aside, one topic that seems to have been settled without much discussion was the question of independent resolution vs. consistent resolution; that is, if you have two uses of the same concept in an AFT (as in void foo(Number, Number);), are they required to be the same concrete type (“consistent”), or two potentially different types that both model the concept (“independent”). The Concepts TS has consistent resolution, but many people prefer independent resolution. I co-authored a paper arguing for independent resolution a while back; that sentiment was subsequently reinforced by another paper, and also in a section of the Sortable{S} proposal. Somewhat to my amusement, the topic was never actually formally discussed and voted on; the idea of independent resolution just seemed to slowly, over time, win people over, such that by this meeting, it was kind of treated as a done deal, that any AFT proposal going into C++20 will, in fact, have independent resolution.)

Coroutines

As mentioned above, EWG had a discussion about merging the Coroutines TS into C++20.

The main pushback was due to a set of concerns described in this paper (see also this response paper). The concerns fell into three broad categories:

  • Performance concerns. As currently specified, coroutines perform a dynamic allocation to store the state that needs to be saved in between suspensions. The dynamic allocation can be optimized away in many cases, but it was argued that for some use cases, you want to avoid the dynamic allocation by construction, without relying on your optimizer. An analogy can be made to std::vector: sure, compilers can sometimes optimize the dynamic allocation it performs to be a stack allocation, but we still have stack arrays in the language to guarantee stack allocation.

    One particularly interesting use case that motivates this performance guarantee, is using coroutines to implement a form of error handling similar to Rust’s try! macro / ? operator. The general idea is to hook the coroutine customization points for a type like expected<T> (the proposed C++ analogue of Rust’s Result), such that co_await e where e has type expected<T> functions like try!(e) would in Rust (see the paper for details). However, no one would contemplate using such an error handling mechanism if it didn’t come with a guarantee of not introducing a dynamic allocation.
  • Safety concerns. The issue here is that reference parameters to a coroutine may become dangling after the coroutine is suspended and resumed. There is a desire to change the syntax of coroutines to make this hazard more obvious.
  • Syntax concerns. There are several minor syntactic concerns related to the choice of keywords (co_await, co_yield, and co_return), having to use co_return instead of plain return, and the precedence of the co_await operator. There is a suggestion to address these by replacing co_await with a punctuation-based syntax, with both prefix and postfix forms for better composition (compare having both * and -> operators for pointer dereferencing).

The paper authors plan to bring forward a set of modifications to the Coroutines TS that address these concerns. I believe the general idea is to change the syntax in such a way that you can explicitly access / name the object storing the coroutine state. You can then control whether it’s allocated on the stack or the heap, depending on your use case (e.g. passing it across a translation unit boundary would require allocating it on the heap, similar to other compiler-generated objects like lambdas).

EWG expressed interest in seeing the proposed improvements, while also expressing a strong preference for keeping coroutines on track to be merged into C++20.

Modules

EWG spent an entire day on Modules. With the Modules TS done, the focus was on post-TS (“Modules v2”) proposals.

  • Changing the term “module interface”. This paper argued that “module interface” was a misnomer because a module interface unit can contain declarations which are not exported, and therefore not conceptually part of the module’s interface. No functional change was proposed. EWG’s reaction was “don’t care”.
  • Modules: dependent ADL. The current name lookup rules in the Modules TS have the consequence that argument-dependent lookup can find some non-exported functions that are declared in a module interface unit. This proposal argued this was surprising, and suggested tightening the rules. EWG was favourable, and asked the author to come back with a specific proposal.
  • Modules: context-sensitive keyword. This proposed making module a context-sensitive keyword rather than a hard keyword, to avoid breaking existing code that uses module as an identifier. The general approach was that if a use of module could legally be a module declaration, it is, otherwise it’s an identifier. EWG disliked this direction, because the necessary disambiguation rules were too confusing (e.g. two declarations that were only subtly different could differ in whether module was interpreted as a keyword or an identifier). It was suggested that instead an “escape mechanism” be introduced for identifiers, where you could “decorate” an identifier as something like __identifier(module) or @module to keep it an identifier. It was also pointed out that adopting relevant parts of the “Another take on modules” proposal (see below) would make this problem moot by restricting the location of module declarations to a file’s “preamble”.
  • Unqualified using declarations. This proposed allowing export using name;, where name is unqualified, as a means of exporting an existing name (such as a name from an included legacy header). EWG encouraged exploration of a mechanism for exporting existing names, but wasn’t sure this would be the right mechanism.
  • Identifying module source code. This requires that any module unit either start with a module declaration, or with module; (which “announces” that this is a module unit, with a module declaration to follow). The latter form is necessary in cases where the module wants to include legacy headers, which usually can’t be included in the module’s purview. This direction was previously approved by EWG, and this presentation was just a rubber-stamp.
  • Improvement suggestions to the Modules TS. This paper made several minor improvement suggestions.
    • Determining whether an importing translation unit sees an exported type as complete or incomplete, based on whether it was complete or incomplete at the end of the module interface unit, rather than at the point of export. This was approved.
    • Exporting the declaration of an inline function should not implicitly export the definition as well. There was no consensus for this change.
    • Allow exporting declarations that don’t introduce names; an example is a static_assert declaration. Exporting such a declaration has no effect; the motivation here is to allow enclosing a group of declarations in export { ... }, without having to take care to move such declarations out of the block. This was approved for static_assert only; EWG felt that for certain other declarations that don’t introduce names, such as using-directives, allowing them to be exported might be misleading.
    • A tweak to the treatment of private members of exported types. Rejected because private members can be accessed via reflection.

That brings us to what I view as the most significant Modules-related proposal we discussed: Another take on modules (or “Atom” for short). This is a proposal from Google based on their deployment experience with Clang’s implementation of Modules; it’s a successor to previous proposals like this one. It aims to make several changes – some major, some minor – to the Modules TS; I won’t go through all of them here, but they include changes to name lookup and visibility rules, support for module partitions, and introducing the notion of a “module preamble”, a section at the top of a module file that must contain all module and import declarations. The most significant change, however, is support for modularized legacy headers. Modularized legacy headers are legacy (non-modular) headers included in a module, not via #include as in the Modules TS, but via import (as in import "file" or import <file>). The semantics is that, instead of textually including the header contents as you would with an #include, you process them as an isolated translation unit, produce a module interface artefact as-if it was a module (with all declarations exported, I assume), and then process the import as if it were an actual module import.

Modularized legacy headers are primarily a transition mechanism for incrementally modularizing a codebase. The proposal authors claim that without them, you can’t benefit from compile-time improvements of Modules in a codebase (and in fact, you can take a compile time hit!) unless you bottom-up modularize the entire codebase (down to the standard library and runtime library headers), which is viewed as infeasible for many large production codebases.

Importantly, modularized legacy headers also offer a way forward in the impasse about whether Modules should support exporting macros. In the Atom proposal, modularized legacy headers do export the macros they define, but real modules do not. (There is an independent proposal to allow real modules to selectively export specific macros, but for transition purposes, that’s not critical, since for components that have macros as part of their interface, you can just use them as a modularized legacy header.)

There was some discussion of whether the Atom proposal is different enough from the Modules TS that it would make sense to pursue it as a separate (competing) TS, or if we should try to integrate the proposed changes into the Modules TS itself. The second approach had the stronger consensus, and the authors plan to come back with a specific proposed diff against the Modules TS.

It’s too early to speculate about the impact of pursuing these changes on the schedule for shipping Modules (such as whether it can be merged into C++20). However, one possible shipping strategy might be as follows (disclaimer: this is my understanding of a potential plan based on private conversation, not a plan that was approved by or even presented to EWG):

  • Modules v1 is the currently shipping Modules TS. It is not forward-compatible with v2 or v3.
  • Modules v2 would be a modified version of v1 that would not yet support modularized legacy headers, but would be forward-compatible with v3. Targeting C++20.
  • Modules v3 would support modularized legacy headers. Targeting post-C++20, possibly a second iteration of the Modules TS.

Such a way forward, if it becomes a reality, would seem to satisfy the concerns of many stakeholders. We would ship something in the C++20 IS, and people who are able to bottom-up modularize their codebases can start doing so, without fear of further breaking changes to Modules. Others who need the power of modularized legacy headers can wait until Modules v3 to get it.

I’m pretty happy with the progress made on Modules at this meeting. With the Atom proposal having been discussed and positively received, I’m more optimistic about the feature than I have been for the past few meetings!

Papers not discussed

With the meeting being fairly heavily focused on large proposals like Concepts, Modules, and Coroutines, there were a number of others that EWG didn’t get a chance to look at. I won’t list them all (see the pre-meeting mailing for a list), but I’ll call out two of them: feature-test macros are finally on the formal standards track, and there’s an revised attempt to tackle named arguments in C++ that’s sufficiently different from previous attempts that I think it at least might not be rejected out of hand. I look forward to having these, and the other proposals on the backlog, discussed at the next meeting.

Other Working Groups

Library Groups

Having sat in EWG all week, I can’t report on technical discussions of library proposals, but I’ll mention where various proposals are in the processing queue.

I’ve already listed the library proposals that passed wording review and were voted into the C++20 working draft above.

A few proposals targeting Technical Specifications also passed wording review and were merged into the relevant TS working drafts:

The following proposals are still undergoing wording review:

The following proposals have passed design review and await wording review at future meetings:

The following proposals are still undergoing design review:

In addition, there is a fairly long queue of library proposals that haven’t started design review yet. See the committee’s website for a full list of proposals.

Finally, I’ll mention that the Library Evolution Working Group had a joint evening session with SG 14 (Low Latency Programming) to discuss possible new standard library containers in C++20. Candidates included a fixed capacity vector, a vector with a small object optimization, ring buffer, colony, and slot map; the first three had the greatest support.

Study Groups

SG 6 (Numerics)

SG 6 met for a day, and reviewed a number of numerics-related proposals. In addition to the “signed integers are two’s complement” proposal that later came to EWG, it looked at several library proposals. Math constants, constexpr for <cmath> and <cstdlib>, letting strong_order truly be a customization point, and interpolation were forwarded to LEWG (in some cases with modifications). More better operators and floating point value access for std::ratio remain under discussion. Safe integral comparisons have been made moot by operator<=> (the proposal was “abducted by spaceship”).

SG 7 (Compile-Time Programming)

SG 7, the Compile-Time Programming (previously Reflection) Study Group, met for an evening session and reviewed three papers.

The first, called constexpr reflexpr, was an exploration of what the reflexpr static introspection proposal might look like formulated in terms of value-based constexpr programming, rather than template metaprogramming. SG 7 previously indicated that this is the direction they would like reflection proposals to take in the longer term. The paper was reviewed favourably, with encouragement to do further work in this direction. One change that was requested was to make the API value-based rather than pointer based. Some implementers pointed out that unreflexpr, the operator that takes a meta-object and reifies it into the entity it represents, may need to be split into multiple operators for parsing purposes (since the compiler needs to know at parsing time whether the reified entity is a value, a type, or a template, but the meta-object passed as argument may be dependent in a template context). Finally, some felt that the constexpr for facility proposed in the paper (which bears some resemblance to the previously-proposed tuple-based for loop) may be worth pursuing independently.

The second was a discussion paper called “What do we want to do with reflection?” It outlines several basic / frequently requested reflection use cases, and calls for facilities that address these use cases to be added to C++20. SG 7 observed that one such facility, source code information capture, is already shipping in the Library Fundamentals TS v2, and could plausibly be merged into C++20, but for the rest, a Reflection TS published in the 2019-2020 timeframe is probably the best we can do.

The third was an updated version of the metaclasses proposal. To recap, metaclasses are compile-time transformations that can be applied to a class definition, producing a transformed class (and possibly other things like helper classes / functions). At the last meeting, SG 7 discussed how a metaclass should be defined, and decided on it operating at the “value level” (where the input and output types are represented as meta-objects, and the metaclass itself is more or less just a constexpr function). At this meeting, SG 7 focused on the invocation syntax: how you apply a metaclass to your class. The syntax that appeared to have the greatest consensus was class<interface> Foo { ... }; (where interface is an example metaclass name).

SG 15 (Tooling)

This week was the inaugural meeting of the new Tooling Study Group (SG 15), also in an evening session.

Unsurprisingly, the meeting was well attended, and the people there had many, many different ideas for how C++ tooling could be improved, ranging from IDEs, through refactoring and code analysis tools, to build systems and package managers. Much of the meeting was spent trawling through this large idea space to try to narrow down and focus the group’s scope and mission.

One topic of discussion was, what is the best representation of code for tools to consume? Some argued that the source code itself is the only sufficiently general and powerful representation, while others were of the opinion that a more structured, easy-to-consume representation would be useful, e.g. because it would avoid every tool that consumes it being (or containing / invoking) a C++ parser. It was pointed out that the “binary module interface” representation that module files compile into may be a good representation for tools to consume, and we may want to standardize it. Others felt that instead of standardizing the representation, we should standardize an API for accessing it.

In the space of build systems and package managers, the group recognized that building “one build system” or “one package manager” to rule them all is unlikely to happen. Rather, a productive direction to focus efforts might be some sort of protocol that any build or package system can hook into, and produce some sort of metadata that different tools can consume. Clang implementers pointed out that compilation databases are a primitive form of this, but obviously there’s a lot of room for improvement.

In the end, the group articulated a mission: that in 10 years’ time, it would like the C++ community to be in a state where a “compiler-informed” (meaning, semantic-level) code analysis tool can run on a significant fraction of open-source C++ code out there. This implies having some sort of metadata format (that tells the tool “here’s how you run on this codebase”) that a significant enough fraction of open-source projects support. One concrete use case for this would be the author of a C++ proposal that’s a breaking change, to run a query on open-source projects to see how much breakage the change would cause; but of course the value of such infrastructure / tooling goes far beyond this use case.

It’s a fair question to ask what the committee’s role is in all this. After all, the committee’s job is to standardize the language and its libraries, and not peripheral things like build tools and metadata formats. Even the binary module interface format mentioned above couldn’t really be part of the standard’s normative wording. However, a format / representation / API could conceivably be published in the form of a Standing Document. Beyond that, the Study Group can serve as a place to coordinate development and specification efforts for various peripheral tools. Finally, the Standard C++ Foundation (a nonprofit consortium that contributes to the funding of some commitee meetings) could play a role in funding critical tooling projects.

New Study Group: SG 16 (Unicode)

The committe has decided to form a new study group for Unicode and Text Handling. This group will take ownership of proposals such as std::text and std::text_view (types for representing text that know their encoding and expose functions that operate at the level of code points and grapheme clusters), and other proposals related to text handling. The first meeting of this study group is expected to take place at a subsequent committee meeting this year.

Conclusion

I think this was a productive meeting with good progress made on many fronts. For me, the highlights of the meeting included:

  • Tackling important questions about Modules, such as how to transition large existing codebases, and what to do about macros.
  • C++20 gaining foundational Concepts for its standard library, with the rest of the Ranges TS hopefully following soon.
  • C++20 gaining a standard calendar and timezone library
  • An earnest design discussion about Coroutines, which may see an improved design brought forward at the next meeting.

The next meeting of the Committee will be in Rapperswil, Switzerland, the week of June 4th, 2018. Stay tuned for my report!

Other Trip Reports

Some other trip reports about this meeting include Vittorio Romeo’s, Guy Davidson’s (who’s a coauthor of the 2D graphics proposals, and gives some more details about its presentation), Bryce Lelbach’s, Timur Doumler’s, Ben Craig’s, and Daniel Garcia’a. I encourage you to check them out as well!

Featured Song: Octavarium

I mentioned in my last Featured Song post that I’ve been dabbling in progressive metal. That dabbling led to my (re-)discovery of Dream Theater, one of the genre’s defining bands.

I say re-discovery, because I had listened to some Dream Theatre over a decade ago, including to some of their most popular songs at the time, “Pull Me Under” and “Metropolis” (both from their 1992 album Images and Words). I somewhat liked them, but also found them somewhat tedious / boring, and ultimately wasn’t motivated to check out more of their work.

More recently, however, a comment on the “Elysium” video I featured last time prompted me to check out some of their more recent albums, such as Octavarium (2005) and Systematic Chaos (2007), and, thanks to the evolution of both their style and my tastes over time, my impression was quite different: this stuff is great!

The title track of Octavarium, in particular, captivated me immediately, and that is what I’m featuring today.

At 24 minutes (26 in the live performance I’m linking to), this is the longest song I’ve featured to date (I promise, they will get shorter going forward!), and yet I do not find this song tedious at all – each part of it is different and interesting in its own right, and contributes to a very satisfying whole.

One particularly notable passage from this song is the extended fingerboard intro. While I’m not generally a huge fan of synthetic sounds, this passage in this piece is really well placed, and sets the atmosphere for the rest of the song perfectly.

My favourite part of the song, though, is “Intervals” (beginning at around 16:42 in the linked video) – the slow build-up of tension that leads to the screamed “TRAPPED INSIDE THIS OCTAVARIUM” lines – and the climactic sequence / dénouement that follows and takes you to the end of the song.

(What is an “octavarium”, you ask? The only prior use of the term that I could find was in the name of a liturgical book, but the root word is “octave“, and if you listen to the lyrics, the notion of cycles and the end being the beginning (a property which musical octaves have) comes up repeatedly – so I interpret “trapped inside this octavarium” as meaning “trapped in a cycle you can’t break out of”.)

My one complaint about progressive metal is that some of the extended keyboard / guitar solos (such as, in this song, the ones in the 2-3 minutes leading up to the “Intervals” section), while being technically challenging and intricate, lack some of the “interestingness” (for lack of a better word) of similar solos in power metal. For example, DragonForce‘s guitar solos, while being every bit as fast and technically intricate as Dream Theater’s, also have a sense of “movement” that the latter seem to lack. I think this is what I disliked about older Dream Theater songs like “Pull Me Under”, and I like “Octavarium” so much because it covers a lot of other stylistic ground.

Without further ado, I invite to you enjoy this live performance of “Octavarium”:



As one might expect from a song of this length, there is a lot of speculation / discussion of exactly what meaning it intends to convey. If you’re interested in that, or just want to see the lyrics, check out its SongMeanings page.

Control Flow Visualizer (CFViz): an rr / gdb plugin

rr (short for “record and replay”) is a very powerful debugging tool for C++ programs, or programs written in other compiled languages like Rust1. It’s essentially a reverse debugger, which allows you to record the execution of a program, and then replay it in the debugger, moving forwards or backwards in the replay.

I’ve been using rr for Firefox development at Mozilla, and have found it to be enormously useful.

One task that comes up very often while debugging is figuring out why a function produced a particular value. In rr, this is often done by going back to the beginning of the function, and then stepping through it line by line.

This can be tedious, particularly for long functions. To help automate this task, I wrote – in collaboration with my friend Derek Berger, who is learning Rust – a small rr plugin called Control Flow Visualizer, or CFViz for short.

To illustrate CFViz, consider this example function foo() and a call site for it:

example code

With the CFViz plugin loaded into rr, if you invoke the command cfviz while broken anywhere in the call to foo() during a replay, you get the following output:

example output

Basically, the plugin illustrates what path control flow took through the function, by coloring each line of code based on whether and how often it was executed. This way, you can tell at a glance things like:

  • which of several return statements produced the function’s return value
  • which conditional branches were taken during the execution
  • which loops inside the function are hot (were executed many times)

saving you the trouble of having to step through the function to determine this information.

CFViz’s implementation strategy is simple: it uses gdb’s Python API to step through the function of interest and see which lines were executed in what order. In then passes that information to a small Rust program which handles the formatting and colorization of the output.

While designed with rr in mind, CFViz also works with vanilla gdb, with the limitation that it will only visualize the rest of the function’s execution from the point where it was invoked (since, without rr, it cannot go backwards to the function’s starting point).

I’ve found CFViz to be quite useful for debugging Firefox’s C++ code. Hope you find it useful too!

CFViz is open source. Bug reports, patches, and other contributions are welcome!

Footnotes

1. rr also has a few important limitations: it only runs on Intel CPUs, and only on Linux (although there is a similar tool called Time-Travel Debugging for Windows)

A response to “Net Neutrality. No big deal.”

I recently watched this video titled “Net Neturality. No big deal.” by Bryan Lunduke.

I watched this video because, while I am in favour of Net Neutrality, and concerned about the impending repeal of Net Neutrality regulations in the United States, I consciously try to avoid being stuck in an echo chamber of similar views, and try to expose myself to opposing viewpoints as well; particularly when such opposing viewpoints are held by a figure in a community that I respect and identify with (in Bryan’s case, the Linux and free software community).

I found the video interesting and well-presented, but I didn’t find Bryan’s arguments convincing. I decided to write this post to respond to two of the arguments Bryan makes in particular.

The first argument I wanted to address was about the fact that some of the largest companies that are lobbying to keep Net Neutrality rules in place – Google, Netflix, and Microsoft – are also supporters of DRM. Bryan argues, that since these companies support DRM, which is a threat to a free and open internet, we should not take their support for Net Neutrality (which they claim to also be motivated by a desire for a free and open internet) at face value; rather, they only support Net Neutrality regulations because they have a financial incentive for doing so (namely, they run high-bandwidth streaming and similar services that are likely to be first in line to be throttled in a world without Net Neutrality protections).

I don’t dispute that companies like Google, Netflix, and Microsoft support Net Neutrality for selfish reasons. Yes, the regulations affect their bottom line, at least in the short term. But that doesn’t mean there aren’t also good reasons for supporting Net Neutrality. Many organizations – like the Electronic Frontier Foundation, whom I have no reason to suspect to be beholden to the pocketbooks of large tech companies – have argued that Net Neutrality is, in fact, important for a free and open internet. That Netflix supports it for a different reason, doesn’t make that any less the case.

I also think that comparing DRM and the lack of Net Neutrality in this way confuses the issue. Yes, both are threats to a free and open internet, but I think they are qualitatively very different.

To explain why, let’s model an instance of communication over the internet as being between two parties: a sender or producer of the communication, and its receiver or consumer. DRM exists to give the producer control over how the communication is consumed. There are many problems with DRM, but at least it is not intended to interfere with communication in cases where the producer and consumer agree on the terms (e.g. the price, or lack thereof) of the exchange1.

By contrast, in a world without Net Neutrality rules, an intermediary (such as an ISP) can interfere with (such as by throttling) communication between two parties even when the two parties agree on the terms of the communication. This potentially opens the door to all manner of censorship, such as interfering with the communications of political activists. I see this as being a much greater threat to free communication than DRM.

(I also find it curious that Bryan seems to focus particularly on the standardization of DRM on the Web as being objectionable, rather than DRM itself. Given that DRM exists regardless of whether or not it’s standardized on the Web, the fact that it is standardized on the Web is a good thing, because it enables the proprietary software that implements the DRM to be confined to a low-privilege sandbox in the user’s browser, rather than having “full run of the system” as pre-standardization implementations of DRM like Adobe Flash did. See this article for more on that topic.)

The second argument Bryan makes that I wanted to address was that Net Neutrality rules mean the U.S. government being more involved in internet communications, such as by monitoring communications to enforce the rules.

I don’t buy this argument for two reasons. First, having Net Neutrality rules in place does not mean that internet communications need to be proactively monitored to enforce the rules. The role of the government could very well be limited to investigating and corrections violations identified and reported by users (or organizations acting on behalf of users).

But even we assume there will be active monitoring of internet communications to enforce the rules, I don’t see that as concerning. Let’s not kid ourselves: the U.S. government already monitors all internet communications it can get its hands on; axing Net Neutrality rules won’t cause them to stop. Moreover, users already have a way to protect the content of their communications (and, if desired, even the metadata, using tools like Tor) from being monitored: encryption. Net Neutrality rules don’t change that in any way.

In sum, I enjoyed watching Bryan’s video and I always appreciate opposing viewpoints, but I didn’t find the arguments that Net Neutrality is not a big deal convincing. For the time being, I continue to believe that the impending rollback of U.S. Net Neutrality rules is a big deal.

Footnotes

1. I am thinking here of cases where the content being communicated is original content, that is, content originated by the producer. I am, of course, aware, that DRM can and does interfere with the ability of two parties to communicate content owned by a third party, such as sending a movie to a friend. To be pedantic, DRM can even interfere with communication of original content in cases where such content is mistakenly identified as belonging to a third party. I’m not saying DRM is a good thing – I’m just saying it doesn’t rise to being the same level of threat to free communication as not having Net Neutrality protections does.