Category Archives: trip-reports

Trip Report: C++ Standards Meeting in Lenexa, May 2015

Summary / TL;DR

 

Project What’s in it? Status
C++14 C++14 Published!
C++17 Various minor features. More substantial features under consideration include default comparisons and operator . On track for 2017
Networking TS Sockets library based on Boost.ASIO In late stages of design review
Filesystems TS Standard filesystem interface Published!
Library Fundamentals TS I optional, any, string_view, and more Voted out for publication!
Library Fundamentals TS II source code information capture, array_view, and more Expected 2016 or 2017
Array Extensions TS Old proposals (arrays of runtime bound (ARBs) and dynarray) abandoned. New direction being explored: a magic type that acts like an ARB and can be used as a local variable only.
Parallelism TS Parallel versions of STL algorithms Voted out for publication!
Concurrency TS improvements to future, latchesand barriers, atomic smart pointers Voted out for balloting by national standards bodies
Transactional Memory TS transaction support Voted out for publication!
Concepts (“Lite”) TS constrained templates In process of addressing comments from national standards bodies. Publication expected late 2015.
Reflection Code introspection and (later) reification mechanisms Still in the design stage, no ETA
Graphics 2D drawing API Standard wording expected to be ready for review at the next meeting.
Modules A component system to supersede the textual header file inclusion model Microsoft and Clang continuing to iterate on their implementations and converge on a design. Ship vehicle (C++17 or a TS) uncertain.

Introduction

Last week I attended a meeting of the ISO C++ Standards Committee in Lenexa, Kansas. This was the first committee meeting in 2015; you can find my reports on 2014’s meetings here (February, Issaquah), here (June 2014, Rapperswil), and here (November, Urbana-Champaign). These reports, particularly the Urbana one, provide useful context for this post.

The focus of this meeting was iterating on the various ambitious proposals in progress, and beginning to form an idea of which of them will be ready for C++17. In addition, several of the Technical Specifications (TS) in flight have reached their final stages in the committee, while work continues on others.

C++14

C++14 was officially published as an Internal Standard by ISO in December 2014. Its timely publication is evidence that the committee’s plan for adopting a three-year publication cycle post-C++11 has, thus far, been successful.

C++17

What Will Make It In?

When the committee originally announced, shortly after C++11 was published, the schedule for the next two standard revisions, C++14 was described as a “minor revision”, and C++17, a “major” one.

A few things have happened since then:

  • C++14 ended up being not so minor, with fairly substantial features like generic lambdas and variable templates making it in.
  • For features not going through the TS process, there’s no intrinsic reason the three years between C++14 and C++17 would be any more or less productive for the committee than the three years between C++11 and C++14.
  • For features going through the TS process, that process has in some cases been taking somewhat longer than initially envisioned, and thus it’s not clear whether they will be published in time – and particularly, if there will be enough time after their publication to get more implementation and use experience with them – to merge them into C++17. I would say the following TS’s stand a chance of being merged into C++17:
    • Filesystems, which has already been published
    • Library Fundamentals I, Parallelism, and Transactional Memory, which were voted for publication at this meeting
    • Concepts, which is expected to be voted for publication in a post-meeting telecon, or at the next meeting (in October in Kona, Hawaii) at the latest

    However, I don’t think any of these mergers is a given. For example, there is some opposition to merging the Filesystems TS into the Standard in its current form on the basis that its design isn’t a good fit for certain less common varieties of filesystems. Concepts also has some pending design issues that may be revisited prior to a merger. In all cases, feedback from implementors and users will be key and will likely decide the TS’s fate.

As a result, I think it’s likely that C++17 will end up being approximately as “major” or “minor” of a revision as C++14 was.

Modules making it into C++17 is less likely. Due to the significance of the change that they bring to the language, there have been calls – including by the authors of one of the two current modules implementations – to pursue the feature initially as a Technical Specification, giving the design the opportunity to be hardened via implementation and use experience. If this avenue is pursued, the timing effectively rules out such a TS being published and merged into the IS by C++17. That said, from a user’s perspective, I don’t think this is any reason to despair: the feature is still likely to become available to users in the 2017 timeframe, even if it’s in the form of a TS.

Reflection is still very much in the design stage, and SG 7 indicated a preference for a TS as the initial ship vehicle. C++17 is practically out of the question.

(I talk about modules and reflection in more detail below, for those interested.)

That said, there are features not going through the TS process which are expected to be in C++17.

What Has Made It In?

My Urbana report lists the language and library features that have made it into C++17 as of the last meeting.

No new language features were voted into C++17 this meeting, in the sense of standard wording for them being merged into the C++17 working draft (a small, language lawyer-ish change, making exception specifications part of the type system came close, but was deferred to Kona due to some concerns brought up during the closing plenary session). However, there are numerous language features in the design or wording review stages that are expected to be voted into C++17 in upcoming meetings; I talk about these in the “Evolution Working Group” section below.

There were some small library features voted into C++17 at this meeting:

Technical Specifications

Procedural changes

There has been a change to ISO’s procedure for publishing a Techincal Specification since the last meeting. The process used to involve two rounds of balloting by national standards bodies, called PDTS (Preliminary Draft TS) and DTS (Draft TS). Recently, the DTS round has been dropped, leaving just the PDTS round, and making for a more agile TS publication process.

Crossed the finish line!

As a result of this procedural change, some TS’s which had successfully finished their PDTS ballots became eligible to be voted out for publication at this meeting, and after addressing PDTS ballot comments, vote them out we did! Library Fundamentals I, Transactional Memory, and Parallelism I have all been sent to ISO for official publication, which should happen within a few months.

A couple of other TS’s haven’t quite crossed the finish line yet, but are very close.

Concepts

The Concepts TS garnered a rich and opinionated collection of PDTS ballot comments. Among them were your usual editorial and wording-level technical comments, but also some design-level comments which were brought before the Evolution Working Group (EWG) for consideration.

It’s rare for the committee to make design changes to a TS or Standard at such a late stage in the publication cycle, and indeed most design-level comments were deferred (meaning, they will not be addressed in this version of the TS, but they could be revisited in a future version, or if the TS comes up for merging into the Standard). One comment, however, which was essentially a request for a (small) feature, was approved. The feature will allow using a concept name as type name in a variable declaration:

ConceptName var = expression;

The semantics is that the type of var is deduced from the type of expression (much like with auto), but the code is ill-formed if the deduced type does not satisfy the concept.

I was mildly surprised that EWG was willing to approve this addition at this late stage, but pleasantly so: I think this feature is very useful. To top off the good news, Andrew Sutton (the primary author of the Concepts TS), who couldn’t make it to the meeting itself, reported only two days later that he added support this feature in his GCC-based Concepts TS implementation! (Implementation experience is immensely valuable for guiding the committee’s decisions, because issues with a proposal feature often come up during implementation.)

As a result of this new addition, and a multitude of wording-level comments, the Core Working Group (CWG) didn’t have time to prepare final wording for the Concepts TS by the end of the meeting, so it couldn’t be voted out for publication just yet. Rather, CWG plans to hold a post-meeting teleconference to (hopefully) complete the final wording, after which the options are to hold a committee-wide teleconference to vote it out for publication, or to wait until Kona to vote on it.

Either way, the Concepts TS is practically at the brink of completion! Very exciting.

There’s also good news on the implementation side: GCC front-end developer Jason Merrill says Andrew’s Concepts implementation is expected to merge into mainline GCC within a month or so. Meanwhile, IBM, who have switched to using clang as the front-end for their newer products, announced their intention to kick off a clang-based implementation.

Concurrency I

Good progress here, too: the first Concurrency TS was sent out for its PDTS ballot! Assuming a successful ballot, it should be ready to be voted for publication in Kona.

Evolution Working Group

As usual, I spent most of the meeting in the Evolution Working Group, which does design review for proposed language features. EWG once again had a full plate of proposals to look at.

Recapping from previous posts, the outcome of an EWG design review is one of the following:

  • Approved. The proposal is approved without design changes. They are sent on to the Core Working Group (CWG), which revises them at the wording level, and then puts them in front of the committee at large to be voted into whatever IS or TS they are targeting.
  • Further Work. The proposal’s direction is promising, but it is either not fleshed out well enough, or there are specific concerns with one or more design points. The author is encouraged to come back with a modified proposal that is more fleshed out and/or addresses the stated concerns.
  • Rejected. The proposal is unlikely to be accepted even with design changes.

Here’s how this meeting’s proposals fared:

Accepted:

  • A proposal to make emplace_back() and similar play nice with aggregate types. Note that this is a library proposal for which EWG input was solicited, so it was sent to the Library Working Group (LWG) rather than CWG.
  • Microsoft’s resumable functions (a.k.a. coroutines) proposal. More about this in the “Coroutines” section below.
  • A proposal to make exception specifications part of the type system. This resolves a long-standing issue where exception specifications sort-of contribute to a function’s type but not quite, and as such their handling in various contexts (passing template arguments, conversions between function pointer types, and others) requires special rules.
  • A minor change to the semantics of inheriting constructors, which makes the semantics more intuitive and consistent with inheritance of other members.
  • A proposal for inline variables, which are basically static storage duration variables (either at namespace scope, or static data members) whose definition can appear inline, and can be defined in a header. This can already be accomplished using a static variable defined locally in an inline function, this proposal just exposes that semantics under a more straightforward syntax. The proposal had some minority opposition (on the basis that it reuses the keyword inline for a purpose somewhat unrelated to its existing use, and that it encourages the use of static storage duration variables to begin with), but otherwise had fairly strong support and was accepted.
  • A proposal to remove the keyword register, but reserve it for future use.

Further work:

  • A tweak to the folding expressions feature added last meeting, which would restrict the set of operators for which a unary fold with an empty parameter pack is valid. Support for this hinges on defining an identity element for the affected operators, but it’s not clear that such a notion is sensible in the presence of operator overloading. For example, consider the following function:
        template <typename... Strings>
        auto concatenate(Strings... strings)
        {
          return strings + ...;
        }
    

    With the current rules, when this function is called with 1 or more strings, it returns the concatenation of its arguments (because strings overload operator + to do concatenation), but when called with no arguments, it returns the integer 0, because that’s defined as the identity element for the addition operator.

    The proposal in question would make it ill-formed to call this function with no arguments; if the author wants that to work, their recourse is to change the unary fold to the binary fold strings + ... + "".

    There was consensus that addition, multiplication, and bitwise operators should be treated this way, but others were more contentious. For example, it was argued that for the logical operators && and ||, you shouldn’t be overloading them to return things that aren’t bool anyways, so the identities true and false remain appropriate.

    A particularly interesting case is the comma operator, for which the specified identity is void(). Comma folds are expected to be common as a technique to emulate statement folding, as in the following example:

        template <typename... Functions>
        auto call_all_functions(Functions... functions)
        {
          functions() , ...;  // expands to function1() , function2() , functionN();
        }
    

    On the one hand, it would be a shame to make people write functions , ... , void() every time they want to do this, and the comma operator generally shouldn’t be overloaded anyways, so keeping the void() identity should be reasonable. On the other hand, if people want to do statement folding, perhaps the language should allow them to do that directly, rather than relying on the comma operator to emulate it with expression folding.

    As there was no consensus on the precise course of action, the matter was referred for further work.

  • The latest modules proposal from the Microsoft folks. More about this in the “Modules” section below.
  • The alternative coroutines proposal that I presented (the author being absent). More about this in the “Coroutines” section below.
  • Contract programming. Two main proposals were discussed, with different flavours. One of them provided an assert-like facility, to be used inside function bodies, primarily for the purpose of runtime checking. The other (there was also a third very similar to it) proposed a syntax for declaring preconditions, postconditions, and invariants for a function in its interface (i.e. in its declaration), primarily for the purpose of static analysis and enabling compiler optimizations. There was consensus that both sets of goals, and both places for writing contracts (interfaces and implementations) are desirable, but there was some debate about whether the proposals should be considered as orthogonal and allowed to proceed independently, or whether the authors should collaborate and come up with a unified proposal that satisfies both use cases. In the end, the direction was to strive for a unified proposal.
  • Default comparisons. Bjarne presented the latest version of his proposal for automatically generating comparison operators for class types. The main features of the proposal are (1) that it’s opt-out, meaning you get the operators by default but you can declare them as = delete if you don’t want them; and (2) it’s carefully designed to avoid breaking existing code as follows: for any comparison operator call site, if name lookup under current rules finds a user-declared operator, it will continue finding that operator under the new rules, rather than using any auto-generated one. The proposal had strong concensus, which was a (welcome) surprise after the lack of consensus on earlier versions (and other proposals in the area) at the previous two meetings. It came close to being approved and sent to CWG, but some details of the semantics remained to be hashed out, so Bjarne was asked to return with an updated proposal in Kona.
    There was another proposal related to comparisons, which pointed out that we are currently waving our hands about matters such as different types of equality and different types of order (total, weak, partial, etc.). To rectify this, it proposed using named functions (e.g. one for each kind for order) instead of operators for comparisons. The feedback was that such machinery is useful to have, but we also want reasonable defaults which are spelt == and <, and as such, the proposed set of named functions can be developed independently of Bjarne’s proposal.
  • A proposal to extend aggregate initialization to be allowed for types with base classes in cases where the base classes are default-constructible. EWG’s feedback was to revise the proposal to also address the use case of providing values to initialize the base classes with in cases where they are not default-constructible.
  • Unified call syntax. This proposal, by Bjarne, seeks to unify the member (x.f(y)) and non-member (f(x, y)) call syntaxes by allowing functions of either kind to be invoked by syntax of either kind. The approach is to have the x.f(y) syntax look for member functions first, and fall back to looking for non-member functions only if the member lookup yields no results; conversely, f(x, y) would look for non-member functions first, and fall back to a member lookup. The resulting semantics are asymmetric (they don’t make x.f(y) and f(x, y) completely interchangeable), but fully backwards compatible. (This design was one of several alternatives Bjarne presented at the previous meeting, and it seemed to have the greatest chance for gaining consensus, primarily due to its backwards compatibility.)

    Beyond aesthetics (“I prefer my function calls to look this way”) and tooling reasons (“member call syntax gives me IntelliSense”), the primary motivation for this feature is facilitating generic proramming, which is expected to become more popular than ever with Concepts. When defining requirements on a template parameter type, either informally in today’s C++ or explicitly with Concepts, you currently have to choose whether the operations on the type are expressed as member of non-member functions. Either choice constrains users of your template: if you choose member functions, they can’t adapt third-party types that they can’t modify to model your concept; if you choose non-member functions, they will likely have to provide a lot of non-member adapters for types that would otherwise automatically model your concept. You could choose to allow either one (this is what C++11 does with the “range” concept used in the range-based for loop: the required operation of getting an iterator to the first element of the range can be spelt either begin(range) or range.begin()), but then your call sites become very unreadable because you need a lot of template/SFINAE magic to express “call X if it exists, otherwise call Y”. A unified call syntax would allow template implementers to use whichever call syntax they like, while users of the template can use either member functions or non-member functions to model the concepts, as they desire / are able to. (C++0x Concepts had a feature called “concept maps” which solved this problem by acting as a bridge between the operations in a concept definition (which is what generic code would use) and the actual operations on a type modelling the concept. However, concept maps were removed from later Concepts designs because they proved very tricky to specify and implement.)

    Unfortunately, this is a very risky change to make to the language. While the proposal itself doesn’t break any existing code, new code that takes advantage of the proposal (that is, code that invokes a non-member function via a member call syntax, or vice versa) is considerably more prone to breakage. For example, adding a new member function to a class can easily break user code which was calling a non-member function of that name via the member function syntax; this breakage can manifest as a compiler error, or as a silent behaviour change, depending on the situation.

    A lot of the finer points of the proposed semantics remain to be nailed down as well. How does the fallback mechanism work – is it activated only if the initial lookup doesn’t find any results, or also if it finds results but they’re all, say, SFINAE’d out? What is the interaction with two-phase name lookup? What happens when the member call syntax is used on an incomplete type?

    EWG was very divided on this proposal; consensus seemed fairly far off. Some people suggested changes to the proposal that would allay some of their concerns with it; one of them was to have classes opt-in to unified call syntax, another to restrict the set of non-member functions that can be found via a member call syntax to those found by ADL. Bjarne said that he intends to continue iterating on the idea.

  • A proposal for overloading operator dot. This would allow creating “smart references”, much as the ability to overload operator -> gives us smart pointers, as well enable many other patterns that take advantage of interface composition. The proposal was generally very well-received; one feature that was somewhat controversial was the ability to declare multiple “overloads” of operator dot that return different types, with the effect of bringing the interfaces of both types into the scope of the declaring type (much how multiple inheritance from the two types would). The author (also Bjarne) was asked to come back with standard wording.
  • A proposal to allow template argument deduction for constructors. The idea here is to avoid having to define factory functions for templated types, such as make_pair(), for the sole purpose of not having to explicitly write out the template argument types in a constructor call of the form pair<T, U>(t, u); the proposal would allow simply pair(t, u).This proposal has been on the table for a while, but it’s been plagued by the problem that for a lot of classes, deduction based on existing constructors wouldn’t work. For example, if a class container<T> has a constructor that takes arguments of type container<T>::iterator, that type is a non-deduced context, so T could not be deduced from a constructor call of the form container(begin, end). The latest version addresses this by allowing class authors to optionally define “canonical factory functions” that define how the class’ template parameters are deduced from constructor arguments. Here’s how one might look like (the syntax is hypothetical):
    template <typename Iter>
    container(Iter a, Iter b) 
        -> container<typename iterator_traits<Iter>::value_type>;

    This basically says “if container is constructed from two iterators, the class’ template parameter is the value type of those iterators”. The question of where to place such a declaration came up; EWG favoured placing it at namespace scope, so as to allow third parties to provide them if desired.

    Another point that was brought up was that a constructor call of the form classname(arguments) where classname is a template class, already has a meaning inside the scope of classname: there, classname without any template arguments means “the current instantiation” (this is called injected class names in standard terminology). The proposal needs to specify whether such a constructor would change meaning (i.e. deduction would be performed instead) or not. The consensus was to try to perform deduction, and fall back to the current instantiation if that fails; this would technically be a breaking change, but the hope is that the scope of any breakage would be minor.

    Overall, the proposal had strong support and is expected to move forward.

  • A proposal to allow a template to have a non-type template parameter whose type is deduced. EWG expressed preference to the syntax template <auto x> and encouraged the author to continue iterating on the idea.
  • A restricted form of static_if; the restrictions are that (1) it can only be used at local scope, (2) each branch of it forms a scope of its own, and (3) non-dependent constructs need to be well-formed in both branches. The proposal was well-received, and the author will continue working on it. It was noted that the Concepts TS doesn’t currently allow evaluating a concept outside a requires-expression, so something like static_if (ConceptName<T>) wouldn’t necessarily work, but hopefully that restriction will be lifted in the near future.
  • Extending static_assert to allow taking for the error message not just a string literal, but any constant expression that can be converted to a string literal. The idea is to allow performing compile-time string formatting to obtain the error message.
  • noexcept(auto), which basically means “deduce the noexcept-ness of this function from the noexcept-ness of the functions it calls. Like return type deduction, this requires the body of the function being available in each translation unit that uses the function. It was brought up that, together with the proposal for making exception specifications part of the type system, this would mean that modifying the function’s body could change the function’s type (again similarly to return type deduction), but people weren’t overly concerned about that.

Rejected:

  • A proposal for allowing return type deduction for explicitly-defaulted and -deleted special member functions. This was rejected because the group realized that it would introduce a footgun: a copy or move assignment operator with an auto return type would return by value!
  • No-op constructors and destructors, which are basically a language hack that would allow certain library optimizations; a previous attempt at enabling said optimizations, destructive move, was presented at the previous meeting. EWG’s feedback was much the same as last time: though it’s dressed differently, the proposal is still an attempt to mess with the language’s lifetime rules, which people are extremely wary of doing. The proposal as written will not move forward, but Chandler Carruth (Google’s lead clang developer) had some ideas about how to allow the desired optimizations by other means, and will discuss them with the author.
  • A proposal for relaxing the rules for forming an array type declarator to allow omitting a dimension anywhere; this would allow forming types such as int[][3][][7], though not instantiating them. The author was seeking to write a multi-dimensional array class where each dimension could be determined statically or dynamically, and use a type of this form as a template parameter and interpret it as a description for which dimensions were static. EWG didn’t find this motivation compelling (the information can be conveyed by other means, such as Array<int, Dimensions<0, 3, 0, 7>>) and was generally wary of adding to the set of types that can be formed but not instantiated (an existing example of such a type is a function type whose return type is a function type).
  • A proposal for generalized dynamic assumptions. EWG liked the use cases, but felt it would make more sense as part of a unified contracts proposal than a stand-alone feature, as contracts also need a syntax to express assumptions.
  • Allowing goto in constexpr functions. The intent here was to plug gratuitous holes between what can be done in a constant expression, and what can be done in regular code. EWG liked the motivation, but preferred to see it together with proposals that plug other holes, such as using lambdas and virtual functions in constant expressions. At least one of those (lambdas) is expected to be proposed in Kona. (Bjarne jokingly wondered whether some day people would be proposing launching threads in a constexpr function.)
  • Delayed evaluation parameters, which is a proposal for evaluating function arguments in a lazy rather than eager fashion (i.e. only evaluating them when their value is needed inside the function, not before calling the function). EWG was intrigued by the idea, but the proposal wasn’t nearly fleshed out enough to be considered as a conrete proposal. Interested people are encouraged to continue exploring the design space.
  • A revised proposal to allow arrays of runtime bound as data members wasn’t looked at specifically, but its presence on the agenda prompted a broader discussion about the Arrays TS, which I talk about in the “Arrays TS” section below.

Modules

There are two groups currently working on modules: Microsoft, in their compiler, and Google, in clang (extending Apple’s clang modules implementation for C and Objective-C, to C++). Microsoft has a draft proposal based on their design; Google hasn’t submitted a proposal based on their design yet.

The two designs differ slightly in philosophy. Microsoft’s design feels like what modules might have looked like if they were part of C++ from the beginning. It’s clean, and promises to be a good fit for new code written in a modular fashion. Google’s design, on the other hand, is geared towards making it possible to incrementally modularize large existing codebases without requiring a significant refactoring or other major changes (at least in the absence of egregiously non-modular design patterns). In other words, Microsoft’s design is more idealistic and pure, and Google’s is more practical.

Most notably, Microsoft’s design essentially requires modularizing a codebase from the bottom-up. For example, if a component of your program uses the C++ standard library, then modularizing that component requires first modularizing the C++ standard library; if the C++ standard library in turn uses the C standard library, then that too must be modularized (which is particularly unfortunate, for two reasons: (1) C standard library headers tend to be notoriously difficult to modularize due to their use of the preprocessor, and (2) they need to remain consumable by non-C++ code). Google’s design, on the other hand, specifically allows modular code to include non-modular code, so you could modularize your program component without having to modularize the C++ standard library.

To be sure, this feature of Google’s design introduces significant implementation complexity. (In my Rapperswil report, I reported that Google claimed their modules implementation was complete. I now understand what they meant was their implementation of a subset of the design that did not include this feature was complete.) I don’t have a deep understanding of the technical issues involved, but from what I’ve gathered, the difficulty is taking multiple copies of entities defined by the same non-modular code included in different modules and “merging” them to view them as a single entity.

There are other differences between the two designs, too. For example, Google’s allows exporting macros from a module, while Microsoft’s does not. Google’s design also supports cyclic dependencies between module interfaces, resolved by forward-declaring an entity from one module in the interface of the other; Microsoft’s proposal has no such support.

EWG spent half a day (and folks from the two groups additional time offline) discussing and trying to reconcile these design differences. The outcome was relatively hopeful about reaching convergence. The Microsoft folks conceded that some abilities, such as forward declaring entities from another module, are necessary. The Google folks conceded that some abilities geared towards making modules work with existing codebases, such as allowing the export of macros, don’t have to be supported directly by the language (they could be handled by compiler flags and such). The two groups agreed to produce a combined design paper for Kona.

In terms of a ship vehicle, the Google folks characterized modules as “the feature with the single greatest implementation impact so far”, and expressed a strong preference for going through a Technical Specification; this route would effectively rule out modules being in C++17, though as a TS the feature would still likely be available to users in the 2017 timeframe. That said, no official decision has been made yet regarding a ship vehicle.

Coroutines

You may recall from my Urbana report that the outcome of the coroutines discussion there was that two flavours of coroutines, stackful and stackless (see the report for an explanation of the distinction) were sufficiently different and both sufficiently motivated by use cases that they deserved to be developed as independent proposals, with a small minority favouring trying to unify them.

Since Urbana there has been progress in all of these directions, with four papers coming back for consideration at this meeting: an iteration on the stackless proposal, an iteration on the stackful proposal, and two different attempts to unify the two approaches. EWG looked at two of these.

The stackless proposal, called “resumable functions” and championed by Microsoft, is the most mature one. It has already gone through numerous rounds of review in SG 1 (the Concurency Study Group), and is close to the stage where standard wording for it can be written. Its discussion in EWG mostly concerned details such as what to call the traits and functions involved in the proposal (there was no consensus to change from the current coroutine_ prefix), whether deducing that a function is resumable by the presence of await expressions in its body without annotating the declaration with a keyword like resumable is implementable (implementers agreed that it was as long as return statements in such a function were spelt differently), and whether yield is a reasonable keyword to standardize (consensus was that it was not, and so we’re going to get keywords prefixed with co- such as coyield and coreturn instead). Ultimately, the proposal author was given the go-ahead to write standard wording and go to CWG.

The other proposal EWG looked at was one of the attempts to unify stackful and stackless coroutines, called resumable expressions. I presented this paper because the author, Chris Kohlhoff, couldn’t make it to the meeting and I was somewhat familiar with the topic as a result of corresponding with him. Unlike resumable functions, this proposal was in the early design stage. The premise was that you could “have your cake and eat it too” by leveraging the compiler’s ability to analyze your code to avoid annotating calls to resumable functions at every level the way you have to do with await (the weakness of stackless coroutines compared to stackful), while still only requiring the allocation of as much memory as you need (the advantage of stackless over stackful). The problem was that the compiler analysis can’t see past translation unit boundaries, thus still requiring annotations there. There were also concerns about the performance cross-translation unit calls compared to resumable functions; Chris was convinced that it was no slower than resumable functions, but unfortunately I didn’t have a sufficiently good comparative understanding of the implementation models to successfully argue this point. The final opinion on the proposal was divided: some people saw imagination in it, and wanted to see it developed further; others didn’t appreciate the fact that a competing proposal to resumable functions was brought up at such a late stage, risking the standardization of the latter.

You might ask how it even makes sense for resumable functions to be sent to CWG without resumable expressions being categorically rejected. The answer is twofold; first, it’s looking like resumable functions will target a Technical Specification rather than C++17, which means there’s room for competing proposals to be developed in parallel. Second, even if it were targeting the standard, it’s conceivable that multiple kinds of coroutines can co-exist in the language (certainly in Urbana the consensus was the stackful and stackless coroutines should coexist). In any case, Chris plans to attend the Kona meeting and presumably present and updated version of the resumable expressions proposal.

The other two papers (the stackful one and a different unification attempt) were only looked at briefly by SG 1, as the author (same person for both) wasn’t present.

Arrays TS

The Array TS, which contains a language feature called “arrays of runtime bound” (ARBs) that’s essentially a toned-down version of C’s variable-length arrays (VLAs), and a library class dynarray for wrapping such a thing into a container interface, has been in limbo for the past year, as attempts to implement dynarray ran into difficulties, and proposals trying to replace it with something implementable got shot down one after the other.

At this meeting, EWG reviewed the status quo and addressed the question of what will happen to the Arrays TS going forward.

The status quo is this:

  • Many people want simple stack arrays. Emphasis on simple (no “making it a member of a class”) and stack (no “it might be on the stack or it might be on the heap” business.)
  • Some people want to be able to wrap such things into a class interface, so it knows its size, and doesn’t automatically decay to a pointer.
  • Some people additionally want to be able to copy this class and make it a member of other classes.
    • Implementation attempts have essentially demonstrated that this latter thing is impossible.

Given this state of affairs, the features currently in the Arrays TS are not going to be accepted in their current form; EWG recommended stripping the TS of its current contents, and waiting for a workable proposal to come along.

A promising direction for such a workable proposal is to have a “magic type” that acts like an ARB but knows its size and does not decay to a pointer (the implementable features that people wanted from a class wrapper). The type in question could only used for a local variable, and the underlying ARB itself wouldn’t be exposed. Several people expressed an interest in collaborating on a proposal in this direction.

Libary / Library Evolution Working Groups

With all the exciting action in EWG, I didn’t have much of a chance to follow progress on the library side in any detail, but here’s what I’ve gathered during the plenary sessions.

Note that I listed the library features accepted into C++17 at this meeting in the “C++17” section above.

The following proposals were accepted into the second Library Fundamentals TS:

The following proposals failed to gain consensus:

  • A proposal to include certain special math functions which have been standardized independently, into C++17. The primary objection was the cost to implementors for what was perceived by some as a relatively niche user base.
  • Multidimensional bounds, offset and array_view was proposed for acceptance into Library Fundamentals II, but was voted down over issues that still remain to be addressed.

The list of proposals still under review is very long, but here are some highlights:

  • Eric Niebler’s suggested design for customization points was reviewed favourably; Eric was encouraged to experiment more with the idea and come back.
  • A proposal for nothrow-swappable traits was reviewed favourably, and the author was given guidance to put forward all of the traits mentioned in the paper.
  • A lot of time was spent reviewing a proposal for a variant class; as one of the few “vocabulary types” still missing from the standard library, this is considered very important. A lot of the discussion centred around whether variant should have an empty state, and if not, how to deal with the scenario where during assignment, the copy constructor of the right-hand side object throws. boost::variant deals with this by incurring heap allocation, which is widely considered undesirable. I believe the prevailing consensus was to have an empty state, but only allow it to arise in this exceptional situation (pun intended), and make accessing the variant in its empty state (other than assigning a new value to it) undefined behaviour; this way, ordinary code isn’t burdened with having to worry about or check for the empty state.
  • LEWG is continuing to review the Networking TS based on Boost.ASIO.

Ranges

I split out ranges into its own section because I believe it deserves special mention.

As I described in my Urbana report, Eric Niebler came to that meeting with a detailed and well fleshed-out design for ranges in the standard library. It was reviewed very favourably, and Eric was asked to “conceptify” it – meaning express the concepts defined by the proposal using the features of the Concepts TS – and develop the result into a TS. This TS would form part of an “STLv2” refresh of the standard library which wouldn’t be subject to strict backwards-compatibility constraints with the current standard library.

Eric did not delay in doing so: he came back in Lenexa with a conceptified proposal written up as a draft TS. LEWG began a design review of this proposal, and made good progress on it; they hope to complete the review during a post-meeting teleconference and then forward the proposal to LWG.

Study Groups

Note that some Study Groups whose proposals have progressed to the main working groups and which don’t plan on looking at new proposals, are no longer considered active – for example, SG 2 (Modules) is in this category, as the modules proposal is now in front of EWG. I talk a bit about the ones still active below.

SG 1 (Concurrency)

With the Parallelism TS published and the Concurrency TS sent out for its PDTS ballot, SG 1 has been working on the second iterations of both TS’s.

Here are some of the proposals being considered for Parallelism II:

And for Concurrency II:

There are numerous other proposals in front of the SG as well which don’t have a target ship vehicle yet.

SG 5 (Transactional Memory)

SG 5 has accomplished commendable work by publishing the Transactional Memory TS, but they aren’t quite calling it quits! They plan to meet again in Kona to discuss proposals for possible future standardization in the area.

SG 6 (Numerics)

SG 6 met and looked at three specific proposals:

The first, as I described in the “Library” section above, was unsuccessfully proposed for acceptance into C++17.

The other two are being considered for a Numerics TS, along with a number of other proposals not specifically discussed at this meeting, such as unbounded-precision integer types, rational numbers, fixed-point arithmetic, decimal floating-point support, various utility functions (GCD/LCM, constexpr bitwise operations), random numbers (a couple of different proposals), and other topics for which there aren’t concrete proposals yet. (Obviously, not everything in this list will be in a single Numerics TS.)

Said Numerics TS is still in a relatively early stage; a working draft is not yet expected in Kona.

SG 7 (Reflection)

SG 7 had a busy evening session where they looked at a number of reflection-related proposals:

  • A language feature for argument stringization (misleadingly titled “parameter stringization”), attempting to obsolete one of the few remaining uses for the preprocessor. The proposal was well-received; of the various syntactic options presented, SG 7 preferred the approach of adding an annotation to a function declaration that makes the stringized arguments corresponding to all parameters available in the function body under a syntax such as function_name.param<N>()
  • Potential improvements to source-code information capture. Two specific improvements were discussed: the ability to query the offset relative to the start of the file (in addition to a line number + offset relative to the source of the line), which meet with approval, and fine-grained intrinsics (e.g. querying the line number and the function name separately), for which SG 7 recommended waiting until the original proposal has implementation experience.
  • A proposal for language features to facilitate writing test code; consensus was that this topic is not sufficiently relevant to reflection, and should be pursued elsewhere (such as in EWG).
  • An updated version of a detailed, comprehensive proposal for static reflection (see also a slightly shorter paper outlining use cases). When the original version was presented in Urbana, it was given very short shrift, mostly because it had no presenter, and no one had had time to study it in any amount of detail. This time around, participants seemed to be more informed about it, and ended up being rather favourable to the overall approach. Most notably, the use of a “magic operator” (spelled mirrored(entity) in the proposal) rather than templates (such as reflect<entity> as the previously-discussed type property queries proposal did) opens the doors to reflecting more kinds of entities, such as typedefs (as distinct from their underlying types), uninstantiated templates, and namespaces, which SG 7 viewed as valuable. Interest in further development of the proposal was expressed.
  • Another reflection proposal, type reflection via variadic template expansion. Due to time constraints, this could only be presented very briefly. SG 7 expressed interest in a comparative analysis of the expressive power of this proposal compared to the “type property queries” and “static reflection” proposals.

Notably absent from the agenda was the latest version of the “type property queries” proposal, which had appeared to be the leading comprehensive reflection proposal in the past few meetings. The main reason it wasn’t presented was that the author couldn’t make it to the meeting, though one could also argue that SG 7 was already familiar with the overall design (the changes since the last version having been only minor), so time was better spent looking at proposals with alternative designs that still needed analysis.

SG 10 (Feature Test)

SG 10 is continuing to maintain the list of feature testing macros and keep it up to date with new things like C++17 features.

They also have a proposal for some new preprocessor features to aid feature detection: a __has_include() predicate for testing for the presence of an include, and a __has_cpp_attribute() predicate for testing for support for a particular attribute.

SG 12 (Undefined Behaviour)

SG 12 is still active, but did not meet this week as its members were busy advancing other proposals.

SG 13 (I/O)

SG 13 did not meet this week; the author of the 2D graphics proposal plans to complete standard wording for it for Kona.

*NEW* SG 14 (Game Development & Low-Latency Applications)

C++ aims to be a “zero-overhead abstraction” language, where you don’t pay for what you don’t use. It does a formidable job at this, but for some communities of C++ users, it could do an even better job.

The big sticking points are exceptions and RTTI, two language features for which you pay a bit even if you don’t use them. Projects concerned about this overhead commonly use compiler flags like -fno-exceptions and -fno-rtti, but the committee views these as nonconforming and doesn’t give them a lot of consideration. As a result, for example, a lot of standard library features require exceptions, and are therefore unusable in these projects.

There is also desire for more guarantees from the standard library, such as “empty containers never allocate” or “strings employ the small-string optimization”.

EWG looked at a a wishlist containing the above and more, and recommended creating a new Study Group to explore the area further.

The new Study Group, SG 14, will tentatively be called “Game Development & Low-Latency Applications”, because these are the most impacted communities, and the communities from which the committee hopes to get significant input.

There is a tentative plan for SG 14 to meet independently of the committee at CppCon and GDC, the idea being that members of the game development community are more likely to be able to make it to those events than to committee meetings.

Next Meeting

The next meeting of the Committee will be in Kona, Hawaii, the week of October 19th, 2015.

Conclusion

On the whole, this was a very exciting meeting! My highlights:

  • The Concepts TS being very close to publication. Concepts has been the feature I’ve been waiting for the most; I think it will revolutionize generic programming and allow C++ users to unleash the power of templates in unprecedented ways.
  • C++17 starting to take shape. While it’s uncertain whether some larger pieces like modules and coroutines will make it in, I think it will still have a good complement of features. Among other things, the success of the most recent default comparisons proposal, after the lack of consensus that plagued the ones that came before, is very encouraging – a good success story for the standards process.
  • Modules being on track to be available in the 2017 timeframe, even if it’s uncertain whether it will be part of C++17 itself or in the form of a Technical Specification.

Things are only going to get more interesting as C++17 starts to take more concrete shape, and more Technical Specifications cross the finish line. Stay tuned!

Author’s Note (2015-06-10): In the original version of this post, I made a claim about Modules not making it into C++17 that sounded authoritative, when in fact it was largely speculation. I have edited the post to clarify that no official decision has yet been made about the ship vehicle for Modules. My apologies for this mischaracterization, and my thanks to Gabriel Dos Reis for pointing it out.

Advertisements

Trip Report: C++ Standards Meeting in Urbana-Champaign, November 2014

Summary / TL;DR

Project Status
C++14 Finalized and approved, will be published any day now
C++17 Some minor features so far. Many ambitious features are being explored. for (e : range) was taken out.
Networking TS Sockets library based on Boost.ASIO moving forward
Filesystems TS On track to be published early 2015
Library Fundamentals TS Contains optional, any, string_view and more. No major changes since last meeting. Expected 2015.
Library Fundamentals TS II Follow-up to Library Fundamentals TS; will contain array_view and more. In early stage, with many features planned.
Array Extensions TS Continues to be completely stalled. A new proposal was looked at but failed to gain consensus.
Parallelism TS Progressing well. Expected 2015.
Concurrency TS Progressing well. Expected 2015. Will have a follow-up, Concurrency TS II.
Transactional Memory TS Progressing well. Expected 2015.
Concepts (“Lite”) TS Progressing well. Expected 2015.
Reflection Looking at two different proposals. Too early to say anything definitive.
Graphics 2D Graphics TS based on cairo moving forward
Modules Microsoft and Clang have implementations at various stages of completeness. They are iterating on it and trying to converge on a design.
Coroutines Proposals for both stackless and stackful variants will be developed, in a single TS.

Introduction

Last week I attended another meeting of the ISO C++ Standards Committee at the Univeristy of Illinois at Urbana-Champaign. This was the third and last Committee meeting in 2014; you can find my reports on the previous meetings here (February 2014, Issaquah) and here (June 2014, Rapperswil). These reports, particularly the Rapperswil one, provide useful context for this post.

The focus of this meeting was moving forward with the various Technical Specifications (TS) that are in progress, and looking ahead to C++17.

C++14

C++14 was formally approved as an Internal Standard in August when it passed its final ballot (the “DIS”, or Draft International Standard, ballot; see my Issaquah report for a description of the procedure for publishing a new language standard).

It will take another few weeks for ISO to publish the approved standard; it’s expected to happen before the end of the year.

C++17

Strategic Vision

With C++14 being approved, the Committee is turning its attention towards what its strategic goals are for the next revision of the language standard, C++17.

As I explained in my Rapperswil report, most major new features are targeted for standardization in two steps: first, as a Technical Specification (TS), an experimental publication vehicle with no backwards-compatibility requirements, to gain implementation and use experience; and then, by incorporation into an International Standard (IS), such as C++17.

Therefore, a significant amount of the content of C++17 is expected is to consist of features being published as Technical Specifications in the near future. It’s not immediately clear which TS’s will be ready for inclusion in C++17; it depends on when the TS itself is published, and whether any concerns about it come up as it’s being implemented and used. Hopefully, at least the ones being published over the next year or so, such as Filesystems, Concepts, Parallelism, Library Fundamentals I, and Transactional Memory, are considered for inclusion in C++17.

In addition, there are some major features that do not yet have a Technical Specification in progress which many hope will be in C++17: namely, Modules and Reflection. Due to the size and scope of these features, it is increasingly likely that the committee will deem it safer to standardize these as TS’s first as well, rather than targetting them directly at C++17. In this case, there may not be time for the additional step of gaining experience with the TS and merging it into the IS in time for C++17; however, it’s too early to know with any confidence at this point.

Minor Features

That said, C++17 will certainly contain some language and library features, and some smaller ones have already made it in. I mentioned a few in my Rapperrswil report, but some new ones came out of this meeting:

  • Language features
    • The most notable and exciting feature in my books is folding expressions. These give you the ability to expand a parameter pack over a binary operator. For example, if Args is a non-type parameter pack of booleans, then Args &&... is a new expression which is the ‘and’ of all the booleans in the pack. All binary operators support this; for operators that have a logical identity element (e.g. 0 for addition), an empty pack is allowed and evaluates to that identity.
    • Another notable change was not an addition, but a removal: the terse form of the range-based for loop, for (elem : range) (which would have meant for (auto&& elem : range)), was removed. (Technically, it was never added, because the C++ working draft was locked for additions in Rapperswil while the C++14 DIS ballot was in progress. However, there was consensus in the Evolution and Core Working Groups in Rapperswil to add it, and there was wording ready to be merged to the working draft as soon as the ballot concluded and it was unlocked for C++17 additions. That consensus disappeared when the feature was put up for a vote in front of full committee in Urbana.) The reason for the removal was that in for (elem : range), there is no clear indication that elem is a new variable being declared; if there already is a variable named elem in scope, one can easily get confused and think the existing variable is being used in the loop. Proponents of the feature pointed out that there is precedent for introducing a new name without explicit syntax for declaring it (such as a type) in generalized lambda captures ([name = init](){ ... } declares a new variable named name), but this argument was not found convincing enough to garner consensus for keeping the feature.
    • std::uncaught_exceptions(), a function that allows you to determine accurately whether a destructor is being called due to stack unwinding or not. There is an existing function, std::uncaught_exception() (note the singular) that was intended for the same purpose, but was inaccurate by design in some cases, as explained in the proposal. This is considered a language feature even though it’s exposed as a library function, because implementing this function requires compiler support.
    • Attributes for namespaces and enumerators. This fills a grammatical hole in the language, where most entities could have an attribute attached to them, but namespaces and enumerators couldn’t; now they can.
    • A shorthand syntax for nested namespace definition.
    • u8 character literals.
    • A piece of official terminology, “forwarding references”, was introduced for a particular use of rvalue references. Some educators have previously termed this use “universal references”, but the committee felt the term “forwarding references” was more accurate.
    • Allowing full constant expressions in non-type template arguments. This plugs a small hole in the language where the template arguments for certain categories of non-type template parameters were restricted to be of a certain form without good reason.
  • Library features

Evolution Working Group

As usual, I spent most of my time in the Evolution Working Group (EWG), which concerns itself with the long-term evolution of the core language. In spite of there being a record number of proposals addressed to EWG in the pre-Urbana mailing, EWG managed to get through all of them.

Incoming proposals were categorized into three rough categories:

  • Accepted. The proposal is approved without design changes. They are sent on to the Core Working Group (CWG), which revises them at the wording level, and then puts them in front of the committee at large to be voted into whatever IS or TS they are targeting.
  • Further Work. The proposal’s direction is promising, but it is either not fleshed out well enough, or there are specific concerns with one or more design points. The author is encouraged to come back with a modified proposal that is more fleshed out and/or addresses the stated concerns.
  • Rejected. The proposal is unlikely to be accepted even with design changes.

Accepted proposals (note: I’m not including here the ones which also passed CWG the same meeting and were voted into the standard – see above for those):

  • Source code information capture, a proposal to provide a replacement for the __FILE__, __LINE__, and __FUNCTION__ macros that doesn’t involve the preprocessor. I think this proposal constitutes a major advance because it removes one of the main remaining uses of the preprocessor.
  • Alias-set attributes, a mechanism to pass information to the optimizer about pointer aliasing (like restrict in C, but better). Some design feedback was given, but generally the proposal was considered baked enough that the next revision can go directly to CWG.
  • A few small design changes to the Transactional Memory TS.
  • A proposal to specify that the behaviour of standard library comparison function objects for comparing pointers is consistent with the behaviour of the built-in comparison operators, where the latter behaviour is defined. This was a matter of tweaking the specification to say something that people took for granted to begin with.
  • A modification to the Concepts Lite TS: removing constexpr constraints, which were one of the kinds of constraints allowed in requires-expressions. The reason for the removal is that they are tricky to specify and implement, and have no major motivating uses.
  • A compile-time string class, templated only on the string length, which stores its data in a constexpr character array. This one was one two competing compile-time string proposals, the other one being a variadic char... template class which encodes the string contents in the template arguments themselves. The two proposals present a tradeoff between expressiveness and compile-time efficiency: one the one hand, encoding the string contents in the template arguments allows processing the string via template metaprogramming, while in the other proposal the string can only be processed with constexpr functions; on the other hand, the variadic approach involves creating lots of template instantiations for string processing, which can slow down compile times signficantly. EWG’s view was that the compile-time efficiency consideration was the more important one, especially as constexpr functions are getting more and more powerful. Therefore, the constexpr array-based proposal was selected to move forward. As the proposal has both core language and library components, it will be going to LEWG for design review of the library components before being sent to CWG and LWG.

Proposals for which further work is encouraged:

  • Destructive move, which addresses classes for which an operation that moves from an object and destroys the moved-from object at the same time is more efficient than moving and destroying separately, because the intermediate (moved-from but not yet destroyed) state would require extra object state to represent.
  • Default comparisons. Three different proposals on this topic were presented: one which would automatically give all classes comparison operators unless they opted out by =delete-ing them, or defined their own; one which would allow opting in to compiler-defined comparison operators via =default; and one which would synthesize comparison operators using reflection. As suggested by the variety of the proposals, this is a feature that everyone wants but no one can agree exactly how it should work. Design considerations that came up included opt-in vs. opt-out, special handling for certain types of fields (such as mutable fields and pointers), special handling for classes with a single member, compile-time performance, and different strengths of ordering (such as weak vs. total orders). After discussing the proposal for half a day, we ran out of time, and decided to pick up at the next meeting in Lenexa, possibly armed with revised proposals. There was one poll taken which provided fairly clear guidance on a single aspect of the proposal: there was much stronger consensus for opt-in behaviour than for opt-out.
  • A [[noreturn]] attribute for main(), designed for programs that are never meant to finish, such as some software running on embedded systems. This would allow the optimizer to remove code for running some cleanup such as the destructors of global objects. EWG liked the proposal, and sent it to CWG with one change, naming the attribute [[noexit]] instead. CWG, however, pointed out that global destructors are potentially generated by all translation units, not just the one that defines main(), and therefore the proposal is not implementable without link-time optimization. EWG discussed the proposal further, but didn’t reach any consensus, and decided to put it off until Lenexa.
  • A paper concerning violations of the zero-overhead principle in exception handling. The motivation behind this discussion was resource-constrained systems such as embedded systems, where the overhead associated with exception handling was unwelcome. The general feedback given was to try to evaluate and address such overhead in a comprehensive manner, rather than trying to avoid running into it in a few specific cases.
  • Proposals for a unified function call syntax. Two alternative proposals were presented: one for partial unification (calling non-member functions with member function call syntax), and one for complete unification (either kind of function can be called with either syntax); the latter would either involve breaking code, or having separate name lookup rules for the two syntaxes (and thus not fully achieving the intended unification in spirit). People were somewhat in favour of the first proposal, and a lot more cautious about the second. There seemed to be enough interest to encourage further exploration of the idea.
  • A proposal to allow initializer lists with elements of a move-only type. There was consensus that we want some way to do this, but no consensus for this specific approach; it was not immediately clear what a superior approach would be.
  • Overloading the member access operator (operator .), similarly to how operator -> can be overloaded. This would enable writing “smart reference” classes, much like how overloading operator -> enables writing smart pointer classes. This would be a significant new feature, and many design considerations remain to be explored; however, there was general interest in the idea.
  • Mechanisms for selecting from parameter packs. This proposal has two parts. The first part is a simple syntax for indexing into a parameter pack: if Ts is a parameter pack, and N is a compile-time integral constant, Ts.[N] is the parameter at index N in Ts (or a SFINAE-eligible error if the index N is out of range). The dot is necessary for disambiguation (if the syntax were simply Ts[N], then consider Ts[Ns]..., where Ns is a parameter pack of size equal to Ts; is this a pack of array types T_1[N_1], T_2[N_2], ..., or is it T_(N_1), T_(N_2), ...?). While people weren’t ecstatic about this syntax (the dot seemed arbitrary), there weren’t any better suggestions raised, and people preferred to have the feature with this syntax than to not have it at all. The second part of the proposal was less baked, and concerned “subsetting” a parameter pack with a pack of indices to yield a new pack; EWG encouraged further thought about this part, and suggested exploring two aspects separately: pack literals (for example 0 ...< 5 might be hypothetical syntax for a pack literal which expands to 0, 1, 2, 3, 4) and pack transformations, which are operations that take a parameter pack as input, and transform it to another parameter pack.
  • A proposal to fix a counter-intuitive aspect of the definition of “trivially copyable”.
  • Supporting custom diagnostics for SFINAE-eligible errors. This proposal aimed to resolve a long-standing deficiency in template design: you had to pick between making an incorrect use of a template SFINAE-eligible (expressing the constraint via enable_if or similar), or giving a custom diagnostic for it (expressing the constraint via a static_assert). The specific suggestion was to allow annotating a = delete-ed function with a custom error message that would be shown if it were chosen as the best match in overload resolution. EWG felt that this was a problem worth solving, but preferred a more general solution, and encouraged the author to come back with one.
  • A proposal to specify the order of evaluation of subexpressions within an expression for some types of expressions. EWG felt this change would be valuable, as the order of evaluation being currently unspecified is a common cause of surprise and bugs, but the exact rules still need some thought.
  • Another proposal for classes with runtime size. Unfortunately, EWG continues to be pretty much deadlocked on this topic:
    • People want arrays of runtime bound, together with a mechanism for them to be used as class members.
    • There is no consensus for having arrays of runtime bound without such a mechanism.
    • There are hard technical problems that need to be solved to allow classes of runtime size. One of the biggest challenges is that some platforms’ ABIs would have to be rearchitected to accomodate classes with a runtime-size data member in the middle (this includes class hierarchies where one of the subobjects that’s not at the end has a runtime-sized member at the end).
    • No one has yet come up with a comprehensive solution to these technical problems.
    • There is a divide between two ways of looking of these proposals: one is to say that stack allocation is an optimization, and implementations are free to place runtime-sized arrays on the heap in situations where placing them on the stack is too difficult; the other is to want a guarantee that the allocation is on the stack. Proponents of the second view argue that we don’t need a new syntax for the “stack allocation is an optimization” use case; we should instead improve our optimizers so they can optimize uses of std::vector and similar into stack allocations.

    Given this state of affairs, the future of classes with runtime size (and of arrays of runtime bound, which people want to tie to classes with runtime size) continues to be uncertain.

  • Inline variables. After some discussion, it became clear that this was a proposal for two separate features with a single syntax: a way to declare and initialize global constants in a header file without having to define them in a .cpp file (which is something everyone wants); and a way to define “expression aliases”. EWG expressed interest in these problems, and encouraged fleshing out separate proposals for them.
  • Categorically qualified classes. This proposal provides a mechanism to express that a class’s objects are meant to be used only as named objects, not temporaries (useful for “scope guard”-type classes), or that a class’s objects are meant to only be used as temporaries (useful for expression templates). For classes in the latter category, it’s useful to provide a mechanism to convert objects of this type to objects of another type when initializing a named variable; as such, this part of the proposal overlaps with the operator auto proposal that was discussed (and encouraged for further work) in Rapperswil. EWG felt that the two use cases (scope guards and expression templates) weren’t sufficiently similar to necessitate fixing them the same way, and that the design questions raised during the operator auto discussion weren’t adequately addressed in this proposal; encouragement was given to continue exploring the problem space, being open to different approaches for the two use cases.
  • Generalized lifetime extension. This paper outlined a set of rules for determining whether the result of an expression refers to any temporaries that appear as subexpressions, and proposed that when the result of an expression is bound to a named variable of reference type (at local scope), the temporaries referred to by the result have their lifetimes extended to the lifetime of the variable. A very limited form of this exists in C++ today; this proposal would generalize it considerably. I found this proposal to be very interesting; it has the potential to dramatically reduce the number of use-after-free errors that occur due to temporaries being destroyed earlier than we intend them to be. On the other hand, if not done carefully, the proposal would have the potential to cause programmers to be more laissez-fare about their analysis of temporary lifetimes, possibly leading to more errors. For EWG, the sticking point was that performing the refers-to analysis for function call expressions where the function body is in another translation unit requires the co-operation of the function author. The paper proposed annotating parameters with the keyword export to indicate that the function’s return value refers to this parameter. EWG didn’t like this, feeling that these annotations would be “exception specifications all over again”, i.e. components of a function declaration that are not quite part of its type, for which we need ad-hoc rules to determine their behaviour with respect to redeclarations, function pointers, overrides in derived classes, being passed as non-type template arguments, and so on. The conclusion was that the problem this proposal addresses is a problem we want solved, but that this approach was not in the right direction for solving the problem.

Rejected proposals:

  • A proposal to let return {expr} be explicit, in the sense that it would allow invoking constructors of the function’s return type even if they were explicit. This proposal had support in Rapperswil, but after several new papers argued against it, EWG decided to shelve it.
  • The proposal for named arguments that Ehsan and I wrote. While a few people in the room liked the idea, the majority had reservations about it; prominent among these were the proposal encouraging functions with many parameters, and the additional maintenance burden on library authors caused by parameter name changes breaking code.
  • A proposal for a null-coalescing conditional operator, a ?: b, which would have been equivalent to a ? a : b. EWG felt the utility wasn’t sufficiently compelling to warrant a language change.
  • Checked-dereference conditions. This would have made if (T x : expr) { S } equivalent to if (auto p = expr) { T x = *p; S } (and similarly for while loops and the test-expressions of for loops). EWG felt this shorthand wasn’t sufficiently compelling, and could cause confusion due to the similarity of the syntax to the range-based for loop.
  • A proposal for uniform handling of subobjects. This would allow data members and bases of a class to be interleaved in any order. EWG felt this change was too intrusive and insufficiently motivated.

Contracts

EWG held a special evening session on the topic of contracts, as there was a lot of interest in them at this meeting. Several papers on the topic were presented; a couple of others were not due to lack of time or a presenter.

The only proposal that was specifically considered was a proposal to turn the assert macro into a compiler-recognized operator with one of a specified set of semantics based on the value of the NDEBUG macro; it was rejected, mostly on the basis that it was infeasible to muck with assert and NDEBUG for backwards-compatibility reasons.

Other than that, the discussion was more about high-level design aspects for contract programming rather specific proposals. Some issues that came up were:

  • Where to specify the contracts for a function – in the declaration, in the implementation, or potentially either – and what the implications are.
  • Whether optimizers should be allowed to assume that contracts are obeyed, such that non-obeyance (e.g. precondition violation) implies undefined behaviour.
  • Whether the standard should specify different modes of behaviour (e.g. “release” vs. “debug”) with respect to contract checking (and if so, how to control the mode, or if this should be left implementation-defined).
  • What the behaviour should be upon contract violation (“keep going” but undefined behaviour, custm handler, terminate, throw, etc.).

The discussion closed with some polls to query the consensus of the room:

  • There was consensus that we want some form of contracts.
  • There was consensus that ensuring correctness and realizing performance gains are both important goals of a contracts proposal, with correctness being with primary one.
  • There was consensus that we need to support contracts in interfaces / declarations (at least).
  • There was no consensus for having some notion of “build modes” specified in the standard to control aspects of contract checking.

These views will likely guide future proposals on this topic.

Coroutines

Coroutines was another topic with a lot of interest at this meeting. There were three proposals on the table: “resumable functions”, “resumable lambdas”, and a library interface based on Boost.Coroutine. These proposals started out under the purview of SG 1 (Concurrency), but then they started growing into a language feature with applications unrelated to concurrency as well, so the proposals were presented in an evening session to give EWG folks a chance to chime in too.

The coroutines proposals fall into two categories: stackful and stackless, with the “resumable functions” and “resumable lambdas” proposals being variations on a stackless approach, and Boost.Coroutine proposal being a stackful approach.

The two approaches have an expressiveness/performance tradeoff. Stackful coroutines have more overhead, because a stack needs to be reserved for them; the size of the stack is configurable, but making it too small risks undefined behaviour (via a stack overflow), while making it too large wastes space. Stackless coroutines, on the other hand, use only as much space as they need by allocating space for each function call on the heap (these are called activation frames; in some cases, the heap allocation can be optimized into stack allocation). The price they pay in expressiveness is that any function that calls a resumable function (i.e. a stackless coroutine) must itself be resumable, so the compiler knows to allocate activation frames on the heap when calling it, too. By contrast, with the stackful approach, any old function can call into a stackful coroutine, because execution just switches to using the coroutine’s side stack for the duration of the call.

Within the “stackless” camp, the difference between the “resumable functions” and “resumable lambdas” approaches is relatively small. The main difference is that the “resumable lambdas” approach allows coroutines to be passed around as first-class objects (since lambdas are objects).

The authors of the “resumable functions” and Boost.Coroutine proposals have attempted to come up with a unified proposal that combines the power of “stackful” with the expressiveness of “stackless”, but haven’t succeeded, and in fact have come to believe that the tradeoff is inherent. In light of this, and since both approaches have compelling use cases, the committee was of the view that both approaches should be pursued independently, both targetting a single Coroutines Technical Specification, with the authors co-operating to try to capture any commonalities between their approaches (if nothing else then a common, consistent set of terminology) even if a unified proposal isn’t possible. For the stackless approach, participants were polled for a preference between the “resumable functions” and “resumable lambdas” approaches; there was stronger support for the “resumable functions” approach, though I think this was at least in part due to the “resumable lambdas” approach being newer and less well understood.

I had a chance to speak to Chris Kohlhoff, the author of the “resumable lambdas” proposal, susbequent to this session. He had an idea for combining the “stackless” and “stackful” approaches under a single syntax that I found very interesting, which he plans to prototype. If it pans out, it might end up as the basis of compelling unified proposal after all.

I’m quite excited about the expressivity coroutines would add to the language, and I await developments on this topic eagerly, particularly on Chris’s unified approach.

Embedded Systems

The topic of forming a Study Group to explore ways to make C++ more suitable for embedded systems came up again. In addition to the two papers presented on the topic, some further ideas in this space were containers that can be stored in ROM (via constexpr), and having exceptions without RTTI. It was pointed out that overhead reductions of this sort might be of interest to other communities, such as gaming, graphics, real-time programming, low-latency programming, and resource-constrained systems. EWG encouraged discussion across communities before forming a Study Group.

Library/Library Evolution Working Groups (LWG and LEWG)

I mentioned the library features that are targeted for C++17 in the “C++17” section above. Here I’ll talk about progress on the Library Fundamentals Technical Specifications, and future work.

Library Fundamentals TS I

The first Library Fundamentals TS has already gone through its first formal ballot, the PDTS (Preliminary Draft Technical Specification) ballot. LWG addressed comments sent in by national standards bodies in response to the ballot; the resulting changes were very minor, the most notable being the removal of the network byte-order conversion functions (htonl() and friends) over concerns that they clash with similarly-named macros. LWG will continue addressing the comments during a teleconference in December, and then they plan to send out the specification for its DTS (Draft Technical Specification) ballot, which, if successful, will be its last before publication.

Library Fundamentals TS II

The second Library Fundamentals TS is in the active development stage. Coming into the meeting, it contained a single proposal, for a generalized callable negator. During this meeting, several new features were added to it:

There will very likely be more features added at the next meeting, in May 2015; the TS is tentatively scheduled to be sent out for its PDTS ballot at the end of that meeting.

Future Work

In addition to the proposals which have already been added into C++17 or one of the TS’s, there are a lot of other library proposals in various stages of consideration.

Proposals approved by LEWG and under review by LWG:

Proposals approved by LEWG for which LWG review is yet to start:

Proposal for which LEWG is encouraging further work:

Proposals rejected by LEWG:

There will be a special library-only meeting in Cologne, Germany in February to allow LWG and LEWG to catch up a bit on all these proposals.

Study Groups

SG 1 (Concurrency)

SG 1’s main projects are the Concurrency TS and the Parallelism TS. As with the Library Fundamentals TS, both are likely to be the start of a series of TS’s (so e.g. the Parallelism TS will be followed by a Parallelism TS II).

Besides coroutines, which I talked about above, I haven’t had a chance to follow SG 1’s work in any amount of detail, but I will mention the high-level status:

The Parallelism TS already had its PDTS ballot; comments were addressed this week, resulting in minor changes, including the addition of a transform-reduce algorithm. SG 1 will continue addressing comments during a teleconference in December, and then plans to send the spec out for its DTS ballot. As mentioned above, there are plans for a Parallelism TS II, but no proposals have been approved for it yet.

The Concurrency TS has not yet been sent out for its PDTS ballot; that is now planned for Lenexa.

Some library proposals that have been approved by LEWG for the Concurrency TS:

Task regions are still being considered by LEWG, and would likely target Concurrency TS II.

A major feature being looked at by SG 1 is executors and schedulers, with two competing proposals. The two approaches were discussed, and SG 1 felt that at this stage there’s still design work to be done and it’s too early to make a choice. This feature is targeting the second Concurrency TS as it’s unlikely to be ready in time for Lenexa, and SG 1 doesn’t want to hold up the first Concurrency TS beyond Lenexa.

Coroutines are also a concurrency feature, but as mentioned above, they are now targeting a separate TS.

SG 2 (Modules)

EWG spent an afternoon discussing modules. At this point, Microsoft and Clang both have modules implementations, at various levels of completion. The Microsoft effort is spearheaded by Gabriel Dos Reis, who summarized the current state of affairs in a presentation.

The goals of modules are:

  • componentization
  • isolation from macros
  • improving build times
  • making it easier to write semantics-aware developer tools
  • being a code distribution mechanism is, at the moment, an explicit non-goal

The aspects of a modules design that people generally agree on at this point are:

  • modules are not a scoping mechanism (i.e. they are independent of namespaces)
  • while performing template instantiation while compiling a module, the compiler has access to the full module being compiled, but only to the interfaces of imported modules
  • the interface of a module can be separated from its implementation
  • module interfaces cannot have cyclic dependencies
  • only one module owns the definition of an entity

Design points that still need further thought are:

  • visibility of private class members across module boundaries
  • ordering of static/dynamic initialization
  • can macros flow into modules? (e.g. NDEBUG)
    • one view on this is that there should be no standard way to provide an input macro to a module, but implementations can provide implementation-defined mechanisms, such as defining NDEBUG on the compiler command line to build a debug version of a module
    • another option is to “parameterize” a module on certain input parameters (such as the value of the NDEBUG macro)
      • this in turn raises the question of a more general parameterization mechanism for modules, akin to templates
  • can macros flow out of modules? (e.g. can the Boost.Preprocessor library be packaged up into a module?)
  • semantics of entities with internal linkage in a module interface
  • can a module interface be spread across several files?
  • the syntax for defining a module
  • how to deal with #includes in a module

EWG was generally pleased with the progress being made, and encouraged implementors to continue collaborating to get their designs to converge, and report back in Lenexa.

The Clang folks also reported promising performance numbers from their implementation, but detailed/comprehensive benchmarks remain to be performed.

SG 3 (Filesystems)

SG 3 did not meet in Urbana. The Filesystems TS is waiting for its DTS ballot to close; assuming it’s successful (which is the general expectation), it will be published early next year.

Proposals targeting a follow-up Filesystems TS II are welcome; none have been received so far.

SG 4 (Networking)

Organizationally, the work of SG 4 has been conducted directly by LEWG over the past few meetings. This arrangement has been formalized at this meeting, with SG 4’s chair, Kyle Kloepper, retiring, and the SG becoming “dormant” until LEWG decides to reactivate it.

In Rapperswil, LEWG had favourably reviewed a proposal for a C++ networking library based on Boost.ASIO, and asked the author (Chris Kohlhoff, whom I’ve talked about earlier in the context of coroutines) to update the proposal to leverage C++14 language features. Chris has done so, and presented an updated proposal to LEWG in Urbana; this update was also received favourably, and was voted to become the initial working draft of the Networking TS, which now joins the roster of Technical Specifications being worked on by the committee. In other words, we’re one step closer to having a standard sockets library!

SG 5 (Transactional Memory)

I haven’t been following the work of SG 5 very closely, but I know the Transactional Memory TS is progressing well. Its working draft has been created based on two papers, and it’s going to be sent out for its PDTS ballot shortly (after a review conducted via teleconference), with the intention being that the ballot closes in time to look at the comments in Lenexa.

SG 6 (Numerics)

Topics of discussion in SG 6 included:

  • a replacement for std::rand which combines the security of the C++11 <random> facilities with the simple interface of std::rand
  • special math functions for C++17
  • typedefs similar to int16_t for floating-point types
  • bignums, ratios, fixed-point arithmetic

A Numerics TS containing proposals for some of the above may be started in the near future.

There is an existing TR (Technical Report, an older name for a Technical Specification) for decimal floating-point arithmetic. There is a proposal to integrate this into C++17, but there hasn’t been any new progress on that in Urbana.

SG 7 (Reflection)

SG 7 looked at two reflection proposals: an updated version of a proposal for a set of type traits for reflecting the members of classes, unions, and enumerations, and a a significantly reworked version of a comprehensive proposal for static reflection.

The reflection type trait proposal was already favourably reviewed in Rapperswil. At this meeting, additional feedback was given on two design points:

  • Access control. There was consensus that reflection over inaccessible members should be allowed, but that it should occur via a separate mechanism that is spelt differently in the code (for example, there might be one namespace called std::reflect which provides traits for reflecting accessible members only, and another called std::reflect_invasively which provides traits for reflecting all members including inaccessible ones). The rationale is that for some use cases, reflecting only over accessible members is appropriate, while for others, reflecting over all members is appropriate, and we want to be able to spot uses of an inappropriate mechanism easily. Some people also expressed a desire to opt-out from invasive reflection on a per-class basis.
  • Syntax. The proposal’s syntax for e.g. accessing the name of the second member of a class C is std::class_member::name<C,1>. A preference was expressed a) for an additional level of grouping of reflection-related traits into a namespace or class reflect, e.g. std::reflect::class_member::name<C,1>, and b) for not delaying the provision of all inputs until the last component of the trait, e.g. std::reflect<C>::class_member<1>::name. (This last form has the disadvantage that it would actually need to be std::reflect<C>::template class_member<1>::name; some suggestions were thrown around for avoiding this by making the syntax use some compiler magic (as the traits can’t be implemented purely as a library anyways)).

It was also reiterated that this proposal has some limitations (notably, member templates cannot be reflected, nor can members of reference or bitfield type), but SG 7 remains confident that the proposal can be extended to fill these gaps in due course (in some cases with accompanying core language changes).

The comprehensive static reflection proposal didn’t have a presenter, so it was only looked at briefly. Here are some key points from the discussion:

  • This proposal allows reflection at a much greater level of detail – often at the level of what syntax was used, rather than just what entities were declared. For example, this proposal allows distinguishing between the use of different typedefs for the same type in the declaration of a class member; the reflection type traits proposal does not.
  • No one has yet explored this area enough to form a strong opinion on whether having access to this level of detail is a good thing.
  • SG 7 is interested in seeing motivating use cases that are served by this proposal but not by the reflection type traits proposal.
  • Reflecting namespaces – a feature included in this proposal – is viewed as an advanced reflection feature that is best left off a first attempt at a reflection spec.
  • The author is encouraged to do further work on this proposal, with the above in mind. Splitting the proposal into small components is likely to help SG 7 make progress on evaluating it.

There is also a third proposal for reflection, “C++ type reflection via variadic template expansion”, which sort of fell off SG 7’s radar because it was in the post-Issaquah mailing and had no presenter in Rapperswil or Urbana; SG 7 didn’t look at it in Urbana, but plans to in Lenexa.

SG 8 (Concepts)

The Core Working Group continued reviewing the Concepts TS (formerly called “Concepts Lite”) in Urbana. The fundamental design has not changed over the course of this review, but many details have. A few changes were run by EWG for approval (I mentioned these in the EWG section above: the removal of constexpr constraints, and the addition of folding expressions). The hope was to be ready to send out the Concepts TS for its PDTS ballot at the end of the meeting, but it didn’t quite make it. Instead, CWG will continue the review via teleconferences, and possibly a face-to-face meeting, for Concepts only, in January. If all goes well, the PDTS ballot might still be sent out in time for the comments to arrive by Lenexa.

SG 9 (Ranges)

As far as SG 9 is concerned, this has been the most exciting meeting yet. Eric Niebler presented a detailed and well fleshed-out proposal for integrating ranges into the standard library.

Eric’s ranges are built on top of iterators, thus fitting on top of today’s iterator-based algorithms almost seamlessly, with one significant change: the begin and end iterators of a range are not required to be of the same type. As the proposal explains, this small change allows a variety of ranges to be represented efficiently that could not be under the existing same-type model, including sentinel- and predicate-based ranges.

The main parts of the proposal are a set of range-related concepts, a set of range algorithms, and a set of range views. The foundational concept is Iterable, which corresponds roughly to what we conversationally call (and also what the Boost.Range library calls) a “range”. An Iterable represents a range of elements delimited by an Iterator at the beginning and a Sentinel at the end. Two important refinements of the Iterable concept are Container, which is an Iterable that owns its elements, and Range, which is a lightweight Iterable that doesn’t own its elements. The range algorithms are basically updated versions of the standard library algorithms that take ranges as Iterables; there are also versions that take (Iterator, Sentinel) pairs, for backwards-compatibiltiy with today’s callers. Finally, the range views are ways of transforming ranges into new ranges; they correspond to what the Boost.Range library calls range adaptors. There is also a suggestion to enhance algorithms with “projections”; I personally see this as unnecessary, since I think range views serve their use cases better.

Eric has fully implemented this proposal, thus convincingly demonstrating its viability.

Importantly, this proposal depends on the Concepts TS to describe the concepts associated with ranges and define algorithms and views in terms of these functions. (Eric’s implementation emulates the features of the Concepts TS with a C++11 concepts emulation layer.)

The proposal was overall very well received; there was clear consensus that Eric should pursue the high-level design he presented and come back with a detailed proposed specification.

An important practical point that needed to be addressed is that this proposal is not 100% backwards-compatible with the current STL. This wasn’t viewed as a problem, as previous experience trying to introduce C++0x concepts to the STL while not breaking anything has demonstrated that this wasn’t possible without a lot of contortions, and people have largely accepted that a clean break from the old STL is needed to build a tidy, concepts-enabled “STL 2.0”. Eric’s proposal covers large parts of what such an STL 2.0 would look like, so there is good convergence here. The consensus was that Eric should collaborate with Andrew Sutton (primary author and editor of the Concepts TS) on a proposal for a Technical Specification for a concepts-enabled ranges library; the exact scope (i.e. whether it will be just a ranges library, or a complete STL overhaul) is yet to be determined.

SG 10 (Feature Test)

The Feature Test Standing Document (the not-quite-a-standard document used by the committee to specify feature test macros) has been updated with C++14 features.

The feature test macros are enjoying adoption by multiple implementors, including GCC, Clang, EDG, and others.

SG 12 (Undefined Behaviour)

SG 12 looked at:

SG 13 (I/O, formerly “Graphics”)

SG 13 has been working on a proposal for 2D Graphics TS based on cario’s API. In Urbana, an updated version of this proposal which included some proposed wording was presented to LEWG. LEWG encouraged the authors to complete the wording, and gave a couple of pieces of design advice:

  • If possible, put in place some reasonable defaults (e.g., a default foreground color that’s in place if you don’t explicitly set a foreground color) so a “Hello world” type program can be written more concisely.
  • Where the API differs from a mechanical transliteration of the cairo API, document the rationale for the difference.

Next Meeting

The next full meeting of the Committee will be in Lenexa, Kansas, the week of May 4th, 2015.

There will also be a library-only meeting in Cologne, Germany the week of Feberuary 23rd, and a Concepts-specific meeting in Skillman, New Jersey from January 26-28.

Conclusion

This was probably the most action-packed meeting I’ve been to yet! My personal highlights:

  • The amount of interest in coroutines, and the green-light that was given to develop proposals for both the stackful and stackless versions. I think coroutines have the potential to revolutionize how C++ programmers express control flow in many application domains.
  • Eric Niebler’s proposal for bringing ranges to the standard library. It’s the first cohesive proposal I’ve seen that addresses all the tough practical questions involved in such an endeavour, and it was accordingly well-received.
  • The continuing work on modules, particularly the fact that Microsoft and Clang both have implementations in progress and are cooperating to converge on a final design.

Stay tuned for further developments!

Trip Report: C++ Standards Committee Meeting in Rapperswil, June 2014

Summary / TL;DR

Project Status
C++14 On track to be published late 2014
C++17 A few minor features so far, including for (elem : range)
Networking TS Ambitious proposal to standardize sockets based on Boost.ASIO
Filesystems TS On track to be published late 2014
Library Fundamentals TS Contains optional, any, string_view and more. Progressing well, expected early 2015
Library Fundamentals TS II Follow-up to Library Fundamentals TS; will contain array_view and more. In early stage
Array Extensions TS Completely stalled. No proposal related to runtime-sized arrays/objects currently has consensus
Parallelism TS Progressing well; expected 2015
Concurrency TS Executors and resumable functions need more work
Transactional Memory TS Progressing well; expected 2015-ish
Concepts (“Lite”) TS Progressing well; expected 2015
Reflection A relatively full-featured compile-time introspection proposal was favourably reviewed. Might target a TS or C++17
Graphics Moving forward with a cairo-based API, to be published in the form of a TS
Modules Clang has a complete implementation for C++, plan to push it for C++17

Introduction

Last week I attended another meeting of the ISO C++ Standards Committee in Rapperswil, Switzerland (near Zurich). This is the third Committee meeting I have attended; you can find my reports about the previous two here (September 2013, Chicago) and here (February 2014, Issaquah). These reports, particularly the Issaquah one, provide useful context for this post.

With C++14’s final ballot being still in progress, the focus of this meeting was the various language and library Technical Specifications (TS) that are planned as follow-ups to C++14, and on C++17.

C++14

C++14 is currently out for its “DIS” (Draft International Standard) ballot (see my Issaquah report for a description of the procedure for publishing a new language standard). This ballot was sent out at the end of the Issaquah meeting, and will close mid-August. If no national standards body poses an objection by then – an outcome considered very likely – then the standard will be published before the end of the year.

Since a ballot was in progress, no changes to the C++14 draft were made during the Rapperswil meeting.

C++17, and what’s up with all these TS’s?

ISO procedure allows the Committee to publish two types of documents:

  • International Standards (IS). These are official standards with stringent backwards-compatibility requirements.
  • Technical Specifications (TS) (formerly called Technical Reports (TR)). These are for things that are not quite ready to be enshrined into an official standard yet, and have no backwards-compatibility requirements. Specifications contained in a TS may or may not be added, possibly with modifications, into a future IS.

C++98 and C++11 are IS’s, as will be C++14 and C++17. The TS/TR form factor has, up until recently, only been used once by the Committee: for TR1, the 2005 library spec that gave us std::tr1::shared_ptr and other library enhancements that were then added into C++11.

Since C++11, in an attempt to make the standardization process more agile, the Committee has been aiming to place significant new language and library features into TS’s, published on a schedule independent of the IS’s. The idea is that being in a TS allows the feature to gain user and implementation experience, which the committee can then use to re-evaluate and possibly revise the feature before putting it into an IS.

As such, much of the standardization work taking place concurrently with and immediately after C++14 is in the form of TS’s, the first wave of which will be published over the next year or so, and the contents of which may then go into C++17 or a subsequent IS, as schedules permit.

Therefore, at this stage, only some fairly minor features have been added directly to C++17.

The most notable among them is the ability to write a range-based for loop of the form for (elem : range), i.e. with the type of the element omitted altogether. As explained in detail in my Issaquah report, this is a shorthand for for (auto&& elem : range) which is almost always what you want. The Evolution Working Group (EWG) approved this proposal in Issaquah; in Rapperswil it was also approved by the Core Working Group (CWG) and voted into C++17.

Other minor things voted into C++17 include:

  • static_assert(condition), i.e. with the message omitted. An implementation-defined message is displayed.
  • auto var{expr}; is now valid and equivalent to T var{expr}; (where T is the deduced type)
  • A template template parameter can now be written as template <...> typename Name in addition to template <...> class Name, to mirror the way a type template parameter can be written as typename Name in addition to class Name
  • Trigraphs (an obscure feature that allowed certain characters, such as #, which are not present on some ancient keyboards, to be written as a three-character sequence, such as ??=) were removed from the language

Evolution Working Group (EWG)

As with previous meetings, I spent most of time in the Evolution Working Group, which spends its time looking at proposals for new language features that either do not fall into the scope of any Study Group, or have already been approved by a Study Group. There was certainly no lack of proposals at this meeting; to EWG’s credit, it got through all of them, at least the ones which had papers in the official pre-Rapperswil mailing.

Incoming proposals were categorized into three rough categories:

  • Approved. The proposal is approved without design changes. They are sent on to CWG, which revises them at the wording level, and then puts them in front of the committee at large to be voted into whatever IS or TS they are targeting.
  • Further Work. The proposal’s direction is promising, but it is either not fleshed out well enough, or there are specific concerns with one or more design points. The author is encouraged to come back with a modified proposal that is more fleshed out and/or addresses the stated concerns.
  • Rejected. The proposal is unlikely to be accepted even with design changes.

Accepted proposals:

  • Opening two nested namespaces with namespace A::B { in addition to namespace A { namespace B {
  • “Making return explicit”. This means that if a class A has an explicit constructor which takes a parameter of type B, then, in a function whose return type is A, it is valid to write return b; where b has type B. (Currently, one has to writen return A(b);.) The idea is to avoid repetition; a very common use case is A being std::unique_ptr<T> for some T, and B being T*. This proposal was relatively controversial; it passed with a weak consensus in EWG, and was also discussed in the Library Evolution Working Group (LEWG), where there was no consensus for it. I was surprised that EWG passed this to CWG, given the state of the consensus; in fact, CWG indicated that they would like additional discussion of it in a forum that includes both EWG and LEWG members, before looking at it in CWG.
  • A preprocessor feature for testing for the presence of a (C++11-style) attribute: __has_cpp_attribute(attribute_name)
  • A couple that I already mentioned because they were also passed in CWG and voted into C++17 at the same meeting:

Proposals for which further work is encouraged:

  • A proposal to make C++ more friendly for embedded systems development by reducing the overhead of exception handling, and further expanding the expressiveness of constexpr. EWG encouraged the author to gather people interested in this topic and form a Study Group to explore it in more detail.
  • A proposal to convey information to the compiler about aliasing, via attributes. This is intended to be an improvment to C99’s restrict.
  • A way to get the compiler to, on an opt-in basis, generate equality and comparison operators for a structure/class. Everyone wanted this feature, but there were disagreements about how terse the syntax should be, whether complementary operators should be generated automatically (e.g. != based on ==), how exactly the compiler should implement the operators (particularly for comparison – questions of total order vs. weaker orders came up), and how mutable members should be handled.
  • A proposal for a per-platform portable C++ ABI. I will talk about this in more detail below.
  • A way to force the compiler to omit padding between two structure fields
  • A way to specify that a class should be converted to another type in auto initialization. That is, for a class C, to specify that in auto var = c; (with c having type C), the type of var should actually be some other type D. The motivating use here is expression templates; in Matrix X, Y; auto Z = X * Y; we want the type of Z to be Matrix even if the type of X * Y is some expression template type. EWG liked the motivation, but the proposal tried to modify the semantics of template parameter deduction for by-value parameters so as to remain consistent with auto, and EWG was concerned that this was starting to encroach on too many areas of the language. The author was encouraged to come back with a more limited-scope proposal that concerned auto initialization only.
  • Fixed-size template parameter packs (typename...[K]) , and packs where all parameters must be of the same type (T...[N]). EWG liked the idea, but had some concerns about syntactic ambiguities. The proposal also inspired an offshoot idea of subscripting parameter packs (e.g. Pack[0] gives you the first parameter), to avoid having to use recursion to iterate over the parameters in many cases.
  • Expanding parameter packs as expressions. Currently, if T is a parameter pack bound to parameters A, B, and C, then T... expands to A, B, C; this expansion is allowed in various contexts where a comma-separated list of things (types or expressions, as the parameters may be) is allowed. The proposal here is to allow things like T +... which would expand to A + B + C, which would be allowed in an expression context.

Rejected proposals:

  • Objects of runtime size. This would have allowed a pure library implementation of something like std::dynarray (and allowed users to write similar classes of their own), but it unfortunately failed to gain consensus. More about this in the Array Extensions TS section.
  • Additional flow control mechanisms like break label;, continue label; and goto case value;. EWG thought these encouraged hard-to-follow control flow.
  • Allowing specifiers such as virtual, static, override, and some others, to apply to a group of members the way acess specifiers (private: etc.) currently do. The basis for rejection here was that separating these specifiers from the members they apply to can make class definitions less readable.
  • Specializing an entity in a different namespace without closing the namespaces you are currently in. Rejected because it’s not clear what would be in scope inside the specialization (names from the entity’s namespace, the current namespace, or both).
  • <<< and >>> operators for circular bit-shifts. EWG felt these would be more appropriate as library functions.
  • A rather complicated proposal for annotating template parameter packs that claimed to be a generalization of the proposal for fixed-size template parameter packs. Rejected because it would have made the language much more complicated, while the benefit would mostly have been for template metaprogramming; also, several of the use cases can be satisfied with Concepts instead.
  • Throwing an exception on stack exhaustion. The implementers in the room felt this was not implementable.

I should also mention the proposal for named arguments that Ehsan and I have been working on. We did not prepare this proposal in time to get it into the pre-Rapperswil mailing, and as such, EWG did not look at it in Rapperswil. However, I did have some informal discussions with people about it. The main concerns were:

  • consideration for constructor calls with {...} syntax and, by extension, aggregate initialization
  • the relationship to C99 designated initializers (if we are covering aggregate initializtion, then these can be viewed as competing syntaxes)
  • most significantly: parameter names becoming part of a library’s interface that library authors then have to be careful not to break

Assuming we are able to address these concerns, we will likely write an updated proposal, get it into the pre-Urbana mailing (Urbana-Champaign, Illinois is the location of the next Committee meeting in November), and present it at the Urbana meeting.

Portable C++ ABI

One of the most exciting proposals at this meeting, in my opinion, was Herb Sutter’s proposal for a per-platform portable C++ ABI.

A per-platform portable ABI means that, on a given platform (where “platform” is generally understood to a mean the combination of an operating system, processor family, and bitness), binary components can be linked together even if they were compiled with different compilers, or different versions of the same compiler, or different compiler options. The current lack of this in C++ is, I think, one of C++’s greatest weaknesses compared to other languages like C or Java.

More specifically, there are two aspects to ABI portability: language and library. On the language side, portability means that binary components can be linked together as long as, for any interface between two components (for example, for a function that one component defines and the other calls, the interface would consist of the function’s declaration, and the definitions of any types used as parameters or return type), the two components are compiled from identical standard-conforming source code for that interface. On the library side, portability additionally means that interfaces between components can make use of standard library types (this does not follow solely from the language part, because different compilers may not have identical source code for their standard library types).

It has long been established that it is out of scope for the C++ Standard to prescribe an ABI that vendors should use (among other reasons, because parts of an ABI are inherently platform-specific, and the standard cannot enumerate every platform and prescribe something for each one). Instead, Herb’s proposal is that the standard codify the notions of a platform and a platform owner (an organization/entity who controls the platform); require that platform owners document an ABI (in the case of the standard library, this means making available the source code of a standard library implementation) which is then considered the platform ABI; and require compiler and standard library vendors to support the platform ABI to be conformant on any given platform.

In order to ease transitioning from the current world where, on a given platform, the ABI can be highly dependent on the compiler, the compiler version, or even compiler options, Herb also proposes some mechanisms for delineating a portion of one’s code which should be ABI-portable, and therefore compiled using the platform ABI. These mechanisms are a new linkage (extern "abi") on the language side, and a new namespace (std::abi, containing the same members as std) on the library side. The idea is that one can restrict the use of these mechanisms to code that constitutes component interfaces, thereby achieving ABI portability without affecting other code.

This proposal was generally well-received, and certainly people agreed that a portable ABI is something C++ needs badly, but some people had concerns about the specific approach. In particular, implementers were uncomfortable with the idea of potentially having to support two different ABI’s side-by-side in the same program (the platform ABI for extern "abi" entities, and the existing ABI for other entities), and, especially, with having two copies of every library entity (one in std and one in std::abi). Other concerns about std::abi were raised as well, such as the performance penalty arising from having to convert between std and std::abi types in some places, and the duplication being difficult to teach. It seemed that a modified proposal that concerned the language only and dropped std::abi would have greater consensus.

Array Extensions TS

The Array Extensions TS was initally formed at the Chicago meeting (September 2013) when the committee decided to pull out arrays of runtime bound (ARBs, the C++ version of C’s VLAs) and dynarray, the standard library class for encapsulating them, out of C++14 and into a TS. This was done mostly because people were concerned that dynarray required too much compiler magic to implement. People expressed a desire for a language feature that would allow them to implement a class like dynarray themselves, without any compiler magic.

In Issaquah a couple of proposals for such a language feature were presented, but they were relatively early-stage proposals, and had various issues such as having quirky syntax and not being sufficiently general. Nonetheless there was consensus that a library component is necessary, and we’d rather not have ARBs at all than get them without a library component to wrap them into a C++ interface.

At this meeting, a relatively fully fleshed-out proposal was presented that gave programmers a fairly general/flexible way to define classes of runtime size. Unfortunately, it was considered a complex change that touches many parts of the language, and there was no consensus for going forward with it.

As a result, the Array Extensions TS is completely stalled: ARBs themselves are ready, but we don’t want them without a library wrapper, and no proposal for a library wrapper (or mechanism that would enable one to be written) has consensus. This means that the status quo of not being able to use VLAs in C++ (unless a vendor enables C-stle VLAs in C++ as an extension) will remain for now.

Library / Library Evolution Working Groups (LWG and LEWG)

Library work at this meeting included the Library Fundamentals TS (and its planned follow-up, Library Fundamentals II), the Filesystems TS and Networking TS (about which I’ll talk in the SG 3 and SG 4 sections below), and reviewing library components of other projects like the Concurrency TS.

The Library Fundamentals TS was in the wording review stage at this meeting, with no new proposals being added to it. It contains general library utilities such as optional, any, string_view, and more; see my Issaquah report for a full list. The current draft of the TS can be found here. At the end of the meeting, the Committee voted to send out the TS for its first national body ballot, the PDTS (Preliminary Draft TS) ballot. This ballot concludes before the Urbana meeting in November; if the comments can be addressed during that meeting and the TS sent out for its second and final ballot, the DTS (Draft TS) ballot, it could be published in early 2015.

The Committee is also planning a follow-up to the Library Fundamentals TS, called the Library Fundamentals TS II, which will contain general utilities that did not make it into the first one. Currently, it contains one proposal, a generalized callable negator; another proposal, containing library facilities for contract programming, got rejected for several reasons, one of them being that it is expected to be obsoleted in large part by reflection. Proposals under consideration to be added include:

Study Groups

SG 1 (Concurrency)

SG 1 focuses on two areas, concurrency and parallelism, and there is one TS in the works for each.

I don’t know much about the Parallelism TS other than it’s in good shape and was sent out for its PDTS ballot at the end of the meeting, which could lead to publication in 2015.

The status of the Concurrency TS is less certain. Coming into the Rapperswil meeting, the Concurrency TS contained two things: improvements to std::future (notably a then() method for chaining a second operation to it), and executors and schedulers, with resumable functions being slated for addition.

However, the pre-Rapperswil mailing contained several papers arguing against the existing designs for executors and resumable functions, and proposing alternative designs instead. These papers led to executors and schedulers being removed from the Concurrency TS, and resumable functions not being added, until people come to a consensus regarding the alternative designs. I’m not sure whether publication of the Concurrency TS (which now contains only the std::future improvements) will proceed, leaving executors and resumable functions for a follow-up TS, or be stalled until consensus on the latter topics is reached.

For resumable functions, I was in the room during the technical discussion, and found it quite interesting. The alternative proposal is a coroutines library based on Boost.Coroutine. The two proposals differ both in syntax (new keywords async and await vs. the magic being hidden entirely behind a library interface), and implementation technique for storing the local variables of a resumable function (heap-allocated “activation frames” vs. side stacks). The feedback from SG 1 was to disentangle these two aspects, possibly yielding a proposal where either syntax could be matched with either implementation technique.

There are also other concurrency-related proposals before SG 1, such as ostream buffers, latches and barries, shared_mutex, atomic operations on non-atomic data, and a synchronized wrapper. I assume these will go into either the current Concurrency TS, or a follow-up TS, depending on how they progress through the committee.

SG 2 (Modules)

Modules is, in my opinion, one of the most sorely needed features in C++. They have the potential of decreasing compe times by an order of magnitude or more, thus bringing compile-time performance more in line with more modern languages, and of solving the combinatorial explosion problem, caused by macros, that hampers the development of powerful tooling such as automated refactoring.

The standardization of modules has been a slow process for two reasons. First, it’s an objectively difficult problem to solve. Second, the solution shares some of the implementation difficulties of the export keyword, a poorly thought-out feature in C++98 that sought to allow separate compilation of templates; export was only ever implemented by one compiler (EDG), and the implementation process revealed flaws that led not only to other compilers not even bothering to implement it, but also to the feature being removed from the language in C++11. This bad experience with export led people to be uncertain about whether modules are even implementable. As a result, while some papers have been written proposing designs for modules (notably, one by Daveed Vandevoorde a couple of years back, and one by Gabriel Dos Reis very recently), what everyone has really been holding their breaths for was an implementation (of any variation/design), to see that one was possible.

Google and others have been working on such an implementation in Clang, and I was very excited to hear Chandler Carruth (head of Google’s team working on Clang) report that they have now completed it! As this work was completely very recently prior to the meeting, they did not get a chance to write a paper to present at this meeting, but Chandler said one will be forthcoming for the next meeting.

EWG held a session on Modules, where Gaby presented his paper, and the Clang folks discussed their implementation. There were definitely some differences between the two. Gaby’s proposal came across as more idealistic, “this is what a module system should look like if you’re writing new code from scratch with modules”. Clang’s implementation is more practical, “this is how we can start using modules right away in existing codebases”. For example, in Gaby’s proposal, a macro defined in a module is not visible to an importing module; in Clang’s implementation, it is, reflecting the reality that today’s codebases still use macros heavily. As another example, in Gaby’s proposal, a module writer must say which declarations in the module file are visible to importing modules by surrounding them in an export block (export here just being a keyword, not to be confused with the failed language feature I talked about above); in Clang’s implementation, a header file can be used as a module without any changes, using an external “module map” file to tell the compiler it is a module.

Another interesting design question that came up was whether private members of a class exported from a module are “visible” to an importing module (in the sense that importing modules need to be recompiled if such a private member is added or modified); in Clang’s implementation, this is the case, but there would certainly be value in avoiding this (among other things, it would obsolete the laborious “Pimpl” design pattern).

The takeaway was that, while everyone wants this feature, and everyone is excited about there finally being an implementation, several design points still need to be decided. EWG deemed that it was too early to take any polls on this topic, but instead encouraged the two parties (the Clang folks, and Gaby, who works for Microsoft and hinted at a possible Microsoft implementation effort as well) to collaborate on future work. Specifically, EWG encourages that the following papers be written for Urbana: one about what is common to the various proposals, and one or more about the remaining deep technical issues. I eagerly await such future work and papers.

SG 3 (Filesystems)

At this meeting, the Library Working Group finished addressing the ballot comments for the Filesystem TS’s PDTS ballot, and sent out the TS for the final “DTS” ballot. If this ballot is successful, the Filesystems TS will be published by the end of 2014.

Beman (the SG 3 chair) stated that SG 3 will entertain new filesystem-related proposals that build upon the Filesystems TS, targetting a follow-up Filesystems TS II. To my knowledge no such proposals have been submitted so far.

SG 4 (Networking)

SG 4 had been working on standardizing basic building blocks related to networking, such as IP addresses and URIs. However, these efforts are stalled.

As a result, the LEWG decided at this meeting to take it upon itself to advance networking-related proposals, and they set their sights on something much more ambitious than IP addresses and URIs: a full-blown sockets library, based on Boost.ASIO. The plan is basically to pick up Chris Kohlhoff’s (the author of ASIO) 2007 paper proposing ASIO for standardization, incorporating changes to update the library for C++11, as well as C++14 (forthcoming). This idea received very widespread support in LEWG; the group decided to give people another meeting to digest the new direction, and then propose adopting these papers as the working paper for the Networking TS in Urbana.

This change in pace and direction might seem radical, but it’s in line with the committee’s philosophy for moving more rapidly with TS’s. Adopting the ASIO spec as the initial Networking TS working paper does not mean that the committee agrees with every design decision in ASIO; on the contrary, people are likely to propose numerous changes to it before it gets standardized. However, having a working paper will give people something to propose changes against, and thus facilitate progress.

SG 5 (Transactional Memory)

The Transactional Memory TS is progressing well through the committee. CWG began reviewing its wording at this meeting, and referred one design issue to EWG. (The issue concerned functions that were declared to be neither transaction-safe nor transaction-unsafe, and defined out of line (so the compiler cannot compute the transaction safety from the definition). The state of the proposal coming into the discussion was that for such functions, the compiler must assume that they can be either transaction-safe or transaction-unsafe; this resulted in the compiler sometimes needing to generate two versions of some functions, with the linker stripping out the unused version if you’re lucky. EWG preferred avoiding this, and instead assuming that such functions are transaction-unsafe.) CWG will continue reviewing the wording in Urbana, and hopefully sendout the TS for its PDTS ballot then.

SG 6 (Numerics)

Did not meet in Rapperswil, but plans to meet in Urbana.

SG 7 (Reflection)

SG 7 met for an evening session and looked at three papers:

  • The latest version of the source code information capture proposal, which aims to replace the __LINE__, __FILE__, and __FUNCTION__ macros with a first-class language feature. There was a lot of enthusiasm for this idea in Issaquah, and now that it’s been written up as a paper, SG 7 is moving on it quickly, deciding to send it right on to LEWG with only minor changes. The publication vehicle – with possible choices being Library Fundamentals TS II, a hypothetical Reflection TS, or C++17 – will be decided by LEWG.
  • The type member property queries proposal by Andrew Tomazos. This is an evolution of an earlier proposal which concerned enumerations only, and which was favourably reviewed in Issaquah; the updated proposal extends the approach taken for enumerations, to all types. The result is already a quite featureful compile-time introspection facility, on top of which facilities such as serialization can be built. It does have one significant limitation: it relies on forming pointers to members, and thus cannot be used to introspect members to which pointers cannot be formed – namely, references and bitfields. The author acknowledged this, and pointed out that supporting such members with the current approach would require language changes. SG 7 did not deem this a deal-breaker problem, possibly out of optimism that such language changes would be forthcoming if this facility created a demand for them. Overall, the general direction of the proposal had basically unanimous support, and the author was encouraged to come back with a revised proposal that splits out the included compile-time string facility (used to represent names of members for introspection) into a separate, non-reflection-specific proposal, possibly targeted at Library Fundamentals II. The question of extending this facility to introspect things other than types (notably, namespaces, although there was some opposition to being able to introspect namespaces) also came up; the consensus here was that such extensions can be proposed separately when desired.
  • A more comprehensive static reflection proposal was looked at very, very briefly (the author was not present to speak about it in detail). This was a higher-level and more featureful proposal than Tomazos’ one; the prevailing opinion was that it is best to standardize something lower-level like Tomazos’ proposal first, and then consider standardizing higher-level libraries that build on it if appropriate.

SG 8 (Concepts)

The Concepts TS (formerly called “Concepts Lite”, but then people thought “Lite” was too informal to be in the title of a published standard) is still in the CWG review stage. Even though CWG looked at it in Issaquah, and the author and project editor, Andrew Sutton, revised the draft TS significantly for Rapperswil, the feature touches many areas of the language, and as such more review of the wording was required; in fact, CWG spent almost three full days looking at it this time.

The purpose of a CWG review of a new language feature is twofold: first, to make sure the feature meshes well with all areas of the language, including interactions that the author and EWG may have glossed over; and second, to make sure that the wording reflects the author’s intention accurately. In fulfilling the first objective, CWG often ends up making minor changes to a feature, while staying away from making fundamental changes to the design (sometimes, recommendations for more significant changes do come up during a CWG review – these are run by EWG before being acted on).

In the case of the Concepts TS, CWG made numerous minor changes over the course of the review. It was initially hoped that there would be time to revise the wording to reflect these changes, and put the reivsed wording out for a PDTS ballot by the end of the meeting, but the changes were too numerous to make this feasible. Therefore, the PDTS ballot proposal was postponed until Urbana, and Andrew has until then to implement the wording changes.

SG 9 (Ranges)

SG 9 did not meet in Rapperswil, but does plan to meet in Urbana, and I anticipate some exciting developments in Urbana.

First, I learned that Eric Niebler, who in Issaquah talked about an idea for a Ranges proposal that I thought was very exciting (I describe it in my Issaquah report), plans to write up his idea as a proposal and present it in Urbana.

Second, one of the attendees at Rapperswil, Fabio Fracassi, told me that he is also working on a (different) Ranges proposal that he plans to present in Urbana as well. I’m not familiar with his proposal, but I look forward to it. Competition is always healthy when it comes up early-stage standards proposal / choosing an approach to solving a problem.

SG 10 (Feature Test)

I didn’t follow the work of SG 10 very closely. I assume that, in addition to the __has_cpp_attribute() preprocessor feature that I mentioned above in the EWG section, they are kept busy by the flow of new features being added into working papers, for each of which they have to decide whether the feature deserves a feature-test macro, and if so standardize a name for one.

Clark (the SG 10 chair) did mention that the existence of TS’s complicates matters for SG 10, but I suppose that’s a small price to pay for the benefits of TS’s.

SG 12 (Undefined Behaviour)

Did not meet in Rapperswil, but plans to meet in Urbana.

SG 13 (Human Interaction, formerly “Graphics”)

SG 13 met for a quarter-day session, during which Herb presented an updated version of the proposal for a cairo-based 2D drawing API. A few interesting points came up in the discussion:

  • The interface being standardized is a C++ wrapper interface around cairo that was generated using a set of mechanical transformation rules applied to cairo’s interface. The transformation rules are not being standardized, only their result (so the standard interface can potentially diverge from cairo in the future, though presumably this wouldn’t be done without a very good reason).
  • I noted that Mozilla is moving away from cairo, in part due to inefficiencies caused by cairo being a stateful API (as explained here). It was pointed out that this inefficiency is an issue of implementation (due to cairo using a stateless layer internally), not one of interface. This is a good point, although I’m not sure how much it matters in practice, as standard library vendors are much more likely to ship cairo’s implementation than write their own. (Jonathan Wakely said so about libstdc++, but I think it’s very likely the case for other vendors as well.)
  • Regarding the tradeoff between a nice interface and high performance, Herb said the goal was to provide a nice interface while providing as good of a performance as we can get, without necesarily squeezing every last ounce of performance.
  • The library has the necessary extension points in place to allow for uses such as hooking into drawing onto a drawing surface of an existing library, such as a Qt canvas (with the cooperation of the existing library, cairo, and the platform, of course).

The proposal is moving forward: the authors are encouraged to come back with wording.

TS Content Guidelines

One mildly controversial issue that came to a vote in the plenary meeting at the end of the week, is the treatment of modifications/extensions to standard library types in a Technical Specification. One group held that the simplest thing to do for users is to have the TS specify modifications to the types in std:: themselves. Another group was of the view that, in order to make life easier for a third-party library vendor to implement a TS, as well as to ensure that it remains practical for a subsequent IS to break the TS if it needs to, the types that are modified should be cloned into an std::experimental:: namespace, and the modifications applied there. This second view prevailed.

Next Meeting

The next Committee meeting (“Urbana”) will be at the University of Illinois at Urbana-Champaign, the week of November 3rd.

Conclusion

The highlights of the meeting for me, personally, were:

  • The relevation that clang has completed their modules implementation, that they will be pushing it for C++17, and that they are fairly confident that they will be able to get it in. The adoption of a proper modules system has the potential to revolutionize compilation speeds and the tooling landscape – revolutions that C++ needs badly.
  • Herb’s proposal for a portable C++ ABI. It is very encouraging to see the committee, which has long held this issue to be out of its scope, looking at a concrete proposal for solving a problem which, in my opinion, plays a significant role in hampering the use of C++ interfaces in libraries.
  • LEWG looking at bringing the entire Boost.ASIO proposal into the Networking TS. This dramatically brings forward the expected timeframe of having a standard sockets library, compared to the previous approach of standardizing first URIs and IP addresses, and then who knows what before finally getting to sockets.

I eagerly await further developments on these fronts and others, and continue to be very excited about the progress of C++ standardization.

Trip Report: C++ Standards Committee Meeting in Issaquah, February 2014

TL;DR

  • C++14 on track to be published this year, no further wording changes expected
  • Filesystems TS expected to be published in 2014-2015 timeframe
  • Library Fundamentals TS containing ‘optional’, ‘any’, ‘string_view’, polymorphic allocators and more expected early 2015
  • Array Extensions TS stalled while ‘dynarray’ is being replaced with something better, still under design
  • Networking TS for URIs and IP addresses progressing slowly; byte-order conversion moved to Library Fundamentals TS
  • Concurrency, Parallelism, and Transactional Memory TS’s making progress, expected 2015-ish
  • Concepts Lite proposal progressing well, a few more additions/changes possible; TS likely in early 2015 timeframe
  • New Concepts-enabled STL on the horizon, separate from current STL (to maybe replace it one day)
  • Reflection study group actively considering proposals for C++17

Introduction

Last week I attended a meeting of the ISO C++ Standards Committee (formally called ISO/IEC JTC1/SC22/WG21) in Issaquah, Washington (near Seattle). This was the second Committee meeting I attended, the first one having been the Chicago meeting last September. For some context for this post, you may want to look at my thread on mozilla.dev.platform about that meeting, particularly my posts about what is in C++14, what is in the various follow-up Technical Specifications, and how the Standards Committee is structured and operates.

This time, I thought I’d report on this meeting in the form of a blog post rather than on mozilla.dev.platform, both as an excuse to start blogging (the three posts I wrote three years ago don’t really count), and because my report may also be of interest outside the Mozilla community.

The focus of this meeting was to finalize C++14, make progress on various Technical Specifications (TS) that are planned as follow-ups to C++14, and work on language and library features headed for C++17.

C++14

The Committee voted to approve a draft of C++14 which will likely become the final published standard, without any further changes, later this year. The Committee is composed of delegations from national standards bodies, and the procedure for putting out a new language standard is to put out a draft, give the national bodies a chance to study the draft and give feedback on it, act on the feedback and put on a new draft, and so on until there is consensus to publish a draft, with no further changes, as an International Standard. ISO procedure mandates a fixed number of these formal rounds of feedback, called “balloting”. C++14 has already gone through one round of balloting, and the draft approved in Issaquah incorporated the feedback from that round. This draft is now out for another round of balloting, and if there are no further comments, which is considered likely, it will be published as a standard later this year. In the unlikely event that new  issues come up during this round of balloting, another draft and another ballot with be necessary, causing “C++14” to slip to early 2015 (in which case it would then be called C++15). Again, this slippage is considered unlikely.

The changes made to the C++14 draft during the Issaquah meeting were very minor in nature. They were handled by the Core and Library Working Groups, and were typically things likely clarifying wording in the standard to reflect existing intent. No new features, nontrivial changes to existing features, or anything else I think is worth calling out here, were made.

Evolution Working Group / SG 8 (Concepts)

I spent most of my time in the Evolution Working Group (EWG), eloping from time to time to sit in on a meeting of a Study Group (SG) that I thought was interesting. I should note that SG 8 (Concepts), the group I am the most interested in, did not meet independently of EWG; rather, Concepts-related proposals were presented to and discussed by EWG at large.

EWG is, appropriately enough, chaired by Bjarne Stroustrup, the designer and original implementor of C++. The general procedure of EWG is to consider proposals, which typically take the form of papers formally submitted to the Committee, one by one. The consideration of a proposal begins with the proposal’s author (or, if the author is not present, then someone who is present who the author has asked to champion their proposal) presenting the proposal. The group then discusses it, taking into account considerations such as whether the proposal breaks any existing code, how it affects the complexity of the language, how it interacts with other features, and what alternatives there are for its intended use cases. The chair tries to guide the discussion towards the path of greatest consensus. After discussion, straw polls are commonly taken to establish the prevailing opinion of the group. A straw poll can be between two alternative designs in a proposal, between suggesting that the proposal’s author continue iterating on their proposal or not (in the earlier stages of a proposal), or between sending the proposal onward to the Core Working Group (CWG) or not (in the later stages of a proposal). If there is consensus in EWG to send the proposal onward to the CWG, the proposal’s author writes standard wording (if they haven’t done so already); having CWG review and approve the wording is the final step before a motion can be brought in front of the full committee to add the wording into the draft standard (such a motion then typically passes as the EWG and CWG having approved the proposal tends to be good enough for others in the committee).

With C++14 being feature-complete, EWG spent the week looking at the Concepts Lite proposal, language changes in other TS’s, and proposals for language features in C++17.

Concepts Lite, headed for a post-C++14 TS, was in a stage where design discussions had already taken place at previous meetings, and the proposal’s primary author, Andrew Sutton, already had proposed wording for a draft TS coming into the meeting. This wording underwent three rounds of review over the course of the week: one in EWG, an informal one in CWG, and finally the formal one in CWG. Each round of review generated feedback that Andrew promptly incorporated into the wording before the next round. The outcome of the final CWG review was that CWG approved the wording with the modifications recommended in that round. The next step of the process is for Andrew to prepare a draft TS based on the modified wording, which will then be presented to the committee for adoption as a working paper of the TS at the beginning of the next meeting (which will be in June, in Rapperswil, Switzerland).

Note that having a working paper does not mean that a TS is finalized. The committee can amend the working paper as many times as they like before advancing it to “Preliminary Draft TS” (PDTS) status, at which point a balloting procedure similar to that used for the Standard itself, begins. This means that there is still room for changes/extensions to be made to the proposal that can go into the TS.

I presented a proposal for some extensions to Concepts Lite, co-authored with Andrew, to EWG. This proposal was in a much earlier stage than the rest of Concepts Lite – it was being presented to EWG for the first time, to see if they like the direction. The intent of my proposal was to add a few related extensions that increase the cohesion and consistency of the various parts of Concepts Lite, while also gaining expressive power. Interestingly, it led some participants to suggest going even further than what I proposed, and making some of the new syntax that I was proposing as alternatives to existing syntax, mandatory instead. This in turn was quite controversial, and the discussion that followed brought to light some differences of opinion between members of EWG, such as whether it is better for code to be short and clean but implicit, or longer but explicit, and whether it is better to give language users more or less choice in how they use the language. One thing the discussion did not generate was consensus, and as such Bjarne recommended that we revisit this proposal in Rapperswil, after people have had time to think about the points that were brought up.

In addition to the extensions I proposed, Andrew is planning to propose some extensions of his own to Concepts Lite in Rapperswil, such as the ability to use a concept as the return type of a function, the ability to use a concept as the type of a variable (an ability that I alluded to as a possible future feature in section 2.3 of my proposal), and the ability to pass a overload set as a function argument (this last one is not strictly related to Concepts and as such may be proposed directly for C++17 rather than for the Concepts Lite TS). I look forward to each of these extensions, as I believe they will contribute to making Concepts a more cohesive and well-rounded language feature.

Looking a little more into the future, after the Concepts Lite TS is finalized, the next step for Concepts is to revise the standard library, particularly the STL, to take advantage of the language features added in Concepts Lite. Concepts was originally planned for inclusion in C++11, but for various reasons it ended up being backed out. One of the reasons for this was that it was too ambitious, trying to introduce a major new language feature and incorporate it into into the standard library, while maintaining perfect compatibility with existing code using the library. Concepts Lite has learned from this mistake, and scaled back the magnitude of the language changes as well as leaving library changes for a separate proposal. Importantly, the prevailing thought among the proponents of Concepts now is that such a library will not be a modification of the existing STL, but a new, separate libary built from the ground up using Concepts Lite. Such a library would aim to provide the same functionality as the current STL, but not maintain perfect compatibility with code written for the current STL. Existing codebases could stick with the current STL or adopt the new one at their leisure, and new code would presumably be written against the new one. Eventually (this will undoubtedly take many years) the current STL would be deprecated and ultimately removed in favour of the new one. I should stress that at this point, all this is speculation. The focus is on getting the Concepts Lite TS published first.

Besides Concepts Lite, EWG considered some language proposals for C++17. A couple of them were really bug fixes for C++14, but it was too late to get them into C++14. It is expected that compilers will implement them promptly and, in practice, C++14 users will be able to use them. The first is to fix a rule that said that in auto x{1};, the type of x would be std::initializer_list. The rationale for this was that auto x{1} should be consistent with auto x{1, 2}, which should in turn be consistent with auto x = {1, 2}, which should in turn be consistent with std::initializer x = {1, 2}. While an argument can be made for each of these points, the result that auto x{1} makes x an std::initializer_list is very counterintuitive. Worse, C++14 introduces generalized lambda captures, with the initialization of the new variable following the same rules as auto, so in [x{1}](){ ... }, the type of x would also be std::initializer_list, which is even more counterintuitive, and such code is likely to be very common as people start to adopt “uniform initialization”, i.e. using the {...} syntax for initialization everywhere. The agreed-upon resolution was that in auto x{1} (and [x{1}](){ ... }), x would be an int, auto x{1, 2} would be invalid, and auto x = {1, 2} would continue to make x an std::initializer_list. I think this is a very reasonable compromise.

The other bugfix/feature-let was to allow static_assert to be used without a string argument; that is, in addition to writing static_assert(some_expression, "some string"), you can also write just static_assert(some_expression). The error message displayed by the second form is implementation-defined, with an obvious choice being the text of some_expression.

In terms of new features proper, EWG looked at a proposal to extend the syntax of range-based for loops to allow for (elem : range) to be a shorthand for for (auto&& elem : range). The rationale is that for (auto&& elem : range) almost always does what you want (makes elem a reference of the appropriate type and binds it to each element of the range without copying the element; see Scott Meyers’ article about “universal references” for how this works), while other forms do not (for (auto elem : range) copies the element; for (auto& elem : range) does not work for ranges with rvalue elements; and for (const auto& elem : range) does not allow you to modify the element), so for (auto&& elem : range) should be the default form of the range-based for loop, and particularly the form used in generic code. However, it’s awkward to have to teach people to use the funky-looking && syntax by default. The proposal met some resistance and being unnecessary, but ultimately it garnered consensus and will progress to the CWG at the next meeting.

EWG also considered enforcing the ‘Rule of 5’ in C++17, which says that if any one of the 5 “special member functions” of a class (copy constructor, copy assignment operator, move constructor, move assignment operator, and destructor) are user-declared, none of the other 5 are auto-generated by the compiler. It is already recommended that if you write one of the 5, you should write all of the 5, but this is not currently enforced. EWG decided that, unfortunately, due to backwards compatibility constraints, it is not feasible to start enforcing this even in the C++17 timeframe. The proposal may resurface for a the next version of C++ after C++17 (likely C++20 if the committee continues its 3-year cadence).

SG 13 (Graphics)

On Tuesday afternoon, I stepped out of EWG to attend the meeting of SG 13 (Graphics), which is currently working on standardizing a lightweight 2D drawing API for C++.

SG 13’s main intended purpose for such an API is to allow people learning C++ and writing graphical applications to do so easily, without having to rely on third-party libraries or learning complex APIs. In the long-term, however, they would like the drawing API to be useful for people writing application frameworks and high-performance applications as well.

Of course, there is more to being able to write graphical applications easily than having a 2D drawing API, and in this respect the SG’s name “Graphics” is a bit misleading: the group also aims to eventually standardize a means of listening to input events in a graphical application, and more. This was pointed out at the meeting, and it was decided that the SG’s name would be changed to something else like “Interaction”. Such projects, however, are farther in the future – for now the SG is focused on a 2D drawing API, and their first product will be a TS specifying such an API.

This TS is still in the early design stage. The SG is looking at using cairo as the basis of the drawing API, with the interface C++-ified using a set of clear and consistent rules. See the current draft of the proposal.

It’s important to note that the thing being standardized here is an interface, not an implementation. Given that the interface is that of cairo, a standard library vendor is likely to find it very convenient to implement the interface by using cairo itself, but this is by no means required – vendors are free to implement the interface however they choose.

Note that this freedom has important performance implications. One of the performance problems cairo suffers from is that it is a stateful API, but internally it has a stateless layer. If it is using a backend implementation which is itself stateful, we get unnecessary dupliation of state (Robert O’Callahan explains this nicely here). The standard API, however, would not neecessarily suffer from this problem, because a vendor intending to use a stateful backend API would be free to implement the interface using that API directly, and thus avoid this duplication of state.

It is also important to note that standardizing an API that is based on the current version of cairo does not imply being locked in to continue using an API based on cairo forever. The Standards Committee would reserve the right to modify the API in ways that make it diverge from cairo if it sees fit. In turn this means that while implementers may find it convenient to use cairo itself as the implementation at first, they are not guaranteed to continue to be able to use cairo (unmodified) as the implementation forever.

Finally, note that while this cairo proposal is being worked on actively, and SG 13 likes its direction, the group has not yet committed to it. If someone wishes to write a competing proposal based on a different API (for example, Moz2D), there is still time to do so.

SG 9 (Ranges)

SG 9, tasked with developing a cohesive set of abstractions and a library for working with them for Ranges (in the sense of the range-based for loop and Boost.Range), did not have a formal meeting in Issaquah. In fact, as far as I am aware, the progress of the group is stalled. The last piece of activity that I am aware of is a lengthy discussion on the group’s mailing list about proposals based on a range primitive (in contrast to today’s ranges which are built on top of an iterator primitive), and/or a position primitive (somewhat but not quite like an iterator primitive). While there are valid abstraction and performance reasons for building a range library based on range and/or position primitives rather than an iterator primitive, these approaches have the disadvantage that they go against the grain of a very established tradition of iterator-based ranges which is enshrined in the STL and many other libraries like Boost.Range, and no one seems to have gotten a range/position-based design quite right yet.

However, I participated in an informal conversation with Eric Niebler, a prominent member of the Boost community and author of the Boost.Proto and Boost.Xpressive libraries, and Andrew Sutton, regarding a library Eric has been developing which sounded very exciting to me.

The library in question is a prototype reimplementation of large parts of the STL, plus more range functionality (some of it similar to the facilities in Boost.Range), based on a slightly modified but still iterator-based Range concept. While a traditional Range consists of two Iterators of the same type, Eric introduces a relaxation of the Range concept which he calls Iterable, which consists of two Iterators of potentially different types. The motivation behind this is to allow efficient support for so-called sentinel ranges, which consist of a single iterator that represents a position in the range, and a predicate that tests whether this position is at the end of the range. The classic example of a sentinel range is a C-style (i.e. null-terminated) string, where the iterator is a pointer to the first character in the string, and the predicate tests whether the character at the pointer is the null character. Of course, a C-style string can be represented by a conventional pair of iterators (a pair of pointers to the first and past-the-end characters), but building this representation requires walking the string in linear time to find its end, while the sentinel representation does not.

You can shoehorn a sentinel representation into the conventional Range representation (two iterators of the same type) by constructing an iterator type which is essentially the union type of the iterator type in the sentinel representation, and a sentinel type. The constructed iterator type’s operator== would then check if the two iterators being compared are a non-sentinel and a sentinel, respectively, and if so invoke the predicate that determines whether the former is at the end of the range. However, this incurs a runtime penalty, as the operator==, which is called once per iteration over the range, needs to perform a runtime check to determine whether the iterator is a sentinel or not. By allowing the begin and end iterators to have different types, however, we can encode the sentinel-ness of an iterator in its type, and avoid this runtime penalty.

Of course, not all algorithms are suitable for use with sentinel ranges. For example reverse() is not, because it needs to walk the range backwards, but a sentinel iterator cannot be moved backwards. For algorithms like this, Eric keeps around the conventional Range concept, which is now a refinement of the Iterable concept that requires the begin and end iterators to be of the same type.

Eric’s library also supports infinite ranges, which are useful e.g. for representing generator ranges (i.e. the range x, f(x), f(f(x)), f(f(f(x))), ... for some initial value x and function f). Sentinel ranges are well-suited to represent infinite ranges: you simply have a predicate that always returns false, so you are never at the end of the range. A sore point for representing infinite ranges with the current Range concept is what type to use as the difference_type of a range. Eric resolves this by inventing an integer-like type which can also have “infinity” as a possible value, not unlike how regular floating-point types are able to represent infinity. (Update: Eric has informed me that he has actually changed his approach to the difference_type of infinite ranges. Stay tuned.)

I am very excited by Eric’s work, because it seems to accomplish the goal of providing powerful range abstractions, and unlike the approaches brought up on the SG 9 mailing list, this approach is something that is implemented and working, and meshes well the existing Range concept used in the STL.

If the committee ends up liking Eric’s approach for a standard set of abstractions for ranges, the question of how these new abstractions will fit into the current STL will likely arise. If the idea I described above about a Concepts-enabled STL that makes a clean break with the current STL goes forward, it may be a nice opportunity to also break with the standard’s current Range concepts and use Eric’s new ones from the start. Two birds, one stone.

Update: Since I started writing this blog post, Eric has also blogged about his range abstractions, and he explains them in a lot more detail:

SG 7 (Reflection)

SG 7 looked at two proposals concerning compile-time reflection, a small but detailed one and a more ambitious but vague one.

The smaller proposal concerned getting compile-time information about an enumeration, such as the number of elements, and the name and value of each element. This proposal received positive feedback, and the author was encouraged to prepare are updated proposal for the Library Evolution Working Group (LEWG) to look at in Rapperswil.

The more ambitious paper proposed:

  • an introspection mechanism whereby the programmer can inspect a program construct in the form of an AST node
  • a reification mechanism that processes arbitrary C++ tokens at compile time with HTML-template-engine-like syntax (containing constructs such as $for, $if, and $switch) and which can use the AST nodes mentioned above as variables
  • a mechanism to extend the set of valid inputs to the introspection mechanism from C++ code to arbitrary text, by essentially defining new “AST nodes”; the idea here was to enable support for domain-specific languages (DSLs)

This was some of the group’s feedback on the paper:

  • It conflated the notion of an “AST” as an abstract/idealized representation of program structure with the notion of an “AST” as the concrete internal representation used by a compiler (the paper went as far as to link to a node of the clang AST as reference for the sorts of methods you can call on an AST node during introspection). While the former may conceivably be standardized, it is unreasonable to expect that the latter can be.
  • Operating at the level of tokens is not optimal for a reification mechanism. It is much better to operate on higher-level, semantic entities.
  • The DSL part of the proposal is out of scope for SG 7, which is looking to be able to introspect the language, not extend it. It was noted, however, that mechanisms that take C++ code as input and produce transformed C++ code as output are in scope for SG 7, albeit probably at a more advanced stage.

Given this feedback, it is very unlikely that the proposal will go forward in its current form. However, there are some good ideas and use cases in there, and it will probably serve to inform future proposals in useful ways.

In addition to these two formal proposals, two informal ideas were floated by the participants, and both garnered strong support from the group. The first was introducing a compile-time string literal class; this would be similar to a regular string class, but all the operations on it could be performed at compile time. The second was providing non-macro replacements for __LINE__, __FILE__, and __FUNCTION__, which are currently one of the few remaining unavoidable uses of the preprocessor. The group expects proposals for these two ideas to be forthcoming.

Library / Library Evolution Working Groups (LWG and LEWG)

There is much happening on the library side, with 4 TS’s in the works.

The Filesystems TS is in very good shape. The TS had already been sent out for its first round of balloting; LWG spent much of its time in Issaquah addressing the ensuing comments. The plan is to send out a revised TS for a second and hopefully final round of balloting, leading to publication in late 2014/early 2015.

The Library Fundamentals TS started out containing just optional (a standardized version of Boost.Optional), but it picked up a number of new goodies in Issaquah:

The Library Fundamentals TS is on track to be sent out for its first ballot in Rapperswil, which means if all goes well it’s looking at publication in the first half of 2015.

The Array Extensions TS currently contains “arrays of runtime bound” (the C++ flavour of variable-length arrays) and dynarray. However, dynarray has been facing implementation challenges, and EWG is in the process of designing a better replacement for it that doesn’t require compiler magic to implement (by introducing a new language feature proper). While this happens, the Array Extensions TS is not moving forward. It did, however, pick up an additional library utility in Issaquah: an array_view class that is similar to string_view but not specific to characters.

The Networking TS is progressing more slowly than expected. As mentioned above, the network byte order conversions part of it was moved into Library Fundamentals, leaving URIs and IP addresses in the Networking TS.

SG 1 (Concurrency) and SG 5 (Transactional Memory)

I don’t follow these groups closely and I’m not very familiar with their work. I do know that SG 1 is working on a Parallelism TS and a Concurrency TS (based on this paper and this one), and SG 5 is working on a TS. These are probably expected sometime next year.

SG 6 (Numerics)

SG 6 reviewed a proposal for adding decimal floating-point support to C++, and is sending it on to be presented to EWG in Rapperswil. It is an open question whether this would be standardized as a TS or go directly into C++17.

Other Study Groups

I covered the work of SG 3 (Filesystem) and SG 4 (Networking) under the Library section above. SG 2 (Modules), SG 10 (Feature Test), and SG 12 (Undefined Behaviour) did not meet in Issaquah. SG 11 (Databases) was disbanded due to no one being available to chair it; presumably LEWG will still consider database-related proposals directly.

Next Meeting

As I mentioned above, the next meeting of the C++ Standards Committee will be in Rapperswil, Switzerland, the week of June 16th.

Conclusion

I think this is a really exciting time for C++, for several reasons:

  • The 3-year release cadence that the Committee seems to  be converging on (C++11, C++14, C++17, etc.) is refreshing after the 13-year wait between C++98 and C++11.
  • The emerging use of Technical Specifications as a way of trying out significant new features while retaining the ability to change them before accepting them into the actual standard allows a lot more work to be done in parallel. The Committee is looking at having no fewer than eight different TS’s published over the next two years (Filesystems, Networking, Concepts, Library Fundamentals, Array Extensions, Concurrency, Parallelism, and Transactional Memory). It is very unlikely that all these features would have made it into an actual standard published in the same timeframe.
  • Compilers’ implementation of Standard C++ is rapidly accelerating. Cland 3.4 already implements all of C++14 (in the state it was in at the end of the last meeting; the minor changes made in Issaquah will presumably be implemented in 3.5). GCC is not far behind. (Of course, both already implement C++11). MSVC doesn’t fully support even C++11 yet, but it seems determined to catch up.

I will continue to follow the progress of C++ standardization and blog about interesting things I discover.