Trip Report: C++ Standards Committee Meeting in Issaquah, February 2014

TL;DR

  • C++14 on track to be published this year, no further wording changes expected
  • Filesystems TS expected to be published in 2014-2015 timeframe
  • Library Fundamentals TS containing ‘optional’, ‘any’, ‘string_view’, polymorphic allocators and more expected early 2015
  • Array Extensions TS stalled while ‘dynarray’ is being replaced with something better, still under design
  • Networking TS for URIs and IP addresses progressing slowly; byte-order conversion moved to Library Fundamentals TS
  • Concurrency, Parallelism, and Transactional Memory TS’s making progress, expected 2015-ish
  • Concepts Lite proposal progressing well, a few more additions/changes possible; TS likely in early 2015 timeframe
  • New Concepts-enabled STL on the horizon, separate from current STL (to maybe replace it one day)
  • Reflection study group actively considering proposals for C++17

Introduction

Last week I attended a meeting of the ISO C++ Standards Committee (formally called ISO/IEC JTC1/SC22/WG21) in Issaquah, Washington (near Seattle). This was the second Committee meeting I attended, the first one having been the Chicago meeting last September. For some context for this post, you may want to look at my thread on mozilla.dev.platform about that meeting, particularly my posts about what is in C++14, what is in the various follow-up Technical Specifications, and how the Standards Committee is structured and operates.

This time, I thought I’d report on this meeting in the form of a blog post rather than on mozilla.dev.platform, both as an excuse to start blogging (the three posts I wrote three years ago don’t really count), and because my report may also be of interest outside the Mozilla community.

The focus of this meeting was to finalize C++14, make progress on various Technical Specifications (TS) that are planned as follow-ups to C++14, and work on language and library features headed for C++17.

C++14

The Committee voted to approve a draft of C++14 which will likely become the final published standard, without any further changes, later this year. The Committee is composed of delegations from national standards bodies, and the procedure for putting out a new language standard is to put out a draft, give the national bodies a chance to study the draft and give feedback on it, act on the feedback and put on a new draft, and so on until there is consensus to publish a draft, with no further changes, as an International Standard. ISO procedure mandates a fixed number of these formal rounds of feedback, called “balloting”. C++14 has already gone through one round of balloting, and the draft approved in Issaquah incorporated the feedback from that round. This draft is now out for another round of balloting, and if there are no further comments, which is considered likely, it will be published as a standard later this year. In the unlikely event that new  issues come up during this round of balloting, another draft and another ballot with be necessary, causing “C++14” to slip to early 2015 (in which case it would then be called C++15). Again, this slippage is considered unlikely.

The changes made to the C++14 draft during the Issaquah meeting were very minor in nature. They were handled by the Core and Library Working Groups, and were typically things likely clarifying wording in the standard to reflect existing intent. No new features, nontrivial changes to existing features, or anything else I think is worth calling out here, were made.

Evolution Working Group / SG 8 (Concepts)

I spent most of my time in the Evolution Working Group (EWG), eloping from time to time to sit in on a meeting of a Study Group (SG) that I thought was interesting. I should note that SG 8 (Concepts), the group I am the most interested in, did not meet independently of EWG; rather, Concepts-related proposals were presented to and discussed by EWG at large.

EWG is, appropriately enough, chaired by Bjarne Stroustrup, the designer and original implementor of C++. The general procedure of EWG is to consider proposals, which typically take the form of papers formally submitted to the Committee, one by one. The consideration of a proposal begins with the proposal’s author (or, if the author is not present, then someone who is present who the author has asked to champion their proposal) presenting the proposal. The group then discusses it, taking into account considerations such as whether the proposal breaks any existing code, how it affects the complexity of the language, how it interacts with other features, and what alternatives there are for its intended use cases. The chair tries to guide the discussion towards the path of greatest consensus. After discussion, straw polls are commonly taken to establish the prevailing opinion of the group. A straw poll can be between two alternative designs in a proposal, between suggesting that the proposal’s author continue iterating on their proposal or not (in the earlier stages of a proposal), or between sending the proposal onward to the Core Working Group (CWG) or not (in the later stages of a proposal). If there is consensus in EWG to send the proposal onward to the CWG, the proposal’s author writes standard wording (if they haven’t done so already); having CWG review and approve the wording is the final step before a motion can be brought in front of the full committee to add the wording into the draft standard (such a motion then typically passes as the EWG and CWG having approved the proposal tends to be good enough for others in the committee).

With C++14 being feature-complete, EWG spent the week looking at the Concepts Lite proposal, language changes in other TS’s, and proposals for language features in C++17.

Concepts Lite, headed for a post-C++14 TS, was in a stage where design discussions had already taken place at previous meetings, and the proposal’s primary author, Andrew Sutton, already had proposed wording for a draft TS coming into the meeting. This wording underwent three rounds of review over the course of the week: one in EWG, an informal one in CWG, and finally the formal one in CWG. Each round of review generated feedback that Andrew promptly incorporated into the wording before the next round. The outcome of the final CWG review was that CWG approved the wording with the modifications recommended in that round. The next step of the process is for Andrew to prepare a draft TS based on the modified wording, which will then be presented to the committee for adoption as a working paper of the TS at the beginning of the next meeting (which will be in June, in Rapperswil, Switzerland).

Note that having a working paper does not mean that a TS is finalized. The committee can amend the working paper as many times as they like before advancing it to “Preliminary Draft TS” (PDTS) status, at which point a balloting procedure similar to that used for the Standard itself, begins. This means that there is still room for changes/extensions to be made to the proposal that can go into the TS.

I presented a proposal for some extensions to Concepts Lite, co-authored with Andrew, to EWG. This proposal was in a much earlier stage than the rest of Concepts Lite – it was being presented to EWG for the first time, to see if they like the direction. The intent of my proposal was to add a few related extensions that increase the cohesion and consistency of the various parts of Concepts Lite, while also gaining expressive power. Interestingly, it led some participants to suggest going even further than what I proposed, and making some of the new syntax that I was proposing as alternatives to existing syntax, mandatory instead. This in turn was quite controversial, and the discussion that followed brought to light some differences of opinion between members of EWG, such as whether it is better for code to be short and clean but implicit, or longer but explicit, and whether it is better to give language users more or less choice in how they use the language. One thing the discussion did not generate was consensus, and as such Bjarne recommended that we revisit this proposal in Rapperswil, after people have had time to think about the points that were brought up.

In addition to the extensions I proposed, Andrew is planning to propose some extensions of his own to Concepts Lite in Rapperswil, such as the ability to use a concept as the return type of a function, the ability to use a concept as the type of a variable (an ability that I alluded to as a possible future feature in section 2.3 of my proposal), and the ability to pass a overload set as a function argument (this last one is not strictly related to Concepts and as such may be proposed directly for C++17 rather than for the Concepts Lite TS). I look forward to each of these extensions, as I believe they will contribute to making Concepts a more cohesive and well-rounded language feature.

Looking a little more into the future, after the Concepts Lite TS is finalized, the next step for Concepts is to revise the standard library, particularly the STL, to take advantage of the language features added in Concepts Lite. Concepts was originally planned for inclusion in C++11, but for various reasons it ended up being backed out. One of the reasons for this was that it was too ambitious, trying to introduce a major new language feature and incorporate it into into the standard library, while maintaining perfect compatibility with existing code using the library. Concepts Lite has learned from this mistake, and scaled back the magnitude of the language changes as well as leaving library changes for a separate proposal. Importantly, the prevailing thought among the proponents of Concepts now is that such a library will not be a modification of the existing STL, but a new, separate libary built from the ground up using Concepts Lite. Such a library would aim to provide the same functionality as the current STL, but not maintain perfect compatibility with code written for the current STL. Existing codebases could stick with the current STL or adopt the new one at their leisure, and new code would presumably be written against the new one. Eventually (this will undoubtedly take many years) the current STL would be deprecated and ultimately removed in favour of the new one. I should stress that at this point, all this is speculation. The focus is on getting the Concepts Lite TS published first.

Besides Concepts Lite, EWG considered some language proposals for C++17. A couple of them were really bug fixes for C++14, but it was too late to get them into C++14. It is expected that compilers will implement them promptly and, in practice, C++14 users will be able to use them. The first is to fix a rule that said that in auto x{1};, the type of x would be std::initializer_list. The rationale for this was that auto x{1} should be consistent with auto x{1, 2}, which should in turn be consistent with auto x = {1, 2}, which should in turn be consistent with std::initializer x = {1, 2}. While an argument can be made for each of these points, the result that auto x{1} makes x an std::initializer_list is very counterintuitive. Worse, C++14 introduces generalized lambda captures, with the initialization of the new variable following the same rules as auto, so in [x{1}](){ ... }, the type of x would also be std::initializer_list, which is even more counterintuitive, and such code is likely to be very common as people start to adopt “uniform initialization”, i.e. using the {...} syntax for initialization everywhere. The agreed-upon resolution was that in auto x{1} (and [x{1}](){ ... }), x would be an int, auto x{1, 2} would be invalid, and auto x = {1, 2} would continue to make x an std::initializer_list. I think this is a very reasonable compromise.

The other bugfix/feature-let was to allow static_assert to be used without a string argument; that is, in addition to writing static_assert(some_expression, "some string"), you can also write just static_assert(some_expression). The error message displayed by the second form is implementation-defined, with an obvious choice being the text of some_expression.

In terms of new features proper, EWG looked at a proposal to extend the syntax of range-based for loops to allow for (elem : range) to be a shorthand for for (auto&& elem : range). The rationale is that for (auto&& elem : range) almost always does what you want (makes elem a reference of the appropriate type and binds it to each element of the range without copying the element; see Scott Meyers’ article about “universal references” for how this works), while other forms do not (for (auto elem : range) copies the element; for (auto& elem : range) does not work for ranges with rvalue elements; and for (const auto& elem : range) does not allow you to modify the element), so for (auto&& elem : range) should be the default form of the range-based for loop, and particularly the form used in generic code. However, it’s awkward to have to teach people to use the funky-looking && syntax by default. The proposal met some resistance and being unnecessary, but ultimately it garnered consensus and will progress to the CWG at the next meeting.

EWG also considered enforcing the ‘Rule of 5’ in C++17, which says that if any one of the 5 “special member functions” of a class (copy constructor, copy assignment operator, move constructor, move assignment operator, and destructor) are user-declared, none of the other 5 are auto-generated by the compiler. It is already recommended that if you write one of the 5, you should write all of the 5, but this is not currently enforced. EWG decided that, unfortunately, due to backwards compatibility constraints, it is not feasible to start enforcing this even in the C++17 timeframe. The proposal may resurface for a the next version of C++ after C++17 (likely C++20 if the committee continues its 3-year cadence).

SG 13 (Graphics)

On Tuesday afternoon, I stepped out of EWG to attend the meeting of SG 13 (Graphics), which is currently working on standardizing a lightweight 2D drawing API for C++.

SG 13’s main intended purpose for such an API is to allow people learning C++ and writing graphical applications to do so easily, without having to rely on third-party libraries or learning complex APIs. In the long-term, however, they would like the drawing API to be useful for people writing application frameworks and high-performance applications as well.

Of course, there is more to being able to write graphical applications easily than having a 2D drawing API, and in this respect the SG’s name “Graphics” is a bit misleading: the group also aims to eventually standardize a means of listening to input events in a graphical application, and more. This was pointed out at the meeting, and it was decided that the SG’s name would be changed to something else like “Interaction”. Such projects, however, are farther in the future – for now the SG is focused on a 2D drawing API, and their first product will be a TS specifying such an API.

This TS is still in the early design stage. The SG is looking at using cairo as the basis of the drawing API, with the interface C++-ified using a set of clear and consistent rules. See the current draft of the proposal.

It’s important to note that the thing being standardized here is an interface, not an implementation. Given that the interface is that of cairo, a standard library vendor is likely to find it very convenient to implement the interface by using cairo itself, but this is by no means required – vendors are free to implement the interface however they choose.

Note that this freedom has important performance implications. One of the performance problems cairo suffers from is that it is a stateful API, but internally it has a stateless layer. If it is using a backend implementation which is itself stateful, we get unnecessary dupliation of state (Robert O’Callahan explains this nicely here). The standard API, however, would not neecessarily suffer from this problem, because a vendor intending to use a stateful backend API would be free to implement the interface using that API directly, and thus avoid this duplication of state.

It is also important to note that standardizing an API that is based on the current version of cairo does not imply being locked in to continue using an API based on cairo forever. The Standards Committee would reserve the right to modify the API in ways that make it diverge from cairo if it sees fit. In turn this means that while implementers may find it convenient to use cairo itself as the implementation at first, they are not guaranteed to continue to be able to use cairo (unmodified) as the implementation forever.

Finally, note that while this cairo proposal is being worked on actively, and SG 13 likes its direction, the group has not yet committed to it. If someone wishes to write a competing proposal based on a different API (for example, Moz2D), there is still time to do so.

SG 9 (Ranges)

SG 9, tasked with developing a cohesive set of abstractions and a library for working with them for Ranges (in the sense of the range-based for loop and Boost.Range), did not have a formal meeting in Issaquah. In fact, as far as I am aware, the progress of the group is stalled. The last piece of activity that I am aware of is a lengthy discussion on the group’s mailing list about proposals based on a range primitive (in contrast to today’s ranges which are built on top of an iterator primitive), and/or a position primitive (somewhat but not quite like an iterator primitive). While there are valid abstraction and performance reasons for building a range library based on range and/or position primitives rather than an iterator primitive, these approaches have the disadvantage that they go against the grain of a very established tradition of iterator-based ranges which is enshrined in the STL and many other libraries like Boost.Range, and no one seems to have gotten a range/position-based design quite right yet.

However, I participated in an informal conversation with Eric Niebler, a prominent member of the Boost community and author of the Boost.Proto and Boost.Xpressive libraries, and Andrew Sutton, regarding a library Eric has been developing which sounded very exciting to me.

The library in question is a prototype reimplementation of large parts of the STL, plus more range functionality (some of it similar to the facilities in Boost.Range), based on a slightly modified but still iterator-based Range concept. While a traditional Range consists of two Iterators of the same type, Eric introduces a relaxation of the Range concept which he calls Iterable, which consists of two Iterators of potentially different types. The motivation behind this is to allow efficient support for so-called sentinel ranges, which consist of a single iterator that represents a position in the range, and a predicate that tests whether this position is at the end of the range. The classic example of a sentinel range is a C-style (i.e. null-terminated) string, where the iterator is a pointer to the first character in the string, and the predicate tests whether the character at the pointer is the null character. Of course, a C-style string can be represented by a conventional pair of iterators (a pair of pointers to the first and past-the-end characters), but building this representation requires walking the string in linear time to find its end, while the sentinel representation does not.

You can shoehorn a sentinel representation into the conventional Range representation (two iterators of the same type) by constructing an iterator type which is essentially the union type of the iterator type in the sentinel representation, and a sentinel type. The constructed iterator type’s operator== would then check if the two iterators being compared are a non-sentinel and a sentinel, respectively, and if so invoke the predicate that determines whether the former is at the end of the range. However, this incurs a runtime penalty, as the operator==, which is called once per iteration over the range, needs to perform a runtime check to determine whether the iterator is a sentinel or not. By allowing the begin and end iterators to have different types, however, we can encode the sentinel-ness of an iterator in its type, and avoid this runtime penalty.

Of course, not all algorithms are suitable for use with sentinel ranges. For example reverse() is not, because it needs to walk the range backwards, but a sentinel iterator cannot be moved backwards. For algorithms like this, Eric keeps around the conventional Range concept, which is now a refinement of the Iterable concept that requires the begin and end iterators to be of the same type.

Eric’s library also supports infinite ranges, which are useful e.g. for representing generator ranges (i.e. the range x, f(x), f(f(x)), f(f(f(x))), ... for some initial value x and function f). Sentinel ranges are well-suited to represent infinite ranges: you simply have a predicate that always returns false, so you are never at the end of the range. A sore point for representing infinite ranges with the current Range concept is what type to use as the difference_type of a range. Eric resolves this by inventing an integer-like type which can also have “infinity” as a possible value, not unlike how regular floating-point types are able to represent infinity. (Update: Eric has informed me that he has actually changed his approach to the difference_type of infinite ranges. Stay tuned.)

I am very excited by Eric’s work, because it seems to accomplish the goal of providing powerful range abstractions, and unlike the approaches brought up on the SG 9 mailing list, this approach is something that is implemented and working, and meshes well the existing Range concept used in the STL.

If the committee ends up liking Eric’s approach for a standard set of abstractions for ranges, the question of how these new abstractions will fit into the current STL will likely arise. If the idea I described above about a Concepts-enabled STL that makes a clean break with the current STL goes forward, it may be a nice opportunity to also break with the standard’s current Range concepts and use Eric’s new ones from the start. Two birds, one stone.

Update: Since I started writing this blog post, Eric has also blogged about his range abstractions, and he explains them in a lot more detail:

SG 7 (Reflection)

SG 7 looked at two proposals concerning compile-time reflection, a small but detailed one and a more ambitious but vague one.

The smaller proposal concerned getting compile-time information about an enumeration, such as the number of elements, and the name and value of each element. This proposal received positive feedback, and the author was encouraged to prepare are updated proposal for the Library Evolution Working Group (LEWG) to look at in Rapperswil.

The more ambitious paper proposed:

  • an introspection mechanism whereby the programmer can inspect a program construct in the form of an AST node
  • a reification mechanism that processes arbitrary C++ tokens at compile time with HTML-template-engine-like syntax (containing constructs such as $for, $if, and $switch) and which can use the AST nodes mentioned above as variables
  • a mechanism to extend the set of valid inputs to the introspection mechanism from C++ code to arbitrary text, by essentially defining new “AST nodes”; the idea here was to enable support for domain-specific languages (DSLs)

This was some of the group’s feedback on the paper:

  • It conflated the notion of an “AST” as an abstract/idealized representation of program structure with the notion of an “AST” as the concrete internal representation used by a compiler (the paper went as far as to link to a node of the clang AST as reference for the sorts of methods you can call on an AST node during introspection). While the former may conceivably be standardized, it is unreasonable to expect that the latter can be.
  • Operating at the level of tokens is not optimal for a reification mechanism. It is much better to operate on higher-level, semantic entities.
  • The DSL part of the proposal is out of scope for SG 7, which is looking to be able to introspect the language, not extend it. It was noted, however, that mechanisms that take C++ code as input and produce transformed C++ code as output are in scope for SG 7, albeit probably at a more advanced stage.

Given this feedback, it is very unlikely that the proposal will go forward in its current form. However, there are some good ideas and use cases in there, and it will probably serve to inform future proposals in useful ways.

In addition to these two formal proposals, two informal ideas were floated by the participants, and both garnered strong support from the group. The first was introducing a compile-time string literal class; this would be similar to a regular string class, but all the operations on it could be performed at compile time. The second was providing non-macro replacements for __LINE__, __FILE__, and __FUNCTION__, which are currently one of the few remaining unavoidable uses of the preprocessor. The group expects proposals for these two ideas to be forthcoming.

Library / Library Evolution Working Groups (LWG and LEWG)

There is much happening on the library side, with 4 TS’s in the works.

The Filesystems TS is in very good shape. The TS had already been sent out for its first round of balloting; LWG spent much of its time in Issaquah addressing the ensuing comments. The plan is to send out a revised TS for a second and hopefully final round of balloting, leading to publication in late 2014/early 2015.

The Library Fundamentals TS started out containing just optional (a standardized version of Boost.Optional), but it picked up a number of new goodies in Issaquah:

The Library Fundamentals TS is on track to be sent out for its first ballot in Rapperswil, which means if all goes well it’s looking at publication in the first half of 2015.

The Array Extensions TS currently contains “arrays of runtime bound” (the C++ flavour of variable-length arrays) and dynarray. However, dynarray has been facing implementation challenges, and EWG is in the process of designing a better replacement for it that doesn’t require compiler magic to implement (by introducing a new language feature proper). While this happens, the Array Extensions TS is not moving forward. It did, however, pick up an additional library utility in Issaquah: an array_view class that is similar to string_view but not specific to characters.

The Networking TS is progressing more slowly than expected. As mentioned above, the network byte order conversions part of it was moved into Library Fundamentals, leaving URIs and IP addresses in the Networking TS.

SG 1 (Concurrency) and SG 5 (Transactional Memory)

I don’t follow these groups closely and I’m not very familiar with their work. I do know that SG 1 is working on a Parallelism TS and a Concurrency TS (based on this paper and this one), and SG 5 is working on a TS. These are probably expected sometime next year.

SG 6 (Numerics)

SG 6 reviewed a proposal for adding decimal floating-point support to C++, and is sending it on to be presented to EWG in Rapperswil. It is an open question whether this would be standardized as a TS or go directly into C++17.

Other Study Groups

I covered the work of SG 3 (Filesystem) and SG 4 (Networking) under the Library section above. SG 2 (Modules), SG 10 (Feature Test), and SG 12 (Undefined Behaviour) did not meet in Issaquah. SG 11 (Databases) was disbanded due to no one being available to chair it; presumably LEWG will still consider database-related proposals directly.

Next Meeting

As I mentioned above, the next meeting of the C++ Standards Committee will be in Rapperswil, Switzerland, the week of June 16th.

Conclusion

I think this is a really exciting time for C++, for several reasons:

  • The 3-year release cadence that the Committee seems to  be converging on (C++11, C++14, C++17, etc.) is refreshing after the 13-year wait between C++98 and C++11.
  • The emerging use of Technical Specifications as a way of trying out significant new features while retaining the ability to change them before accepting them into the actual standard allows a lot more work to be done in parallel. The Committee is looking at having no fewer than eight different TS’s published over the next two years (Filesystems, Networking, Concepts, Library Fundamentals, Array Extensions, Concurrency, Parallelism, and Transactional Memory). It is very unlikely that all these features would have made it into an actual standard published in the same timeframe.
  • Compilers’ implementation of Standard C++ is rapidly accelerating. Cland 3.4 already implements all of C++14 (in the state it was in at the end of the last meeting; the minor changes made in Issaquah will presumably be implemented in 3.5). GCC is not far behind. (Of course, both already implement C++11). MSVC doesn’t fully support even C++11 yet, but it seems determined to catch up.

I will continue to follow the progress of C++ standardization and blog about interesting things I discover.

Advertisements

About botondballo

I am a recent graduate of the University of Toronto, with a degree in Computer Science. I currently work for Mozilla as a software engineer.
This entry was posted in c++, programming and tagged , , . Bookmark the permalink.

One Response to Trip Report: C++ Standards Committee Meeting in Issaquah, February 2014

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s