YugabyteDB coding style

YugabyteDB is primarily written in C++ (the distributed storage and transactions layer and the YCQL query layer) and C (the YSQL layer based on PostgreSQL), with some parts of the build system and test suite written in Python, Java, and Bash. We also use Protocol Buffers to define some data and network message formats.

Language-agnostic style guidelines

Variable names

Avoid rarely used abbreviations. Think about whether all other potential readers of the code know about any abbreviations you're using.

Comments

Start full sentences with a capital letter, and end them with a period. This rule doesn't apply if the comment is a single phrase on the same line as a code statement.

Functions and classes in header files should typically have detailed comments, but the comments should not duplicate what is already obvious from the code. In fact, if the code can be restructured, or if functions or classes could be renamed to reduce the need for comments, that is the preferred way. Obvious comments like the following don't add anything useful:

Functions and classes in .cc files don't have to be commented as extensively as code in header files. However, do add comments and examples for anything that might be non-obvious to a potential new reader of your code.

C coding style

For the modified PostgreSQL C codebase residing inside the YugabyteDB codebase, we adhere to the PostgreSQL Coding Conventions. Note that PostgreSQL code uses tabs for indentation and we follow that rule in the src/postgres subdirectory; we use spaces for indentation everywhere else in YugabyteDB code.

C++ coding style

Line length

Use a 100-character line length limit.

Formatting function declarations and definitions

Use one of the following formatting styles for function declarations and definitions.

All arguments on one line

Aligned, one argument per line

All arguments aligned with the opening parenthesis, one argument per line.

Four-space indentation

One argument per line, with four-space indentation for each argument.

Packed

Pack arguments into the fewest number of lines—but not exceeding the maximum line width—with each line indented by four spaces.

Don't break the argument list arbitrarily, and only break the list if the next argument won't fit within the line-length limit.

Formatting function calls and macro invocations

Use one of the following formatting styles for functions calls and macro invocations.

All arguments on one line

Aligned arguments

All arguments aligned with the opening parenthesis, one argument per line.

One argument per line, four-space indentation

Packed arguments, four-space indentation, wrapping at margin

Start a new line after the opening parenthesis, with a four-space indentation, and pack arguments into as few lines as possible.

Function calls within function calls

The preceding formatting styles also apply to nested function calls.

String substitution functions

For string substitution and formatting functions (Format, Substitute, StringPrintf, and so on) avoid putting substitution parameters on the same line as the format string, unless the entire function call fits on one line. For example:

Expressions

Indent multi-line expressions like this:

Or like this:

The following style is also widely used in our codebase, so it's acceptable to leave it as-is when modifying the surrounding code, but the two previous options are preferable for new code.

Ternary operator

For expressions involving the ternary operator (? and :), prefer one of the following formatting styles:

or

Command-line flag definitions

Put the flag name on the same line as DEFINE_... to make the code more grep-friendly.

Follow function-like macro invocation styles with either all arguments on one line or with aligned arguments as defined earlier, depending on whether all three arguments of the flag definition fit on one line.

Note that in this code style, we've aligned the first line of the string constant (the flag description) with its other lines. Some coding styles, including CLion's standard behavior, would indent the second and later lines by four spaces, but we prefer to keep them aligned.

Forward declarations

You can use forward declarations in a header file, if the class you are referencing is to be defined in the related .cc file, essentially making it a private class implementation. This helps keep the header file cleaner, and keeps the definition and implementation of the private class together with the actual implementation of the main class.

We also frequently use special header files named ..._fwd.h that forward-declare various classes and declare some types, including enums. This helps to avoid including full class declarations wherever possible, and reduces compilation time.

Parameter ordering

Normally, in function parameter lists in YugabyteDB, we put input parameters first, followed by output parameters.

However, this rule doesn't apply to parameters that aren't pointers to variables to be assigned to by the functions, but are simply non-const objects that the function calls some methods on that modify its state.

For example, in the following function it is OK that writer is in the middle of the parameter list, because the function is not calling the assignment operator on *writer but simply calls some member functions of *writer that modify its internal state, even though this is how this particular function produces its "output".

Pointer and reference parameters and variables

Put a space on either side of * and & (but not on both sides).

Both of the following examples are correct:

Similarly, for variable declarations and definitions:

Get prefix for getters

Some C++ coding styles (such as Google's) use the Get prefix for functions returning a value, and some don't (Boost, STL). In YugabyteDB code it is allowed to either use or not use the Get prefix.

There are many different function names in our codebase with the Get prefix.

The Get prefix is especially useful if the function name could be misinterpreted as a verb or a verb phrase without it, but is optional in all cases.

If you can't decide whether to use the Get prefix for functions, look at other functions in the same class, file, or subsystem, and adhere to the prevalent naming style in the surrounding code.

Code duplication

Try to reduce code duplication by extracting repeated code into reusable functions, classes, or templates. Reducing code duplication with macros is also allowed, but try not to overuse them; use non-macro-based ways to reuse code duplication as much as possible. Undefine macros that are only used in an isolated section of code when they're no longer needed.

Switch statements over enums

If you don't use the default statement in a switch over an enum, the compiler will warn you if some values aren't handled (and we have made that warning an error). This allows the compiler to enforce that all enum values are being handled by a switch over an enum, if that's your intention. This complicates default case handling a bit, though. If every case is followed by a return, you can simply move the default handler to right after the end of the switch statement, such as:

In this case, when you add a new request type, such as OperationType::kUpdate, you'll get a compile error asking you to add it to the switch statement. In case there is no return following each case handler, we can still detect the default case by (for example) setting a boolean flag.

Note that FATAL_INVALID_ENUM_VALUE will terminate the process, so for functions returning a Status, you should handle invalid enum values gracefully, e.g.:

Arguments passed by value

Don't use const in function (including member function) declarations when the argument is passed by value.

Do use const in function (including member function) definitions wherever appropriate to indicate (and enforce) that the object is not modified inside the function body.

In widget.h:

In widget.cc:

Or in widget.h:

And in widget.cc:

Here's an example of what not to do (in widget.h):

The using keyword and namespaces

You can use the using directive for inner utility namespaces in .cc files (but not in header files), e.g.:

This allows you to use various convenient literals, such as 100ms to denote "100 milliseconds".

Unnamed namespaces vs static keyword

When you define functions that should only be accessible within a .cc file, prefer using unnamed namespaces to using the static keyword. See for example this discussion.

Static and global variables

Using static objects is allowed in our coding style with a few limitations, and with an understanding of a static object's lifecycle. The limitations are:

• Static global variables must be totally independent. The order of constructions/destructions should not matter.
• Static objects must not lock anything or allocate global resources (except memory) during their construction.
• Static objects should be as simple as possible.

If you are adding new static objects, remember:

• Global static objects (as well as static class fields) are constructed before the main() function call.

• Local static objects are constructed when they first come into variable scope. This option should be preferred. For local static objects, there is also a locking mechanism needed so that a concurrent execution waits for initialization completion if the static variable is already being initialized by another thread. For example, for a simple static variable defined in a function

  void f() {
      static string s = "asdf";
  }
  

  The generated code might be using a lock automatically to guard the static variable initialization:

    call    __cxa_guard_acquire
          test    eax, eax
          setne   al
          test    al, al
          je      .L6
          mov     ebx, 0
          lea     rax, [rbp-17]
          mov     rdi, rax
          call    std::allocator<char>::allocator()
          lea     rax, [rbp-17]
          mov     rdx, rax
          mov     esi, OFFSET FLAT:.LC0
          mov     edi, OFFSET FLAT:f()::s
          call    std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)
          mov     edi, OFFSET FLAT:guard variable for f()::s
          call    __cxa_guard_release
  

• Static objects are destroyed after the end of the main() function.

• The order of global static objects construction is undefined, and can even change from build to build.

• The order of static objects destructions is reverse of the order of constructions, so it is also undefined.

Multiple inheritance

Use protected or private inheritance if you can.

Multiple inheritance is allowed in general, subject to a few limitations:

• Don't down-cast pointers or references (casting from a base to a derived class).

  
  
  

  Note: The only way to do this safely in C++ is using the dynamic_cast<...> operator, which relies on run-time type information (RTTI), and we may decide to disable RTTI in release builds at some point for performance reasons.

• Don't use multiple inheritance if a base class is part of this object from an architectural point of view. Instead, prefer composition (making the "part" object a member field in the "whole" object) in such cases.

Testing whether a pointer is null

It's OK to use implicit boolean conversion for raw, unique, or shared pointers:

Another example:

Error checking macros

We use various macros for error/invariant checking.

Checking a condition and returning a Status with SCHECK

SCHECK (shorthand for "Status CHECK") checks a condition and returns a Status with the appropriate message if the condition is not true. It can only be used within a function that returns a Status or a Result.

The SCHECK macro is a good way to check for errors that are expected to happen under some conditions, e.g. with invalid input, and the errors need to be ultimately reported to the user.

There are also variants the SCHECK macro for equality checks and comparisons: SCHECK_EQ, SCHECK_NE, SCHECK_GT, SCHECK_LT, SCHECK_GE, and SCHECK_LE.

Returing a Status in release mode, but triggering a fatal error in debug mode, with RSTATUS_DCHECK

RSTATUS_DCHECK works similarly to SCHECK in release mode, but triggers a fatal error and a log message in debug mode, terminating program execution. Similarly to SCHECK, it also has variants for checking for equality and inequality. RSTATUS_DCHECK can be used for invariant checks and sanity checks where the error is not expected to happen under normal circumstances but there is a recovery path from this error in a production situation. In these cases it is OK to cause a unit test to crash in debug mode when the error is encountered.

Checking an invariant that must always hold with CHECK

For really important invariants that are difficult to recover from while still maintaining correctness, we sometimes use the CHECK macro and its variants. It is enabled in both debug and release modes and causes program termination if the condition is not true. It should be used really carefully to avoid causing unnecessary server restarts in release mode.

Only checking a condition in debug mode with DCHECK

This macro expands to a no-op in release mode. This is reserved for checking invariants or preconditions in performance-critical code, and typically only in cases when we already expect the condition to be true because other code guarantees it.

We sometimes use DCHECKs to verify function prerequisites. If you never expect an incorrect parameter value to be passed into a function, because there is validation happening in the calling function, it's OK to keep that as a DCHECK.

However, if you could theoretically get bad data in production at a certain point in the code, then:

• If you can recover from this error, return an error Status (e.g. using SCHECK or RSTATUS_DCHECK).
• If this is a severe invariant violation and you can't recover from it, this could be a CHECK.

PREDICT_TRUE and PREDICT_FALSE

PREDICT_TRUE and PREDICT_FALSE macros expand to hints to the compiler that a particular codepath is likely or unlikely. In theory these macros could allow better compiler optimizations. However, we don't use them in new code as it is difficult to check if they really improve performance.

Result vs Status with output parameters

For new code, use Result, e.g.:

Much of our code is wired to return Status, so we are able to get a sense of whether a function completed successfully (OK), or if there was some kind of an error. However, we sometimes want to also get legitimate output from these functions. We used to do this by using function signatures such as

However, now we have a better way to achieve the same goal; the Result type can encapsulate both an output parameter (in the successful case) and a Status (in case of an error):

String formatting

Use the Format function to produce formatted strings, rather than the older function Substitute.

While the two functions have similar syntax, with inline substitution parameters $0, $1, and so on, Format has several advantages:

• It uses the ToString utility, so it can convert many different types of objects to strings, such as collections, protobufs, or any class with a ToString member function.
• You don't need to call arg.ToString(). Just pass arg to the Format function as-is, and it will call ToString for you.
• Format is a bit faster than Substitute on some benchmarks.

consensus::OpId

consensus::OpId is just an alias for yb::OpIdPB, a protobuf class. Use this only where you really need to use protobuf, for example inside other protobuf messages. For the rest of the code, use yb::OpId, a normal C++ class.

Thread safety analysis

We use Clang's Thread Safety Analysis annotations in parts of our code.

Thread safety analysis is a C++ extension implemented as part of Clang Static Analyzer that provides compile-time checks for potential race conditions in code. Annotations are extremely useful for code maintainability, because they make locking semantics explicit, and the compiler warns about accessing memory without holding the necessary mutexes.

A few more things to keep in mind:

• std::unique_lock doesn't work with thread safety annotations out of the box. We wrap it into our custom yb::UniqueLock wrapper.
• Similarly, for std::shared_lock we have the yb::SharedLock wrapper.
• Occasionally, you may need to annotate some functions where thread safety analysis cannot be properly applied, with NO_THREAD_SAFETY_ANALYSIS, so that we can still use thread safety analysis in the surrounding code. The situations in which thread safety analysis might not work include conditional locking and complex locking semantics where a unique lock is being passed around between member functions of different classes.

Our build scripts enable thread safety analysis for Clang version 11 and above; earlier versions of the Clang compiler don't support certain features that we need. Thread safety analysis works very well on macOS with modern Clang compilers, providing instant hints if your environment is set up properly.

Unused C/C++ features

• C++ exceptions. In most of our code, we don't use C++ exceptions. However, in some cases, we still have to use C++ standard library functions that throw exceptions, and we catch those exceptions as early as possible and convert them to Status or Result return values.
• assert C library macro. We use our own set of macros for invariant checking.

Other related coding guidelines for C++

In addition to the guidelines outlined on this page, the YugabyteDB C++ coding style is based on Google's C++ style guide.

Protocol buffers coding style

Name protobuf structures and enums with a PB suffix. This is for the following reasons:

• Better "greppability" (searchability) means fewer false positive results when searching for TableTypePB and not TableType.
• We can have structs/classes with the same name, but without PB suffix for usage in our code.
• We can have enum wrappers similar to YB_DEFINE_ENUM with the same name as the protobuf enum, but without the PB suffix.