Flow 1.0.0
Flow project: Public API.
Namespaces | Classes | Enumerations | Functions
flow::log Namespace Reference

Flow module providing logging functionality. More...

Namespaces

namespace  fs
 Short-hand for namespace boost::filesystem.
 

Classes

class  Async_file_logger
 An implementation of Logger that logs messages to a given file-system path but never blocks any logging thread for file I/O; suitable for heavy-duty file logging. More...
 
class  Buffer_logger
 An implementation of Logger that logs messages to an internal std::string buffer and provides read-only access to this buffer (for example, if one wants to write out its contents when exiting program). More...
 
class  Component
 A light-weight class, each object storing a component payload encoding an enum value from enum type of user's choice, and a light-weight ID of that enum type itself. More...
 
class  Config
 Class used to configure the filtering and logging behavior of Loggers; its use in your custom Loggers is optional but encouraged; supports dynamically changing filter settings even while concurrent logging occurs. More...
 
class  Log_context
 Convenience class that simply stores a Logger and/or Component passed into a constructor; and returns this Logger and Component via get_logger() and get_log_component() public accessors. More...
 
class  Logger
 Interface that the user should implement, passing the implementing Logger into logging classes (Flow's own classes like net_flow::Node; and user's own logging classes) at construction (plus free/static logging functions). More...
 
struct  Msg_metadata
 Simple data store containing all of the information generated at every logging call site by flow::log, except the message itself, which is passed to Logger::do_log() assuming Logger::should_log() had returned true. More...
 
class  Ostream_log_msg_writer
 Utility class, each object of which wraps a given ostream and outputs discrete messages to it adorned with time stamps and other formatting such as separating newlines. More...
 
class  Simple_ostream_logger
 An implementation of Logger that logs messages to the given ostreams (e.g., cout or an ofstream for a file). More...
 
class  Verbosity_config
 Optional-use structure encapsulating a full set of verbosity config, such that one can parse it from a config source (like an options file) in concise form and apply it to a log::Config object. More...
 

Enumerations

enum class  Sev : size_t {
  S_NONE = 0 , S_FATAL , S_ERROR , S_WARNING ,
  S_INFO , S_DEBUG , S_TRACE , S_DATA ,
  S_END_SENTINEL
}
 Enumeration containing one of several message severity levels, ordered from highest to lowest. More...
 

Functions

void swap (Log_context &val1, Log_context &val2)
 Log_context ADL-friendly swap: Equivalent to val1.swap(val2). More...
 
std::ostream & operator<< (std::ostream &os, Sev val)
 Serializes a log::Sev to a standard output stream. More...
 
std::istream & operator>> (std::istream &is, Sev &val)
 Deserializes a log::Sev from a standard input stream. More...
 
void beautify_chrono_logger_this_thread (Logger *logger_ptr)
 Sets certain chrono-related formatting on the given Logger in the current thread that results in a consistent, desirable output of durations and certain types of time_points. More...
 
size_t deep_size (const Msg_metadata &val)
 Estimate of memory footprint of the given value, including memory allocated on its behalf – but excluding its shallow sizeof! – in bytes. More...
 
std::istream & operator>> (std::istream &is, Verbosity_config &val)
 Deserializes a Verbosity_config from a standard input stream by invoking val.parse(is). More...
 
std::ostream & operator<< (std::ostream &os, const Verbosity_config &val)
 Serializes a Verbosity_config to a standard output stream. More...
 
bool operator== (const Verbosity_config &val1, const Verbosity_config &val2)
 Checks for exact equality of two Verbosity_config objects. More...
 
bool operator!= (const Verbosity_config &val1, const Verbosity_config &val2)
 Returns !(val1 == val2). More...
 

Detailed Description

Flow module providing logging functionality.

While originally intended to be used from within the flow::net_flow module's implementation, it can be used by general user code as well. All other Flow modules expose flow::log concepts when logging is relevant, so one way or another the user is likely to have at least limited contact with flow::log if they use Flow at all. (In particular, classes and free/static functions in various Flow modules often take a Logger pointer as a constructor or function argument respectively. Hence to use such APIs one must instantiate a concrete Logger, the simplest choice being probably Simple_ostream_logger.)

(The general user [in their own, non-Flow-related code] may well prefer to use another logging system directly instead; or use boost.log. However, we humbly recommend taking a look at flow::log as a possibility, as it is highly usable, yet fast, yet small and elegant enough to ensure complete visibility into its low-level behavior, including how that affects performance. Note also that this system easily integrates with others via log::Logger interface, so one can stack this on top of perhaps a lower-level logging facility. The performance cost of this is essentially 1-2 virtual pointer lookups per log-message call.)

Log philosophy of Flow is as follows. The following is a list of goals; and for each item, sub-item(s) explain how this goal is accomplished.

Enumeration Type Documentation

◆ Sev

enum class flow::log::Sev : size_t
strong

Enumeration containing one of several message severity levels, ordered from highest to lowest.

Generally speaking, volume/verbosity is inversely proportional to severity, though this is not enforced somehow.

As the underlying type is size_t, and the values are guaranteed to be 0, 1, ... going from lowest to highest verbosity (highest to lowest severity), you may directly use log::Sev values to index into arrays that arrange one-to-one values in the same order.

The supplied ostream<< operator, together with this enum, is suitable for util::istream_to_enum(). ostream>> operator is built on the latter and is also supplied. Hence I/O of Sev is available out of the box; this enables parsing in boost.program_options among other things.

Formal semantics of log::Sev

From the point of view of all Flow code, the formal semantics of log::Sev are as follows:

  • Sev::S_NONE has value 0 and is a start-sentinel.
  • Non-sentinel severities follow, starting at 1 with no gaps.
    • Ordinal comparison is meaningful; lower is termed as more severe/less verbose; higher is therefore termed as less severe/more verbose. These are, formally, only terminology.
  • Sev::S_END_SENTINEL is the end-sentinel.
  • Sev::S_WARNING is and always shall be considered the least severe "abnormal" condition. Any additional "abnormal" conditions shall always have more-severe (numerically lower) values. As of this writing, within flow::log Flow module, this has only one practical meaning (but that could change): Simple_ostream_logger directs WARNING-and-more-severe messages to its configured error stream (os_for_err); while all the rest are directed to its configured other (regular) stream (os). (For example these might be set to std::cerr and std::cout respectively; or both to std::cout.)

Informal semantics of log::Sev

The following are not enforced by any code logic in the flow::log Flow module. However we suggest users understand and consider these, as a haphazard approach to severity selection for a given log call site can cause significant operational problems in a high-volume production environment (at least).

Generally, there are 2 types of log call sites: ones inside Flow; and ones outside (user code). The former (inside Flow) are deliberately minimalistic, so as to use as few severities as possible and thus make severity selection maximally simple and, importantly, unambiguous.

However, logging code outside Flow may require more sophisticated in-use severity sets. Informally we recommend keeping it as simple as inside-Flow's scheme... if it is sufficient. If it is not sufficient, the other severities may also be used.

The log::Sev values used inside Flow (the minimalistic set) are as follows. (Brief meanings accompany them; see their individual doc headers for more detail.) From most to least severe:

  • Sev::S_FATAL: the program will abort shortly due to the condition that is being logged. Usually an assert(false) and/or std::abort() follows. If you've disabled abort-on-assertion-trip (NDEBUG is defined), and there is no std::abort() or equivalent, then the program may continue, but subsequent behavior is undefined.
  • Sev::S_WARNING: abnormal condition is being logged, and its aggregate volume is not high enough to be classified as TRACE instead to avoid perf impact. (Other than possibly FATAL – used in extreme cases – no other abnormal-condition Sev are in-use inside Flow nor will they be.)
  • Sev::S_INFO: non-abnormal condition is being logged, and it's not so frequent in practice that enabling this log level shall adversely impact performance.
  • Sev::S_TRACE: like INFO, but enabling this log level can adversely impact performance. However, entire dumps of potentially large (in aggregate) data (such as packet contents) being processed are not included.
  • Sev::S_DATA: like TRACE, but entire contents of potentially large (in aggregate) data (such as packet contents) being processed are included in the message, which may lead to particularly large log output (if enabled).

As of this writing the following log::Sev values are available for use (by user/non-Flow code) beyond the above minimalistic set. Potential subjective meanings are included, but user code can use whatever conventions that suit them best.

  • Sev::S_ERROR: A non-FATAL abnormal condition subjectively more severe than WARNING.
  • Sev::S_DEBUG: A non-abnormal condition with, perhaps, non-perf-affecting verbosity (a-la INFO) but subjectively of less interest to a human glancing over a large-ish log snippet (a-la TRACE).
Enumerator
S_NONE 

Sentinel log level that must not be specified for any actual message (at risk of undefined behavior such as assert failure); but can be used for sentinel purposes such as specifying a log filter wherein no messages (not even Sev::S_WARNING ones) are shown.

S_FATAL 

Message indicates a "fatally bad" condition, such that the program shall imminently abort, typically due to immediately-following assert(false) and possibly std::abort(); or if aborts are disabled (such as via defining NDEBUG), and there is no explicit std::abort() or equivalent, then further program behavior is undefined.

Note
This severity is part of the minimalistic (in-use within Flow) severity set as discussed in log::Sev doc header, "Informal semantics" section.
S_ERROR 

Message indicates a "bad" condition with "worse" impact than that of Sev::S_WARNING.

Note
If it's "bad" but frequent enough for TRACE, we strongly recommend to make it TRACE – not ERROR.
This severity is NOT part of the minimalistic (in-use within Flow) severity set as discussed in log::Sev doc header, "Informal semantics" section. Informally we recommend projects do not use it unless it is holistically necessary.
S_WARNING 

Message indicates a "bad" condition that is not frequent enough to be of severity Sev::S_TRACE.

These typically should occur with less frequency than INFO messages; however, it's not a hard rule.

Note
If it's "bad" but frequent enough for TRACE, it's TRACE by definition – not WARNING.
This severity is part of the minimalistic (in-use within Flow) severity set as discussed in log::Sev doc header, "Informal semantics" section.
S_INFO 

Message indicates a not-"bad" condition that is not frequent enough to be of severity Sev::S_TRACE.

Note
That is, it's identical to WARNING except doesn't represent a "bad" situation.
This severity is part of the minimalistic (in-use within Flow) severity set as discussed in log::Sev doc header, "Informal semantics" section.
S_DEBUG 

Message indicates a condition with, perhaps, no significant perf impact if enabled (like Sev::S_INFO) but of subjectively less interest to a human reader than INFO (hence, like Sev::S_TRACE in that respect).

Note
This severity is NOT part of the minimalistic (in-use within Flow) severity set as discussed in log::Sev doc header, "Informal semantics" section. Informally we recommend projects do not use it unless it is holistically necessary.
S_TRACE 

Message indicates any condition that may occur with great frequency (thus verbose if logged).

The line between Sev::S_INFO or Sev::S_WARNING and TRACE is as follows: The former is not allowed to classify messages such that in realistic scenarios they would degrade performance, from processor cycles or log file I/O.

See also
Sev::S_DATA for an even-more-verbose severity, when it's frequent and potentially large in size in aggregate.
Warning
One MUST be able to set max severity level to INFO and confidently count that logging will not affect performance.
Note
This severity is part of the minimalistic (in-use within Flow) severity set as discussed in log::Sev doc header, "Informal semantics" section.
S_DATA 

Message satisfies Sev::S_TRACE description AND contains variable-length structure (like packet, file) dumps.

If these are allowed to be logged, resulting log file might be roughly similar to (or even larger than) the data being transmitted by the logging code. Packet dumps are obvious examples.

Note that just because it's a variable-length structure dump doesn't mean it's for Sev::S_DATA severity. If it's not frequent, it's fine for it to even be INFO. E.g., if I decide to dump every RST packet, it's probably okay as Sev::S_INFO, since RSTs are practically rare; that need not be Sev::S_DATA. On the other hand, if I dump every DATA packet as anything except Sev::S_DATA severity, then I should be arrested and convicted.

When this level is disabled, consider logging a TRACE message instead but summarize the contents you'd dump; e.g., log a hash and/or a size and/or the few first bytes instead of the entire thing. Something is often better than nothing (but nothing is still safer than the whole thing).

Note
This severity is part of the minimalistic (in-use within Flow) severity set as discussed in log::Sev doc header, "Informal semantics" section.
S_END_SENTINEL 

Not an actual value but rather stores the highest numerical payload, useful for validity checks.

Function Documentation

◆ beautify_chrono_logger_this_thread()

void flow::log::beautify_chrono_logger_this_thread ( Logger logger_ptr)

Sets certain chrono-related formatting on the given Logger in the current thread that results in a consistent, desirable output of durations and certain types of time_points.

The effect is that of util::beautify_chrono_ostream().

See also
flow::async in which new threads are set to use this formatting automatically. However you'll want to do this explicitly for the startup thread.
Parameters
logger_ptrThe Logger to affect in this thread. Null is allowed; results in no-op.

◆ deep_size()

size_t flow::log::deep_size ( const Msg_metadata val)

Estimate of memory footprint of the given value, including memory allocated on its behalf – but excluding its shallow sizeof! – in bytes.

Parameters
valValue.
Returns
See above.

◆ operator!=()

bool operator!= ( const Verbosity_config val1,
const Verbosity_config val2 
)

Returns !(val1 == val2).

Parameters
val1Object to compare.
val2Object to compare.
Returns
See above.

◆ operator<<() [1/2]

std::ostream & operator<< ( std::ostream &  os,
const Verbosity_config val 
)

Serializes a Verbosity_config to a standard output stream.

Parameters
osStream to which to serialize.
valValue to serialize.
Returns
os.

◆ operator<<() [2/2]

std::ostream & flow::log::operator<< ( std::ostream &  os,
Sev  val 
)

Serializes a log::Sev to a standard output stream.

The output string is compatible with the reverse istream>> operator.

Parameters
osStream to which to serialize.
valValue to serialize.
Returns
os.

◆ operator==()

bool operator== ( const Verbosity_config val1,
const Verbosity_config val2 
)

Checks for exact equality of two Verbosity_config objects.

(Note that this is not maximally clever, in that if (val1 == val2), then they definitely produce the same Config all else being equal; but if the reverse is true, then it is possible they differently expressed values producing the same actual result. E.g., something like ALL:INFO differs from ALL:WARN;ALL:INFO – yet they have the same effect. However component names are normalized internally when parsing, so that won't produce false inequality.)

Only Verbosity_config::component_sev_pairs() is significant in this comparison; last_result_message() is not.

Parameters
val1Object to compare.
val2Object to compare.
Returns
true if definitely equal; false if possibly not equal.

◆ operator>>() [1/2]

std::istream & flow::log::operator>> ( std::istream &  is,
Sev val 
)

Deserializes a log::Sev from a standard input stream.

Reads up to but not including the next non-alphanumeric-or-underscore character; the resulting string is then mapped to a log::Sev. If none is recognized, Sev::S_NONE is the result. The recognized values are:

  • "0", "1", ...: Corresponds to the int conversion of that log::Sev (e.g., 0 being NONE).
  • Case-insensitive encoding of the non-S_-prefix part of the actual log::Sev member; e.g., "warning" (or "Warning" or "WARNING" or...) for S_WARNING. This enables a few key things to work, including parsing from config file/command line via and conversion from string via boost::lexical_cast.
Parameters
isStream from which to deserialize.
valValue to set.
Returns
is.

◆ operator>>() [2/2]

std::istream & operator>> ( std::istream &  is,
Verbosity_config val 
)

Deserializes a Verbosity_config from a standard input stream by invoking val.parse(is).

val.last_result_message() can be used to glean the success or failure of this operation.

Parameters
isStream from which to deserialize.
valValue to set.
Returns
is.

◆ swap()

void flow::log::swap ( Log_context val1,
Log_context val2 
)

Log_context ADL-friendly swap: Equivalent to val1.swap(val2).

Parameters
val1Object.
val2Object.