Flow-IPC 3.0.0
Flow-IPC project: Full implementation reference.
native_socket_stream_batch.hpp
Go to the documentation of this file.
1/* Flow-IPC: Core
2 * Copyright 2023 Akamai Technologies, Inc.
3 *
4 * Licensed under the Apache License, Version 2.0 (the
5 * "License"); you may not use this file except in
6 * compliance with the License. You may obtain a copy
7 * of the License at
8 *
9 * https://www.apache.org/licenses/LICENSE-2.0
10 *
11 * Unless required by applicable law or agreed to in
12 * writing, software distributed under the License is
13 * distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
14 * CONDITIONS OF ANY KIND, either express or implied.
15 * See the License for the specific language governing
16 * permissions and limitations under the License. */
17
18/// @file
19#pragma once
20
28#include <flow/error/error.hpp>
29#include <flow/util/stat/histo.hpp>
30#include <type_traits>
31#include <boost/array.hpp>
32#include <boost/move/unique_ptr.hpp>
33#include <boost/noncopyable.hpp>
34#include <cassert>
35#include <ostream>
36#include <utility>
37
38namespace ipc::transport
39{
40
41// Types.
42
43/**
44 * `Msg_batch_in` concept implementation for at least sync_io::Native_socket_stream (as taken by APIs
45 * Native_socket_stream::async_receive_blob_batch() and Native_socket_stream::async_receive_native_handle_batch())
46 * for high-speed *natively batched* receiving.
47 *
48 * @see Native_handle_receiver concept doc header for an explanation of receive-batching (including what
49 * native batching is; how `Msg_resource_t` template-param relates to things); and how the general design
50 * enables high perf.
51 *
52 * It features the following public APIs as of this writing:
53 *
54 * - For user (including internal user struc::sync_io::Channel): `size_t max_msg_count`-taking ctor,
55 * initialized(), n_used(), full(), clear_used(), prepare_target_payload(), target_payload_size(),
56 * result_payload_blob(), result_payload_hndl().
57 * - For receiver-engine, namely Native_socket_stream internals: a `private` `this->` API.
58 * - The expected order of ops for each batched-read (after reaching initialized() originally) is:
59 * -# User invokes sync_io::Native_socket_stream::async_receive_blob_batch() or
60 * sync_io::Native_socket_stream::async_receive_native_handle_batch() (or similar) which internally invokes
61 * a private `*this` API (spoiler alert: `nb_read()`).
62 * -# User consumes results: result_payload_blob() and possibly result_payload_hndl() (the latter only
63 * in the `async_receive_native_handle_batch()` case).
64 * -# User resets for next batch-read:
65 * -# Save N = n_used().
66 * -# clear_used().
67 * -# prepare_target_payload() (for each of the first N slots only).
68 *
69 * @internal
70 * ### Privileged access to nb_read() ###
71 * nb_read() is the `private` API that performs the actual reading-into `*this`, after the user has prepped
72 * via prepare_target_payload() et al and before they consume the results of the read.
73 *
74 * Flow-IPC internals may access it through `detail/`-residing `friend` facade type
75 * Native_socket_stream_msg_batch_in_privileged. It is very simple.
76 *
77 * ### Impl notes ###
78 * Internally a `*this` just stores a asio_local_stream_socket::Msg_batch_in called #m_batch. It is the
79 * general workhorse dealing with the local socket and native OS API(s) and so on. We "just" build slightly
80 * on top of that to support our specific *slightly* higher-level protocol which adds some special control messages
81 * needed for sync_io::Native_socket_stream, like auto-ping and graceful-close.
82 *
83 * To understand `*this` impl, then, it should be sufficient to grok asio_local_stream_socket::Msg_batch_in
84 * and then read inline comments in `*this`.
85 *
86 * @endinternal
87 *
88 * @tparam Msg_resource_t
89 * Same as for Generic_msg_batch_in.
90 */
91template<typename Msg_resource_t>
93 private boost::noncopyable
94{
95public:
96 // Types.
97
98 /// Alias for template param `Msg_resource_t`.
99 using Msg_resource = Msg_resource_t;
100
101 /// Buffer(s) description type (conforming to boost.asio `MutableBufferSequence` concept) supported by `*this`.
103
104 // Constructors/destructor.
105
106 /**
107 * Identical to ctor of Generic_msg_batch_in.
108 * @param max_msg_count
109 * See above.
110 */
111 explicit Native_socket_stream_msg_batch_in(size_t max_msg_count);
112
113#ifdef IPC_DOXYGEN_ONLY
114 /// Clears all resources -- including any #Msg_resource currently attached to any receive-slot.
116#endif
117
118 // Methods.
119
120 /**
121 * Identical to Generic_msg_batch_in.
122 * @return See above.
123 */
124 bool initialized() const;
125
126 /**
127 * Identical to Generic_msg_batch_in.
128 * @return See above.
129 */
130 size_t n_used() const;
131
132 /**
133 * Identical to Generic_msg_batch_in.
134 * @return See above.
135 */
136 bool full() const;
137
138 /// Identical to Generic_msg_batch_in.
139 void clear_used();
140
141 /**
142 * Identical to Generic_msg_batch_in.
143 * @param target_blob
144 * See above.
145 * @param msg_resource
146 * See above.
147 * @param idx
148 * See above.
149 */
150 void prepare_target_payload(const Mutable_buffer_sequence& target_blob, Msg_resource&& msg_resource,
151 size_t idx = -1);
152
153 /**
154 * Identical to Generic_msg_batch_in.
155 * @return See above.
156 */
157 size_t target_payload_size() const;
158
159 /**
160 * Identical to Generic_msg_batch_in, except `idx >= n_used()` is undefined behavior (assertion may trip).
161 *
162 * @param idx
163 * See above.
164 * @param msg_resource_ptr
165 * See above.
166 * @return See above.
167 */
168 size_t result_payload_blob(size_t idx, Msg_resource** msg_resource_ptr = nullptr);
169
170 /**
171 * Identical to Generic_msg_batch_in, except `idx >= n_used()` is undefined behavior (assertion may trip).
172 *
173 * @param idx
174 * See above.
175 * @return See above.
176 */
177 Native_handle result_payload_hndl(size_t idx) const;
178
179 /**
180 * Identical to Generic_msg_batch_in.
181 * @param os
182 * See above.
183 */
184 void to_ostream(std::ostream* os) const;
185
186 /**
187 * Stats: Histogram tracking how many raw in-dgrams were received by each raw OS batch-read (`recvmmsg()` in Linux).
188 * The would-block outcome results in the value 0 (and should be rare to non-existent, if a read is engaged only
189 * when the kernel-socket event-tracking system reports readiness). So does a socket error, but this would
190 * hose the socket usually. 1 means no actual batching occurred (only 1 message was ready).
191 *
192 * @return See above.
193 */
194 const flow::util::stat::Histogram_counter& histo_raw_read_n_msgs() const;
195
196 /**
197 * Stats: Histogram tracking how many of the in-dgrams counted by histo_raw_read_n_msgs() were then filtered-out as
198 * not representing user-reportable in-messages; as of this writing either auto-ping or graceful-close tokens.
199 *
200 * A graceful-close token appears at most once per `*this`. An auto-ping token can be ubiquitous but only under
201 * low-traffic conditions, as auto-pings are sent only when a connection is not being kept-alive by user
202 * traffic (e.g., if the auto-ping interval is 5 sec, then an auto-ping would be sent 5 sec after the last
203 * user message or auto-ping, whichever was last).
204 *
205 * Hence, outside of short-lived connections and low-traffic ones, this histogram should show results near zero.
206 * It can be viewed as a sanity-check of the algorithm.
207 *
208 * @return See above.
209 */
210 const flow::util::stat::Histogram_counter& histo_raw_read_n_msgs_filtered_out() const;
211
212 /**
213 * Stats: Histogram tracking how many OS batch-reads (`recvmmsg()` in Linux) resulted from each
214 * Native_socket_stream::async_receive_blob_batch() or Native_socket_stream::async_receive_native_handle_batch()
215 * call. The outcome 1 is the minimum and should be by far the most frequent. A 2nd read would occur if and only
216 * if *all* of the following happened in the 1st read:
217 * - `this->full() == true` immediately following it (filled entire `*this`, so don't know if we're now in
218 * would-block or happened to grab exactly the number of in-dgrams that were ready);
219 * - at least 1 of those was auto-ping or graceful-close (or any other non-user-message dgram, if the protocol
220 * features more such possibilities; not as of this writing).
221 *
222 * A 3rd read would occur if and only if this happened again; and so forth.
223 *
224 * Hence, outside of short-lived connections and low-traffic ones, this histogram should show results near 1.
225 * It can be viewed as a sanity-check of the algorithm.
226 *
227 * @return See above.
228 */
229 const flow::util::stat::Histogram_counter& histo_usr_read_n_raw_reads() const;
230
231private:
232 // Friends.
233
234 /// Facade type that exposes specific `private` APIs of `*this` to other internal Flow-IPC code.
236
237 // Types.
238
239 /**
240 * Alias (for more accurate naming in context; brevity) for type of the message-type internal field, sent/received
241 * in as the first bytes of every valid dgram. (Despite the mention of length in the alias target, in our case
242 * this has nothing to do with any length of anything; but the same type is used in the alternative non-dgram-based
243 * mode, and there it can be a length; see Native_socket_stream_cfg if you want to know more... but really we don't
244 * care here other than the fact it's mandatory to use the same type for both modes/purposes, as some at least
245 * send-side common code in Native_socket_stream impl handles both modes.)
246 */
248
249 /**
250 * The `Msg_resource` type attached to each slot of our internal #m_batch. Naturally this includes the user's
251 * chosen #Msg_resource type; but we also add the backing (tiny) area where the first in-dgram bytes are always
252 * written, containing the #msg_type_t.
253 */
255 {
256 /// Per-slot resource moved from user's object via `prepare_target_payload()`.
258
259 /**
260 * This slot's potential/actual in-dgram's message-type field, where the actual OS receive-batch call
261 * shall place (and/or has placed) the first bytes of the in-dgram.
262 *
263 * ### Rationale for indirect storage via smart-pointer handle ###
264 * There is exactly one reason it's not simply `msg_type_t m_msg_type;`: the address of the leading
265 * scatter/gather buffer (that we pass-up to #m_batch in our prepare_target_payload() wrapper around
266 * asio_local_stream_socket::Msg_batch_in::prepare_target_payload()) will continue to be accurate,
267 * even as `*this` Msg_resource_impl is moved in memory. This requirement is documented on
268 * the template param's doc in `Msg_batch_in` class doc header; the explanation is around there too
269 * (spoiler alert: `reuse_result_payloads()`, which we use, is a culprit; as is the initial load
270 * via `prepare_target_payload(Msg_resource_impl&&)` -- which is unavoidable as-is).
271 *
272 * The main cost is that of [de]allocating these little guys per-`prepare_target_payload()`. Perf-wise it's
273 * okay, but actually it is a bit annoying nevertheless. Could it be avoided? Formally, as-is, no:
274 * it's a requirement as noted above. Could we eliminate the requirement? Maybe, but it is
275 * no joke: to begin with, the #Mutable_buffer_sequence_impl given to prepare_target_payload() has to remain
276 * accurate, as our `Msg_resource_impl` is moved-into #m_batch. Then there are the swaps via
277 * reuse_result_payloads(). At the very least some serious thinking is involved (let's not try to do it here).
278 *
279 * That said let's have some perspective. Yes, it's a tiny buffer, but it's a buffer; it is pretty normal
280 * to expect receive-APIs (even as relatively complex as ours here) to assume the target buffer(s) do not
281 * move around in memory. It is probably just an acceptable cost in the end.
282 */
283 boost::movelib::unique_ptr<msg_type_t> m_msg_type;
284 }; // struct Msg_resource_impl
285
286 /**
287 * Scatter/gather structure subsuming the user's "big" `Blob_mutable` preceded by our internal-needs
288 * tiny #msg_type_t prefix.
289 */
290 using Mutable_buffer_sequence_impl = boost::array<util::Blob_mutable, 2>;
291
292 // Methods.
293
294 /**
295 * The API for the receiver-engine (sync_io::Native_socket_stream) to perform the core batch-receive.
296 * Access through internal facade helper type Native_socket_stream_msg_batch_in_privileged.
297 *
298 * This guy does the bulk of (batched) receive-side logic for sync_io::Native_socket_stream (including
299 * even dealing with its internal Protocol_negotiator receive-side). So all that big-guy has to do is
300 * manage getting to non-would-block, and knowing to abandon the pipe on error; once it is time to read
301 * (even at the very start), this `nb_read()` will take care of business.
302 *
303 * However! It is very important to understand this `nb_read()`'s contract semantics -- particularly around
304 * how reading data and emitting errors works when combined. Please read carefully.
305 *
306 * So: nb_read() performs a high-perf non-blocking batched-receive into the batch area with indices
307 * [n_used(), `max_msg_count`). See ctor for the latter. Usually, but not necessarily, `this->n_used() == 0`,
308 * as usually one would clear_used() first, or `*this` was just initialized(). Returns `false` if
309 * the aforementioned range is empty -- that is full() is `true` and no-ops modulo logging; otherwise returns
310 * `true` and proceeds. Further documentation assumes the latter.
311 *
312 * Let `n_payloads` be the size of the aforementioned range (thus `n_payloads >= 1`). This receives
313 * at most `n_payloads` user in-messages. If no error but no in-messages are pending, it is would-block
314 * (see specific code below; *note* we choose to emit the low-level boost.asio code, not
315 * error::Code::S_SYNC_IO_WOULD_BLOCK; performing the translation is up to the caller).
316 * If no error, and messages were pending, the number is not explicitly returned
317 * but rather reflected in n_used() increasing by that # (at least 1, at most `n_payloads`). If error,
318 * and no messages were pending, emits that error and leaves n_used() unchanged. If messages were received,
319 * and *then* an error was detected, then emits that error *and* modifies n_used() as described earlier
320 * in the paragraph.
321 *
322 * @warning Read that again please! It *is* a possible outcome that messages were emitted (n_used() changed)
323 * *and* an error is emitted. We do *not* defer the error as some other APIs. (Rationale omitted but
324 * trust us.) This possibility includes *exactly* the following errors:
325 * (1) `Native_socket_stream`-protocol graceful-close (error::Code::S_RECEIVES_FINISHED_CANNOT_RECEIVE) and
326 * (2) low-level graceful-close a/k/a EOF (`boost::asio::error::eof`). The caller of course can (and as of
327 * this writing does) do the deferring themselves if appropriate. All errors outside of those 2 are *not*
328 * combined with emitting 1+ messages and can informally be thought of as truly exceptional; though
329 * informally it is likely best not to rely on this in your code.
330 *
331 * @note Be aware of the *implied would-block* condition. See full() doc header. Spoiler alert:
332 * if we yielded data/no error, then it wasn't would-block... but if `full() == false` post-op, then
333 * the in-pipe is nevertheless in would-block state. It would be wasteful to try another nb_read().
334 * Conversely, though, if `full()` then you can and should nb_read() again (assuming the goal is to
335 * drain the in-pipe such as in edge-triggered loops).
336 *
337 * ### Additional context, for Native_socket_stream implementer/maintainer ###
338 * Arguably the most complex thing this does is emit only user in-messages while
339 * transparently but performantly filtering-out control messages (including auto-ping and graceful-close).
340 * For completeness/context here is how some control messages are handled. Arguably these are impl details,
341 * but we're all friends (but not `friend`s) here.
342 *
343 * - Leading Protocol_negotiator stuff is checked as appropriate (emitting negotiation error if appropriate,
344 * continuing transparently as appropriate).
345 * - Auto-pings are ignored (their slots are reset for the next nb_read() performantly)
346 * except implying non-idleness (see `not_idle_on_would_block` arg).
347 * - Our-protocol graceful-close message is detected and causes emission of
348 * error::Code::S_RECEIVES_FINISHED_CANNOT_RECEIVE (as to be delivered to user).
349 *
350 * @param logger_ptr
351 * Logger to which to log (or null).
352 * @param peer_socket_ptr
353 * The (presumably) connected dgram-based socket.
354 * @param protocol_negotiator
355 * The Native_socket_stream protocol-negotiator which is fully managed by this method.
356 * @param no_hndls
357 * If `false`, receiving a native handle in an in-dgram is normal and is performed when relevant;
358 * if `true`, receiving a native handle triggers the appropriate error.
359 * @param not_idle_on_would_block
360 * Out-arg which has meaning if and only if we emit `boost::asio::error::would_block`; in that
361 * case: `true` means that, although (per `would_block` definition above) no user in-messages were
362 * read + no further data are pending, nevertheless internally *some* data *were* received
363 * (as of this writing: auto-ping(s)), and therefore the connection is not idle per
364 * the Blob_receiver concept definition of "idle." (Any other non-error-emitting outcome by definition
365 * means user in-messages were received and thus => not idle.) (Hence `false` means that given it's a
366 * would-block, also no traffic at all internally was received.)
367 * @param err_code
368 * See `flow::Error_code` docs for error reporting semantics. Generated codes:
369 * error::Code::S_INVALID_ARGUMENT (`!this->initialized()` at entry);
370 * error::Code::S_LOW_LVL_INTERNAL_PROTOCOL_INVALID_HEADER (illegal in-dgram: other side misbehaved);
371 * error::Code::S_BLOB_RECEIVER_GOT_NON_BLOB (`no_hndls == true`, but other side sent a handle: misbehaved);
372 * error::Code::S_RECEIVES_FINISHED_CANNOT_RECEIVE (graceful-close received: other side did
373 * `Blob_sender::async_end_sending()` or equivalent);
374 * those emitted by asio_local_stream_socket::Msg_batch_in::nb_read();
375 * `boost::asio::error::would_block` (none of the above errors apply -- including `eof` error from prev bullet
376 * point, and not even 1 user in-message was pending).
377 * @param message_flags
378 * See nb_read_some_with_native_handle().
379 * @param rcv_stats
380 * If not null, `nb_read()` accumulates into `*rcv_stats`.
381 * @return `false` if pre-condition `!this->full()` is failed; `true` otherwise. See above.
382 */
383 bool nb_read(flow::log::Logger* logger_ptr,
385 Protocol_negotiator* protocol_negotiator,
386 bool no_hndls, bool* not_idle_on_would_block, Error_code* err_code = nullptr, int message_flags = 0,
387 stat::Blob_rcv_stats* rcv_stats = nullptr);
388
389 /**
390 * nb_read() helper: Checks in-dgram at given index, returning `true` if and only if the dgram represents anything
391 * except a valid message suitable to pass-through to Native_socket_stream::async_receive_blob_batch() (et al)
392 * user.
393 *
394 * @param logger_ptr
395 * See nb_read().
396 * @param idx
397 * See result_payload_blob().
398 * @param no_hndls
399 * See nb_read().
400 * @return `false` if vanilla user in-message, `true` otherwise.
401 */
402 bool is_not_user_message(flow::log::Logger* logger_ptr, size_t idx, bool no_hndls) const;
403
404 // Data.
405
406 /**
407 * The natively-batched batch containing target buffer(s) from the user plus a bit of added buffer-age of ours
408 * (namely the #msg_type_t per slot) needed to support the Native_socket_stream protocol.
409 */
411
412 /**
413 * Determined permanently by the first prepare_target_payload(), this is the max size in bytes of each
414 * in-message (per slot); 0 before then.
415 */
417
418 /// See histo_raw_read_n_msgs().
419 flow::util::stat::Histogram_counter m_histo_raw_read_n_msgs;
420
421 /// See histo_raw_read_n_msgs_filtered_out().
422 flow::util::stat::Histogram_counter m_histo_raw_read_n_msgs_filtered_out;
423
424 /// See histo_usr_read_n_raw_reads().
425 flow::util::stat::Histogram_counter m_histo_usr_read_n_raw_reads;
426
427 /**
428 * The results in #m_histo_raw_read_n_msgs et al shall be logged opportunistically inside `nb_read()` once
429 * the time passes this time-point; `zero()` indicates `nb_read()` has not yet been called, so this will be
430 * first set when it is called. Always `zero()` and unused if
431 * Native_socket_stream_cfg::S_STATS_LOG_ENABLED is `false`.
432 */
434}; // class Native_socket_stream_msg_batch_in
435
436// Template implementations.
437
438template<typename Msg_resource_t>
440 m_batch(max_msg_count),
441 m_target_payload_sz(0),
442 /* A bucket for 0 in-dgrams (would-block), then buckets for all outcomes up to and including getting the max
443 * (each one covering BUCKET_SZ adjacent outcomes; e.g., 2 => [1, 2][3, 4][5, 6]...[63, 64]). */
444 m_histo_raw_read_n_msgs(1 + (max_msg_count / Native_socket_stream_cfg::S_STATS_HISTO_MSG_CT_BUCKET_SZ),
445 1, Native_socket_stream_cfg::S_STATS_HISTO_MSG_CT_BUCKET_SZ, 0),
446 // Ditto (this covers a subset of events tracked in m_histo_raw_read_n_msgs).
447 m_histo_raw_read_n_msgs_filtered_out(1 + (max_msg_count / Native_socket_stream_cfg::S_STATS_HISTO_MSG_CT_BUCKET_SZ),
448 1, Native_socket_stream_cfg::S_STATS_HISTO_MSG_CT_BUCKET_SZ, 0),
449 /* A bucket for an nb_read() leading to 1 raw read; then one for each outcome 2, 3, ..., 9; and lastly a bucket
450 * for 10-or-more raw reads. Really any result other than 1 is quite unlikely... 10+ is almost science fiction. */
451 m_histo_usr_read_n_raw_reads(10, 1, 1, 1)
452{
453 // OK.
454}
455
456template<typename Msg_resource_t>
458 (const Mutable_buffer_sequence& target_blob, Msg_resource&& msg_resource, size_t idx)
459{
460 using util::Blob_mutable;
461
462 if (target_payload_size() == 0)
463 {
464 m_target_payload_sz = target_blob.size();
465 }
466 else
467 {
468 assert((target_payload_size() == target_blob.size())
469 && "Msg_batch_in concept requires the max message size be the same across all slots.");
470 }
471
472 /* Wrap their Msg_resource into a wrapper Msg_resource_t of our own which is the
473 * the struct containing their Msg_resource and our msg_type_t value target. And, make a simple 2-array
474 * of Blob_mutable (iovec-like) things which describe this scatter/gather blob target:
475 * [msg_type_t][their Blob_mutable target]
476 * So then a given in-dgram will be received into the 1st fixed-size thing (the message type), and any remaining
477 * bytes (possibly none) will go to the user's target buffer. this->nb_read() will then populate as many
478 * such in-dgrams as possible -- then check each one (is it a user message? a ping? etc.). Spoiler alert:
479 * illegal in-dgrams aside, the user buffer will be touched only when it's a user in-message with meta-blob;
480 * all the other legal in-messages by definition feature no meta-blob, so it won't be written-to. */
481
482 Msg_resource_impl msg_resource_impl{ std::move(msg_resource),
483 boost::movelib::unique_ptr<msg_type_t>(new msg_type_t{0}) };
484 Mutable_buffer_sequence_impl buf_seq{ Blob_mutable{msg_resource_impl.m_msg_type.get(), sizeof(msg_type_t)},
485 target_blob };
486
487 m_batch.prepare_target_payload(buf_seq, std::move(msg_resource_impl), idx);
488}
489
490template<typename Msg_resource_t>
492{
493 return m_target_payload_sz;
494}
495
496template<typename Msg_resource_t>
497size_t
499 Msg_resource** msg_resource_ptr)
500{
501 assert((idx < n_used()) && "Attempt by outside user to access a result in a slot that was not received-to "
502 "and/or holds no user in-message; could also be an internal bug in *this class.");
503
504 size_t n_rcvd;
505 if (msg_resource_ptr)
506 {
507 Msg_resource_impl* msg_resource_impl;
508 n_rcvd = m_batch.result_payload_blob(idx, &msg_resource_impl);
509 *msg_resource_ptr = &(msg_resource_impl->m_msg_resource);
510 }
511 else // Don't waste time on grabbing msg_resource_impl, if we won't use it.
512 {
513 n_rcvd = m_batch.result_payload_blob(idx);
514 }
515
516 assert((n_rcvd >= sizeof(msg_type_t))
517 && "Bug in *this class? Apparently gave user access to not-a-user-in-message.");
518 return n_rcvd - sizeof(msg_type_t);
519}
520
521template<typename Msg_resource_t>
523{
524 assert((idx < n_used()) && "Attempt by outside user to access a result in a slot that was not received-to; "
525 "could also be an internal bug in *this class.");
526
527 return m_batch.result_payload_hndl(idx);
528}
529
530template<typename Msg_resource_t>
532{
533 return m_batch.initialized();
534}
535
536template<typename Msg_resource_t>
538{
539 return m_batch.n_used();
540}
541
542template<typename Msg_resource_t>
544{
545 return m_batch.full();
546}
547
548template<typename Msg_resource_t>
550{
551 m_batch.clear_used();
552}
553
554template<typename Msg_resource_t>
556 (flow::log::Logger* logger_ptr,
558 Protocol_negotiator* protocol_negotiator,
559 bool no_hndls, bool* not_idle_on_would_block, Error_code* err_code, int message_flags,
560 stat::Blob_rcv_stats* rcv_stats)
561{
562 using flow::log::Sev;
563 using util::Blob_mutable;
564 constexpr auto PING = Native_socket_stream_cfg::S_PING_SENTINEL;
565 constexpr auto TYPE0 = msg_type_t{0x0000};
566 constexpr auto MSG_TYPE_SZ = sizeof(msg_type_t);
567
568 FLOW_ERROR_EXEC_AND_THROW_ON_ERROR(bool, nb_read,
569 logger_ptr, peer_socket_ptr, protocol_negotiator, no_hndls,
570 not_idle_on_would_block, _1, message_flags, rcv_stats);
571 // ^-- Call ourselves and return if err_code is null. If got to present line, err_code is not null.
572
573 assert(not_idle_on_would_block);
574
575 if (full())
576 {
577 return false; // As promised special degenerate case.
578 }
579 // else: Now we needn't worry about it.
580
581 FLOW_LOG_SET_CONTEXT(logger_ptr, Log_component::S_TRANSPORT);
582
583 bool not_idle_val = false; // This is for *not_idle_on_would_block.
584
585 if (protocol_negotiator->negotiated_proto_ver() == Protocol_negotiator::S_VER_UNKNOWN)
586 {
587 /* No data have been received on this connection, so we've yet to do protocol negotiation in this direction.
588 * Reminder: It means we must receive an in-dgram exactly MSG_TYPE_SZ bytes long which shall contain
589 * a version number; and *protocol_negotiator will either accept it or spit out a truthy Error_code.
590 * In the latter case we emit that and GTFO before trying to receiving anything else.
591 *
592 * We choose to do this with a separate 1-msg read as opposed to trying to fold it into the main m_batch.nb_read()
593 * below (which accepts potentially many in-dgrams in one go). Could we do it? Probably yes; we could
594 * detect it using is_not_user_message() below (along with auto-pings, graceful-closes, etc.) and so on.
595 * However this'd break the rationale/conventions/procedure behind the Protocol_negotiator pattern; the whole
596 * point is to read, ahead of anything else, one version-bearing thing whose format is agreed-upon indefinitely.
597 * If it doesn't pass Protocol_negotiator check, then we don't (formally speaking anyway) know anything about
598 * further messages that might be received and should not be checking anything about them -- it could even lead
599 * to a crash, for example, if we make some assumption about lengths or who knows what.
600 *
601 * Furthermore it is probably simpler to do it separately like this; no need to worry about stuff about protocol
602 * negotiation being written into user buffers... that type of thing. Is there a perf penalty? Answer: Not really;
603 * doing a single separate read at the start of the connection is negligible. */
604
605 Native_handle hndl_or_null;
606 msg_type_t proto_ver_as_msg_type;
607 const auto n_rcvd
608 = asio_local_stream_socket::nb_read_some_with_native_handle<asio_local_stream_socket::Protocol_pkt_stream>
609 (logger_ptr, peer_socket_ptr, &hndl_or_null, Blob_mutable{&proto_ver_as_msg_type, MSG_TYPE_SZ},
610 err_code, message_flags);
611 if (*err_code)
612 {
613 /* Fatal error => obviously overall fatal error. Would-block => can't read any more messages either.
614 * So we are done here. Though in would-block case the contract requires the following. */
615 (*err_code == boost::asio::error::would_block) && (*not_idle_on_would_block = false); // Contract fulfilled.
616 return true;
617 }
618 /* else: Got something (an in-dgram and possibly a handle). Check for various protocol misbehaviors first.
619 * (Incidentally: If there was an in-dgram but too large for sizeof(msg_type_t), that would've been detected
620 * by nb_read_some_with_native_handle() and emitted as the appropriate error. So we needn't worry about that.) */
621 if (n_rcvd < MSG_TYPE_SZ)
622 {
624 FLOW_LOG_WARNING("Native_socket_stream_batch [" << *this << "]: "
625 "Native_socket_stream batch nb-read of protocol-negotiation dgram: "
626 "illegal too-short in-dgram.");
627 hndl_or_null.close(); // If there's also a handle in there for some reason, don't leak it.
628 return true;
629 }
630 // else if (n_rcvd == MSG_TYPE_SZ):
631
632 if (!hndl_or_null.null())
633 {
634 FLOW_LOG_WARNING("Native_socket_stream_batch [" << *this << "]: "
635 "Native_socket_stream batch nb-read of protocol-negotiation dgram: "
636 "Expecting protocol-negotiation (first) in-dgram "
637 "to contain *only* a meta-blob: but received Native_handle is non-null which is "
638 "unexpected.");
639 hndl_or_null.close(); // Let's not leak it.
640
641 #ifndef NDEBUG
642 const bool ok =
643 #endif
645 assert(ok && "Protocol_negotiator breaking contract? Bug?");
646 assert(*err_code
647 && "Protocol_negotiator should have emitted error given intentionally bad version.");
648 return true;
649 } // if (hndl_or_null)
650 // else: All cool format-wise; check the version finally.
651
652 /* Protocol_negotiator handles everything (invalid value, incompatible range...); we just know
653 * the encoding is to shove the version number into what is normally the msg_type_t field. */
654#ifndef NDEBUG
655 const bool ok =
656#endif
657 protocol_negotiator->compute_negotiated_proto_ver
658 (static_cast<Protocol_negotiator::proto_ver_t>(proto_ver_as_msg_type), err_code);
659 assert(ok && "Protocol_negotiator breaking contract? Bug?");
660
661 if (*err_code)
662 {
663 return true; // Protocol negotiation did not pass. It logged.
664 }
665 // else: Cool; done with protocol negotiation forever (it is marked in *protocol_negotiator).
666
667 not_idle_val = true; // Certainly we got an in-message.
668
669 // Protocol-negotiation dgram = MSG_TYPE_SZ low-level bytes (not a user message though).
670 rcv_stats && (rcv_stats->m_total_low_lvl_bytes += MSG_TYPE_SZ);
671
672 /* Now what? Answer: We asked for 1 msg and got 1 msg; so now it might be in would-block or not. Essentially
673 * it's as-if protocol negotiation was *not* necessary after all; and now we begin the actual nb-read
674 * into m_batch. So, outside of having already marked not_idle_val as true, we "start" fresh by
675 * simply falling-through. */
676 } // if (*protocol_negotiator indicated we need to do protocol negotiation) [Inside {}, we made it so we don't now.]
677
678 // See justification for this thing in similar spot around `do_log_trace` in Msg_batch_in::nb_read().
679 const bool do_log_trace = logger_ptr && logger_ptr->should_log(Sev::S_TRACE, get_log_component());
680
681 /* Possible legal payloads (m_msg_type, meta-blob-length, hndl-or-none):
682 * - 0x0000, 0, hndl => user in-message (with handle but no meta-blob) [NOTE: Illegal if no_hndls is true.]
683 * - 0x0000, >0, none => user in-message (no handle but with meta-blob)
684 * - 0x0000, >0, hndl => user in-message (with handle and meta-blob) [NOTE: Illegal if no_hndls is true.]
685 * - 0x0000, 0, none => graceful-close [NOTE: This + no-other-error => emit graceful-close error.]
686 * - 0xFFFF, 0, none => auto-ping [NOTE: Ignore it, other than it too implies non-idleness.]
687 * - Others = illegal.
688 * We want the top 3 to be at the start, so we will use m_batch.reuse_result_payloads() to swap stuff around,
689 * moving the others (including illegal ones, which should essentially not exist, but we'll still check) at the end
690 * of [0, this->n_used()) range. The ones at the start can then just be left alone (user will grab 'em as desired).
691 * Then, we can scan the trailing ones: any illegals => error; graceful-close => graceful-close error;
692 * auto-ping => ignore.
693 *
694 * Caveats/stuff:
695 * - Generally, would-block means we shall emit would-block as per contract. Recall however that
696 * while there is the standard would-block semantic (got 0 messages + would-block error), there is also the
697 * special subtle (perf-saving) additional semantic, wherein m_batch.nb_read() yields no error but also
698 * does not fill all available slots => would-block. No problem; we just need to detect this.
699 * - In any case: would-block is just another error really, and we emit it as such. Just remember that
700 * certainly we must make any user-messages also acquired available as opposed to somehow thrown away.
701 * - However it is also possible (and if n_used() is such that there is exactly 1 slot left, common) that
702 * m_batch.nb_read() *does* fill all available slots. Now we do *not* know whether the connection reached
703 * would-block; maybe it has, maybe it hasn't. Well, no problem: after all, all slots filled => nothing more
704 * we can read-into, so we have to stop (the caller will do what it wants; spoiler alert: typically try another
705 * read after handling all the stuff we got this time)...
706 * - ...but a corner case occurs where, yes, m_batch.nb_read() *does* fill all available slots; but then
707 * the aforementioned m_batch.reuse_result_payloads() eliminates a bunch of non-user-messages but encounters
708 * no error (i.e., it finds 1+ auto-pings). So now on one hand we don't know whether connection is in
709 * would-block, but we *do* have 1+ available slots again. So in this case we must do another m_batch.nb_read()
710 * and repeat the algorithm. So that's why it must be a `while` loop that stops either on error
711 * (including would-block, or graceful-close for that matter) or running out of slots -- *after*
712 * weeding out the auto-pings.
713 * - Our-protocol graceful-close error is annoyingly special. If there are no user messages, then no prob. But if
714 * there is 1+ user message and then graceful-close, by contract we must emit the error *and also* leave the
715 * normal messages for the user. (By contract what to do subsequently is the caller's responsibility.) That said
716 * we need not really do anything special; nothing can follow graceful-close, so there's no further reading -- as
717 * with any other error -- but still we must be mindful of that situation and follow our contract in terms of
718 * post-conditions. In a sense would-block is similar actually, as far as we are concerned (not so much the
719 * caller, which would stop messing with this in-pipe on graceful-close/other errors but not would-block).
720 * - Low-level graceful-close (`eof`) is also annoyingly special. m_batch.nb_read() emits this condition by setting
721 * our low_graceful_close=true, and it can do this while also emitting 1+ messages (or 0 of them, too). We need
722 * to detect this and emit `eof` while keeping any in-messages as well.
723 * - All other errors (so not would-block, not graceful-close of either kind) we specifically advertised as
724 * being uncombinable with emitting 1+ in-messages. They indicate some kind of catastrophic exceptional hosedness
725 * (like a truncated message maybe, or even EBADF) and we promised to emit no messages then; so if iteration 1
726 * emits some, then iteration 2 finds such an exceptional problem, we have to undo any n_used() change from
727 * iteration 1. */
728
729 bool got_a_user_msg = false; // For a particular corner-case condition check: true on 1+ user msgs received.
730 /* For each iteration mark true if and only if overall this->nb_read() yielded the situation wherein we've received 1+
731 * user messages total, yet the last actual nb-read did not fill all available slots. */
732 bool implied_would_block;
733
734 unsigned int n_iters = 0;
735 const auto orig_n_used = m_batch.n_used(); // Set back to this on exceptional error in iteration 2+.
736
737 do
738 {
739 ++n_iters;
740
741 /* It's a tricky algorithm to reason about, so let's go in detail at each step. At this point:
742 * - Before the .nb_read():
743 * - There is no error on the connection (including from any earlier loop iteration in
744 * the present function, as that stops the loop; see below).
745 * - .n_used() is not necessarily 0; it might be the first iteration, but for some reason they
746 * ran us with some slots already filled; or it might be iteration 2+, and the preceding .nb_read()
747 * or further processing yielded no error of any kind (including would-block) but did as a result of
748 * the "further processing" free up 1+ slot(s) at the end. In any case .n_used() is not max_msg_count,
749 * so we should .nb_read(). */
750
751 const auto n_used_pre_read = m_batch.n_used();
752 bool low_graceful_close;
753 size_t iter_low_lvl_bytes;
754#ifndef NDEBUG
755 const bool ok =
756#endif
757 m_batch.nb_read(logger_ptr, peer_socket_ptr, &low_graceful_close, err_code, message_flags,
758 &iter_low_lvl_bytes);
759 assert(ok && "For 1st iteration we should've returned pre-loop; "
760 "for others we should've stopped the loop (b/c full()) before trying another nb-read.");
761
762 /* At this point the possibilities are:
763 * - *err_code is false, but 0 messages read (=> `low_graceful_close == true`), then it is as-if
764 * m_batch.nb_read() emitted `eof`; so just pretend that is what happened: assign it accordingly, then
765 * execute the next bullet point. (`low_graceful_close == true` with 1+ messages is handled below, not here.)
766 * - *err_code is true:
767 * - If would-block: No more to read, period. Loop should stop.
768 * Official would-block means 0 in-messages read; so that's it; nothing to post-process.
769 * - Emit the would-block via *err_code, if no prior iteration has produced even 1 user in-message.
770 * Otherwise clear *err_code. (Caller can still detect the overall "implied would-block.")
771 * However, mark that it is "implied would-block": loop should stop.
772 * - If other error: No more to read (socket hosed), period. Loop should stop.
773 * Emit the error via *err_code; *and* reset n_used() to orig_n_used.
774 * - However, any legal messages from any prior iterations have been recorded. It is up to the caller to
775 * proceed how they want.
776 * - *err_code is false; and 1+ messages read (but low_graceful_close may be true or false).
777 * - `not_idle_val = true`: The connection is not idle.
778 * - If slots still remain, it is "implied would-block": loop should stop. But do *not* emit this in
779 * *err_code; and the next bullet points still apply.
780 * - Must post-process them:
781 * - Place all regular user messages at the start of the read-to range, swapping-away the others
782 * to the end of that range: m_batch.reuse_result_payloads().
783 * - For each swapped-away slot (if any):
784 * - If illegal, set *err_code to reflect it (stop scanning the range; loop should stop).
785 * In addition: reset n_used() to orig_n_used.
786 * - If graceful-close, set *err_code to reflect it (stop scanning the range; loop should stop).
787 * (Note: We are reasonably but not maximally paranoid about checking for legality; that is e.g.
788 * graceful-close means no need to check further; but there could be slots after that which are illegal.
789 * We could keep going and search for illegality or assume the best and stop for perf.
790 * Philosophically these steps are for safety, not security, so we extend some trust to the other side
791 * and check for illegality opportunistically/defensively (against bugs by the other side) only.)
792 * - Note: All else being appropriate (basically: if all these slots are just auto-pings), they can be
793 * reused for future reads without any further work required on them. The target blob areas are still
794 * fine to read-to, etc.
795 * - If the post-processing did not emit a truthy *err_code, check low_graceful_close. If it's true, then
796 * set *err_code to `eof`; otherwise (mainstream case) nothing special to do.
797 * - If by now we have truthy *err_code, loop should now stop. However, any legal messages
798 * have been recorded. It is up to the caller to proceed how they want.
799 * - Else (falsy *err_code still):
800 * - If, after the post-processing, we see that "all regular user messages" = "0 such messages,"
801 * then:
802 * - If "implied would-block" detected above:
803 * - If no prior iteration has produced even 1 user in-message: Set *err_code = would-block.
804 * (Loop will soon stop due to *err_code.)
805 * - Else: just continue. (Loop will soon stop due to "implied would-block.")
806 * - Else: just continue. (Loop might continue: no slots remained, but post-processing might have freed
807 * some up. Otherwise it'll stop due no slots remaining.)
808 * - Else (if got 1+ user-messages): just continue.
809 * - If *err_code still falsy:
810 * - If "implied-would-block" detected above: loop stops.
811 * - Else: If *after post-processing* no slots remain: loop stops.
812 * - Else: Gotta do another .nb_read() / loop iteration (slots remain; no would-block yet detected).
813 * - Else (if *err_code is truthy): loop stops. */
814
815 // Read did proceed: slots were available.
816
817 const auto n_used_post_read = m_batch.n_used();
818
819 /* This is, in a sense, a locally-available helper method of `*this`'s class. Perf-wise:
820 * - Its existence is free (no captures).
821 * - Its execution is not free -- but errors are rare.
822 *
823 * Call it below, on error, to put *this back to pre-us state. ("Pre-us state" = in the semantically
824 * meaningful sense. Meaningless values -- in slots [n_used(), ...) -- may be modified, but it does not matter.) */
825 const auto cleanup_on_error = [](auto this_obj, size_t orig_n_used, size_t n_used_post_read)
826 {
827 auto& batch = this_obj->m_batch;
828
829 /* Main thing is undo .n_used() having potentially advanced in previous iterations. Thus any slots we would
830 * have changed are meaningless again => same state as pre-us state. The one thing this leaves is that
831 * we've received, potentially, native-handle copies; these would leak. Un-leak them via .close().
832 * Do remember that, while prev iterations may have produced leaked-handles, this iteration's
833 * m_batch.nb_read() may well have done the same. Point being: un-leak anything from prev iterations and
834 * up to where the above .nb_read() populated: [orig_n_used, n_used_post_read). The potential
835 * pruning step (reuse_result_payloads()) below is irrelevant; it's an error => we emit nothing, so we
836 * .close() every slot's handle from any .nb_read().
837 *
838 * (What about un-leaking handles in messages discarded by reuse_result_payloads() *without* a subsequent
839 * error being detected in that same iteration? We won't be called -- no error -- so who'll unleak them?
840 * Answer: Legal, yet discarded, messages -- auto-ping, graceful-close -- do not come with handles. Thus,
841 * no such thing.) */
842
843 for (auto idx = orig_n_used; idx != n_used_post_read; ++idx)
844 {
845 /* If there's a handle in there, un-leak it.
846 * (Stored value in m_batch is not touched; but it remains in [n_used(), ...) <=> meaningless.
847 * @todo For cleanliness/defensiveness it would be good to also nullify it; m_batch lacks the required
848 * receiver-engine-facing API at the moment; could add it (see to-do on
849 * asio_local_stream_socket::Msg_batch_in::result_payload_hndl()).) */
850 batch.result_payload_hndl(idx).close();
851 }
852 batch.clear_used(orig_n_used);
853 }; // auto cleanup_on_error =
854
855 m_histo_raw_read_n_msgs.record_value(n_used_post_read - n_used_pre_read);
856
857 if (*err_code)
858 {
859 assert((n_used_post_read == n_used_pre_read) && "Expected low-level .nb_read() to emit error <=> no in-msgs.");
860
861 if ((*err_code == boost::asio::error::would_block) && got_a_user_msg)
862 {
863 // Interesting and rare enough for INFO log-level.
864 FLOW_LOG_INFO("Native_socket_stream_batch [" << *this << "]: "
865 "Native_socket_stream batch nb-read: raw nb-read got would-block in iteration 2+; but earlier "
866 "iteration got a user in-message + filled all slots + enough auto-pings to free some slot(s) "
867 "for further iteration(s). Will not emit official would-block but will stop reading due to "
868 "overall implied would-block.");
869
870 err_code->clear();
871 implied_would_block = true;
872 }
873 else
874 {
875 implied_would_block = false; // Don't log; m_batch->nb_read() logged enough.
876
877 /* If e_c==would_block && !got_a_user_msg: m_batch.n_used() has not been modified from orig_n_used.
878 * So the following statement is a no-op.
879 * If e_c!=would_block: m_batch.n_used() >= orig_n_used (from preceding iteration(s)), but m_batch.nb_read()
880 * contract is that any non-would-block error is exceptional, meaning *this* iteration did not itself
881 * increase m_batch.n_used(). Therefore we similarly -- as promised for any non-graceful-close error --
882 * guarantee that this overall this->nb_read() does not modify m_batch.n_used(). Hence this statement.
883 * A/k/a: Un-emit any in-messages from prev iteration(s): exceptional error. */
884 cleanup_on_error(this, orig_n_used, n_used_post_read);
885 }
886
887 m_histo_raw_read_n_msgs_filtered_out.record_value(0);
888 } // if (*err_code) [But it may have become falsy { inside }.]
889
890 else if (n_used_pre_read == n_used_post_read) // && !*err_code
891 {
892 assert(low_graceful_close && "Basic nb_read() yielding no error but no messages must mean low_graceful_close.");
893
894 /* Per algorithm described above -- low-level graceful-close *not* following any in-msgs is simple enough to
895 * be handled almost the same as on truthy *err_code just above; simpler really. That is: Keep any in-messages
896 * from prev iteration(s) (do not reset m_batch.n_used()), and it is not would-block of any kind; most saliently
897 * emit this as the specific `eof` error. */
898
899 *err_code = boost::asio::error::eof;
900 implied_would_block = false; // Don't log; m_batch->nb_read() logged enough.
901
902 m_histo_raw_read_n_msgs_filtered_out.record_value(0);
903 }
904
905 else // if (!*err_code) && (n_used_pre_read < n_used_post_read)
906 {
907 not_idle_val = true; // Got anything => not idle.
908 implied_would_block = !full(); // m_batch->nb_read() logged enough.
909
910 // Heuristic check BEGIN -->
911
912 /* To continue grokking the impl, in terms of correctness/functionality, skip to "Heuristic check END."
913 * Before then we perform a heuristic pre-check such that, if it passes, we can skip everything past
914 * "Heuristic check END," replacing it with about ~2 statements and `continue`. Now to explain what we do
915 * here and why:
916 *
917 * Basically what we would do now (as prescribed in the large comment higher-up) is
918 * reuse_result_payloads(F), where F is a functor wrapping this->is_not_user_message(). That might have
919 * filtered-out 1+ non-user-in-messages (which we must then scan) and lowered n_used() as a result.
920 * That code is written (we feel) quite tightly, but nevertheless there are many branches and busy-work
921 * including setting up the functor F. Now consider the eventuality wherein reuse_result_payloads(F)
922 * ends up being a no-op. Then, if you run through that code, you'll see all that work amounts to
923 * `got_a_user_msg = true`. Since the present code is *extremely* perf-sensitive under load, it'd be
924 * an improvement if we could make a computation such that if it results in TRUE, then reuse_result_payloads()
925 * would *definitely* be a no-op; and therefore we could in fact skip that expensive-ish processing;
926 * set got_a_user_msg to true; and `continue`. Naturally key questions must be answered for this to be
927 * worthwhile. To wit:
928 * - Is there a quick such computation? Answer: Yes. If (and only if, actually) every message in
929 * range [n_used_pre_read, n_used_post_read) is a legal user in-message, then reuse_result_payloads()
930 * would no-op. (Among other optimizations -- as you'll see shortly -- we need not assemble a functor
931 * and can make some simple inline checks instead.)
932 * - Will it result in TRUE almost all of the time? Answer: Yes but it requires reasoning/explanation:
933 * When would it return FALSE? The only non-vanilla (legal user in-message) dgrams are: illegal (~never
934 * happens in reality), graceful-close (happens at most once per entire connection's lifetime), and
935 * auto-ping. Therefore only auto-pings matter in practice, perf-wise.
936 * - When under any kind of load (the only case where the added compute of a FALSE-resulting pre-check would
937 * matter in practice) there shall be *no* auto-pings, by definition (they are sent after ~seconds of
938 * idleness).
939 * - When there is no load there will indeed be auto-pings. But in that case the FALSE-resulting pre-check,
940 * while technically a (slight) waste of compute, has ~no impact on overall perf.
941 *
942 * Oh, and another possibility is low_graceful_close (low-level graceful-close), similarly rare as the
943 * high-level graceful-close we just discussed. So we simply add the check for !low_graceful_close; if that is
944 * not the case, then the pre-check immediately fails, and we do the full algorithm (which will handle
945 * low_graceful_close as applicable).
946 *
947 * We've shown that this is a worthwhile endeavour. So here's the computation. */
948
949 /* idx_past_user_msg shall point to right after the last *contiguous* user in-message (if any) in
950 * [n_used_pre_read, n_used_post_read). Meaning, it'll point to the first message from the start that is *not*
951 * a user message. (There may be user messages past that (but not contiguously with the leading ones)!)
952 * - == n_used_pre_read => there are 0 such messages.
953 * - == n_used_post_read => they are all such messages (fast-path). */
954 size_t idx_past_user_msg;
955 if (low_graceful_close)
956 {
957 idx_past_user_msg = n_used_pre_read; // That'll fail the pre-check.
958 }
959 // Some code is duplicated as a result, but by checking this outside loop we avoid repeating `no_hndls` check.
960 else if (no_hndls) // && !low_graceful_close
961 {
962 /* Go through each in-dgram. Break out of loop if and only if a non-vanilla one is found.
963 * If post-loop idx_past_user_msg indicates we were able to get through all in-dgrams,
964 * pre-check result is TRUE, else FALSE.
965 *
966 * Refer to "Possible legal payloads" table above, specifically the legal user in-message rows. */
967 for (idx_past_user_msg = n_used_pre_read; idx_past_user_msg != n_used_post_read; ++idx_past_user_msg)
968 { // Note: We know this loop has 1+ iterations.
969 /* no_hndls allowed; therefore only the following is a vanilla msg:
970 * - 0x0000, >0, none => user in-message (no handle but with meta-blob)
971 * Thus: dgram size must *exceed* MSG_TYPE_SZ; first two bytes must equal 0x0000; and there must be no
972 * native handle. */
973 Msg_resource_impl* msg_resource_impl;
974 const auto n_rcvd = m_batch.result_payload_blob(idx_past_user_msg, &msg_resource_impl);
975 if ((n_rcvd <= MSG_TYPE_SZ) || (*msg_resource_impl->m_msg_type != TYPE0)
976 || (!m_batch.result_payload_hndl(idx_past_user_msg).null()))
977 {
978 break;
979 }
980 // else: Record user message in histogram of user bytes per message, so we don't have to loop yet again.
981 if (rcv_stats) { rcv_stats->m_histo_payload_sz.record_value(n_rcvd - MSG_TYPE_SZ); }
982 }
983 }
984 else // if (!no_hndls) // && !low_graceful_close
985 {
986 // Same cmnts as in the other branch except where noted.
987 for (idx_past_user_msg = n_used_pre_read; idx_past_user_msg != n_used_post_read; ++idx_past_user_msg)
988 {
989 /* !no_hndls (handles allowed); therefore only the following are vanilla msgs:
990 * - 0x0000, 0, hndl => user in-message (with handle but no meta-blob)
991 * - 0x0000, >0, none => user in-message (no handle but with meta-blob)
992 * - 0x0000, >0, hndl => user in-message (with handle and meta-blob)
993 * Note that if first two bytes were indeed received and equal 0x0000, then the only in-dgram that
994 * is *not* vanilla is:
995 * - 0x0000, 0, none => graceful-close
996 * Thus: dgram size must >= MSG_TYPE_SZ; first two bytes must equal 0x0000; and either
997 * dgram size *exceeds* MSG_TYPE_SZ (likeliest), or there's a handle, or both. */
998 Msg_resource_impl* msg_resource_impl;
999 const auto n_rcvd = m_batch.result_payload_blob(idx_past_user_msg, &msg_resource_impl);
1000
1001 /* Need this either for ++rcv_stats->m_msgs_with_hndls or for the pre-check or both. Perf-wise:
1002 * - If pre-check passes: Fast-path; definitely need the flag (unless !rcv_stats but in practice
1003 * it's gonna be non-null as of this writing).
1004 * - If it (eventually) fails: Slow-path; so doesn't matter if we needlessly grab this value. */
1005 const bool got_hndl = !m_batch.result_payload_hndl(idx_past_user_msg).null();
1006
1007 if ((n_rcvd < MSG_TYPE_SZ) || (*msg_resource_impl->m_msg_type != TYPE0)
1008 || ((n_rcvd == MSG_TYPE_SZ) && (!got_hndl)))
1009 {
1010 break;
1011 }
1012 // else:
1013 if (rcv_stats)
1014 {
1015 rcv_stats->m_histo_payload_sz.record_value(n_rcvd - MSG_TYPE_SZ);
1016 got_hndl && (++rcv_stats->m_msgs_with_hndls);
1017 }
1018 }
1019 }
1020 if (idx_past_user_msg == n_used_post_read) // Pre-check passed -- we can skip much stuff and (don't even log)....
1021 {
1022 m_histo_raw_read_n_msgs_filtered_out.record_value(0);
1023
1024 // Stats: all dgrams are user messages; 0 filtered out. iter_low_lvl_bytes is the total from m_batch.nb_read().
1025 if (rcv_stats)
1026 {
1027 const auto n_user_msgs = n_used_post_read - n_used_pre_read;
1028 rcv_stats->m_total_msgs += n_user_msgs;
1029 rcv_stats->m_total_low_lvl_bytes += iter_low_lvl_bytes;
1030 rcv_stats->m_total_bytes += (iter_low_lvl_bytes - (MSG_TYPE_SZ * n_user_msgs));
1031 // rcv_stats->m_histo_payload_sz and ->m_msgs_with_hndls are already good to go.
1032 }
1033
1034#if 0
1035 got_a_user_msg = true; // Got 1+ user-message.
1036 continue;
1037#else
1038 /* `continue` is correct, but we can optimize further actually:
1039 * The loop-ending condition is: implied_would_block || *err_code || full(). *err_code is false, so it's:
1040 * `implied_would_block || full()`. However, we've just set: `implied_would_block = !full()`.
1041 * So it's `true || false` or `false || true`; i.e., true; i.e., loop ends. So we can `break` instead of
1042 * `continue`.
1043 *
1044 * Sanity check: How can it even be not true then? Answer: In the non-vanilla case (which we've eliminated),
1045 * full() would be true pre-prune; but 1+ auto-pings being present would make it become false (freeing up
1046 * some space for another iteration). In our case, though, such complex eventualities are not possible.
1047 *
1048 * Lastly: we can also skip `got_a_user_msg = true`, because it would only be checked/used in a subsequent
1049 * iteration; but that will not happen. */
1050 break;
1051#endif
1052 }
1053 /* else if (at least 1 non-vanilla in-dgram, and/or low_graceful_close): Do the full algorithm.
1054 *
1055 * There is also an annoying corner case here regarding rcv_stats->m_histo_payload_sz and ->m_msgs_with_hndls:
1056 * We recorded all (0+) user in-messages in the pre-check loop above, opportunistically. There shall be 0+ more
1057 * user in-messages past that point however. So we need to (1) count those but (2) not count the ones
1058 * already counted. The part that makes this less stressful is that in this slow-path we're much less
1059 * concerned about saving every cycle. E.g., another loop through part of the range = no big deal.
1060 * Look for that below. */
1061
1062 if (do_log_trace)
1063 {
1064 FLOW_LOG_TRACE_WITHOUT_CHECKING
1065 ("Native_socket_stream_batch [" << *this << "]: "
1066 "Native_socket_stream batch iteration [" << n_iters << "] (1-based): raw nb-read yielded 1+ in-dgrams; "
1067 "heuristic-optimization pre-check detected low-graceful-close "
1068 "and/or at least 1 non-vanilla in-dgram; therefore we must execute "
1069 "the full post-processing algorithm (that looks for auto-pings, graceful-close x 2, and illegal msgs); "
1070 "proceeding. This should NOT occur under load, with the exception of a potential 1-time graceful-close.");
1071 }
1072
1073 // <-- Heuristic check END
1074
1075 m_batch.reuse_result_payloads(n_used_pre_read, // @todo Can we scan from idx_past_user_msg? Probably.
1076 [this, logger_ptr, no_hndls](size_t idx) -> bool
1077 { return is_not_user_message(logger_ptr, idx, no_hndls); });
1078
1079 const auto n_used_post_prune = m_batch.n_used();
1080 /* Non-empty range [n_used_pre_read, n_used_post_read) has been split into:
1081 * Ruser = [n_used_pre_read, n_used_post_prune) and Rother = [n_used_post_prune, n_used_post_read). */
1082 assert((n_used_post_prune <= n_used_post_read)
1083 && "Expected .reuse_result_payloads() to lower or not-touch .n_used().");
1084 assert((n_used_pre_read <= n_used_post_prune)
1085 && "Expected .reuse_result_payloads() to operate only on range [n_used_pre_read, ...).");
1086
1087 m_histo_raw_read_n_msgs_filtered_out.record_value(n_used_post_read - n_used_post_prune);
1088
1089 if (n_used_pre_read != n_used_post_prune) // I.e., pre_read<post_prune.
1090 {
1091 got_a_user_msg = true; // Got 1+ user-message (Ruser is not empty).
1092 }
1093
1094 // Post-process the pruned-away a/k/a swapped-away guys if any (Rother). See large-ish comment just above.
1095 size_t rother_auto_pings = 0; // Stats: count auto-pings in Rother.
1096 for (size_t idx = n_used_post_prune; (!*err_code) && (idx != n_used_post_read); ++idx)
1097 { // May well be no-op loop.
1098 Msg_resource_impl* msg_resource_impl;
1099 const auto n_rcvd = m_batch.result_payload_blob(idx, &msg_resource_impl);
1100 if (n_rcvd < MSG_TYPE_SZ)
1101 {
1102 // *msg_resource_impl->m_msg_type was not fully read => illegal.
1104 cleanup_on_error(this, orig_n_used, n_used_post_read); // Un-emit/un-leak: exceptional error.
1105 FLOW_LOG_WARNING("Native_socket_stream_batch [" << *this << "]: "
1106 "Native_socket_stream batch nb-read post-post-processing scan of slot [" << idx << "]: "
1107 "illegal too-short in-dgram.");
1108 continue;
1109 }
1110 // else
1111
1112 const auto& msg_type = *msg_resource_impl->m_msg_type;
1113 if (msg_type == PING)
1114 {
1115 if ((n_rcvd != MSG_TYPE_SZ) || (!m_batch.result_payload_hndl(idx).null()))
1116 {
1117 // Auto-ping sentinel + any data or hndl => illegal.
1119 cleanup_on_error(this, orig_n_used, n_used_post_read); // Un-emit/un-leak: exceptional error.
1120 FLOW_LOG_WARNING("Native_socket_stream_batch [" << *this << "]: "
1121 "Native_socket_stream batch nb-read post-post-processing scan of slot [" << idx << "]: "
1122 "illegal auto-ping-like in-dgram: "
1123 "contains blob payload sized [" << (n_rcvd - MSG_TYPE_SZ) << "] and/or "
1124 "native-handle [" << m_batch.result_payload_hndl(idx) << "].");
1125 }
1126 else
1127 {
1128 ++rother_auto_pings;
1129 }
1130 continue; // Don't log about ping; is_not_user_message() doing so is plenty.
1131 }
1132 // else if (not PING):
1133
1134 if (msg_type != TYPE0)
1135 {
1136 // Unknown msg_type (neither PING nor 0000) => illegal.
1138 cleanup_on_error(this, orig_n_used, n_used_post_read); // Un-emit/un-leak: exceptional error.
1139 FLOW_LOG_WARNING("Native_socket_stream_batch [" << *this << "]: "
1140 "Native_socket_stream batch nb-read post-post-processing scan of slot [" << idx << "]: "
1141 "illegal in-dgram with unexpected msg-type value [" << msg_type << "].");
1142 continue;
1143 }
1144 // else if (msg_type == 0000):
1145
1146 /* Refer to the table earlier-up and also recall that in Rother we have no legal messages (they're in Ruser).
1147 * Hence it's either graceful-close or illegal (due to handle presence).
1148 * - If !no_hndls: The 3/4 possibilities that aren't graceful-close are all legal. So it must be
1149 * graceful-close.
1150 * - Else: The 2/4 illegal cases all are illegal due to having handle. So if no handle, graceful-close.
1151 * Otherwise, illegal due to handle presence. */
1152 if ((!no_hndls) || m_batch.result_payload_hndl(idx).null())
1153 {
1154 *err_code = error::Code::S_RECEIVES_FINISHED_CANNOT_RECEIVE; // Our-protocol (high-level) graceful-close.
1155 // (Keep any in-messages from prev iteration(s). Do not cleanup_on_error().)
1156
1157 /* @todo (Low-priority) As you can see, RECEIVES_FINISHED_CANNOT_RECEIVE is not an exceptional error;
1158 * it's a graceful-close. Yet we drop out of the loop without looking behind it. It's defensible in
1159 * that there's no reason to suspect the other side is buggy; we see the graceful-close and get out;
1160 * everything is Kosher, that we know of anyway. The to-do would be to check the remaining slots: If there
1161 * is something illegal there, cleanup_on_error() and emit *that* error instead of the nice graceful-close
1162 * after all. Otherwise, emit graceful-close (as we do now unconditionally). / We're quite careful to
1163 * check slots for legality already, so in that sense doing so here as well would be consistent. It is
1164 * slightly different here, in that the per-slot checking up to now was necessary if only to test for
1165 * graceful-close -- a totally legal thing that we must check for -- so it's probably easier to do the
1166 * full verification logic rather than try to skip checks in an effort to be more-trusting of the
1167 * other side. (Perf-wise: this whole post-heuristic-check chunk of code, to begin with, only
1168 * executes rarely.) Whereas having reached graceful-close, the rest (in a legal situation) does not
1169 * matter, so at least we're saving the extra logic/code in checking it anyway. / That said: If I'm
1170 * having to defend it at length in English, just freakin' handling it in C++, instead, might be better. */
1171
1172 FLOW_LOG_INFO("Native_socket_stream_batch [" << *this << "]: "
1173 "Native_socket_stream batch nb-read post-post-processing scan of slot [" << idx << "]: "
1174 "got NSS-protocol graceful-close.");
1175 }
1176 else
1177 {
1179 cleanup_on_error(this, orig_n_used, n_used_post_read); // Un-emit/un-leak: exceptional error.
1180 FLOW_LOG_WARNING("Native_socket_stream_batch [" << *this << "]: "
1181 "Native_socket_stream batch nb-read post-post-processing scan of slot [" << idx << "]: "
1182 "illegal in-dgram contains only a native handle "
1183 "[" << m_batch.result_payload_hndl(idx) << "], but handles unexpected in this channel.");
1184 }
1185 } // for (idx in Rother = [n_used_post_prune, n_used_post_read)) [and break if *err_code becomes truthy inside.]
1186
1187 /* Didn't detect exceptional error or higher-level graceful-close; so by the algorithm in the large-ish comment
1188 * above: low-level graceful-close may apply... */
1189 if ((!*err_code) && low_graceful_close)
1190 {
1191 // ...and indeed it does (low_graceful_close == true).
1192 *err_code = boost::asio::error::eof;
1193 // (Keep any in-messages from prev iteration(s). Do not cleanup_on_error(). Same as above.)
1194
1195 /* Quick discussion: Why check low_graceful_close after high-level graceful-close + illegal
1196 * in-messages? Answer: Simply, it would be emitted by opposing side after those out-messages; meaning
1197 * first some write OS-call(s), and only then close OS-call(s). Or another way of thinking about it is
1198 * that in-messages are the user protocol we are trying to speak, while low_graceful_close is the substrate
1199 * protocol; in fact the our-protocol graceful-close is intentionally instituted as a more predictable/civilized
1200 * graceful-close, common to all blob-transport impls including *but not limited to* Native_socket_stream.
1201 * So it's in a sense a fallback. */
1202 }
1203
1204 else if ((!*err_code) && implied_would_block && (!got_a_user_msg)) // && !low_graceful_close
1205 {
1206 if (do_log_trace)
1207 {
1208 FLOW_LOG_TRACE_WITHOUT_CHECKING
1209 ("Native_socket_stream_batch [" << *this << "]: "
1210 "Native_socket_stream batch nb-read: raw nb-read got 0 user in-messages but 1+ auto-pings, "
1211 "with empty slots available (implied would-block); and 0 user in-messages in any preceding "
1212 "iterations also. Since pings are ignorable, this amounts to overall actual would-block; "
1213 "will emit that.");
1214 }
1215 *err_code = boost::asio::error::would_block;
1216 implied_would_block = false;
1217 }
1218 /* else:
1219 * The above logic is a transcription of the wordy algorithm comment above. That said let's contemplate
1220 * the other possibilities, to ensure we're doing the right thing (in doing nothing in this case).
1221 * - If *err_code: It's a high-level graceful-close or illegal message. Of course we should stop and will, when
1222 * *err_code is checked shortly. So the do/while() condition check should proceed normally.
1223 * - If !*err_code but !implied_would_block: The raw nb-read filled all slots, so potentially there are more
1224 * data; loop should continue as long as !full() (which might indeed be the case due to post-processing
1225 * detecting 1+ auto-pings). So the do/while() condition check should proceed normally.
1226 * - If !*err_code and implied_would_block, but got_a_user_msg: Basically, either in this iteration or
1227 * preceding, there was at least 1 user in-message. Loop shall stop due to the implied_would_block; and
1228 * report 1+ user-messages. So the do/while() condition check should proceed normally. */
1229
1230 /* Stats (slow-path/pre-check failed): Rother = control messages; Ruser = user messages.
1231 * Each Rother msg = MSG_TYPE_SZ low-level bytes (header only).
1232 * Ruser low-level bytes = total - Rother low-level bytes.
1233 * On catastrophic error (kinds that un-emit everything from before / no trusted opposing peer behaves this way),
1234 * let's not even touch stats. The calculation would be potentially wrong (can't assume each non-user message
1235 * is MSG_TYPE_SZ bytes), and making it right for something close to being assert()-worthy isn't worth the
1236 * code effort. However, the m_histo_payload_sz and m_msgs_with_hndls counted during the (failed) pre-check
1237 * loop above are already committed even on catastrophic error; that's fine too.
1238 *
1239 * Reminder: When perf matters, the vast majority of the cases this code won't run; so the
1240 * *err_code check (or two) is negligible. */
1241 if (rcv_stats && ((!*err_code)
1243 || (*err_code == boost::asio::error::would_block)
1244 || (*err_code == boost::asio::error::eof)))
1245 {
1246 const auto n_rother = n_used_post_read - n_used_post_prune;
1247 const auto rother_low_lvl_bytes = n_rother * MSG_TYPE_SZ;
1248 const auto n_ruser = n_used_post_prune - n_used_pre_read;
1249
1250 rcv_stats->m_auto_pings += rother_auto_pings;
1251 rcv_stats->m_total_low_lvl_bytes += iter_low_lvl_bytes;
1252 rcv_stats->m_total_msgs += n_ruser;
1253 /* Reminder: m_total_bytes = bytes inside actual user messages, no headers. Low-level bytes in
1254 * user-message-bearing datagrams is the first term (total minus Rother). Then subtract the headers. */
1255 rcv_stats->m_total_bytes += ((iter_low_lvl_bytes - rother_low_lvl_bytes) - (MSG_TYPE_SZ * n_ruser));
1256
1257 /* Per-Ruser-message stats: payload-size histogram and handle counting.
1258 * As foreshadowed before: we must skip the ones we opportunistically counted while doing the
1259 * pre-check loop; and count the rest. We claim: the already-counted ones are exactly the ones now
1260 * sitting at [n_used_pre_read, idx_past_user_msg); so we begin at idx_past_user_msg and go through the
1261 * rest of Ruser. Why is the claim true, even though .reuse_result_payloads() may have moved slots
1262 * around? Answer:
1263 * - The pre-check loop counted the leading run of user messages: slots
1264 * [n_used_pre_read, idx_past_user_msg), in their original positions.
1265 * - .reuse_result_payloads() keeps user messages in their original relative order; moreover it does
1266 * not touch anything before the *first* non-user message. That first non-user message is at
1267 * idx_past_user_msg -- by definition of idx_past_user_msg.
1268 * - Hence the leading run stayed put, position by position. Any user messages the pruning moved
1269 * forward -- the not-yet-counted ones -- landed right after it:
1270 * [idx_past_user_msg, n_used_post_prune). */
1271 for ( ; idx_past_user_msg != n_used_post_prune; ++idx_past_user_msg)
1272 {
1273 rcv_stats->m_histo_payload_sz.record_value(m_batch.result_payload_blob(idx_past_user_msg) - MSG_TYPE_SZ);
1274
1275 (!no_hndls) && (!m_batch.result_payload_hndl(idx_past_user_msg).null())
1276 && (++rcv_stats->m_msgs_with_hndls);
1277 }
1278 } // if (rcv_stats && no catastrophic [non-graceful-close] error)
1279 } // else if (!*err_code) && (n_used_pre_read < n_used_post_read) [But *e_c may have become truthy { inside }.]
1280
1281 /* Refer to the large-ish comment above regarding loop-stop conditions. Basically it comes down to,
1282 * usually we stop; the only case where we don't is that: no error was detected,
1283 * and the actual nb-read yielded all possible slots filled, and the post-processing then freed 1+ slots
1284 * to re-target. Little optimization: implied would-block (i.e., actual nb-read filled not-all-slots) --
1285 * particularly in the first iteration -- is the likeliest stop condition, especially since a batch-receive
1286 * would often be triggered when knowing the socket is readable. Point is check that first. Then a would-block
1287 * is second-likeliest, so check *err_code next. */
1288 }
1289 while ((!implied_would_block) && (!*err_code) && (!full()));
1290
1291 m_histo_usr_read_n_raw_reads.record_value(n_iters);
1292
1293 if (do_log_trace)
1294 {
1295 FLOW_LOG_TRACE_WITHOUT_CHECKING
1296 ("Native_socket_stream_batch [" << *this << "]: "
1297 "Native_socket_stream batch nb-read, after iteration, stopped for one of these reasons: "
1298 "implied would-block? = [" << implied_would_block << "]; "
1299 "error/would-block [" << *err_code << "] [" << err_code->message() << "]; "
1300 "all slots filled? = [" << full() << "].");
1301 }
1302
1303 // As advertised....
1304 (*err_code == boost::asio::error::would_block) && (*not_idle_on_would_block = not_idle_val);
1305
1307 {
1308 constexpr auto STATS_SEV = Native_socket_stream_cfg::S_STATS_LOG_SEV;
1309 if (logger_ptr && logger_ptr->should_log(STATS_SEV, get_log_component()))
1310 {
1311 using boost::chrono::milliseconds;
1312 using boost::chrono::round;
1313 using flow::Fine_clock;
1314 using util::Fine_duration;
1315
1316 const Native_handle peer_hndl{peer_socket_ptr->native_handle()};
1317 const auto now = Fine_clock::now();
1318
1319 if (m_stats_next_output_when.time_since_epoch() == Fine_duration::zero())
1320 {
1321 m_stats_next_output_when = (now + Native_socket_stream_cfg::S_STATS_LOG_PERIOD);
1322 FLOW_LOG_WITHOUT_CHECKING
1323 (STATS_SEV,
1324 "STATS: Native_socket_stream_batch [" << *this << '/' << peer_hndl << "]: "
1325 "First log checkpt; next real one shall be in "
1326 "[" << round<milliseconds>(m_stats_next_output_when - now) << "].");
1327 }
1328 else if (now >= m_stats_next_output_when)
1329 {
1330 const auto since_last_checkpt = round<milliseconds>((now - m_stats_next_output_when)
1332
1333 FLOW_LOG_WITHOUT_CHECKING
1334 (STATS_SEV,
1335 "STATS: Native_socket_stream_batch [" << *this << '/' << peer_hndl << "]: "
1336 "Since last log [" << since_last_checkpt << "]: "
1337 "Histogram: # raw in-dgrams from each raw batch-recv: "
1338 "[" << m_histo_raw_read_n_msgs << "].");
1339 FLOW_LOG_WITHOUT_CHECKING
1340 (STATS_SEV,
1341 "STATS: Native_socket_stream_batch [" << *this << '/' << peer_hndl << "]: "
1342 "Since last log [" << since_last_checkpt << "]: "
1343 "Histogram: # ping/close/bad in-dgrams from each raw batch-recv: "
1344 "[" << m_histo_raw_read_n_msgs_filtered_out << "].");
1345 FLOW_LOG_WITHOUT_CHECKING
1346 (STATS_SEV,
1347 "STATS: Native_socket_stream_batch [" << *this << '/' << peer_hndl << "]: "
1348 "Since last log [" << since_last_checkpt << "]: "
1349 "Histogram: # raw batch-recvs from each higher-lvl batch-recv: "
1350 "[" << m_histo_usr_read_n_raw_reads << "].");
1351
1352 m_stats_next_output_when = (now + Native_socket_stream_cfg::S_STATS_LOG_PERIOD);
1353 }
1354 else if (do_log_trace)
1355 {
1356 FLOW_LOG_TRACE_WITHOUT_CHECKING
1357 ("STATS: Native_socket_stream_batch [" << *this << '/' << peer_hndl << "]: "
1358 "Between checkpts; next one shall be in "
1359 "[" << round<milliseconds>(m_stats_next_output_when - now) << "].");
1360 }
1361 } // if (should_log)
1362 } // if constexpr(Native_socket_stream_cfg::S_STATS_LOG_ENABLED)
1363
1364 return true;
1365} // Native_socket_stream_msg_batch_in::nb_read()
1366
1367template<typename Msg_resource_t>
1369 size_t idx, bool no_hndls) const
1370{
1371 constexpr auto TYPE0 = msg_type_t{0x0000};
1372 constexpr auto MSG_TYPE_SZ = sizeof(msg_type_t);
1373
1374 /* Do not log unless will return `true`. `false` is the fast-path; try to detect it ASAP and GTFO.
1375 *
1376 * That said as of this writing nb_read() has a pre-check that will, whenever calling us could possibly affect
1377 * perf, avoid calling us in the first place. So we can slightly relax including setting up logging and forgoing
1378 * the do_log_trace trick that nb_read() does use (but we don't).
1379 *
1380 * (*That* said this code was originally written before the pre-check existed, so it tries hard to be as tight
1381 * as possible; and there's no reason to undo that; and remaining code and comments are written in that
1382 * perf-jealous spirit.) */
1383 FLOW_LOG_SET_CONTEXT(logger_ptr, Log_component::S_TRANSPORT);
1384
1385 /* No choice but to first ensure we got the entire m_msg_type at least. Pretty much always that'll be the case,
1386 * so pretty much always this won't wastefully set msg_resource_impl (i.e., it'll set it -- then we'll use it). */
1387 Msg_resource_impl* msg_resource_impl;
1388 const auto n_rcvd = const_cast<Native_socket_stream_msg_batch_in*>(this)
1389 ->m_batch.result_payload_blob(idx, &msg_resource_impl);
1390 // (const_cast<> used because we use result_payload_blob() but in an explicitly non-destructive way. @todo Hmmm.)
1391 if (n_rcvd < MSG_TYPE_SZ) // *msg_resource_impl->m_msg_type was not fully read => illegal.
1392 {
1393 FLOW_LOG_WARNING("Native_socket_stream_batch [" << *this << "]: "
1394 "Native_socket_stream batch nb-read post-processing slot [" << idx << "]: "
1395 "illegal in-dgram of only [" << n_rcvd << "] bytes.");
1396 return true;
1397 }
1398 // else: We can safely examine *msg_resource_impl->m_msg_type and the blob's length. [likeliest]
1399
1400 /* Now refer to the table in the comment above (where we outline the possible combinations of payloads).
1401 * We must return false if and only if it is a legal in-message that is in fact a user in-message.
1402 * Anything else => not a regular user message; or just illegal. */
1403
1404 if (*msg_resource_impl->m_msg_type == TYPE0) // [likeliest]
1405 {
1406 // Either user in-message, or graceful-close, or illegal due to handle presence while no_hndls.
1407
1408 if (n_rcvd != MSG_TYPE_SZ) // [likeliest]
1409 {
1410 // Non-empty blob: cannot be graceful close; and is legal if and only if no unexpected handle.
1411
1412 if (m_batch.result_payload_hndl(idx).null() // No handle (even if disallowed) => legal. [likeliest]
1413 || (!no_hndls)) // Got handle but handles allowed => legal.
1414 {
1415 return false;
1416 }
1417 // else if (no_hndls && (got handle)):
1418 FLOW_LOG_WARNING("Native_socket_stream_batch [" << *this << "]: "
1419 "Native_socket_stream batch nb-read post-processing slot [" << idx << "]: "
1420 "illegal in-dgram contains a native handle, but we were instructed that is illegal in this "
1421 "channel.");
1422 return true;
1423 }
1424 /* else: Empty blob. If no handle then graceful-close => true. If yes handle but no_hndls then illegal => true.
1425 * Else, legal user message (with handle but no blob) => false.
1426 * At this point it's pretty much just either/or; can't think of how to get to likeliest answer faster; so
1427 * just check which one it is. In any case the likeliest case was handled above first. */
1428 if ((!no_hndls) && (!m_batch.result_payload_hndl(idx).null()))
1429 {
1430 return false;
1431 }
1432 // else:
1433
1434 FLOW_LOG_TRACE
1435 ("Native_socket_stream_batch [" << *this << "]: "
1436 "Native_socket_stream batch nb-read post-processing slot [" << idx << "]: "
1437 "in-dgram (msg-type 0x0) contains graceful-close, or else it contains only a native handle, but "
1438 "we were instructed handles are illegal in this channel (will emit the proper error code "
1439 "depending).");
1440 return true;
1441 } // if (*m_msg_type == TYPE0) // [likeliest]
1442 // else if (*m_msg_type == PING or illegal value): Either auto-ping or illegal.
1443
1444 FLOW_LOG_TRACE
1445 ("Native_socket_stream_batch [" << *this << "]: "
1446 "Native_socket_stream batch nb-read post-processing slot [" << idx << "]: "
1447 "(msg-type [" << *msg_resource_impl->m_msg_type << "]): "
1448 "either auto-ping or unknown-type or illegal auto-ping-like in-dgram (if 1 of the latter 2: "
1449 "will emit error).");
1450 return true;
1451} // Native_socket_stream_msg_batch_in::is_not_user_message()
1452
1453template<typename Msg_resource_t>
1455{
1456 *os << "Nss[blob_sz[" << target_payload_size() << "] " << m_batch << ']';
1457 // So it looks like "Nss[blob_sz[1024] Lcl[slots-total/rdy/rcvd [...]]@...]" as of this writing.
1458}
1459
1460template<typename Msg_resource_t>
1461const flow::util::stat::Histogram_counter&
1463{
1464 return m_histo_raw_read_n_msgs;
1465}
1466
1467template<typename Msg_resource_t>
1468const flow::util::stat::Histogram_counter&
1470{
1471 return m_histo_raw_read_n_msgs_filtered_out;
1472}
1473
1474template<typename Msg_resource_t>
1475const flow::util::stat::Histogram_counter&
1477{
1478 return m_histo_usr_read_n_raw_reads;
1479}
1480
1481template<typename Msg_resource_t>
1482std::ostream& operator<<(std::ostream& os, const Native_socket_stream_msg_batch_in<Msg_resource_t>& val)
1483{
1484 val.to_ostream(&os);
1485 return os;
1486}
1487
1488} // namespace ipc::transport
Grouping of compile-type config/knobs (such as type aliases, constants) governing some of the behavio...
static constexpr bool S_STATS_LOG_ENABLED
Whether to periodically log various statistics about Native_socket_stream.
static constexpr flow::log::Sev S_STATS_LOG_SEV
If S_STATS_LOG_ENABLED: the log severity for stats output.
static constexpr util::Fine_duration S_STATS_LOG_PERIOD
If S_STATS_LOG_ENABLED: how often to output stats via logs.
uint16_t low_lvl_payload_blob_length_t
The type used to encode the internal meta-blob length/type-indicator for sync_io::Native_socket_strea...
static constexpr low_lvl_payload_blob_length_t S_PING_SENTINEL
For sync_io::Native_socket_stream and derivative(s): Value for the length/type-indicator field in eac...
Msg_batch_in concept implementation for at least sync_io::Native_socket_stream (as taken by APIs Nati...
const flow::util::stat::Histogram_counter & histo_raw_read_n_msgs() const
Stats: Histogram tracking how many raw in-dgrams were received by each raw OS batch-read (recvmmsg() ...
bool initialized() const
Identical to Generic_msg_batch_in.
bool full() const
Identical to Generic_msg_batch_in.
void prepare_target_payload(const Mutable_buffer_sequence &target_blob, Msg_resource &&msg_resource, size_t idx=-1)
Identical to Generic_msg_batch_in.
Msg_resource_t Msg_resource
Alias for template param Msg_resource_t.
bool is_not_user_message(flow::log::Logger *logger_ptr, size_t idx, bool no_hndls) const
nb_read() helper: Checks in-dgram at given index, returning true if and only if the dgram represents ...
bool nb_read(flow::log::Logger *logger_ptr, asio_local_stream_socket::Peer_socket< asio_local_stream_socket::Protocol_pkt_stream > *peer_socket_ptr, Protocol_negotiator *protocol_negotiator, bool no_hndls, bool *not_idle_on_would_block, Error_code *err_code=nullptr, int message_flags=0, stat::Blob_rcv_stats *rcv_stats=nullptr)
The API for the receiver-engine (sync_io::Native_socket_stream) to perform the core batch-receive.
boost::array< util::Blob_mutable, 2 > Mutable_buffer_sequence_impl
Scatter/gather structure subsuming the user's "big" Blob_mutable preceded by our internal-needs tiny ...
const flow::util::stat::Histogram_counter & histo_raw_read_n_msgs_filtered_out() const
Stats: Histogram tracking how many of the in-dgrams counted by histo_raw_read_n_msgs() were then filt...
Native_socket_stream_cfg::low_lvl_payload_blob_length_t msg_type_t
Alias (for more accurate naming in context; brevity) for type of the message-type internal field,...
size_t n_used() const
Identical to Generic_msg_batch_in.
size_t target_payload_size() const
Identical to Generic_msg_batch_in.
Native_handle result_payload_hndl(size_t idx) const
Identical to Generic_msg_batch_in, except idx >= n_used() is undefined behavior (assertion may trip).
void to_ostream(std::ostream *os) const
Identical to Generic_msg_batch_in.
util::Fine_time_pt m_stats_next_output_when
The results in m_histo_raw_read_n_msgs et al shall be logged opportunistically inside nb_read() once ...
const flow::util::stat::Histogram_counter & histo_usr_read_n_raw_reads() const
Stats: Histogram tracking how many OS batch-reads (recvmmsg() in Linux) resulted from each Native_soc...
~Native_socket_stream_msg_batch_in()
Clears all resources – including any Msg_resource currently attached to any receive-slot.
flow::util::stat::Histogram_counter m_histo_raw_read_n_msgs_filtered_out
See histo_raw_read_n_msgs_filtered_out().
Native_socket_stream_msg_batch_in(size_t max_msg_count)
Identical to ctor of Generic_msg_batch_in.
asio_local_stream_socket::Msg_batch_in< Mutable_buffer_sequence_impl, Msg_resource_impl > m_batch
The natively-batched batch containing target buffer(s) from the user plus a bit of added buffer-age o...
util::Blob_mutable Mutable_buffer_sequence
Buffer(s) description type (conforming to boost.asio MutableBufferSequence concept) supported by *thi...
size_t m_target_payload_sz
Determined permanently by the first prepare_target_payload(), this is the max size in bytes of each i...
size_t result_payload_blob(size_t idx, Msg_resource **msg_resource_ptr=nullptr)
Identical to Generic_msg_batch_in, except idx >= n_used() is undefined behavior (assertion may trip).
flow::util::stat::Histogram_counter m_histo_usr_read_n_raw_reads
See histo_usr_read_n_raw_reads().
flow::util::stat::Histogram_counter m_histo_raw_read_n_msgs
See histo_raw_read_n_msgs().
A simple state machine that, assuming the opposing side of a comm pathway uses an equivalent state ma...
int16_t proto_ver_t
Type sufficient to store a protocol version; positive values identify newer versions of a protocol; w...
bool compute_negotiated_proto_ver(proto_ver_t opposing_max_proto_ver, Error_code *err_code=nullptr)
Based on the presumably-just-received-from-opposing-side value of their local_max_proto_ver,...
proto_ver_t negotiated_proto_ver() const
Returns S_VER_UNKNOWN before compute_negotiated_proto_ver(); then either the positive version of the ...
static constexpr proto_ver_t S_VER_UNKNOWN
A proto_ver_t value, namely a negative one, which is a reserved value indicating "unknown version"; i...
Similarly to the transport::Msg_batch_in concept (as taken by API {Blob|Native_handle}_receiver::asyn...
typename Protocol::socket Peer_socket
Short-hand for boost.asio Unix domain peer stream-socket (usually-connected-or-empty guy).
@ S_LOW_LVL_INTERNAL_PROTOCOL_INVALID_HEADER
Unable to receive incoming traffic: Opposing side sent invalid internal per-message header.
@ S_RECEIVES_FINISHED_CANNOT_RECEIVE
Will not receive message: opposing user sent graceful-close via API.
@ S_BLOB_RECEIVER_GOT_NON_BLOB
User protocol-code mismatch: local user expected blob only and no native handle; received at least th...
Flow-IPC module providing transmission of structured messages and/or low-level blobs (and more) betwe...
std::ostream & operator<<(std::ostream &os, const Generic_msg_batch_in< Msg_resource_t, NO_HNDLS > &val)
Prints string representation of the given Generic_msg_batch_in to the given ostream.
Definition: batch.hpp:431
boost::asio::mutable_buffer Blob_mutable
Short-hand for a mutable blob somewhere in memory, stored as exactly a void* and a size_t.
Definition: util_fwd.hpp:137
flow::Fine_time_pt Fine_time_pt
Short-hand for Flow's Fine_time_pt.
Definition: util_fwd.hpp:116
flow::Fine_duration Fine_duration
Short-hand for Flow's Fine_duration.
Definition: util_fwd.hpp:114
flow::Error_code Error_code
Short-hand for flow::Error_code which is very common.
Definition: common.hpp:313
The Msg_resource type attached to each slot of our internal m_batch.
Msg_resource m_msg_resource
Per-slot resource moved from user's object via prepare_target_payload().
boost::movelib::unique_ptr< msg_type_t > m_msg_type
This slot's potential/actual in-dgram's message-type field, where the actual OS receive-batch call sh...
This is the friend facade of Native_socket_stream_msg_batch_in that exposes private APIs hidden from ...
Cumulative stats from the incoming-direction (receive) pipe of a Blob_receiver or Native_handle_recei...
uint64_t m_total_bytes
Total user blob bytes received (sum of in-message blob sizes).
uint64_t m_total_msgs
Total user messages fully received and delivered.
uint64_t m_total_low_lvl_bytes
Total low-level bytes received (user blob bytes plus protocol headers and framing).
uint64_t m_msgs_with_hndls
Number of messages received that carried at least one native handle. Currently at most 1 handle per m...
uint64_t m_auto_pings
Number of auto-pings received from the opposing sender.
flow::util::stat::Histogram_counter m_histo_payload_sz
Payload-size histogram for received user messages.
A monolayer-thin wrapper around a native handle, a/k/a descriptor a/k/a FD.
constexpr bool null() const noexcept
Returns true if and only if m_native_handle equals S_NULL_HANDLE.
void close() noexcept
Little utility that returns m_native_handle to the OS.