|
Flow-IPC 3.0.0
Flow-IPC project: Public API.
|
boost::asio::ip::tcp::socket. Similarly, async-ops could be made available as C++26 std::execution senders. (The sync_io cores would make either adapter fairly natural to build.) boost::has_trivial_destructor_after_move<ipc::shm::stl::Stateless_allocator> specialization: remove that specialization. As of Boost-1.92 the bug is there. Session / Session_server template parameters into a struct; possibly per-Client_app knobs). Its constants may then become run-time and/or per-session-configurable. Server_session::open_channel() and Client_session::open_channel() both emit error::Code::S_SESSION_OPEN_CHANNEL_SERVER_CANNOT_PROCEED_RESOURCE_UNAVAILABLE when unable to acquire the resources (as of this writing likely MQ-related ones) underpinning the would-be channel; but server-side the actual original Error_code could be emitted instead. (Client-side, due to the IPC-mechanics involved, this is less straightforward, as then: (1) it would be ambiguous as to whether it was the server-side-user-yanked-channel-immediately scenario (user's choice, not an environmental problem), or the current-RESOURCE_UNAVAILABLE scenario; and (2) transmitting an Error_code over IPC is a potentially subtle impl task. Hence the to-do is probably aimed at Server_session::open_channel() et al, not the client-side method family.) flow::async::Task_asio_err for shm_channel_error_handler. There is zero functional difference, as internally it is stored as a Task_asio_err (which a Function<>) anyway; but for consistency and maintainability it'd be nice. public (accessible by said internals via a facade/attorney/etc. pattern of choice). A careful effort is required to clean up the API and then classify what's left as either public-facing or otherwise. Suggest also converting currently-used protected-for-unit-tests pattern to the facade/attorney/etc. pattern of choice (if necessary a separate facade/whatever could be used for internal-code consumers versus test-code consumers). E.g., start() is such a test-accessed protected-only-for-that-reason item. / Lastly: the public super-class Owner_shm_pool_collection, and its super-class(es), likely expose some public things that too should be gated-off from end-users. Ipc_arena::create() would be nice to emit a specific fitting error, as opposed to the somewhat ambiguous resource_unavailable_try_again-or-similar, when pool_name_base is malformed. The message could even suggest Shared_name::sanitize() and/or Shared_name::sanitized().
Ipc_arena::create() should follow normalized Flow error emission semantics (take Error_code*, set to success or specific failure, if null then throw on failure, encapsulating the truthy Error_code).
std::abort() if the assertion-trip attempt did not do it. Arena::Pointer shall be a fancy-pointer, but we could support raw pointers also. Suppose Arena is set up in such a way as to map all processes' locally-dereferenceable pointers to the same SHM location to the same numeric value (by specifying each pool's start as some predetermined numerical value in the huge 64-bit vaddr space) in all processes sharing that SHM pool. Now no address translation is needed, and Arena::Pointer could be simply T*. As of this writing some inner impl details assume it being a fancy-pointer, and it's the only active need we have; but that could be tweaked with a bit of effort. asio_local_stream_socket additional feature: APIs that can read and write native sockets together with accompanying binary blobs can be extended to handle an arbitrary number of native handles (per call) as opposed to only 0 or 1. The main difficulty here is designing a convenient and stylish, yet performant, API. Having achieved this we could also similarly extend the Native_handle_sender, Native_handle_receiver concepts and their impls.
Analogously to how Msg_batch_in supports batch-receipt, we could add API(s) for batch-send. Having achieved this we could also similarly extend the Blob_sender, Native_handle_sender concepts and their impls. Without getting into details: sending and receiving, especially in batches, are not really mirror images of each other conceptually. Hence lacking one but not the other is not necessarily an omission due to lack of time; the need for it is not as clear (we feel); and the API(s) would probably be designed fairly differently.
Comparing Blob_sender::send_blob_max_size() (and similar checks in that family of concepts) to test whether the object is in PEER state is easy enough, but perhaps we can have a utility that would more expressively describe this check: in_peer_state() free function or something? It can still use the same technique internally.
In C++20, if/when we upgrade to that, Native_handle_sender (and other such doc-only classes) can become an actual concept formally implemented by class(es) that, today, implement it via the "honor system." Currently it is a class #ifdef-ed out from actual compilation but still participating in doc generation. Note that Doxygen (current version as of this writing: 1.9.3) claims to support doc generation from formal C++20 concepts.
Channel-opening capabilities – the user must come up with their own naming scheme that avoids name clashes; we could supply an ipc::session-facilitated system for providing this service instead. I.e., ipc::session could either expose a facility for generating the Shared_name absolute_name arg to the Native_socket_stream_acceptor ctor (and opposing Native_socket_stream::sync_connect() call). Alternatively it could provide some kind of Native_socket_stream_acceptor factory and corresponding opposing facility. Long story short, within the ipc::session way of life literally only one acceptor exists, and it is set up (and named) internally to ipc::session. We could provide a way to facilitate the creation of more acceptors if desired by helping to choose their Shared_names. (An original "paper" design did specify a naming scheme for this.) false; otherwise it always represents a message-type only. We have kept the existing name arguably form legacy inertia and/or because a better name was not immediately forthcoming. unique_ptr. For example, imagine a Session::Structured_channel Session::structured_channel_upgrade(Owned_channel_t&& channel, ...) that constructs a suitably-typed struc::Channel, subsuming the raw channel just opened in that session::Session, and returns that guy. Currently it would need to return unique_ptr<Session::Structured_channel> or something. struc::Channel::sync_end_sending(), possibly with timeout, since async_end_sending() to completion is recommended (but not required) to execute at EOL of any struc::Channel. The "blocking" part is good because it's convenient to not have to worry about handling completion with async semantics boiler-plate. The "timeout" part is good because it's a best-effort operation, when in a bad/slow environment, and blocking – even during deinit – would be important to avoid. For example, the Session impls' dtors perform this op; we don't want those to block for any significant time if at all avoidable. The reason it's a mere to-do currently is that a bug-free opposing struc::Channel should not let would-block occur for any real length of time; so blocking is presumably unlikely. Nevertheless. struc::Channel::auto_ping() and struc::Channel::idle_timer_run() and, for safety, removing struc::Channel::owned_channel_mutable(). Consider treating a response to a timed-out struc::Channel::sync_request() as an unexpected response: firing the set_unexpected_response_handler() handler locally (perhaps with an added flag indicating expected-but-timed-out) and informing the opposing peer, so that its set_remote_unexpected_response_handler() handler fires. As of this writing such a late response is simply dropped.
Improve ipc::transport::struc::Channel::sync_request(), so that 2 such calls on the same Channel can execute without one having to fully block the other. Note this may be seen as a breaking change of sorts for the purposes of release notes (even though it would presumably be seen as an improvement).
Segment_ptrs element type would ideally be a view into Blob-or-similar instead of actual Basic_blob*, to decouple impl from API more cleanly. One approach might be to develop a flow::util class/template, a peer of the flow::util::Blob hierarchy, that specifically represents a view into a Blob-type-thing (perhaps 1 variant for mutable, 1 variant for immutable), perhaps analogously to how span<uint8_t> + span<const uint8_t> is each a view into vector<uint8_t> (minus the latter's capacity). It would therefore store a pointer, a size, a capacity, and a start-index – representing a main buffer (size() in size), N-byte header/prefix (N = start()), and M-byte postfix (M = capacity() - start() - size()). (A more economical buffer+header version – directly applicable here – might store all of those minus the capacity, meaning there is no postfix area.) Possible names might be Blob_view_const (read-only view into buffer plus header plus postfix), Blob_view_mutable (same plus writable access), Prefixed_byte_span_const (Blob_view_const minus postfix area), Prefixed_byte_span_mutable (Blob_view_mutable minus postfix area). Segment_ptrs would likely be vector<Prefixed_byte_span_mutable>. Blob. The existing one is suitable for the main use-case which is internally by shm::Builder; but Capnp_message_builder is also usable as a capnp::MessageBuilder directly. If a user were to indeed leverage it in that latter capacity, they may want to transmit/store the SHM-handle some other way. n_init_channels_by_srv_req_func arg, it would be a nice improvement to the contract to guarantee issuing the call to that user-function from the KJ-event-loop thread, like the rest of the async action. (E.g., the following async parts execute in the KJ style: promise fulfillment; transport_method_func() call if applicable.) This API change would not be a breaking change. The reason it is not already done is that the Flow-IPC capnp-RPC-support layer treats the non-RPC-aware ipc::session layer (normally used directly by user but not here) as a black box; and the session::Session_server::async_accept() contract – in this particular detail – did not need to be KJ-friendly (or even KJ-aware) in its design, so it was not. (capnp-RPC support did not exist at the time.) So the way session::Session_server invokes n_init_channels_by_srv_req_func() is inverted versus what the KJ flow needs. To implement this improvement, session::Session_server would need an added async hook for the give-me-the-desired-channel-count step, and Context_server::accept() impl would make use of it. capnp::ReaderOptions configurable – analogously to the vanilla capnp::TwoPartyVatNetwork. (Be sure to apply this judiciously to the correct layer of serialization; and document thoughtfully, as the practical perf implications here are different from the vanilla/non-zero-copy use case.) N == 1 to something larger, so that the user can optionally – while still using this global stats-holder – further categorize stats according to their custom needs. Also consider similar steps in such alias(es) in struc::shm::stat. (Note various relevant APIs allow one to simply pass-in a Serializer_stats* (et al) directly – and the user can maintain their own Global_stats or static Stat_set_list and lightning-quickly obtain the aforementioned * therefrom. So flexibility already exists on that front; but here we use a Global_stats for a reason already; can make use of it more fully this way.) One tactical idea is that factories like struc::Channel::heap_fixed_builder_config() can take an optional arg, defaulting to 0, which would cause such a factory to select the Global_stats slot. Slot 0 would continue to, by convention be the default/catch-all one; and the user can designate other slots for their own purposes. ..._receiver in terms of a custom sync_io::..._receiver would cause a build error, unless the writer of the latter hypothetical custom guy added a S_TRANSMIT_NATIVE_HANDLES member. In practice they'd probably know to do so upon seeing the error and the present code; but formally a relevant concept should specify it, so the custom-guy writer would've known to do so from the start. operator>> and operator<< being asymmetrical get one into trouble when using Shared_name with boost.program_options (or flow::cfg which is built on top of it)? Look into it. It may be necessary to make operator<< equal to that of ostream << string after all; though the added niceties of the current << semantics may still at least be available via some explicit accessor. Consider adding a Shared_name_view which would be to Shared_name as String_view is to string; and that could itself be constructed from a String_view (and therefore from const char* [+ length], etc.). The main work would arguably not be the creation of that class itself but rather (1) ensuring proper implicit conversion between the main class and the new one and (2) conversion of the many Flow-IPC APIs that take a const Shared_name& that would (hopefully in essentially non-user-code-breaking ways) instead take Shared_name_view.
It would be good to constexpr-ify Shared_name to the extent possible; it might even make a dent in perf in some contexts. Internally this is annoying until C++20, where std::string itself got constexpr-ified, as we use std::string internally; best left alone until then.
sync_io-pattern use with an old-school reactor-pattern event loop, using poll() and/or epoll_*(). 1.9.4