Feature Request
Add C++ stream deserialization support while preserving existing deserialize(const uint8_t*, size_t) and deserialize(Buffer&) APIs.
Is your feature request related to a problem? Please describe
Current C++ deserialization is buffer-only. Users integrating with gRPC/RPC streams need extra buffering and framing glue before calling Fory.
Design update (based on PR #3307)
Reference: #3307
- API shape
- Keep current deserialize entrypoints unchanged.
- Introduce stream-backed buffer usage via
ForyInputStream + Buffer so existing deserialize(Buffer&) path can consume stream input.
- No mandatory one-shot
deserialize(std::istream&) API in this phase.
- Stream abstraction
- Add
fory::ForyInputStream (cpp/fory/util/stream.h) wrapping std::istream.
ForyInputStreamBuf maintains a growable internal byte window and supports fill_buffer(min_bytes), consume, rewind, reader_index, and size queries.
- Buffer/read integration
- Extend
fory::Buffer with ensure_size(target_size, error) so reads can auto-fill from attached stream when needed.
- Update deserialization read paths (header, string, struct/object reads) to call
ensure_size instead of assuming all bytes are already materialized.
- Keep pure in-memory hot path behavior unchanged when no stream is attached.
- Object-boundary behavior
- Preferred mode is reusing one stream-backed
Buffer across sequential deserializations from the same stream.
- No stream seek rewind requirement in this design; unread prefetched bytes stay in the stream buffer.
- Error handling
- Premature EOF / short read is reported through existing
Result<T, Error> with buffer/out-of-bound or invalid-data style errors.
- Tests
- Add C++ stream tests for:
- tiny-fragmented reads,
- primitive/string/struct roundtrip,
- reference graph behavior,
- sequential multi-object decode from one stream,
- truncated stream error handling.
Describe alternatives you've considered
- Force callers to read full payload into memory first.
- Add only a one-shot stream API with seek rewind.
Additional context
Relevant files (PR #3307):
cpp/fory/util/stream.h
cpp/fory/util/buffer.h
cpp/fory/serialization/serializer.h
cpp/fory/serialization/stream_test.cc
Feature Request
Add C++ stream deserialization support while preserving existing
deserialize(const uint8_t*, size_t)anddeserialize(Buffer&)APIs.Is your feature request related to a problem? Please describe
Current C++ deserialization is buffer-only. Users integrating with gRPC/RPC streams need extra buffering and framing glue before calling Fory.
Design update (based on PR #3307)
Reference: #3307
ForyInputStream+Bufferso existingdeserialize(Buffer&)path can consume stream input.deserialize(std::istream&)API in this phase.fory::ForyInputStream(cpp/fory/util/stream.h) wrappingstd::istream.ForyInputStreamBufmaintains a growable internal byte window and supportsfill_buffer(min_bytes),consume,rewind,reader_index, and size queries.fory::Bufferwithensure_size(target_size, error)so reads can auto-fill from attached stream when needed.ensure_sizeinstead of assuming all bytes are already materialized.Bufferacross sequential deserializations from the same stream.Result<T, Error>with buffer/out-of-bound or invalid-data style errors.Describe alternatives you've considered
Additional context
Relevant files (PR #3307):
cpp/fory/util/stream.hcpp/fory/util/buffer.hcpp/fory/serialization/serializer.hcpp/fory/serialization/stream_test.cc