Tool 3: Framing with Protobuf

The details from the paragraph on framing related with Protobuf are implemented for Protobuf-net in the package slStreamUtilsProtobuf, with contains tools for (de)serialization of a continuous flux of objects in a parallel manner using message framing.

Usage examples:

 

       

using slProto = slStreamUtils.MultiThreadedSerialization.Protobuf;
// ...
private static IEnumerable GetSampleArray()
{
    return Enumerable.Range(0, 10).Select(f => new X() { b1 = f % 2 == 0, i1 = f, l1 = f % 3 });
}
// ...
IEnumerable arr = GetSampleArray();
using var s = File.Create(fileName);
//...

// original implementation
foreach (var obj in arr)
    ProtoBuf.Serializer.SerializeWithLengthPrefix(s, obj, ProtoBuf.PrefixStyle.Base128, 1);

// framed multithreaded implementation
await using var ser = new CollectionSerializerAsync(s, new FIFOWorkerConfig(maxConcurrentTasks: 2));
foreach (var item in arr)
    await ser.SerializeAsync(item);