Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

MemoryStream pooling?

Discussion in 'Scripting' started by TwoTen, Jul 25, 2017.

  1. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    Hello!
    I have just changed Networking Library for our project.
    For serialization, we use Protobuf as we found it speedy and compact.
    Our current structure is like this:
    A BaseMessage class which contains meta data such as game version etc and what type of packet it is.
    And it also contains a byte[].
    So when we recieve a message. We first deserialize the message into a BaseMessage. Then we check compatibility etc. And then we check the EventType. And depending on that, we deserialze the byte[] into different objects. If per say the EventType is a InputUpdate. We know that the byte[] is going to be deserialized into x object.

    Same goes for sending, you make your object. Serialize it to a byte[]. Then serialize the BaseMessage.

    So for each packet there is two serializations. And every time we serialize or deserialize we allocate a MemoryStream. And I can not see this being good for the performance. So I am wondering. What's the tips here? Anyone worked alot with MemoryStreams? Or have some fancy MemoryStream pooling system or something of the like.

    Thanks, TwoTen
     
  2. ThomasTrenkwalder

    ThomasTrenkwalder

    Joined:
    Jun 18, 2017
    Posts:
    10
    I simply wrote my own class to write/read stuff (all basic data types: ints, floats, ...) inside byte arrays, and pretty much write the serialization/deserialization code of each message type or whatever by hand.
    The class can be used for both reading/writing at the same time, and can be set to use another byte[] whenever I want. I could even make that class a struct, so if I just need an instance somewhere and don't have one yet, I can just make one without allocating.

    As a result, absolutely nothing in my networking code generates any garbage.

    Writing everything by hand sounds like a lot of work, but I think the wonderful performance, as well as all the control over serialization it gives me, is both really worth it.
    If tons of different message types need to be handled and typing actually becomes too tedious or error-prone, I could still easily write a code generator to do the work for me. Slightly less control but can be made to not impact performance at all.
     
  3. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    But I have to use Memory Streams to serialize and deserialize. That's the problem.
     
  4. ThomasTrenkwalder

    ThomasTrenkwalder

    Joined:
    Jun 18, 2017
    Posts:
    10
    It's been a long time since I worked with MemoryStream, but I quickly went through the doc again. If you absolutely have to use it, there might be a way to make it garbage-free, although probably a bit tedious.

    You could try constructing the MemoryStream with the constructor that allows you to pass your own byte[].
    Then you just create a struct/class to keep track of both the MemoryStream instance and the byte[] you passed to construct it, and create a pool of that. Make sure to use seek methods etc correctly to reset the memory stream though when you retrieve an existing one from the pool.
    That way you can use the MemoryStream to do serialization/deserialization and still have access to the plain byte[] whenever you need it.

    Hope that helps!

    EDIT:
    Note that, iirc, the MemoryStream will allocate a new byte[] whenever you would write over the current arrays bounds, and then uses that instead, so it basically won't work in that case.
    But I'm not sure about this behavior, better check the docs and/or test it properly :)
     
  5. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    Yes I NEED to use MemoryStreams, atleast for Protobuf which is what has been decided to use.
    But all the data is different in each one. So how would your pooling suggestion work?
    Like I never need to reuse the byte[].
    Basically what I want is to not having to reallocate a new byte[] everytime. Seems quite difficult TBH.
     
  6. ThomasTrenkwalder

    ThomasTrenkwalder

    Joined:
    Jun 18, 2017
    Posts:
    10
    Oh I see, I must have misread something.
    If you're stuck with protobuf, it's possible that it actually prevents you from doing clever pooling altogether, depending on how its C# implementation and api is designed ... but I haven't used it myself so I can't give proper advice unfortunately.
     
  7. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    I found this repo which was developed by a MS Employee.
    https://github.com/Microsoft/Microsoft.IO.RecyclableMemoryStream
     
  8. R1PFake

    R1PFake

    Joined:
    Aug 7, 2015
    Posts:
    533
    (I know this thread is old but I found it through google and wanted to post a answer if someone else finds this thread too)

    A MemoryStream has different constructors, one where you can pass your own byte buffer and one where you can pass a capacity and the MemoryStream will manager it's own buffer.

    I pool these by just creating some MemoryStreams with the capacity constructor (the default size depends on your needs) then I just use them for read/write with a BinaryReader/Writer and then store the MemoryStream in a pool (simple stack) to reusem them.

    If you need the actual byte array then you can just use the GetBuffer method (do NOT use the ToArray method, it will create a new array -> garbage)

    This way the MemoryStream will not generate any additional garbage and reuse it's own byte array as long as there is enough space, if it runs out of space it will resize the buffer, yes this will generate garbage but even if you make your own class you will at some point have to handle this "problem" and the only way to deal with this is to somehow generate more space, but if you pass a solid default capacity it shouldn't happen too often and since you will reuse the MemoryStream again it shouldn't be a problem.

    This might be off topic now, but I have seen some people avoid the MemoryStream and BinaryReader/Writer in Unity claiming that the generate so much garbage, can anyone explain why? Am I missing something? Are they not reusing them and generate new instances every time? Or was that a problem in older versions? Because im looking at the current code reference https://referencesource.microsoft.com/ and I don't see any place where they are "wasting" memory or generating additional garbage (aside from the situation where they run out of space) for example the BinaryReader also has it's own buffer which it reuses for the primitive types, so they don't generate any additional garbage if you reuse them.

    The link posted above with the RecyclableMemoryStream will reuse the byte buffers, but not the actual streams, so everytime you want a new stream it generates a new stream instance (which get's a pooled buffer) so it will still generate (some) garbage by creating new stream instances every time.

    I know that the current Unity networking is out-dated but even they have their own buffer with a comment "used instead of MemoryStream and BinaryReader/BinaryWriter to avoid allocations" without any additonal comment why their buffer generates less allocations than a MemoryStream, because their buffer also allocations new arrays if it runs out of space, so it's not really "better" than the MemoryStream, again unless im missing something very obvious?
    https://bitbucket.org/Unity-Technol...Buffer.cs?at=5.3&fileviewer=file-view-default
     
    Last edited: Feb 3, 2019
    BagoDev and alex_roboto like this.