Performance of Google’s Protocol Buffers in Flex

Update: I ran tests again using newer version of protoc-gen-as3. They can be found here.

For last couple of days I have been investigating Google’s Protocol Buffers with intention to use it in Flex application. While in Flex world such idea instantly raises at least a couple of questions, let me explain our situation a little bit more.

We are looking into using Kaazing, which is quite low level and data format agnostic solution, to provide us with streaming capability. This means we have a choice of format we want to use for our messages (and streaming protocol, but that’s a topic for another post). We have never really considered XML because of message size, performance and Flash Player bugs (apparently fixed in version 10.1). Using AMF3 would give us a lot of developer friendliness and client-side performance, thanks to native FP support, but in our case would also add additional (and probably not insignificant) load on the server. As Protocol Buffers are already used to communicate between deeper layers of backend, we decided we could try to push PB messages up to the client, without having to translate them into another format. As that solution would also tick a couple of boxes on our nice-to-have list, we decided to give it a go.

During my investigation I concentrated on performance as this is very important factor for us. Our application will need to receive more than hundred messages per second and update view with minimal latency. As outbound traffic will not ever be as high, I was mostly interested in deserialization performance. To have some point of reference, I decided to test Protocol Buffers against AMF3, which in Flex world is a synonym for “as fast as you can get”. I used protoc-gen-as3 project, which seems to be most complete and is continuously maintained by it’s author.

Test application ran in total 150 tests per each format, 50 per each of three test classes. All tests classes have different size, so trend could be observed. Following listing shows definition for test class I refer to as Small in results section below. In terms of number of fields Medium is twice the size and Large thrice.

public class InstrumentPriceSmall
    public var numeric1 : int;
    public var numeric2 : int;
    public var text1 : String;
    public var text2 : String;
    public var date1 : ValueDate;
    public var date2 : ValueDate;

public class ValueDate
    public var year : int;
    public var month : int;
    public var day : int;

During each test an object was created, filled with data, serialized 10 000 times and then deserialized 10 000 times. Test application was compiled in release mode so that Protocol Buffers serialization and deserialization logic, which is completely coded in Action Script, could run with maximum efficiency. To minimize impact of garbage collector, System.gc() was called before running each test.

In terms of over the wire size, PB is quite efficient for simple messages. However it’s advantage over AMF3 would quickly go away for objects that hold multiple references to the same value. This is fortunately not relevant for our use case, but could be for other scenarios.

AMF 3 Protocol Buffers
Class: Small Medium Large Small Medium Large
198 290 371 31 80 123

On average deserialization times for PB are very close to AMF3. It also seems to scale quite well (although curve is steeper than AMF3). One thing I found surprising was that PB times were more stable (standard deviation smaller by factor of 2 to 6, depending on test object size). This is very desired feature because sometimes (like in our case) not only average timing is important, but also how far off best and worse case timings are.

AMF 3 Protocol Buffers
Class: Small Medium Large Small Medium Large
444.92 955.06 1348.38 332.46 977.12 1457.12
Std dev:
18.19 20.88 16.32 3.10 6.77 7.42

Serialization times look much less promising. Protocol Buffers are much slower, even 5 to 10 times depending on test object size.

AMF 3 Protocol Buffers
Class: Small Medium Large Small Medium Large
165.30 235.08 320.66 835.82 2385.44 3619.78
Std dev:
22.22 17.15 16.57 17.89 63.02 81.80

I will keep my summary short. Small message size could mean short over the wire transfer time, deserialization times comparable with AMF3 could mean low processing overhead on the client. No need to translate each message to AMF3 could reduce server load. In our scenario Protocol Buffers could work.

EDIT: There’s one more thing that might be important. AMF3 serialization and deserialization is done by Flash Player on a separate thread while Protocol Buffers messages are processed by AVM on GUI thread. This means that large volumes of data might influence framerate and disrupt animations.

  1. Anil said:

    Interesting comparison, Do you mind sharing the test data used in this test? Also, does this test involve AMF on the server or is this just the client?


    • I’ll soon update the post with test classes that I’ve used. All results are client-side only.

    • Added a bit about test data. Let me know if there’s anything more you want to know.

  2. Atry said:

    Hi, I am author of protoc-gen-as3, serialization performance has been optimized after 0.8.6, I think now it would much faster than AMF3.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: