Protocol Buffers

Protocol Buffers are language-neutral, platform-neutral extensible mechanisms for serializing structured data.

How it looks: (filed number + wiretype)(value) = ( field number << 3 | wiretype)(value) precise view.

There are three terms here:

• Serializing structured data:

Suppose you have an object in Java:

classUser {
intid=101;
Stringname="Anurag";
}

Enter fullscreen mode Exit fullscreen mode

Computers cannot directly send Java objects over a network.

So we convert the object into a format that can be transmitted or stored.

This conversion process is called S*erialization*.

Example:

{
  "id":101,
  "name":"Anurag"
}

Enter fullscreen mode Exit fullscreen mode

Protocol Buffers serialize the same data into a compact binary format instead of JSON.

• Language-neutral:

You define the data structure once in a .proto file:

message User {
  int32 id = 1;
  string name = 2;
}

Enter fullscreen mode Exit fullscreen mode

From this single file, Protocol Buffers can generate classes for:

• Java
• Python
• Go
• C++
• C#
• JavaScript
• Kotlin

So a Java service and a Python service can communicate using the same contract.

One definition → many programming languages.

Protocol buffers binary encoding

Let’s suppose:

message User {
   int32 age = 1;
}

Enter fullscreen mode Exit fullscreen mode

And suppose the value of age that is send is : 25

What protobuf actually sends

Instead of sending

{
    "age":"25"
}

Enter fullscreen mode Exit fullscreen mode

protobuf sends:

08 19

Only 2 bytes!

Let’s understand where they come from.

• Part 1: Field number – It is the just the numbering/ordering of fields that is set in “.proto” file. Like as age is at 1 then the field number is 1.
• Part 2: Wire Type – A wire type tells protobuf what kind of data follows.

Data Type Wire Type int32, int64, bool 0 string 2 embedded message 2

For:

int32 age = 1;

Enter fullscreen mode Exit fullscreen mode

wire type =

0

Enter fullscreen mode Exit fullscreen mode

because it’s an integer.

• Part 3: Combine Field Number + Wire Type

Protobuf stores them together:

Formula:

(field_number << 3) | wire_type

Enter fullscreen mode Exit fullscreen mode

For our example:

field_number = 1
wire_type = 0

1 << 3 | 0 = ( 1 * 2 ^ 3 ) | 0 = 8 | 0 = 1000 | 0 = 1000 = 8
So, the decimal we get is 8 we need to get it Hexadecimal representation: 08.

Enter fullscreen mode Exit fullscreen mode

• Part 4: Encode Value

Value:

25

Enter fullscreen mode Exit fullscreen mode

25 in hexadecimal:

19

Enter fullscreen mode Exit fullscreen mode

So protobuf stores:

19

Enter fullscreen mode Exit fullscreen mode

Final Payload

Combine both parts:

08 19

Enter fullscreen mode Exit fullscreen mode

Interpretation:

08 -> Field #1, Integer
19 -> Value 25

Enter fullscreen mode Exit fullscreen mode