Primitive Message Types: Google Protobuf

Below are the primitive Message Types and their meaning, as used in Google Protobuf.

S No.	Type	Java Mapping	Usage
1	double	double
2	float	float
3	int32	int	Variable length. Not good for negatives
4	int64	long	Variable length. Not good for negatives
5	uint32	int	Variable length
6	uint64	long	Variable length
7	sint32	int	Variable length. Better for Negatives
8	sint64	long	Variable length. Better for Negatives
9	fixed32	int	4 bytes
10	fixed64	long	8 bytes
11	sfixed32	int	4 bytes
12	sfixed64	long	8 bytes
13	bool	boolean
14	string	String
15	bytes	ByteString	Variable length

Google Protobuf: proto3 enhancements

Proto3 is the new version of Google Protobuf with various enhancements and additions. Let's go over them one by one.

1. Since all fields are optional by default. Hence "optional" keyword has been done away with. If nothing is specified, it will be considered optional. (Protof files just became a tad smaller)

2. Support to group fields are not longer there. Group was just another way of nesting information in messages. But it was inefficient way to do (better way, Nested Messages). Hence is no longer supported.

3. Packed repeated fields were introduced in 2.1.0. Repeated fields needed to be explicitly set packed=true to use this encoding. In proto3, repeated primitive fields have been made packed by default. They have to be set false to be disabled. 

4. Common proto types (timestamp.proto etc) have been added. They can be used after importing. This takes a step towards usable library in protobuf. See below,



5. reserverd keyword introduced. By this, user has the control over reserved field_names and field_numbers. Once declared as reserved, these can't be used anywhere in the message.

Below, the field_numbers and names can't be used anywhere in the message again. Gives us more control.


6. Support for Objective-C and C# introduced in proto3. yay!

7. There are other language specific changes for various languages.

Google Protobuf: Introduction

Google Protobuf has simplified a historically cumbersome process of sending messages from one component of an application to another.

They have made the communication, data storage, platform independent. An object serialized in Java can easily be de-serialized in C++.

Google protobuf aids in managing structured data in a better way. Definitions are easily extensible and simple to use.

.Proto File
Data Definition is created once in a simple format. It is stored as .proto file. These proto files are much simpler to write and manage than xml files that we have been using till now.

This file is compiled and we get generated source code in whatever programming language we want. We can then populate the object using Builder/Getter-Setter, Serialize it and send/store. Later de-serialize it in a different language (or same) and retrieve the information.

 An example below



This .proto can now be compiled and language specific Data Access Classes are created. These classes have auto-generated getter/setter functions for members.

We can declare package name at the top of protofiles, dependency on other protofiles, just like any Java code. Compiler takes care of it all.

More about Google Protobuf below;

Google Protobuf: Historical Significance

With the advent of distributed applications, the messaging between one component to another has become all too prominent. There are a lot of middleware mechanisms to do so. Tibco EMS, ZMQ, etc. I have worked on projects involving a few of such mechanisms. All have their own positives and negatives depending on the problem at hand.

While communicating between components, we need data units to be sent to and fro. What those data units will be? You might have used xml units send across. Remember how cumbersome was to write parsing code at both places. And if the components are written in different languages (Java and C++ say), then it becomes all too messy. Writing receivers at both ends and parsing logic too. Handling exceptions and what not. 

Some might have used json. Programmers of the old might remember Serialization/De-serialization and sending Java as a whole. But what to do when other component is not in Java? Again it becomes messy.

Well Google Protobuf's is one such endeavour to put those issues to an end.

Google provides libraries for various programming languages. A .proto file is created (looks somewhat like json, but not all too much), in which we can define structured data structure of the Object to be move to and fro. The same proto file can be compiled in different languages into class objects. 

These classes can now be used simply, Call setters and populate the object. Send it across. Easy de-serialization at other end, and we get local language specific Object. Call getters and get the data.

Thus a Java Object created Serialized, Sent Across, Received, De-Serialized as C++ Object.

Thus from messy history, this seems like a fresh breeze. Doesn't it?