Remote Repository vs Local Repository GIT

Remote Repository is the Central Location of the code. We can clone remote repo into a local repo and start working on local. This local can be synched from time to time with the remote repo. We can push changes from local to remote.

Local repository is the cloned copy of the remote repo. All Version control functions are applicable to the local repo. We can create branch on a local repo. This branch will not be visible to remote, neither will have the parent in the remote. Local branches can be merged with local if required and local in-turn can push the changes to remote.

Putting changes to remote

This is a multi-step process. First changes are moved to the staging area, which is then committed to local. The local is then pushed to remote. Thus we commit to local and not to remote.  git add and git commit are two commands for that.

Distributed Version Control System

Distributed Version Control System

Unlike CVS/SVN, Distributed Version Control System, does not hold the data at a central location. A central copy is kept but each user has a local copy of his/her own. This copy is an exact replica of the central server with required history.

This saves the time to go to-and-fro to the central server among other things. GIT is a Distributed Version Control System.

What is Version Control?

What is Version Control?

As the name suggests it is a mechanism to control the versions (older/current). Consider it like a database which stores the copy of the copy at various times in life (snapshots). It gives the difference between two snapshots in time. Also it helps to do development in parallel on the same codebase.

To make it simple, VCS (Version Control System) hold copies of the code at various times on life. Consider it just like a photo album which contains your pic at various times in life. You can readily compare two pics and see the change in height/weight/hair etc in two images. Same is VCS.

Benefits of Version Control?

1. Group Work

Imagine a world without VCS. You would be working on a shared excel file. Who writes what and what happens to the data, is unknown even to God. A maintenance nightmare. With VCS these maintenance issues are taken care of. All code, revisions, everything at a central location.

2. Versioning

With VCS we maintain the versions over the time. Old copy of the code can be retrieved at a later date. We can come to know who changed what and can restore the code in case of bad changes.

Java 8: Gone are the days of For Loops: Stream API

Java 8 seems to have skipped a generation. So much changes bundled together and so much to explore.

java.util.stream adds support for functional-style operations on streams of elements. This opens up huge possibilities. A major outcome is that traditional for-loops might not exist as we know them. Gone are their days. So what come as replacement?

There you go! Right question. Already on the mark.

Let's see some examples.



Above piece of code is a traditional for loop, which returns the Opening Batsmen.. Same code can now be written as below without the for loop.




Nice! Used filter to filter out the Opening Batsmen and return the first one of all.

There are two Opening Batsmen. How to get both of them.

Below is the code both in traditional for-loop and the new Streams



There can be innumerable example of replacing for-loops. Explore them and start using the power of Java 8.

How to execute Java code Inside Comments?

Hi folks,

An interesting observation. It's possible to execute Java code from inside what looks like comment line. You may call it  a Java design flaw or a trick, below is the code piece 

So what's going on there? The trick is the unicode character in front of the comment line. It is parsed by Java compiler as a newline and hence the above code is read a below by the Compiler. 

In Java Unicode decoding occurs earlier than any lexical translation. So the whole code gets converted into a unicode (not exactly, but it's easier to understand this way). This was done for Java code to be encoding free and truly platform independent.

Hence it's possible to write an entire code of Java in unicodes only. Still compiler will understand and run the program.

Java 20 year old and still going strong

Java has turned 20 and it's a time to celebrate. It wouldn't be wrong to say that Java created an entire generation of coders that thrived on this celestial language including the author.

Let's see why Java has stood the test of last 20 years in this fast evolving world.

1. Platform independent: Before the advent of Java, half the time of programmers (C++) went in getting the written code complied and run. No one even dreamt of running the same binary across platforms. I remember having done a project which was to migrate the code from Sun OS to Linux OS. It went on for good six moths given the C++ code was like a big garbage dump, where each day we started our journey and each evening we found ourselves back to square one.

Java changed all this. Same binary now runs across platforms via JVM. 

2. Automated memory allocation and Garbage Collection: Memory allocation/ deallocation/ garbage collection was one big headache of C++ developers. Java changed all that. It made programming fun. I am not sure I would have taken programming but for Java.

3. The trinity of James Gosling, Mike Sheridan, Patrick Naughton who worked on Sun's Green Project are G.O.D (Generator, Observer, Destroyer) of Java world. Kudos! to them.

4. Broke the shackles: C++ always came with the burden if running legacy code and hence was never able to break itself free of cumbersome header files, multiple inheritance, pointers etc. Java broke those chains. Unburdened with the past, it wrote a new history. Free of the ghosts and simple as ever.

5. Remember the applets: The initial push to Java was given by applets. The user interactive applets running within the webpage was like a dream come true at that time. With time however applets lost the race to much better options. But still they had their own charm. They attracted a lot of us to start coding in Java.

6. Daring of the Investment Banks: Java has been a darling of back-end of investment banks. Investment banks are where there is money. Hence Java is in the pool of money and hence a strong backing, To beat this, other languages have to do something out of the box. 

7. JUnit: How simple unit testing of code became with JUnit. It started a revolution of Test Driven Development (TDD). It opened new avenues.

Spring Batch Technical Concepts

The technical aspects of a Spring Batch comprises of the Language and libraries used. Libraries consists of JobLauncher, Job, Step, JobRepository, ItemReader, ItemProcessor, ItemWriter.

A Job is launched by a JobLauncher. A Job consists of multiple steps. Each step has a Reader/Processor/Writer. Metadata about the process is stored on a JobRepository.

Job

Spring Job represents entire batch process. A job contains steps which are executed within a batch. It stands at the top of the table in the spring batch processing. A wrapper/container for steps to be executed one by one or simultaneously. 

Job can be defined in an xml file listing the steps to be executed. Job has a Name, Steps, Restartable (Yes/No). execute(...) method executes the Job.

Job Instance

Spring JobInstance represents one iteration of a Job. It's lifecycle consists of multiple failures and ends with one successful run. A successful Job can't be restarted. A failed Job may be restarted depending on other factors.                                        

Consider an example of a Cricket match. A match starts (JobInstance). Runs are scored, players get injured. And then it rains. Players go off the field (Job paused). Now either the rain subsides and same JonInstance starts again. If it rains a lot then No-Result (JobFailure). In this case we can start a match again next day. Either we continue from the score we left off (Start off the Failed Instance). or we start the match again (Start a new JobInstance).

Only one match (JobInstance) between two given teams should be played a given time.

getJobName() give the name of the Job.

getInstanceId() will give the Id for a given JobInstance.

JobParameters

JobParameters  are associated with a given JobInstance. They are the run time parameters passed to a Job. They identify an Job Instance uniquely. They hold the reference data required to run a Job.

In the above cricket match example. A match that starts today starts on day sat 'T'. If it restarts the next day, then it starts on day 'T+1'. 

These Parameters need to be persisted to be able to referenced again for restart etc. 

JobParameters is immutable/thread-safe class. It overrides hascode and equals method to be able to compare between two JobParameters.

JobExecution

JobExecutions contains what exactly happened during a run. For a give JobInstance, if it is run multiple times (with same JobParameters), a unique JobExecution takes placed. Thus JobInstance and JobExecution has one-many relationship.

JobExecution contains information such as StartTime, EndTime, Status, JobInstanceId etc. JobExecution can be stopped by stop().

A typical scenario looks like below,

S No.	JobExecutionId	JobInstanceId	StartTime	EndTime	Status
1	101	12	X	X+1	FAILED
2	102	12	X+2	X+3	COMPLETED
3	103	12	X+4	X+5	COMPLETED

Step

Spring Batch Step contains the configuration of a Step in Batch Processing. When a complete Job is to play a match. First Step can be Toss, 2nd Step can be the Powerplay overs and so on.

Job vs Step

Just like a Job, execute() starts a Step. You can consider it just a baby-step. In an application, first step can be to read the data from a database. Complex steps can be to perform computations on the data and process it. Simple ItemReader/ItemWriter/ItemProcessor to application specific functions are preformed in a Step.

StepExecution

StepExecution is just like JobExecution, but for a Step. Every time a Step is run, a new StepExecution is created. However it contains more details than a JobExecution as proceeding steps depend on this step (unlike JobExecution, when it comprises of entire Job). So it contains transactional data (commit/rollback etc).

ExecutionContext

ExecutionContext consists of the data which is persisted and is required by StepExecution/JobExecution for example, information required to restart.

In case of a failure, we need the state of the last step (or some preceding step). This is stored in format. Both key and values are user defined.

We can store the result of a particular step. For example as below

Consider Execution Context like a generic HashMap. However we have to be careful how much we persist, as it can bubble up Program Memory.

JobRepository

JobRepository is for persisting everything we discussed till now. At the starting, JobExecution is read from JobRepository and intermittently Step/JobExecutions are persisted to JobRepository. JobRepo is omnipresent across the lifecycle of Batch Execution. What is persistence infra for JobRepo depends on implementation. It can be a flat-file/database etc.

ItemReader/ItemWriter/ItemProcessor

Abstractions provided by Spring batch for reading/writing/processing data required for a Step/output from a Step. 

ItemReader has many implementations in Spring Batch to read from Files/Database/Queues etc. Need not be thread-safe as it just reads without modifying data.

ItemWriter has many implementations in Spring Batch to write to Files/Database/Queues etc. Implementing class needs to take care of serialization of the Objects written. 

ItemProcessor has many implementations in Spring Batch as per requirement.

It is possible to write custom reader/writer/processor as we will see later.

Primitive Message Types: Google Protobuf

Below are the primitive Message Types and their meaning, as used in Google Protobuf.

S No.	Type	Java Mapping	Usage
1	double	double
2	float	float
3	int32	int	Variable length. Not good for negatives
4	int64	long	Variable length. Not good for negatives
5	uint32	int	Variable length
6	uint64	long	Variable length
7	sint32	int	Variable length. Better for Negatives
8	sint64	long	Variable length. Better for Negatives
9	fixed32	int	4 bytes
10	fixed64	long	8 bytes
11	sfixed32	int	4 bytes
12	sfixed64	long	8 bytes
13	bool	boolean
14	string	String
15	bytes	ByteString	Variable length

What drives a Spring Batch application?

Batch can be driven by incoming File trigger, Database update, JMS (or some other) incoming message. 

In an application I worked on file trigger/JMS message used to initiate and drive the batch at different steps.

1. File Trigger: On the basis of input file and instruction thereof, action can be taken. Spring batch initiates and drives on the basis of the file data.

2. Database update: An incoming Database updates defines what the applications needs to do and how the batch will proceed.

3. Message: JMS message for example, read from a Queue can drive the batch.

Standard Building Blocks to Divide the Application using Spring Batch

When using Spring Batch for an application, we should divide the business logic such that we can utilize the following building blocks:

1. Transformation Step: Incoming or outgoing data has to be of a certain format. It has to be transformed as per the need of the application. 

2. Validation Step: Data needs to validated for correctness. File headers/trailers checks, checksums etc. can be applied to data for basic validation.

3. Read/Write: Read the Data from Database/File. Filter the data and write it to the Database/File.

4. Read/Modify/Write: Same as above, just that data can be modified before writing.

The modification of data forms the crux of the application and varies from application to application.