Quantcast
Channel: Learning the code way
Viewing all 231 articles
Browse latest View live

Using query language with MongoDB

$
0
0
In the previous posts we have seen how to insert and read data from the MongoDB client and from the Java API. Now to fetch documents by executing queries on the collection.
We saw the find method that can be used to fetch records. The find method can be passed with query like parameters. At its simplest:
Here we have passed a parameter to the find method. The parameter - "{}" resembles an EMPTY JSON object. It is equivalent to the no parameter option - return all records in the collection. This parameter is known as the query document.
To query on specific fields, we can pass fields to the query document:
 Here we have queried on the name field by passing the query document {"name":"apple"}.
However what if we want to fetch multiple fruits? Or rather how to specify or conditions on the database ?
db.fruits.find( { name: { $in: [ 'mango', 'apple' ] } } )
In the above code we have used in operator to indicate an in condition - similar to the in keyword we use in SQL. In a normal SQL application we would have used the or condition.
There is a similar or operator available in MongoDB:
From the MongoDB Docs:
Syntax: { $or: [ { <expression1> }, { <expression2> }, ... , { <expressionN> } ] }
The $or operator performs a logical OR operation on an array of two or more
<expressions> and selects the documents that satisfy at least one of the <expressions>.
The website also recommends when to use the operator:
use the $in operator rather than the $or operator when performing equality checks on 
the same field.
There is also a not in operator:
db.fruits.find( {name: {$nin: ['mango','apple']}} )
The result would be a complement of the records returned by our earlier in based query.
Specifying and conditions is easy in the query document. We simply keep adding more attributes:


Tinkering with blogger - Adding a scrolling headline

$
0
0
OK first and foremost - This is not related to Java. And second - This was all done with the purpose of getting an internship (yes selfish and self serving reasons for this post people).
Now that I have all disclaimers out of the way, let me start the story. I have always wanted a way to post some quick notes on my blog. Like a mini- feed. The way news sites do. (Nothing pompous, don't worry). I looked over the blogger apps and never found anything suitable. So I just let it be.
But now that I am looking for an internship, I felt the need to have that information out here. And also the resume. Like an advertisement for myself :). But how to do the same.
A quick look around the net acquainted me with the marquee tag. (Should be re-acquainted, I remember playing with scrolling message when I first learned HTML.)  The marquee tag would allow me to display a simple message - something like:
<marquee behavior="scroll" direction="left">
My message
<marquee>
But how to put this in my page ?This is where the Blogger provides help. I do not need to go and edit every post to put my scrolling message. I need to just edit my template.

I built the HTML code I needed and decided to add it in. I needed to also provide my resume. So I uploaded it to my Google drive as a public document.
The next step is to identify a suitable place in the template. I decided I wanted to have my headline at the head of the page (nothing innovative there). So a quick search enabled me to locate the right section in the template and I added my code:
Got good help from this blog. Save the template and view - The Headline has been added !

More querying in MongoDB

$
0
0
In the last post we started running queries on MongoDB. To continue on the operators, I decided to do a find on the auto generated id:
> db.fruits.find({ "_id" : ObjectId("52d49575e3a91584ad8f614e")})
{ "_id" : ObjectId("52d49575e3a91584ad8f614e"), "name" : "mango", "color" : "yel
low", "rating" : 2, "weight" : 1.03 }
There are also comparison operators available for use in queries:
> db.fruits.find( { rating: { $gt: 0 } } )
{ "_id" : ObjectId("52d305ab4e386a0b85ac249a"), "name" : "apple", "color" : "red
", "rating" : 4, "weight" : 2.3 }
{ "_id" : ObjectId("52d49575e3a91584ad8f614e"), "name" : "mango", "color" : "yel
low", "rating" : 2, "weight" : 1.03 }
Other operators are $gte,$lt and $lte. Similar to the $in operator, we can also try the $nin operator:
> db.fruits.find( { rating: { $nin: [1,2,3] } } )
{ "_id" : ObjectId("52d305ab4e386a0b85ac249a"), "name" : "apple", "color" : "red
", "rating" : 4, "weight" : 2.3 }
{ "_id" : ObjectId("52d30d464e386a0b85ac249b"), "name" : "lost", "age" : 12 }
{ "_id" : ObjectId("52d30d524e386a0b85ac249c"), "name" : "INS Viraat", "wt" : 50
00, "displacement" : 190.87 }
>
This is interesting. The records returned were those that did not match as well as those that did not have the specified attribute.
There is also the not equal operator:
> db.fruits.find( { rating: { $ne: 1 } } )
{ "_id" : ObjectId("52d305ab4e386a0b85ac249a"), "name" : "apple", "color" : "red
", "rating" : 4, "weight" : 2.3 }
{ "_id" : ObjectId("52d30d464e386a0b85ac249b"), "name" : "lost", "age" : 12 }
{ "_id" : ObjectId("52d30d524e386a0b85ac249c"), "name" : "INS Viraat", "wt" : 50
00, "displacement" : 190.87 }
{ "_id" : ObjectId("52d49575e3a91584ad8f614e"), "name" : "mango", "color" : "yel
low", "rating" : 2, "weight" : 1.03 }
>
The ne operator also returns records that do not possess the specified attribute.
We saw the and operator and the or operator in the earlier post. Next is the not operator:
> db.fruits.find( { weight: { $not: { $gt: 1.00}}})
{ "_id" : ObjectId("52d30d464e386a0b85ac249b"), "name" : "lost", "age" : 12 }
{ "_id" : ObjectId("52d30d524e386a0b85ac249c"), "name" : "INS Viraat", "wt" : 50
00, "displacement" : 190.87 }
>
This is an interesting point from the docs:
{ $not: { $gt: 1.99 } } is different from the $lte operator. { $lte: 1.99 } 
returns only the documents where --- field exists and its value is less
than or equal to 1.99.
As we see, the records returned do not even have the weight field.
There is also a nor operator. The below query will fetch documents colors are not red nor their rating a 4.
> db.fruits.find( { $nor: [ { color: 'red' }, { rating: 4 } ]  } )
{ "_id" : ObjectId("52d30d464e386a0b85ac249b"), "name" : "lost", "age" : 12 }
{ "_id" : ObjectId("52d30d524e386a0b85ac249c"), "name" : "INS Viraat", "wt" : 50
00, "displacement" : 190.87 }
{ "_id" : ObjectId("52d49575e3a91584ad8f614e"), "name" : "mango", "color" : "yel
low", "rating" : 2, "weight" : 1.03 }
>
We specified the multiple conditions using an array.

DBCursor - size,length and count

$
0
0
Consider the below method:
publicstatic String DB_NAME = "fruits";
publicstatic String COLLECTION_NAME = "fruits";
publicstatic void main(String[] args) throws UnknownHostException {
Mongo mongoClient = new Mongo();
DB targetDB = mongoClient.getDB(DB_NAME);

DBCollection collection = targetDB.getCollection(COLLECTION_NAME);
DBCursor cursor = collection.find();
System.out.println("Count of records are : " + cursor.count());
System.out.println("Length of records are : " + cursor.length());
cursor.close();

mongoClient.close();
}
The above code opens a cursor to the fruit collection.I ran code to get result of count and length methods of the cursor.
Count of  records are : 4
Length of records are : 4
I created a sample collection and added like a large number of  objects to the collection.
122934447 records.This is big. If I run my read code on this store now:
Count of  records are : 122934447
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.LinkedHashMap.createEntry(
LinkedHashMap.java:442)
at java.util.HashMap.addEntry(
HashMap.java:856)
at java.util.LinkedHashMap.addEntry(
LinkedHashMap.java:427)
at java.util.HashMap.put(
HashMap.java:484)
at org.bson.BasicBSONObject.put(
BasicBSONObject.java:281)
My code crashed for the length function.
It is not that the above two methods are same. Consider the code for the two methods:
public int count() {
if ( _collection == null )
thrownewIllegalArgumentException("why is _collection null");
if ( _collection._db == null )
thrownewIllegalArgumentException("why is _collection._db null");
return(int)_collection.getCount(this._query, this._keysWanted, getReadPreference());
}
and the length method:
public int length() {
_checkType( CursorType.ARRAY );
_fill( Integer.MAX_VALUE );
return _all.size();
}
The count method simply returns a count of the records associated with the cursor. Length method on the other hand does some more things. The method will fetch all records from mondoDB and add it to a java array. So any next call on the cursor will not be a database hit, instead a simple fetch from the java array.
But trying to execute length function on a collection of  122934447 records meant fetching 122934447 records from the database and putting them in a java array. The result - Out of memory.
There is also a size method available in the cursor API. Like length, it counts the number of objects matching the query. The difference with the size method is that  it  takes limit/skip settings into consideration
publicstatic void queryResultCount() throws UnknownHostException {
Mongo mongoClient = new Mongo();
DB targetDB = mongoClient.getDB(DB_NAME);

DBCollection coll = targetDB.getCollection("Players");
BasicDBObject query = new BasicDBObject();
DBCursor cursor = coll.find(query);
cursor.limit(10);
System.out.println("Count of records are : " + cursor.count());
System.out.println("Size of records are : " + cursor.size());
mongoClient.close();
}
In the above code we have applied a limit on the cursor. If we look at the execution result:
Count of  records are : 122934447
Size of records are : 10
As seen above the result of the size method differed from that of the length method.

Indexes and MongoDB

$
0
0
I have never given too  much thought to database indexing. Its a concept that works silently in the background of database tables and views. Most databases provide indexes by default for the primary key and unique keys.
If it comes to performance concerns, a DBA steps in and does his Index magic. As a java developer it wasn't something I had to worry about. So now that I am learning MongoDB at my own sweet pace, I decided to look at indexing.
So what is an index ? That is from where I decided to start. From the SQL Server docs:
An index is an on-disk structure associated with a table or view that speeds retrieval 
of rows from the table or view. An index contains keys built from one or more columns
in the table or view. These keys are stored in a structure (B-tree) that enables SQL
Server to find the row or rows associated with the key values quickly and efficiently.
This is more or less how all databases are using indexes (I think.) I tried to go all diagram on it
For this database I have created a simple table of user. I have also added an index on the firstname property. All this was done via the database IDE. At a higher level.
At the lower (physical) level we are dealing with files and binary data. The index table for firstname (This data is in a file, I am guessing) provides the database with a short cut to reach the records quickly. So if a query involved something like "where firstname like 'John'", the database looks up the index, locates the entry for 'John'. The associated value for 'John' tells the database to look at the record No 3 in the data file. A quick find of the query result. In the absence of the indexes, the database would have to scan all the records in users and located the result.
MongoDB also has indexes. Big data without indexing would be a nightmare ! In fact the _id attribute of the document has an index associated with it by default. We can also create additional indexes to our collection.
Here I first executed the getIndexes method on the collection to get the available indexes.As seen here for the fruits collection only the _id column has an index. I then added an index using the ensureIndex method. The -1 value indicates the sort order to be used. -1 implies descending sort while 1 is for ascending. There are additional settings available for index creation (detailed here.)
By default, creating an index blocks all other operations on a database. When building an index on a 
collection, the database that holds the collection is unavailable for read or write operations until
the index build completes.
Any operation that requires a read or write lock on all databases
(e.g. listDatabases) will wait for the foreground index build to complete.
For potentially long running index building operations, consider the background operation so that the
MongoDB database remains available during the index building operation.
Accordingly I decided to build an index in the background:
>db.Players.ensureIndex( { name: 1}, {background: true,unique:false} )
The unique property if passed as true will treat the property as a unique one. Its false by default.
Background indexing operations run in the background so that other database operations can run 
while creating the index. However, the mongo shell session or connection where you are creating
the index will block until the index build is complete
. To continue issuing commands to the database,
open another connection or mongo instance.
This pretty much renders my cmd screen useless. ( The perfect excuse for me to stop my exploring too :P )
What I have gone over is just the basics on indexing in MongoDB. Its whole power hasn't been explored yet. The above create index/ view indexes operations can also be done in Java:
publicstatic void main(String[] args) throws UnknownHostException {
Mongo mongoClient = new Mongo();
DB targetDB = mongoClient.getDB(DB_NAME);

DBCollection collection = targetDB.getCollection(COLLECTION_NAME);
List<DBObject> indexes = collection.getIndexInfo();
System.out.println("Indexes on " + collection.getFullName() + " are : ");
for (DBObject index : indexes) {
System.out.println(index);
}
}
The output is :
Indexes on fruits.fruits are : 
{ "v" : 1 , "key" : { "_id" : 1} , "ns" : "fruits.fruits" , "name" : "_id_"}
{ "v" : 1 , "key" : { "name" : -1.0} , "ns" : "fruits.fruits" , "name" : "name_-1"}
To create an index is also pretty straight forward.
publicstatic void main(String[] args) throws UnknownHostException {
Mongo mongoClient = new Mongo();
DB targetDB = mongoClient.getDB(DB_NAME);

DBCollection collection = targetDB.getCollection(COLLECTION_NAME);
collection.createIndex(new BasicDBObject("rating", 1).append("unique", true));// create unique index, ascending
List<DBObject> indexes = collection.getIndexInfo();
System.out.println("Indexes on " + collection.getFullName() + " are : ");
for (DBObject index : indexes) {
System.out.println(index);
}

}
If I run this code than:
Indexes on fruits.fruits are : 
{ "v" : 1 , "key" : { "_id" : 1} , "ns" : "fruits.fruits" , "name" : "_id_"}
{ "v" : 1 , "key" : { "name" : -1.0} , "ns" : "fruits.fruits" , "name" : "name_-1"}
{ "v" : 1 , "key" : { "rating" : 1 , "unique" : true} , "ns" : "fruits.fruits" , "name" : "rating_1_unique_"}
As indexing is a means of speeding up query execution, this snippet from the MongoDB docs is something to keep in mind when writing queries:
Some query operations are not selective. These operations cannot use indexes  effectively 
or cannot use indexes at all. The inequality operators $nin and $ne are not very selective,
as they often match a large portion of the index. As a result, in most cases, a $nin or $ne
query with an index may perform no better than a $nin or $ne query that must scan all
documents in a collection. Queries that specify regular expressions, with inline JavaScript
regular expressions or $regex operator expressions, cannot use an index with one exception.
Queries that specify regular expression with anchors at the beginning of a string can use an
index.

Hello Spring Batch

$
0
0
Batch processing is something that we keep hearing in the enterprise industry. Most software projects will at some point require that certain tasks happen in the background on certain scheduled times. Such offline tasks constitute batch jobs. Send reminder emails to clients, sending job alerts to prospective candidates, fetching transactions from financial systems and loading them into analytical systems. Batch always creeps into the picture.
 Two main parts of the Batch system is the scheduler and the batching mechanism. The Scheduler is the code responsible for triggering jobs at appropriate times. The batch mechanism is our code which executes once triggered by the scheduler.
Just like Spring has creeped in and standardized database operations, messaging, localization it has also made its mark in the  world of Batch. While I do not have enough experience or background to explain the history/ geography of the batching world, I do know it won't be long before I encounter a batch system that needs to be implemented. So I thought why not give Spring Batch a dry run.
(BTW for a good story about batch history check this one)
I decided to do a simple batch job that picks up a file from the system and loads it into memory. Not much of a job - but who cares so long as I understand Spring Batch.
The Spring batch world has something called a Job Repository - This is nothing but a collection of all jobs available for running in the System.
<beans:bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager">
</beans:bean>

<beans:bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<beans:property name="transactionManager" ref="transactionManager"/>
</beans:bean>
I have used an instance of SimpleJobRepository here. The thing about Spring batch is that it uses database tables to store information about the batch - BATCH_JOB_EXECUTION_SEQ, BATCH_JOB_SEQ, BATCH_STEP_EXECUTION_SEQ, BATCH_JOB_EXECUTION_CONTEXT just to name a few. These tables are used by SpringBatch to ensure correct execution and for providing a host of features which we shall not concern ourselves with now.
I didn't want any database so I decided to use the MapJobRepositoryFactoryBean which is a JobRepository implementation that holds data in memory - using maps to be precise.
A note from Spring docs :"Note that the in-memory repository is volatile and so does not allow restart between JVM instances. It also cannot guarantee that two job instances with the same parameters are launched simultaneously, and is not suitable for use in a multi-threaded Job, or a locally partitioned Step. So use the database version of the repository wherever you need those features."
To ensure transaction-like behavior they class ResourcelessTransactionManager is used which uses maps to track which resource is bound to which process. The database version of the repository is JobRepositoryFactoryBean. Once we create the repository we need to add jobs to the repository. For this we define the job as below:
<job id="reportJob" job-repository="jobRepository">
<step id="step1">
<tasklet>
<chunk reader="itemReader" writer="itemWriter" processor="personProcessor"
commit-interval="2"/>
</tasklet>
</step>
</job>
The job at its very basic is composed of a series of steps - here we have only one. Our only step is to read the csv file, get the record, process it and then write it to an ArrayList. The input file is as below:
Robin,Varghese
Rohan,Naidu
Roman,Barlan
Spring is made up of beans, beans and then more beans - The data here would be best represented within the Spring world as a Bean. Accordingly I have defined a simple model class:
publicclass Person {
privateString lastName;
privateString firstName;
// setter getters

}
Each row in the file can be represented by a Person instance. Now what we need is a bean that will read the csv file and generate the Person instance for us:
<beans:bean id="fieldSetMapper"
class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<beans:property name="targetType" value="org.robin.learn.sb.data.Person"/>
</beans:bean>

<beans:bean id="lineTokenizer"
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<beans:property name="names" value="firstName,lastName"/>
</beans:bean>

<beans:bean id="lineMapper"
class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<beans:property name="lineTokenizer" ref="lineTokenizer"/>
<beans:property name="fieldSetMapper" ref="fieldSetMapper"/>
</beans:bean>

<beans:bean id="itemReader"
class="org.springframework.batch.item.file.FlatFileItemReader">
<beans:property name="resource" value="classpath:sample-data.csv"/>
<beans:property name="lineMapper" ref="lineMapper"/>
</beans:bean>
Here we have implemented the ItemReader class From the Spring site:
"Strategy interface for providing the data. Implementations are expected to be 
stateful and will be called multiple times for each batch, with each call
to read() returning a different value and finally returning null when all
input data is exhausted. Implementations need not be thread-safe and clients
of a ItemReader need to be aware that this is the case. "
The ItemReader is composed of three main parts:
  • a resource which is the input data to process 
  • a lineTokenizer which will split the input line into separate tokens and associate them with keys - thus creating a set of key value pairs. 
  • a FieldSetMapper which will instantiate the target bean (Person) and set the properties from the key value pairs generated in the previous step. 
Once the data has been read, we need to process it - i.e. deal with the Person instances created by our FlatFileItemReader.
publicclass PersonItemProcessor implements ItemProcessor<Person, Person> {

// returning null indicates that the item should not be continued to be
// processed.
public Person process(final Person person) throwsException {
finalString firstName = person.getFirstName().toUpperCase();
finalString lastName = person.getLastName().toUpperCase();
final Person transformedPerson = new Person(firstName, lastName);
System.out.println("Converting (" + person + ") into ("
+ transformedPerson + ")");
return transformedPerson;
}

}
This is the transformation class. It receives each person object, transforms it - in this case setting the values to upper case and returns the result. The last step is the saving of this data. I didn't really need a database for this example.An in memory solution was good enough. So:
<beans:bean id="itemWriter"
class="org.springframework.batch.item.support.ListItemWriter"/>
This completes the code for my step1 of Job1 .
publicstatic void main(String[] args) {

ApplicationContext context = new ClassPathXmlApplicationContext("spring-config.xml");
JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
Job job = (Job) context.getBean("reportJob");

try {
JobExecution execution = jobLauncher.run(job, new JobParameters());
System.out.println("Exit Status : " + execution.getStatus());
} catch (JobExecutionAlreadyRunningException | JobRestartException
| JobInstanceAlreadyCompleteException | JobParametersInvalidException e) {
e.printStackTrace();
}

List<? extends Person> results = ((ListItemWriter<Person>) context.getBean("itemWriter"))
.getWrittenItems();
for (Person person : results) {
System.out.println("Found <" + person + "> in the result.");
}
((ConfigurableApplicationContext) context).close();
System.out.println("Done");
}
As can be seen here, to run the Job we needed an instance of JobLauncher class. To summarize
  • A JobLauncher class executes a Job retrieved from a JobRepository passing to it a JobParameters object. 
  • For the executed job, the JobLauncher returns us a JobExecution object which is like a report. 
  • Every job is composed of one or more steps each of which could be composed of read process and write operations. 
The xml configuration is as below:
<?xmlversion="1.0"encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/batch"
xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
 http://www.springframework.org/schema/beans
 http://www.springframework.org/schema/beans/spring-beans.xsd
 http://www.springframework.org/schema/batch
 http://www.springframework.org/schema/batch/spring-batch-2.2.xsd">

<beans:bean id="fieldSetMapper"
class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<beans:property name="targetType"
value="org.robin.learn.spring.batch.data.Person"/>
</beans:bean>

<beans:bean id="lineTokenizer"
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<beans:property name="names" value="firstName,lastName"/>
</beans:bean>

<beans:bean id="lineMapper"
class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<beans:property name="lineTokenizer" ref="lineTokenizer"/>
<beans:property name="fieldSetMapper" ref="fieldSetMapper"/>
</beans:bean>

<beans:bean id="itemReader"
class="org.springframework.batch.item.file.FlatFileItemReader">
<beans:property name="resource" value="classpath:sample-data.csv"/>
<beans:property name="lineMapper" ref="lineMapper"/>
</beans:bean>


<beans:bean id="itemWriter"
class="org.springframework.batch.item.support.ListItemWriter"/>

<beans:bean id="personProcessor"
class="org.robin.learn.spring.batch.PersonItemProcessor"/>


<job id="reportJob" job-repository="jobRepository">
<step id="step1">
<tasklet>
<chunk reader="itemReader" writer="itemWriter"
processor="personProcessor" commit-interval="2"/>
</tasklet>
</step>
</job>

<beans:bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager">
</beans:bean>

<beans:bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<beans:property name="transactionManager" ref="transactionManager"/>
</beans:bean>

<beans:bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<beans:property name="jobRepository" ref="jobRepository"/>
</beans:bean>

</beans:beans>
The console output:
13:58:13.678 [main] INFO  o.s.b.f.xml.XmlBeanDefinitionReader - Loading XML bean definitions 
from class path resource [spring-config.xml]
...
13:58:14.258 [main] DEBUG o.s.t.i.NameMatchTransactionAttributeSource - Adding transactional method
[*] with attribute [PROPAGATION_REQUIRED,ISOLATION_DEFAULT]
13:58:14.258 [main] DEBUG o.s.t.i.NameMatchTransactionAttributeSource - Adding transactional method
[create*] with attribute [PROPAGATION_REQUIRES_NEW,ISOLATION_SERIALIZABLE]
13:58:14.258 [main] DEBUG o.s.t.i.NameMatchTransactionAttributeSource - Adding transactional method
[getLastJobExecution*] with attribute [PROPAGATION_REQUIRES_NEW,ISOLATION_SERIALIZABLE]
...
13:58:14.278 [main] DEBUG o.s.b.f.s.DefaultListableBeanFactory - Finished creating instance of bean 'jobRepository'
...
13:58:14.368 [main] DEBUG o.s.b.f.s.DefaultListableBeanFactory - Creating instance of bean 'reportJob'
...
13:58:14.378 [main] DEBUG o.s.b.f.s.DefaultListableBeanFactory - Creating instance of bean 'jobLauncher'
13:58:14.398 [main] DEBUG o.s.b.s.t.ResourcelessTransactionManager - Creating new transaction with
name [org.springframework.batch.core.repository.support.SimpleJobRepository.getLastJobExecution]:
PROPAGATION_REQUIRES_NEW,ISOLATION_SERIALIZABLE
13:58:14.448 [main] DEBUG o.s.b.s.t.ResourcelessTransactionManager - Initiating transaction commit
13:58:14.448 [main] DEBUG o.s.b.s.t.ResourcelessTransactionManager - Committing resourceless transaction on
[org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@6b7fd49c]
13:58:14.508 [main] INFO o.s.b.c.l.support.SimpleJobLauncher - Job: [FlowJob: [name=reportJob]]
launched with the following parameters: [{}]

13:58:14.508 [main] DEBUG o.s.batch.core.job.AbstractJob - Job execution starting: JobExecution: id=0,
version=0, startTime=null, endTime=null,
lastUpdated=Thu Feb 19 13:58:14 CST 2015, status=STARTING, exitStatus=exitCode=UNKNOWN;

exitDescription=, job=[JobInstance: id=0, version=0, Job=[reportJob]], jobParameters=[{}]
13:58:14.518 [main] DEBUG o.s.b.s.t.ResourcelessTransactionManager - Creating new transaction with name
[org.springframework.batch.core.repository.support.SimpleJobRepository.update]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
...
13:58:14.538 [main] INFO o.s.batch.core.job.SimpleStepHandler - Executing step: [step1]
13:58:14.538 [main] DEBUG o.s.batch.core.step.AbstractStep - Executing: id=1
Converting (firstName: Robin, lastName: Varghese) into (firstName: ROBIN, lastName: VARGHESE)
Converting (firstName: Rohan, lastName: Naidu) into (firstName: ROHAN, lastName: NAIDU)
13:58:14.578 [main] DEBUG o.s.b.c.s.item.ChunkOrientedTasklet - Inputs not busy, ended: false
13:58:14.578 [main] DEBUG o.s.b.core.step.tasklet.TaskletStep - Applying contribution: [
StepContribution: read=2, written=2, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]
...
Converting (firstName: Roman, lastName: Barlan) into (firstName: ROMAN, lastName: BARLAN)
13:58:14.588 [main] DEBUG o.s.batch.core.step.AbstractStep - Step execution success: id=1
...
13:58:14.608 [main] INFO o.s.b.c.l.support.SimpleJobLauncher - Job: [FlowJob: [name=reportJob]]
completed with the following parameters: [{}] and the following status: [COMPLETED]

Exit Status : COMPLETED
Found <firstName: ROBIN, lastName: VARGHESE> in the result.
Found <firstName: ROHAN, lastName: NAIDU> in the result.
Found <firstName: ROMAN, lastName: BARLAN> in the result.
Done

Multiple Steps in a Job

$
0
0
In the previous post we saw how to execute a job with a single step. A job can be composed of multiple steps. So I decided to extend the previous job to include two steps.
  1. Step 1 is same as before - read csv -> process -> write to List.
  2. Step 2 is read from List -> process -> write to CSV file.
To read from the List I introduced a new class :
publicclass MemItemReader extends ListItemReader<Person> {

@SuppressWarnings("unchecked")
public MemItemReader(ListItemWriter<Person> writer) {
super((List<Person>) writer.getWrittenItems());

}

}
The class simply picks up the records from the ListItemWriter. For the processing - I did not want to to do anything complex - so I simply transformed the Person object into a BetterPerson instance:
publicclass BetterPerson extends Person {
privateString title;
//setter getters
}
publicclass BetterPersonItemProcessor implements ItemProcessor<Person, BetterPerson> {

public BetterPerson process(final Person person) throwsException {

final BetterPerson betterPerson = new BetterPerson();

betterPerson.setFirstName(person.getFirstName());
betterPerson.setLastName(person.getLastName());
betterPerson.setTitle("??");
System.out.println("Converting (" + person + ") into ("
+ betterPerson + ")");
return betterPerson;
}

}
Now to configure the new step:
<job id="reportJob" job-repository="jobRepository">
<step id="step1" next="step2">
<tasklet>
<chunk reader="itemReader" writer="itemWriter"
processor="personProcessor" commit-interval="2"/>
</tasklet>
</step>
<step id="step2">
<tasklet>
<chunk reader="memItemReader" writer="cvsFileItemWriter"
processor="betterPersonProcessor" commit-interval="2"/>
</tasklet>
</step>
</job>
As seen here we have step with id step1 followed by step2. The next attribute indicates the link. If the attribute is not specified than we get an error:
Exception in thread "main" org.springframework.beans.factory.parsing.BeanDefinitionParsingException:
Configuration problem: The element [step2] is unreachable
Offending resource: class path resource [spring-config.xml]
The configuration for the new beans is as below:
<beans:bean id="memItemReader" class="org.robin.learn.sb.batch.MemItemReader"
scope="step">
<beans:constructor-arg ref="itemWriter"/>
</beans:bean>


<beans:bean id="betterPersonProcessor"
class="org.robin.learn.sb.batch.BetterPersonItemProcessor"/>

<beans:bean id="cvsFileItemWriter"
class="org.springframework.batch.item.file.FlatFileItemWriter">
<beans:property name="resource" value="file:opt.csv"/>
<beans:property name="shouldDeleteIfExists" value="true"/>
<beans:property name="lineAggregator">
<beans:bean
class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<beans:property name="delimiter" value=","/>
<beans:property name="fieldExtractor">
<beans:bean
class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<beans:property name="names"
value="title, firstName,lastName"/>
</beans:bean>
</beans:property>
</beans:bean>
</beans:property>
</beans:bean>
The cvsFileItemWriter was lifted from this example. It includes a delimiter property - so we can easily generate pipe, hyphen or even a space separated file. The fieldExtractor property gives the sequence in which the fields must be read to write the data.
For the first bean "memItemReader", I have specified the scope. This is a previously unseen scope that has been introduced for Spring batch. If the memItemReader bean were created in the beginning it would be initialized with the empty list of the itemWriter bean. What we need is that bean is created after the first step has executed. So we bind this bean to a step scope. The bean is now created before the step execution:
15:21:16.894 [main] INFO  o.s.batch.core.job.SimpleStepHandler - Executing step: [step2].
...
15:21:16.904 [main] DEBUG o.s.batch.core.scope.StepScope - Creating object in scope=step,
name=scopedTarget.memItemReader
15:21:16.904 [main] DEBUG o.s.b.f.s.DefaultListableBeanFactory - Creating instance of bean
'scopedTarget.memItemReader'
15:21:16.904 [main] DEBUG o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance
of singleton bean 'itemWriter'
As seen here, as the bean is created after Step1, there is data in the itemWriter which is made available to the MemItemReader. On running the code now a csv file is created on the disk with information as
??,ROBIN,VARGHESE
??,ROHAN,NAIDU
??,ROMAN,BARLAN

Introducing database to Spring Batch

$
0
0
Until now we have seen Spring batch without its meta data persistence features. When I say meta-data for Spring batch it includes batch related information - a listing of the steps executed, and their audit information, how many items did the framework read/write, did it skip any items, what was the duration of each step executed etc. This meta data is maintained in a database - persistence.
To keep things easy we used in memory implementations of the batch classes - these implementations ran with Maps. Spring recommends that the Map based implementation be used only for application testing. For batch applications in the real world Spring recommends the database linked code:
"Note that the in-memory repository is volatile and so does not allow restart between JVM instances. 
It also cannot guarantee that two job instances with the same parameters are launched simultaneously,
and is not suitable for use in a multi-threaded Job, or a locally partitioned Step. So use the
database version of the repository wherever you need those features."
So I decided to introduce the database to my batch example. The changes were only needed in the configuration:
<beans:bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<beansroperty name="transactionManager" ref="transactionManager"/>
<beansroperty name="dataSource" ref="dataSource"/>
</beans:bean>
the JobRepository class has been changed from MapJobRepositoryFactoryBean to JobRepositoryFactoryBean. This class needs to have the dataSource property initialized. I decided to go with a postgres database:
<beans:bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<beans:property name="driverClassName" value="org.postgresql.Driver"/>
<beans:property name="url"
value="jdbcostgresql://localhost:5433/batchTest"/>
<bean:psroperty name="username" value="postgres"/>
<bean:psroperty name="password" value="#######"/>
</beans:bean>
The other class that need to be changed is the TransactionManager:
<beans:bean id="transactionManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<beans:property name="dataSource" ref="dataSource"/>
</beans:bean>
This replaces the ResourcelessTransactionManager. If I run the code now:
11:53:45.589 [main] INFO  o.s.j.support.SQLErrorCodesFactory - SQLErrorCodes loaded: 
[DB2, Derby, H2, HSQL, Informix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase]
11:53:45.589 [main] DEBUG o.s.j.support.SQLErrorCodesFactory - Looking up default
SQLErrorCodes for DataSource [org.springframework.jdbc.datasource.DriverManagerDataSource@464d02ee]
11:53:45.589 [main] DEBUG o.s.j.support.SQLErrorCodesFactory - Database product name cached for
DataSource [org.springframework.jdbc.datasource.DriverManagerDataSource@464d02ee]: name is 'PostgreSQL'
11:53:45.589 [main] DEBUG o.s.j.support.SQLErrorCodesFactory - SQL error codes for 'PostgreSQL' found
11:53:45.590 [main] DEBUG o.s.j.s.SQLErrorCodeSQLExceptionTranslator
- Translating SQLException with SQL state '42P01', error code '0', message [ERROR: relation
"batch_job_instance" does not exist

Position: 39]; SQL was [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where
JOB_NAME = ? and JOB_KEY = ?] for task [PreparedStatementCallback]
11:53:45.590 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction rollback
The code failed as Spring batch did not find the tables necessary for the code to run. I had expected that the tables would be auto generated, like Hibernate's DDLcommand. To get the table scripts to execute we need to add the below XML element:
<jdbc:initialize-database data-source="dataSource">
<jdbc:script
location="org/springframework/batch/core/schema-drop-postgresql.sql"/>
<jdbc:script
location="org/springframework/batch/core/schema-postgresql.sql"/>
</jdbc:initialize-database>
This tells Spring to load the scripts from the org.springframework.batch.core package. Within this folder (its the spring-batch-core jar) there are present such drop and create scripts for several databases. Now if I were to run the code:
13:18:00.641 [main] DEBUG o.s.b.f.xml.XmlBeanDefinitionReader - Loaded 20 bean definitions from location pattern [spring-config-db.xml]
...
13:18:00.853 [main] DEBUG o.s.jdbc.datasource.DataSourceUtils - Fetching JDBC Connection from DataSource
13:18:00.853 [main] DEBUG o.s.j.d.DriverManagerDataSource - Creating new JDBC DriverManager Connection to [jdbcostgresql://localhost:5433/batchTest]
13:18:00.928 [main] DEBUG o.s.jdbc.datasource.DataSourceUtils - Returning JDBC Connection to DataSource
13:18:00.928 [main] INFO o.s.b.c.r.s.JobRepositoryFactoryBean - No database type set, using meta data indicating: POSTGRES
13:18:01.013 [main] DEBUG o.s.t.i.NameMatchTransactionAttributeSource - Adding transactional method [*] with attribute
[PROPAGATION_REQUIRED,ISOLATION_DEFAULT]
13:18:01.013 [main] DEBUG o.s.t.i.NameMatchTransactionAttributeSource - Adding transactional method [create*] with attribute
[PROPAGATION_REQUIRES_NEW,ISOLATION_SERIALIZABLE]
13:18:01.013 [main] DEBUG o.s.t.i.NameMatchTransactionAttributeSource - Adding transactional method [getLastJobExecution*]
with attribute [PROPAGATION_REQUIRES_NEW,ISOLATION_SERIALIZABLE]
...
13:18:01.114 [main] DEBUG o.s.jdbc.datasource.DataSourceUtils - Fetching JDBC Connection from DataSource
13:18:01.114 [main] DEBUG o.s.j.d.DriverManagerDataSource - Creating new JDBC DriverManager Connection to [jdbcostgresql://localhost:5433/batchTest]
13:18:01.132 [main] INFO o.s.jdbc.datasource.init.ScriptUtils - Executing SQL script from class path resource
[org/springframework/batch/core/schema-drop-postgresql.sql]

...
13:18:01.143 [main] INFO o.s.jdbc.datasource.init.ScriptUtils - Executed SQL script from class path resource
[org/springframework/batch/core/schema-drop-postgresql.sql] in 10 ms.

13:18:01.143 [main] INFO o.s.jdbc.datasource.init.ScriptUtils - Executing SQL script from class path resource
[org/springframework/batch/core/schema-postgresql.sql]

13:18:01.479 [main] INFO o.s.jdbc.datasource.init.ScriptUtils - Executed SQL script from class path resource
[org/springframework/batch/core/schema-postgresql.sql] in 336 ms.

As seen now, on startup spring batch executed the relevant drop and create scripts. The database schema for batch is as below:
This is when the job runs:
13:18:01.522 [main] DEBUG o.s.jdbc.core.JdbcTemplate - Executing prepared SQL statement [SELECT JOB_INSTANCE_ID, JOB_NAME
from BATCH_JOB_INSTANCE where JOB_NAME = ? and JOB_KEY = ?]
13:18:01.531 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Creating new transaction with name
[org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution]: PROPAGATION_REQUIRES_NEW,ISOLATION_SERIALIZABLE
SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? and JOB_KEY = ?
INSERT into BATCH_JOB_INSTANCE(JOB_INSTANCE_ID, JOB_NAME, JOB_KEY, VERSION) values (?, ?, ?, ?)
INSERT into BATCH_JOB_EXECUTION(JOB_EXECUTION_ID, JOB_INSTANCE_ID, START_TIME, END_TIME,
STATUS, EXIT_CODE, EXIT_MESSAGE, VERSION, CREATE_TIME, LAST_UPDATED, JOB_CONFIGURATION_LOCATION) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
INSERT INTO BATCH_JOB_EXECUTION_CONTEXT (SHORT_CONTEXT, SERIALIZED_CONTEXT, JOB_EXECUTION_ID) VALUES(?, ?, ?)
Initiating transaction commit:
13:18:01.583 [main] INFO  o.s.b.c.l.support.SimpleJobLauncher - Job: [FlowJob: [name=reportJob]] launched with the following parameters: [{}]
13:18:01.583 [main] DEBUG o.s.batch.core.job.AbstractJob - Job execution starting: JobExecution: id=1, version=0, startTime=null, endTime=null,
lastUpdated=Tue Feb 24 13:18:01 CST 2015, status=STARTING, exitStatus=exitCode=UNKNOWN;
exitDescription=, job=[JobInstance: id=1, version=0, Job=[reportJob]], jobParameters=[{}]
13:18:01.586 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Creating new transaction with name
[org.springframework.batch.core.repository.support.SimpleJobRepository.update]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=?
UPDATE BATCH_JOB_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?,
CREATE_TIME = ?, LAST_UPDATED = ? where JOB_EXECUTION_ID = ? and VERSION = ?
SELECT JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, CREATE_TIME, LAST_UPDATED, VERSION,
JOB_CONFIGURATION_LOCATION from BATCH_JOB_EXECUTION where JOB_INSTANCE_ID = ? order by JOB_EXECUTION_ID desc
SELECT JOB_EXECUTION_ID, KEY_NAME, TYPE_CD, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING
from BATCH_JOB_EXECUTION_PARAMS where JOB_EXECUTION_ID = ?
SELECT STEP_EXECUTION_ID, STEP_NAME, START_TIME, END_TIME, STATUS, COMMIT_COUNT, READ_COUNT, FILTER_COUNT, WRITE_COUNT,
EXIT_CODE, EXIT_MESSAGE, READ_SKIP_COUNT, WRITE_SKIP_COUNT, PROCESS_SKIP_COUNT, ROLLBACK_COUNT, LAST_UPDATED, VERSION
from BATCH_STEP_EXECUTION where JOB_EXECUTION_ID = ? order by STEP_EXECUTION_ID
13:18:01.639 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
SELECT JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, CREATE_TIME, LAST_UPDATED, VERSION,
JOB_CONFIGURATION_LOCATION from BATCH_JOB_EXECUTION where JOB_INSTANCE_ID = ? order by JOB_EXECUTION_ID desc
SELECT JOB_EXECUTION_ID, KEY_NAME, TYPE_CD, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING
from BATCH_JOB_EXECUTION_PARAMS where JOB_EXECUTION_ID = ?
SELECT STEP_EXECUTION_ID, STEP_NAME, START_TIME, END_TIME, STATUS, COMMIT_COUNT, READ_COUNT, FILTER_COUNT,
WRITE_COUNT, EXIT_CODE, EXIT_MESSAGE, READ_SKIP_COUNT, WRITE_SKIP_COUNT, PROCESS_SKIP_COUNT, ROLLBACK_COUNT, LAST_UPDATED,
VERSION from BATCH_STEP_EXECUTION where JOB_EXECUTION_ID = ? order by STEP_EXECUTION_ID
Initiating transaction commit
SELECT JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, CREATE_TIME, LAST_UPDATED, VERSION, JOB_CONFIGURATION_LOCATION
from BATCH_JOB_EXECUTION where JOB_INSTANCE_ID = ? order by JOB_EXECUTION_ID desc]
13:18:01.681 [main] DEBUG o.s.jdbc.core.JdbcTemplate - Executing prepared SQL query
13:18:01.681 [main] DEBUG o.s.jdbc.core.JdbcTemplate - Executing prepared SQL statement [SELECT JOB_EXECUTION_ID, KEY_NAME, TYPE_CD, STRING_VAL, DATE_VAL,
LONG_VAL, DOUBLE_VAL, IDENTIFYING from BATCH_JOB_EXECUTION_PARAMS where JOB_EXECUTION_ID = ?]
13:18:01.682 [main] DEBUG o.s.jdbc.core.JdbcTemplate - Executing prepared SQL query
13:18:01.682 [main] DEBUG o.s.jdbc.core.JdbcTemplate - Executing prepared SQL statement [SELECT STEP_EXECUTION_ID, STEP_NAME, START_TIME, END_TIME, STATUS,
COMMIT_COUNT, READ_COUNT, FILTER_COUNT, WRITE_COUNT, EXIT_CODE, EXIT_MESSAGE, READ_SKIP_COUNT, WRITE_SKIP_COUNT, PROCESS_SKIP_COUNT,
ROLLBACK_COUNT, LAST_UPDATED, VERSION from BATCH_STEP_EXECUTION where JOB_EXECUTION_ID = ? order by STEP_EXECUTION_ID]
13:18:01.683 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
13:18:01.684 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.add]:
PROPAGATION_REQUIRED,ISOLATION_DEFAULT
INSERT into BATCH_STEP_EXECUTION(STEP_EXECUTION_ID, VERSION, STEP_NAME, JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, COMMIT_COUNT,
READ_COUNT, FILTER_COUNT, WRITE_COUNT, EXIT_CODE, EXIT_MESSAGE, READ_SKIP_COUNT, WRITE_SKIP_COUNT, PROCESS_SKIP_COUNT, ROLLBACK_COUNT,
LAST_UPDATED) values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
INSERT INTO BATCH_STEP_EXECUTION_CONTEXT (SHORT_CONTEXT, SERIALIZED_CONTEXT, STEP_EXECUTION_ID) VALUES(?, ?, ?)
13:18:01.708 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
13:18:01.711 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.update]:
PROPAGATION_REQUIRED,ISOLATION_DEFAULT
UPDATE BATCH_STEP_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, COMMIT_COUNT = ?, READ_COUNT = ?, FILTER_COUNT = ?, WRITE_COUNT = ?,
EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, READ_SKIP_COUNT = ?, PROCESS_SKIP_COUNT = ?, WRITE_SKIP_COUNT = ?, ROLLBACK_COUNT = ?, LAST_UPDATED = ?
where STEP_EXECUTION_ID = ? and VERSION = ?
SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=?
13:18:01.733 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
Creating new transaction with name [null]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
Converting (firstName: Robin, lastName: Varghese) into (firstName: ROBIN, lastName: VARGHESE)
Converting (firstName: Rohan, lastName: Naidu) into (firstName: ROHAN, lastName: NAIDU)
13:18:01.805 [main] DEBUG o.s.b.c.s.item.ChunkOrientedTasklet - Inputs not busy, ended: false
13:18:01.805 [main] DEBUG o.s.b.core.step.tasklet.TaskletStep - Applying contribution: [StepContribution: read=2, 
written=2, filtered=0, readSkips=0, writeSkips=0, 
processSkips=0, exitStatus=EXECUTING]
UPDATE BATCH_STEP_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE STEP_EXECUTION_ID = ?
13:18:01.808 [main] DEBUG o.s.b.core.step.tasklet.TaskletStep - Saving step execution before commit: StepExecution: id=1, version=1, name=step1, status=STARTED,
exitStatus=EXECUTING, readCount=2, filterCount=0, writeCount=2 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=1
UPDATE BATCH_STEP_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, COMMIT_COUNT = ?, READ_COUNT = ?, FILTER_COUNT = ?, WRITE_COUNT = ?,
EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, READ_SKIP_COUNT = ?, PROCESS_SKIP_COUNT = ?, WRITE_SKIP_COUNT = ?, ROLLBACK_COUNT = ?, LAST_UPDATED = ?
where STEP_EXECUTION_ID = ? and VERSION = ?]
SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=?
13:18:01.810 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
13:18:01.814 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Creating new transaction with name [null]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
Converting (firstName: Roman, lastName: Barlan) into (firstName: ROMAN, lastName: BARLAN)
13:18:01.834 [main] DEBUG o.s.b.core.step.tasklet.TaskletStep - Applying contribution: [StepContribution: read=1, written=1, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]
UPDATE BATCH_STEP_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE STEP_EXECUTION_ID = ?
13:18:01.836 [main] DEBUG o.s.b.core.step.tasklet.TaskletStep - Saving step execution before commit: StepExecution: id=1, version=2, name=step1, status=STARTED,
exitStatus=EXECUTING, readCount=3, filterCount=0, writeCount=3 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=2, rollbackCount=0, exitDescription=
UPDATE BATCH_STEP_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, COMMIT_COUNT = ?, READ_COUNT = ?, FILTER_COUNT = ?,
WRITE_COUNT = ?, EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, READ_SKIP_COUNT = ?, PROCESS_SKIP_COUNT = ?, WRITE_SKIP_COUNT = ?, ROLLBACK_COUNT = ?,
LAST_UPDATED = ? where STEP_EXECUTION_ID = ? and VERSION = ?
SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=?
13:18:01.838 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
UPDATE BATCH_STEP_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE STEP_EXECUTION_ID = ?
13:18:01.861 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
13:18:01.864 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Creating new transaction with name
[org.springframework.batch.core.repository.support.SimpleJobRepository.update]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
UPDATE BATCH_STEP_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, COMMIT_COUNT = ?, READ_COUNT = ?, FILTER_COUNT = ?,
WRITE_COUNT = ?, EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, READ_SKIP_COUNT = ?, PROCESS_SKIP_COUNT = ?, WRITE_SKIP_COUNT = ?,
ROLLBACK_COUNT = ?, LAST_UPDATED = ? where STEP_EXECUTION_ID = ? and VERSION = ?
SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=?
13:18:01.886 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
13:18:01.894 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Creating new transaction with name
[org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
UPDATE BATCH_STEP_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE STEP_EXECUTION_ID = ?
UPDATE BATCH_STEP_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, COMMIT_COUNT = ?, READ_COUNT = ?, FILTER_COUNT = ?,
WRITE_COUNT = ?, EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, READ_SKIP_COUNT = ?, PROCESS_SKIP_COUNT = ?, WRITE_SKIP_COUNT = ?,
ROLLBACK_COUNT = ?, LAST_UPDATED = ? where STEP_EXECUTION_ID = ? and VERSION = ?
SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=?
13:18:01.886 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
13:18:01.894 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Creating new transaction with name
[org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
UPDATE BATCH_JOB_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE JOB_EXECUTION_ID = ?
13:18:01.919 [main] DEBUG o.s.j.d.DataSourceTransactionManager - Initiating transaction commit
...
13:18:01.923 [main] DEBUG o.s.b.c.job.flow.support.SimpleFlow - Completed state=reportJob.step1 with status=COMPLETED
13:18:01.923 [main] DEBUG o.s.b.c.job.flow.support.SimpleFlow - Handling state=reportJob.end1
13:18:01.923 [main] DEBUG o.s.b.c.job.flow.support.SimpleFlow - Completed state=reportJob.end1 with status=COMPLETED
SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=?
SELECT COUNT FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID = ?
UPDATE BATCH_JOB_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, CREATE_TIME = ?,
LAST_UPDATED = ? where JOB_EXECUTION_ID = ? and VERSION = ?
...
13:18:01.948 [main] INFO o.s.b.c.l.support.SimpleJobLauncher - Job: [FlowJob: [name=reportJob]] 
completed with the following parameters: [{}] and the following status: [COMPLETED]
Exit Status : COMPLETED
Found <firstName: ROBIN, lastName: VARGHESE> in the result.
Found <firstName: ROHAN, lastName: NAIDU> in the result.
Found <firstName: ROMAN, lastName: BARLAN> in the result.
...
Done


Lambda

$
0
0
This is kind of late - very late, but then like always - better late than never. So while java lovers world over have dug deep, praised, criticized and in some cases confused the developer community about Lambda expressions in Java 8, I have finally woken and decided to throw my two cents into the pit.
I was kinda too busy/lazy until I got a code review that used Lambdas. With no escape but to review it, I had to start learning Lambdas.  This post is a collection of thoughts I found online and like always just my ready notes to be available for me to refer back when the next code review comes around. With that time to get started.....
Am sure we all had an use case, where we needed a list sorted. Being a one off, writing a top level comparator class and using it seemed an overkill. So we turned to anonymous classes.
public void someMethod() {
List<User> users = getListOfUsers();// some complex code
//Need to sort this List by age ofusers
users.sort(new Comparator<User>() {

@Override
publicint compare(User u1, User u2) {
returnInteger.compare(u1.getAge(), u2.getAge());
}
});
//Continue processing the sorted list
}
I guess other than the UI cases, this is one of the popular areas we see anonymous classes in code. This works. Perfectly. But the amount of code needed to say "compare users on age" is several lines. In production code you would even run across comments.
With Java 8, Oracle introduced the style of functional programming to Java. Before moving forward, I looked up the definition on wikipedia:
In computer science, functional programming is a programming paradigm—a style of building the structure and elements of computer programs—that treats computation as the evaluation of mathematical functions and avoids changing-state and mutable data.
Quite a lot to grasp at one go, so a some points of  note:
  1. Functional programming involves code that is stateless (Think of HTTP and REST)
  2. Functional programming works with immutable data.
Point 2 means that the function is kind of only working with the parameters that you passed to it.While 1 means that your function has no real recall of any operations it did before.

The compare method that we have above fits this paradigm. It only operates on the two parameters that we passed to it. Also it does not have any recall, i.e. it does not care of the results of an previous comparisons.
Accordingly Java 8 has changed the code for the Comparator interface:
@FunctionalInterface
publicinterface Comparator<T> {
As seen above a new annotation has been introduced to mark such interfaces ( have exactly one method and which satisfy the above two rules) as functional interfaces.
So we have heard about Functional  programming, seen java identify code that can be written in a functional manner and have them annotated as functional interface. But what is Lambda expression ?

Lambda Expressions are one of the ways to create instance of Functional Interface. From the Oracle tutorial, Lambda Expressions could be :
This is a really powerful thing - we are creating complex objects through expressions

Now that we have seen the various terms, time to rewrite our Comparator as a lambda expression:
public void someMethod() {
List<User> users = getListOfUsers();// some complex code
// Need to sort this List by age ofusers
users.sort(Comparator.comparing(user -> user.getAge()));
// Continue processing the sorted list
}
As seen, the whole code has been written as a single line.
Thus our simple lambda expression which converts a user instance to a integer instance (user.getAge()) is converted by the compiler into a Function instance. And what we were doing before with about 5 -10 lines of code now become 1 -2 lines.
Like I said before Lambda expression is not the only way to achieve functional programming in java. They also have a technique called Method references.
For instance the above code could be rewritten as:

users.sort(Comparator.comparing(User::getAge));

There is much more to Lambdas. Its presence in Collection Libraries, threading code, Predicates....I have just scraped the surface here.

With help from:
http://www.oracle.com/webfolder/technetwork/tutorials/obe/java/Lambda-QuickStart/index.html
http://www.smashingmagazine.com/2014/07/dont-be-scared-of-functional-programming/
http://howtodoinjava.com/2014/04/18/using-comparator-becomes-easier-with-lambda-expressions-java-8/ 

Consumers in Java Lambda World

$
0
0
A I mentioned in the previous post, Lambda expressions have helped simplify Collection use cases. We saw that with sorting a Collection. Let me explore some more simplifications in this post.
We often need to iterate over the code in collections. General approach is to use a for loop or an enhanced for each loop. Consider the below code to display all user names
for (User user : users){
System.out.println(user.getName() + " of age "
+ user.getAge() + " has a salary of " + user.getSalary());
}
Java 8 has a added a new method called forEach() to the Iterable interface. All objects (such as Collection, some NIO classes etc.) that implement this interface can work with both the for each enhanced loop and the new for each method.

The new code is:
users.forEach(user -> System.out
.println(user.getName() + " of age " + user.getAge()
+ " has a salary of " + user.getSalary()));
As seen from the implementation of the interface, the method takes not a Function instance but a Consumer instance. Consumer class is also a type of FunctionInterface - From the java docs:
 Represents an operation that accepts a single input argument and returns no
result. Unlike most other functional interfaces, Consumer is expected
to operate via side-effects.
As can be seen here we are not really doing any transformation, or returning some output as we did with the Function instance for comparable, instead here we simply consume the string and return nothing.
The code need not be a one liner, could also be a block code.

Set<User> highIncomeUsers = new HashSet<>();
users.forEach(user -> {
if (user.getSalary() > 3000) {
highIncomeUsers.add(user);
} else {
System.out.println(user.getName() + " with salary of "
+ user.getSalary() + " was skipped");
}
});
System.out.println("HIGH INCOME USERS");
highIncomeUsers.forEach(user->System.out.println(user.getName()
+ " with salary of " + user.getSalary()));
As can be seen here, we are introducing a hash set that results that is used outside the scope of this call - kind of side effects that we saw in the class comments.
It is not that we are restricted to using Consumer or other functional interfaces only as parameters to other methods.
Consider the below code:
      Set<User> highIncomeUsers = new HashSet<>();
Consumer<User> highIncomeUserCollector = (user) -> {
if (user.getSalary() > 3000) {
highIncomeUsers.add(user);
} else {
System.out.println(user.getName() + " with salary of "
+ user.getSalary() + " was skipped");
}
};
highIncomeUserCollector.accept(users.get(0));
Just to be clear, Consumer is one type of function object created by the compiler from Lambda expression. It is generated for expressions associated with taking input but not creating any return value.

Suppliers in Java Lambda World

$
0
0
In the previous post I had a look at Consumer function objects and how they were used in the forEach method of the Iterable class. An exactly opposite kind of interface to the Consumer is the Supplier interface. (like the name doesn't give it away)
To be honest I had a tough time coming up with a use case for this.I decided to first use this in a few places.
I first created a simple class Ticket:
publicclass Ticket {

privatestaticint counter = 0;

public Ticket() {
counter++;
}

publicstatic Ticket getSpecialTicket() {
// some processing needed to get ticket
counter++;
      returnnew Ticket();
}

publicstaticint getTicketCount() {
return counter;
}

@Override
publicString toString() {
return"T[" + this.hashCode() +"]";
}
}
The next step was to pass a ticket generator for use in application code. Normal way to do it would be:
publicstatic void performOperation(){
//get details of operation
//generate a ticket
Ticket ticket = new Ticket();
//use the ticket instance to perform operation
System.out.println("Ticket " + ticket + " used for operation");
}
Here we see the ticket being generated and used in code. If we had different sub-classes of tickets than we need to place that logic within the peformOperation method or else pass the ticket as a parameter. But what if we need multiple tickets ?
A more suitable way would be to move this logic into a different abstraction and pass this as a parameter to the performOperation method.
The Supplier interface could be an easy way to achieve this without having to write any code.
publicstatic void performOperation(Supplier<? extends Ticket> ticketSupplier) {
// get details of operation
// generate a ticket
Ticket ticket1 = ticketSupplier.get();
// use the ticket instance to perform operation
System.out.println("Ticket " + ticket1 + " used for operation");
//using the second ticket for
Ticket ticket2 = ticketSupplier.get();
// use the ticket instance to perform operation
System.out.println("Ticket " + ticket2 + " used for operation");
}
As can be seen here an instance of Supplier is passed here. Every time a ticket is needed, a call to the get method of Supplier class is made. As per the java documentation:
Represents a supplier of results. 
There is no requirement that a new or distinct result be returned each time the supplier is invoked.
This is a functional interface whose functional method is get().
The client code for execution would be:
publicstatic void main(String[] args) {
performOperation(Ticket::new);
}
As can be seen here, the code makes two calls to get instance of Supplier - this will result in tow calls to Ticket constructor, which will return us two ticket objects. If I wanted the static method to be used here:
publicstatic void main(String[] args) {
// performOperation(Ticket::new);
performOperation(Ticket::getSpecialTicket);
}
In both the cases every call to get will return a new ticket instance.
Ticket T[617901222] used for operation
Ticket T[1159190947] used for operation
However if a singleton type of method was passed to Supplier, than we would have a single value returned by all calls to get. In both of the above calls, I used method reference. We can also achieve the same using lambda expressions:
performOperation(()->new Ticket());
performOperation(()-> getSpecialTicket());

Interfaces versus Abstract classes : pre and post Java 8

$
0
0
I have been given a contract to implement in Java. As a developer one of the early design decisions needed is, do I want an interface or an abstract class at the base of my hierarchy. This question is very popular in the Java world and often always pops up when doing low level design.
There is no golden rule on when to use what. It is easily possible to find a group of developers who say:
  • Pure Interface looks so much clean.
  • Interface ensures you do not fall to Java's Single Inheritance restriction.
  • Contracts should say what and not how - making it an ideal use case for an interface.
Opposing arguments are mostly concerned with:
  • What if I want to provide a default implementation for some methods ?
  • I don't like my data objects hierarchy starting with an interface. I prefer a class at the base.
  • I don't foresee any use case requiring a second class to be extended by my classes.
  • What if I wanted to force all my implementations to use a certain implementation for one of the contracts.
  • With abstract classes at the base, it is easy to add a new method with a default implementation. This will ensure that all sub classes that do not care for this method don't break, while the affected ones can go ahead and override.
As guessed the people in the second category would prefer to see abstract classes win here. A common compromise, I have seen in most of the projects I have worked in involve:
  • Define all the services, Business objects behind interfaces.
  • Where there was the possibility (genuine) of some class hierarchies having common code, setup an abstract class, and move the common code there - something I have often seen in web controllers and earlier in servlets ( remember the pre - framework days).
  • If needed the design would have a combination of both - a pure interface that defined all the contracts and abstract classes that provided default implementations where possible.
The third scenario seems the safest way to do things as it allows us to have default implementations for some methods which if needed can be overridden by the sub classes. Sub classes can also choose to directly implement the interface, thus allowing for extending some other class. It does means some additional work on the code front though. And if a sub class needs some default implementation code, but also extend a different class, then we are stuck.

With Java 8, Oracle has introduced some major rework in inheritance mechanism, boosting the power of interfaces in Java.
The introduction of default methods will go a long way in easing up the inheritance chain in some of the projects (and could also confuse in other places).

I decided the best place to start with was the Java docs at Oracle:
An abstract class is a class that is declared abstract—it may or may not include 
abstract methods. Abstract classes cannot be instantiated, but they can be
subclassed.
On a direct comparison between the two:
Abstract classes are similar to interfaces. You cannot instantiate them, and they may 
contain a mix of methods declared with or without an implementation. However, with
abstract classes, you can declare fields that are not static and final, and define public,
protected, and private concrete methods. With interfaces, all fields are automatically public,
static, and final, and all methods that you declare or define (as default methods) are public.

In addition, you can extend only one class, whether or not it is abstract, whereas you can
implement any number of interfaces.
So when to use interfaces and when to use abstract classes ?
Again I am quoting from the documentation:
Consider using abstract classes if any of these statements apply to your situation:
-> You want to share code among several closely related classes.
-> You expect that classes that extend your abstract class have many common methods
or fields, or require access modifiers other than public (such as protected and private).
-> You want to declare non-static or non-final fields. This enables you to define methods
that can access and modify the state of the object to which they belong.

Consider using interfaces if any of these statements apply to your situation:
-> You expect that unrelated classes would implement your interface. For example,
the interfaces Comparable and Cloneable are implemented by many unrelated classes.
-> You want to specify the behavior of a particular data type, but not concerned about who
implements its behavior.
-> You want to take advantage of multiple inheritance of type.

An example of an abstract class in the JDK is AbstractMap, which is part of the
Collections Framework.
Its subclasses (which include HashMap, TreeMap, and ConcurrentHashMap) share many methods
(including get, put, isEmpty, containsKey, and containsValue) that AbstractMap defines.
An example of a class in the JDK that implements several interfaces is HashMap, which
implements the interfaces Serializable, Cloneable, and Map<K, V>. By reading this
list of interfaces, you can infer that an instance of HashMap (regardless of the developer
or company who implemented the class) can be cloned, is serializable (which means that it
can be converted into a byte stream; see the section Serializable Objects), and has the
functionality of a map. In addition, the Map<K, V> interface has been enhanced
with many default methods such as merge and forEach that older classes that have implemented this
interface do not have to define.


Note that many software libraries use both abstract classes and interfaces; the HashMap class implements
several interfaces and also extends the abstract class AbstractMap.
Along with all the nuggets of information, the documentation talks of the new feature of default methods.
While the reason for the existance of this feature seems to be Lambdas, it kind of fits in nicely with the use case of providing some default implementations for the interface methods.
Consider the below code:
interface Contract {
publicvoid method1();
publicvoid commonMethod();
}

abstractclass ContractBase implements Contract {

@Override
publicvoid commonMethod() {

}
}

class ContractImpl extends ContractBase {

@Override
publicvoid method1() {
}

}
Here we have an interface, but we would like to provide some default implementation for commonMethod(), one that we expect would help most of the implementations. With pre Java 8, we would have to create an abstract class and place the code there. All our implementations would now be sub classes of ContractBase and not implement Contract directly.
With Java8, default methods helps skip the abstract class.
interface Contract {
publicvoid method1();

defaultpublicvoid commonMethod() {
//the common code here
}
}

class ContractImpl implements Contract {//extends ContractBase {

@Override
publicvoid method1() {
   }

}
The reason for default methods : (again from Oracle documentation)
default methods enable you to add new functionality to the interfaces of your libraries and 
ensure binary compatibility with code written for older versions of those interfaces.
With the introduction of Lambdas, there was a lot of richness that could be added to the Collections interfaces. But adding a new method in such heavily used API meant Oracle would end up breaking all implementations of Collections interfaces everywhere. So to avoid colossal complaints from java developers, Oracle came up with the idea of default methods. This feature allowed them to define all the new Lambda based methods in existing API, as well as provide code for them in the interface, thus ensuring that all implementations everywhere stay as is and can also use the new APIs without any change in code.
Another new feature Oracle introduced for interfaces is to allow them to have static methods. Before Java 8, interface represented a contract - at an instance level. Thus all methods were associated with an object. Now,
interface Contract {
publicvoid method1();

defaultpublicvoid commonMethod() {
//the common code here
}

staticvoid utilityMethod() {
//This is simply an utility method - implicit public method
}
}
The advantage, according to Oracle is:
This makes it easier for you to organize helper methods in your libraries; you can keep 
static methods specific to an interface in the same interface rather than in a separate class.
I can relate to this. I remember defining static methods in the base abstract class on occasions, as I wanted the method to be close to the hierarchy as opposed to a Helper class. But this is more of a design call and needs to be evaluated on a case to case basis. So with this added power, there are some cases where an abstract class could be skipped from the design.

Predicates to determine the truth !

$
0
0
We have seen some of the functional interfaces like Consumer and Supplier in the earlier posts. The next one I wanted to explore is the Predicate interface. This one is used for testing certain conditions on a passed instance and returns a boolean value.
Consider the User class below:
publicclass User {

privateString name;
privateint age;
privatedouble salary;
//setter getters
}
I wanted to use predicates to decide if a user is eligible to vote:
publicstatic boolean isEligibleToVote(User user) {
return user.getAge() > 18;
}
This logic could be wrapped into a Java object using Predicates
   staticPredicate<User> userEligibleToVote = user -> user.getAge() > 18;

publicstatic boolean isEligibleToVote(User user) {
returnuserEligibleToVote.test(user);
}
As seen we wrapped the imperative style code into a Class.The Predicate interface also defines some easy methods to get other conditions created.
static Predicate<User> userEligibleToVote = user -> user.getAge() > 18;
static Predicate<User> userNotEligibleToVote = userEligibleToVote.negate();

static Predicate<User> unemployedUser = user -> user.salary <= 0;

static Predicate<User> unemployedVoter = userEligibleToVote.and(unemployedUser);
static Predicate<User> unemployedOrVoter = userEligibleToVote.or(unemployedUser);
As seen above, the negate method will create a predicate that is the exact negation of the passed predicate, the and method will combine two predicates to create a new predicate that returns the result by performing a boolean and between two predicates. Similarly or method will perform an or operation between tow predicates.
Predicates have been used in the Collection class. Consider the removeIf method below:
public void testRemoveIf(List<User> users) {
int testAge = 24;
System.out.println("Number of users : " + users.size());
users.removeIf(user -> user.getAge() > testAge);
System.out.println("Number of users with age < " + testAge + " : " + users.size());
}
The compiler would be responsible for creating a predicate from the Lambda expression and then testing it on all the members of the passed collection. Those that pass it would be retained while the rest would be removed from it.

The Function Interface

$
0
0
I worked with using lambdas for Comparator interface in my first post on Lambdas. We saw how a Function interface could be used to convert an object of one type into an object of another.
This class is used to extract the compare key for the comparator method.
users.sort(Comparator.comparing(User::getAge));
The method reference will result in a Function instance that converts an User instance to a String instance (user.getAge())
The Collections framework  has utilized this concept in their computeIfAbsent method:
Below is a use case with the code
publicstatic void main(String[] args) {      
Function<Ticket, String> ticketKeyValFn = ticket -> ticket.toString();
Map<Ticket, String> ticketMap = new HashMap<>();
ticketMap.put(getSpecialTicket(), "For Purchase Operation");
Ticket ticket = new Ticket();
//this ticket is not in map
System.out.println("Map contains ticket " + ticket + " ? " + ticketMap.containsKey(ticket));
ticketMap.computeIfAbsent(ticket, ticketKeyValFn);
//now ticket is in map
System.out.println("Map contains ticket " + ticket + " ? " + ticketMap.containsKey(ticket));

}
The output is as below:
Map contains ticket T[1159190947] ? false
Map contains ticket T[1159190947] ? true
If I simply wanted to generate values for ticket than the code would be simply:
String value = ticketKeyValFn.apply(new Ticket());
Thus we have seen that Functions serve as transformers converting one type to another. We could also use it for more complicate operations once we combine this with streams.

There are also some more utility methods provided in the Function interface. Consider the compose method which allows us to apply another function to the input, and then applies the code inside our function.
Ticket ticket = new Ticket();

//This is the before function
Function<Ticket, String> ticketKeyValFn = ticketInstance -> ticketInstance.toString();
String sampleOpt = ticketKeyValFn.apply(ticket);
System.out.println("Output of first function is " + sampleOpt);

Function<String, Long> cleanTicketValFn = inp -> Long.valueOf(inp.substring(2,inp.length()-1));
Long no = cleanTicketValFn.apply(sampleOpt);
System.out.println("Output of second function is " + no);

//If using composites
//cleanTicketValFn = the one that executes last - gets a string and returns a Long
Function<Ticket, Long> compositeFn = cleanTicketValFn.compose(ticketKeyValFn);
System.out.println("The ticket Id is " + compositeFn.apply(ticket));
The output:
Output of first function is T[617901222]
Output of second function is 617901222
The ticket Id is 617901222
As can be seen, I have defined two functions - one that transforms a Ticket into a String and the second that transforms a String into a Long. Using compose we have combined the two.
The transformation is as below:
input : Ticket
Stage 1: Ticket -> String ( the 'before' function)
Stage 2: String -> Long
output: Long
There is also a way to combine, by adding a post function - or a function that runs after our function:
Function<String, Long> toLongFunction = string -> Long.valueOf(string);
Function<Long, Integer> toIntegerCutoffFunction = longVal -> longVal.intValue();
//Now combine the two
Function<String, Integer> compositeFunction = toLongFunction.andThen(toIntegerCutoffFunction);
// Test call
System.out.println(compositeFunction.apply("1234"));
Here first the String -> Long transformation happens and then Long -> Integer transformation.

Variations in java functions

$
0
0
In the previous post, we saw the Function Interface that takes an object of one type and transforms it into an object of another type. Consider the user case where we need to split a string and retain only the second half.

publicstatic void testSingleType() {
String testInp = "Hello";
Function<String, String> splitter = inp -> inp.substring(inp.length() / 2);
System.out.println(splitter.apply(testInp));// will print 'llo'
}
As seen here while we transformed the string content/value, we did not really transform from one type to another. The two type parameters here is kind of redundant. Java 8 has provided for this case by introducing the UnaryOperator class:
From the java docs:
Represents an operation on a single operand that produces a result of the
same type as its operand. This is a specialization of Function for
the case where the operand and result are of the same type.
Accordingly the updated class is :
publicstatic void testSingleType() {
String testInp = "Hello";
UnaryOperator<String> unaryStyleSplitter = inp -> inp.substring(inp.length() / 2);
System.out.println(unaryStyleSplitter.apply(testInp));// will print 'llo'
}
There is also a three parameter version of Function - one that takes two parameters and transforms that to a singe output.
BiFunction<String, Character, Long> cleanNumber = (value, character) 
-> Long.valueOf(value.replaceAll(character+"", ""));
System.out.println(cleanNumber.apply("123#45#12", '#'));
//will print 1234512

The BiVersions of Java FunctionalInterfaces

$
0
0
In the previous post, we saw a version of the Function class that took two parameters into apply instead of one. Similar to the BiFunction interface, we also have BiPredicate, BinaryOperator, and BiSupplier .
The BiPredicate Interface allows for a test on two arguments. Consider the example:
BiPredicate<Integer, String> isEqual = (no, str) -> (no + "").equals(str);
System.out.println("Is True ? " + isEqual.test(45, "45.06"));
This class now performs the test on the two passed input type, checking if the integer value and string value are same.
Next is the BinaryOperator class - this is the binary version of the UnaryOperator. The class is
a type of BiFunction and thus exposes the apply method for two input arguments, which it transforms into an output, and all the three are of same type.
I tried out the class:
BinaryOperator<String> appendOperator = (a1, a2) -> a1 + "" + a2;
System.out.println("Strings " + "Hi &" + " Robin "
+ "on combining results in "
+ appendOperator.apply("Hi", "Robin"));
The lambda expression results in the two strings being summed to give a new string.
There are also two static methods available here that provide for easy min max code between two aguments.
Comparator<Ticket> ticketComparator = Comparator.comparing(t -> t.toString());
BinaryOperator<Ticket> maxOperator = BinaryOperator.maxBy(ticketComparator);
BinaryOperator<Ticket> minOperator = BinaryOperator.minBy(ticketComparator);

Ticket t1 = new Ticket();
Ticket t2 = new Ticket();
System.out.println("T1 is " + t1 + " and T2 is "+ t2);
System.out.println("The max of two tickets is " + maxOperator.apply(t1, t2));
System.out.println("The min of two tickets is " + minOperator.apply(t1, t2));
The output of the code is as below:
T1 is T[1804094807] and T2 is T[951007336]
The max of two tickets is T[951007336]
The min of two tickets is T[1804094807]
Next is the BiConsumer class which is a binary version of the Consumer class we have seen earlier.
{
Map<Ticket, String> ticketMap = new HashMap<>();
BiConsumer<Ticket, String> mapAdder = (ticket, opName) -> {
ticketMap.put(ticket, opName);
};

mapAdder.accept(t1, "This is for Entry");
mapAdder.accept(t2, "This is for Exit");
System.out.println("No of entries in map are " + ticketMap.size()); //returns 2
You dont have a BiSupplier class - you cant return more than one item in java.

Interfaces and Default Methods

$
0
0
In an earlier post, I covered one of Java 8's new features - default methods in interfaces. While I did the pros there were some corner cases to be considered when implementing this feature.
Consider that we have an interface A:
interface A {
defaultString display(String s) {
return s.toLowerCase()+"__";
}
}
Now I have another interface B that extends from A:
interface B extends A {
defaultString display(String s) {
return s.toLowerCase()+"%%";
}
}
It also overrides the method. Now I added a third interface C, that extends from B. Since C extends from both A and B, which implementation of the method display() will C receive ?
publicstatic void main(String[] args) {
System.out.println(new C(){}.display("HELLO"));
}

interface C extends B {
//which version will display() execute ?
}
If I run the main method then the output is as below:
hello%%
This makes sense, as B is the closest to C in the chain, and therefore C gets the method implementation provided by B. This is same as the rules seen with class inheritance. Now consider the below scenario:
interface A {
defaultString display(String s) {
return s.toLowerCase()+"__";
}
}

interface B {
defaultString display(String s) {
return s.toLowerCase()+"%%";
}
}

interface C extends A, B {
//which version will display() execute ?
}
This is a multiple inheritance scenario. Both A and B are independent interfaces and C extends both of them. Which version will C inherit now ?
The above code will throw a compiler error -

Duplicate default methods named display with the parameters (String) and (String)
are inherited from the types B and A

The fix to the issue is to tell the compiler which version of the method must C inherit (This is kind of different to what is done in C++ , where we see the scope resolution operator :: or use explicit casting)
interface C extends A, B {

@Override
defaultString display(String s) {
returnA.super.display(s);
}
}
Here we have to override the display() method and delegate the call to one of the interfaces. On running the code now:
hello__
This is also a new use case for the super keyword - something that we have only associated with classes till now.

Primitives and Lambdas

$
0
0
II have been playing with the lambda expressions and they have all involved working with objects. There are also some classes in the SDK that have been created for working with primitives.
I tested some of the primitive classes supported here :
BooleanSupplier booleanSupplier = () -> Boolean.FALSE;
// will always return the value as false
System.out.println(booleanSupplier.getAsBoolean());
DoubleBinaryOperator sumDouble = (a, b) -> (a + b);
System.out.println(2.56 + " + " + 13.45 + " = " + sumDouble.applyAsDouble(2.56, 13.45));
final int[] val = new int[1];
IntConsumer numberSink = num -> val[0] = num;
numberSink.accept(1); // val[1] = 1
numberSink.accept(2);// val[1] = 2
numberSink.accept(3);// val[1] = 3
LongFunction<String> longToString = l -> l + "";
System.out.println(longToString.apply(189L));
LongPredicate longPredicate = l -> l != 0;
System.out.println("A non zero number ? " + longPredicate.test(134l));
The output is as below:
false
2.56 + 13.45 = 16.009999999999998
189
A non zero number ? true
  1. There is a BooleanSupplier class that supplies boolean values. We also have int,long and double versions of the same.
  2. There is a DoubleBinaryOperator that is summing two double primitives. We also have int and long versions of the same.
  3. We created an IntConsumer which simply consumes all passed int values. I also observed a double and long version of the class.
  4. We used a LongFunction that takes as input long value and transforms it into a different type. We have double and int versions of it.
  5. Last I created a LongPredicate that is used to run tests on long values. We also have int and double versions of these.
There are also some converters that have been provided, eg for double class we have the DoubleToInt  and DoubleToLong interfaces.
DoubleToIntFunction dblToIntFunction1 = l -> ((int)(l*100)) +2;
DoubleToIntFunction dblToIntFunction2 = l -> ((int)(l)*100) +2;
System.out.println(dblToIntFunction1.applyAsInt(10.98));//1100
System.out.println(dblToIntFunction2.applyAsInt(10.98));//1002
There are similar classes for converting from primitive long and  int to double value. And for converting primitive double and int to long values.
IntUnaryOperator intSquarer = i -> i * i;
System.out.println("Square of 10 is " + intSquarer.applyAsInt(10)); // 100

List<String> contents = new ArrayList<>();
ObjIntConsumer<String> inserter = (term, times) -> 
{
for (int i = 0; i < times; i++) {
contents.add(term);
}
};
inserter.accept("Test", 3);// add the string 3 times to the list
ToIntBiFunction<String, Short> computeLength = (str, shortNo) -> str.length() + shortNo;
System.out.println("The value is " + computeLength.applyAsInt("Hello ", newShort((short) 25)));
  1. The IntUnaryOperator will be used to apply a transformation on an int value to return an int value. 
  2. The ObjIntConsumer consumes a Object and a primitive int value. Similary there are ObjDoubleConsumer and ObjLongConsumer for double and long primitive values respectively.
  3. The BiFunction interface has also got an int, long and double variotion where the function transforms the input arguments into primitive values.

Be careful with Sets and equals behavior

$
0
0
I was recently assigned to fix a bug in the code. For the scope of this post I re-framed the problem as below:
Context: There is a set of objects of each type (lets say fruits). Over a series of operations different type of fruits are added to the set. In certain stages certain fruits may be replaced by others in the family. For example during stage 4, we may need to replace a green apple by a red apple. On the final page we display all the fruits selected.
Problem: While the various fruits show up, the changes are not seen. So if we had updated the apple to be a red one, the final result still displays a green apple.
If the code for the same were to be written out I would have a fruit class:
publicclass Fruit {

privateString name;
privateString color;
privateint cost;

@Override
publicint hashCode() {
finalint prime = 31;
int result = 1;
result = prime * result + ((name == null) ? 0 : name.hashCode());
return result;
}

@Override
publicboolean equals(Object obj) {
if (this == obj)
returntrue;
if (obj == null)
returnfalse;
if (getClass() != obj.getClass())
returnfalse;
Fruit other = (Fruit) obj;
if (name == null) {
if (other.name != null)
returnfalse;
} elseif (!name.equals(other.name))
returnfalse;
returntrue;
}

// setter getters here
}
As seen here fruits are unique by their name. Thus the set can only hold one fruit of each type.
Now the main code, where we modify the set in stages
publicstatic void main(String[] args) {
Set<Fruit> fruitBasket = new HashSet<>();
//Stage 1----------
fruitBasket.add(new Fruit("Apple", "green", 3));
fruitBasket.add(new Fruit("Watermelon", "green", 11));

//Stage 2-------------------
//replace a green apple by a red one
fruitBasket.add(new Fruit("Apple", "red", 3));
fruitBasket.add(new Fruit("Mango", "yellow", 5));

//Final Display
System.out.println("The fruits in the basket are");
for (Fruit fruit : fruitBasket) {
System.out.println(fruit.getColor() + " colored " + fruit.getName() + " costs " + fruit.getCost());
}

}
If I were to run this code I would expect to see red Apple, yellow Mango and a red watermelon.
The fruits in the basket are
green colored Apple costs 3
green colored Watermelon costs 11
yellow colored Mango costs 5
There's been a mistake ! I expected a red Apple not a green one. What was the bug in the above code ??

If we look at the behavior of the Set's add method then:
public boolean add(E e) {
return map.put(e, PRESENT)==null;
}
This seems fine. We know that the HashSet internally uses a Map. The add method tries to add the passed Fruit as a key entry in the underlying map. But if we look at the documentation
Adds the specified element to this set if it is not already present (optional operation).
More formally, adds the specified element e to this set if the set contains no element
e2 such that (e==null ? e2==null : e.equals(e2)). If this set already contains the
element, the call leaves the set unchanged and returns false.

This is where things get interesting. Two Fruits are same if they have the same name. Thus for the set, the Green Apple and the Red Apple are identical. The Set will ignore the add request for the Red Apple and hence we see the Green Apple in the result.To fix the same, I changed the add code as below :
publicstatic void main(String[] args) {
Set<Fruit> fruitBasket = new HashSet<>();
//Stage 1----------


//fruitBasket.add(new Fruit("Apple", "green", 3));
add(fruitBasket, new Fruit("Apple", "green", 3));
//fruitBasket.add(new Fruit("Watermelon", "green", 11));
add(fruitBasket, new Fruit("Watermelon", "green", 11));
//Stage 2-------------------
//replace a green apple by a red one
// fruitBasket.add(new Fruit("Apple", "red", 3));
add(fruitBasket, new Fruit("Apple", "red", 3));
// fruitBasket.add(new Fruit("Mango", "yellow", 5));
add(fruitBasket, new Fruit("Mango", "yellow", 5));
//Final Display
System.out.println("The fruits in the basket are");
for (Fruit fruit : fruitBasket) {
System.out.println(fruit.getColor() + " colored " + fruit.getName() + " costs " + fruit.getCost());
}
}

publicstatic void add(Set<Fruit> fruitBasket, Fruit crtFruit) {
if(fruitBasket.contains(crtFruit)) {
fruitBasket.remove(crtFruit);
}

fruitBasket.add(crtFruit);
}
This code will now work fine. We will first remove the Apple if it exists before adding the new/modified apple. The output on running is
The fruits in the basket are
red colored Apple costs 3
green colored Watermelon costs 11
yellow colored Mango costs 5
This is interesting when you compare with Maps. In the map if a key exists, then its value field is updated with the parameters in the current request. But not for sets. You need to explicitly remove and add again.

Servlet 3.x - time to have a look

$
0
0
Its not brand new - in fact it first appeared in December 2009. Since starting to use servlets (actually servlet 2.5) I never really bothered to look at the specs. What started with application development using MVC and having multiple servlets changed to Spring framework with a single controller and lot of JSPs and then less JSPs and more AJAX. Along the way, my web.xml changed to refer to 3.0 and then 3.1.
I even used certain features without really exploring the specs. So now when the world moves towards newer and more powered client side technologies like Angular and mobile development and HTML 5, I decided to take a pause and look at the Servlet specs.
Servlet 3.0 specs came to be in 2009. As of today - 2015, we have Servlets 3.1. I decided to go over and check out the new features in these specs.
The first one I decide to try was the web-fragment feature.
In the Pre Servlet 3.0 days, the web.xml was the single point for all servlets, filters and other web.xml definitions. So if you were using a framework say something like SpringMVC, than you needed to update the web.xml of your project with definitions for the Spring Front Controller. So if the frameworks like Spring MVC, Spring Security, Struts etc start providing jars with appropriate fragments, then we can directly include them in the war and execute the code.
Consider the filter I created here:
publicclass SimpleFilter implements Filter {

@Override
publicvoid init(FilterConfig arg0) throws ServletException {
System.out.println("SimpleFilter initialized");
}

@Override
publicvoid doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain)
throwsIOException, ServletException {
String requestUrl = ((HttpServletRequest) servletRequest).getRequestURL().toString();
System.out.println("url trapped in SimpleFilter " + requestUrl);
filterChain.doFilter(servletRequest, servletResponse);
return;

}

@Override
publicvoid destroy() {
System.out.println("SimpleFilter destroyed");
}
}
The definition for the filter was placed in a separate XML file:
<?xmlversion="1.0"encoding="UTF-8"?>
<web-fragmentxmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
xmlns:webfragment="http://java.sun.com/xml/ns/javaee/web-fragment_3_0.xsd"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
 http://java.sun.com/xml/ns/javaee/web-fragment_3_0.xsd"
id="SimpleFilterWebFragment" version="3.0">

<filter>
<filter-name>SimpleFilter</filter-name>
<filter-class>com.simple.SimpleFilter</filter-class>
</filter>

<filter-mapping>
<filter-name>SimpleFilter</filter-name>
<url-pattern>*.jsp</url-pattern>
</filter-mapping>

</web-fragment>
As seen here we have a root level xml-fragment element which holds the details of the filter. The web-fragment element is in fact is near identical to the web-app element.
The next step is to bundle it into a jar:
The jar structure

The next step is to add the jar to a web application.

Within the web application I placed a similar filter:
publicclass LocalFilter implements Filter {

@Override
publicvoid init(FilterConfig arg0) throws ServletException {
System.out.println("LocalFilter initialized");
}

@Override
publicvoid doFilter(ServletRequest servletRequest, ServletResponse servletResponse,
FilterChain filterChain) throwsIOException, ServletException {
String requestUrl = ((HttpServletRequest) servletRequest).getRequestURL().toString();
System.out.println("url trapped in LocalFilter " + requestUrl);
filterChain.doFilter(servletRequest, servletResponse);
return;

}

@Override
publicvoid destroy() {
System.out.println("LocalFilter destroyed");
}
}
Thus the application has two filters:
  1. SimpleFilter - This filter is placed within the jar and loads based on the web-fragment.
  2. LocalFilter - This filter is defined in the web app and loads based on the web.xml
If we start the server:
LocalFilter initialized
SimpleFilter initialized
INFO: Server startup in 1076 ms
As seen here first the LocalFilter instance defined in web.xml was loaded and read. The Servlet Container than scanned the jars in the project looking for any web-fragments. It detected and loaded the SimpleFilter instance. On sending a request for index.jsp
url trapped in LocalFilter http://localhost:8080/WebFragments/index.jsp
url trapped in SimpleFilter http://localhost:8080/WebFragments/index.jsp
What if we need this sequence changed ? The Oracle blogs has a good entry which discusses on the ordering behavior. It talks about the absolute-ordering element used in web.xml and ordering element in the fragment file.
In the next post I shall look at other new features.
Viewing all 231 articles
Browse latest View live