In the last post I wrote about ways to tackle bottlenecks and to increase the performance of an application. When dealing with a high traffic application servicing tons of concurrent users all requesting the same data, intelligent caching can boost the speed in which requests are processed. As a result, lots of load is taken away from the database.

Querying a database makes much sense when the data requested changes a lot. But imagine the database is backing a blog or news website where the data requested from the database is the same in hundreds or even thousands of cases. The most frequent query sent to the database in such an environment is probably for loading the article on page one, which - once published - probably won’t change a lot. This doesn’t hurt the Database if your private blog has 100 unique visitors per day, but if the frontpage of a popular news website is attacked by 100.000 unique visitors per hour then you are in trouble. You need to use caching in order to take away stress from your database server.

An excellent, widely used open source cache solution which is easy to use and yet powerful is OSCache (http://www.opensymphony.com/oscache/). Besides features like clustering, a servlet filter for caching entire JSP pages or persistent caching it comes with a JSP tag library that enables you to include caching into your jsp based web application so easily it hurts.

All you have to do is:
1)       Add the OSCache jar file to your classpath, put the tld file into the WEB-INF directory of the webapp and declare the OSCache taglib in your jsp page:

2)       mark portions of the jsp-page to be cached in your jsp page with cache tags:

In this example the cached portion of the page is stored under the key defined in the variable cacheKey, it supports multiple languages and expires after 1800 seconds. There are a few other attributes available for configuring the cache tag. More information can be found here http://www.opensymphony.com/oscache/wiki/JSP%20Tags.html

A cache with an invalidation timeout as seen in the above example is great for many situations, but sometimes you need full control on when a cache entry needs invalidating. OSCache offers an API to manage the cache in your Java code. This is an example of how to invalidate it in a Java web application:

Piece of cake!

Take care,

Alex

PS: OSCache is under the OpenSymphony Software License, Version 1.1 which is derived and fully compatible with the Apache Software License.

The performance of a dynamic web application can be quite unpredictable when you don’t know how many People are going to use it and of course how they are going to use it. You can predict some of the users’ behaviours and do your best to prepare your application to cope with the expected load. But sometime after the launch you suddenly find yourself in a situation where the performance of your application – or at least parts of it - is poor and the customer is unhappy. In cases like this typical reasons why you didn’t find the bottlenecks while developing are:

• more users than expected
• unpredicted behaviour of users
• the amount of data stored in the production database compared to the development database
• missing indexes on database tables
• slow queries that need optimizing besides adding indexes
• inefficient code
• insufficient resources on the servers

So before using the brute force approach by throwing more hardware at the problem, the most important task to accomplish first is to find the real bottleneck. Just like in war times good intelligence is your key to winning the battle. So gather information first.

• Find out which parts are slow and under which circumstances the performance is poor. Talking to actual users of the app is a good approach to do so.
• Monitor the servers using tools like munin. CPU and Memory usage can show you which part of the architecture is stressed. In some cases the webserver idles most of the time, while the database machine is under heavy load
• Check the Database for slow queries – most databases provide tools and statistics to determine the slowest queries. This will help you find out where indexes might be missing or which queries need optimizing
• When optimizing queries first look at the explain plan. It can point you to the problem directly.
• Check the DB for the most frequently executed queries – if a query is executed very often this might be a place where the application code needs optimizing or where caching might help. Add caching wherever the same piece of infrequently changing data is shown to many users many times. A great roundup on open source Java cache implementations can be found here.
• Use a profiler to find out where in your code the application gets stuck. Chances are high that you end up locating the place where one of the slow queries is executed

Then start optimizing the parts you expect the most performance boost from. Be careful though when adding indexes on database tables with huge amounts of data – check if there is sufficient disk/table space available on the RDBMS. Indexes can also have effects on the performance of write operations.

If all this brain doesn’t solve the problem you might need to add muscle to your hardware:

• Scale vertically by adding RAM and increasing CPU
• Scale horizontally by adding machines, which is a lot easier for web servers as it is for database servers
• An excellent piece of literature on this topic is “Scalable Internet Architectures” by Theo Schlossnagle

Take care…

Alex

One part of reviewing and refactoring code is finding and removing code smells. An example, which I encounter quite often is the outmoded manner of coding Enumeration-like behaviour using int-constants. Or worse: String constants. The java.util.Calendar-class serves as a great example for this former best practice, before the Enumeration-Type was introduced with Java release 1.5.

Imagine you want to get a java.util.Date object pointing to the 1st of January 2009, midnight. This is one way of doing it:

This has been a clean way of implementing and “imitating” enum-like behaviour in pre 1.5 times and of course in projects that require a lower java version.

Since we’re passing ints to the set method, it is possible to do something like this, without having the compiler complain about it (seen it done before):

And due to the fact that the number of the months in the Calendar class starts with 0 this would create a Date object pointing to the 1st of February. Ouch.  An even more evil way of misusing the Calendar class could be this:

I’ve never seen this done before, but looking at it, I must say it really hurts.

Since the release of Java 1.5 there is a much better way of programming that kind of behaviour using the Enumeration type. This is how a simple Month enumeration could look like:

But we can add a little more brain to the implementation:

We’ll add member variables for the name, index and number of days, a constructor with arguments for each of these values and we’ll provide getters for retrieving them. Additionally we’ll add a method for calculating the correct number of days depending on a given year, considering leap years.

And here’s an example of how the enumeration can be used:

I warmly recommend the use of this great feature of the Java language. It definitely adds more type safety, ease of use and security to your application. But unfortunately it seems to me, that it still hasn’t reached the popularity that it really deserves, yet.

Whether you’re part of an agile team using methods like XP or Scrum, or processing your projects in a more classic way with RUP or even waterfall, in most cases you’re part of a team of developers. Chances that someday you’ll have to work with or (worse) debug code someone else has written are 100%. And who of us hasn’t complained about having to deal with “the mess” someone else has made. And probably some of our co-workers have been scratching their heads with both hands looking at some piece of code we’ve written after midnight with a numb mind in shipping hell. Nobody’s perfect.

So what can we do about this situation? Whining, complaining and pointing fingers won’t make the code any better.

Just recently I read “Clean Code” by Robert C. Martin. Zillions of blog entries were recommending this book, so I had to get it myself. I was probably one the last people on earth who hadn’t read it. It’s a great piece of programming literature. Reading it I found myself nodding my head a lot, agreeing with many of the points made and regretted I hadn’t read it earlier. There are some aspects I do not agree with 100% but they were very few.

One thing that impressed me a lot because of its simpleness is the “boy scout rule”: “Leave the campground cleaner than you found it”. When working with a piece of code, make your changes and commit it a little cleaner to your version control system. Let me give you an example.

I was debugging a class for creating financial charts. The following piece of code was scattered in it with slight variations about 20 times.

The reason I was debugging the class was that the font sizes within the charts did not look right when run on the production environment. Simple change made: remove the hard coded String and the numeric font size, and replace them with constants.

No big deal, but improving the readability and maintainability of the code a little bit. Imagine a whole team following this simple principle. More productivity, less rotten code, better team spirit. And the best thing about it: it’s absolutely free!

Happy coding!
Alex

Anyone who ever seriously started writing unit tests will sooner or later have come to the conclusion, that it helps isolating small testable chunks of code resulting in a clean structure of methods within a class and forcing the developer to write cleaner code. This is easy as long as the tested code does not work with data or configurations that might change depending on the environment in which the test are run.

My two favourite basic rules for writing tests (Thanks uncle bob!) are

• tests must run anywhere. (e.g. on a continuous integration server)
• tests must run fast, so they’re fun running them.

As soon as your tests depend on a database containing specific data or a file system with a certain directory structure you’re in trouble. In my experience many programmers write test for methods that are easy to write tests for, and avoid the more complicated cases requiring setting up test data and such like.

A really cool way for decoupling tests from dependencies like this, speeding up their execution time and finally leading to a much better code coverage are mock objects.

A mock library I really enjoy using is easymock. Let me show you an example of how easy easymock (http://www.easymock.org) really is.

First include this library to your classpath: easymock.jar

In case you want to mock classes that do not implement an interface, additionally add this library: easymockclassextension.jar

Now you can start writing your isolated test making use of mock objects

Add this include to your test class:

or this if you’re mocking interfaces:

This is how a test could look like (example code)

Mock Objects will definitely help you decoupling your tests from platform dependencies. They will make your tests faster and more fun to run.

Alex