The performance of a dynamic web application can be quite unpredictable when you don’t know how many People are going to use it and of course how they are going to use it. You can predict some of the users’ behaviours and do your best to prepare your application to cope with the expected load. But sometime after the launch you suddenly find yourself in a situation where the performance of your application – or at least parts of it - is poor and the customer is unhappy. In cases like this typical reasons why you didn’t find the bottlenecks while developing are:

• more users than expected
• unpredicted behaviour of users
• the amount of data stored in the production database compared to the development database
• missing indexes on database tables
• slow queries that need optimizing besides adding indexes
• inefficient code
• insufficient resources on the servers

So before using the brute force approach by throwing more hardware at the problem, the most important task to accomplish first is to find the real bottleneck. Just like in war times good intelligence is your key to winning the battle. So gather information first.

• Find out which parts are slow and under which circumstances the performance is poor. Talking to actual users of the app is a good approach to do so.
• Monitor the servers using tools like munin. CPU and Memory usage can show you which part of the architecture is stressed. In some cases the webserver idles most of the time, while the database machine is under heavy load
• Check the Database for slow queries – most databases provide tools and statistics to determine the slowest queries. This will help you find out where indexes might be missing or which queries need optimizing
• When optimizing queries first look at the explain plan. It can point you to the problem directly.
• Check the DB for the most frequently executed queries – if a query is executed very often this might be a place where the application code needs optimizing or where caching might help. Add caching wherever the same piece of infrequently changing data is shown to many users many times. A great roundup on open source Java cache implementations can be found here.
• Use a profiler to find out where in your code the application gets stuck. Chances are high that you end up locating the place where one of the slow queries is executed

Then start optimizing the parts you expect the most performance boost from. Be careful though when adding indexes on database tables with huge amounts of data – check if there is sufficient disk/table space available on the RDBMS. Indexes can also have effects on the performance of write operations.

If all this brain doesn’t solve the problem you might need to add muscle to your hardware:

• Scale vertically by adding RAM and increasing CPU
• Scale horizontally by adding machines, which is a lot easier for web servers as it is for database servers
• An excellent piece of literature on this topic is “Scalable Internet Architectures” by Theo Schlossnagle

Take care…

Alex

I spend about one hour a day commuting between my office and my home by car. During drive-time I used to listen to the radio a lot. Someday the flooding of air time with commercials drove me nuts, so I decided to change something about that. After having spent some time with podcasts I found out that audiobooks are the better alternative for me: they are professionally produced,  you can read the reviews on them on amazon before you buy them and they usually - when selected wisely - contain less crap than some podcasts (I am a podcaster myself (www.de-lite.de), and I’m sure many people consider my productions crap, too). The downside is - audiobooks are not free

Anyway - just recently I downloaded and listened to “Getting Things Done: The Art of Stress-Free Productivity ” by David Allen, a book everybody I know obviously had already read, except for me. When I first heard of GTD a couple of years ago, what first came to my mind was some kind of car, a Gran Turismo with a Diesel engine or something like that. Then I learned it was a methodology for self management. Some people I know claim to be using GTD as a method for self-management. Now, looking at some of their desktops (both digital and analog) I know they aren’t.

Many poeple misunderstand GTD simply as using a specific digital tool to store and manage all their todos. In fact it is more of a methodology or even a philosophy on how to organize both your professional and private life. And the toolset for GTD can vary from person to person. In order to be doing GTD of course you need tools, but those can be completely analogue. A printed calendar, a stack of paper, a pen and a bunch of file folders will suffice for some people if used properly.

So what is this GTD thing all about? I’ll try to outline the basic steps of the methodology in my own humble words (and probably missing a lot of points - sorry in advance!):

1) Collect

• collect your thoughts, what’s bothering you at the moment, get it all out of your brain and transfer it to a physical system (e.g. a sheet of paper). When doing this for the first time, it will probably take you hours to finish that process. The good news is that once you’ve got all of your projects and tasks together, adding the additional stuff that pops up each day won’t cost you a lot of time
• stuff like email does not need to be collected. It’s already stored in your inbox. It just has to be processed - and that’s easier than you might think. Before I started following the rules hundreds of emails were lurking around in my inbox.
• the aim is to free your “psychic RAM” from all that’s in there. This will in turn free a lot of energy and take away stress from you. “It’s a lot easier to relax when you know what you’re NOT doing at the moment” is a proverb David Allen uses a lot. And as I experienced this is so true.

2) Process

• look at everything that you’ve collected and decide what you have to do about it
• some of the items are actionable and won’t take longer than two minutes, e.g. “answering that email” or suchlike. Do those immediately and get rid of them
• in case the item is not actionable, consider if it might be actionable someday maybe -> put those on a separate list
• in case the item is not actionable, consider if it might be worth keeping -> throw it away if not, otherwise file it

3) Organize

• assign the actionable items to projects, groupe them by context (Telephone, Office, Home, En route), project, time and energy needed
• if an action needs to be processed on a specific point of time or day in the future, add a reminder to your calendar
• by the way: projects are things where more than one action is needed to accomplish them.

4) Review

• Review your lists every day
• look into your calendar first thing in the morning, so that you know what definitely has to done that day
• Every week reserve about two hours time for reviewing and rethinking your projects, lists, tasks and thoughts from different angles (also the big picture)

5) Do

from the list of actions chose the best one to be done next. This can vary depending on:

• the context: are you online? have you got access to a phone? are you in your car for making errands? etc.
• time available: if you’ve got a ten minutes time slot before a meeting, choose a task you can finish within that amount of time
• energy: if you’re tired, choose a no-brainer, otherwise something more sophisticated.

The audiobook cost me 30 bucks and it was one of my best investments ever. I’ve been following the rules for a week now and my impression is, it seems to be working fine for me. For the first time in years the number of emails in my inbox is 0, NULL, zip, nada. Pure joy! It even made me start this blog - something I’ve been thinking about for months but never got started with.

In one of my next posts I’ll write about my GTD toolset and how I work with it every day.

Take care,
Alex

David Allens website
The book on amazon
The audiobook
about GTD on wikipedia

PS: feel free to leave a comment, if you liked this post. If not, please leave a comment, too and let me know where I can improve. I appreciate it!