Monday, February 11, 2013

Map Reduce Simplified

Yes it is about parallel and distributed computing, there are tonnes of web pages, books articles, diagrams etc. etc with nice buzz words to talk about Map Reduce, here is the most simplified explanation.

Lets take a real life example.

1. Company CEO called all Program Manager's "I need total effort spent this month by noon". Program Manager's no problem sir. Why are Program Manager's not worried because they are going to distribute task :-)

2. Each Program Manager called their project manager asking for effort spent so far.

3. Each Project Manager pulled up effort sheet and provided it to their Program Managers.

4. Program Managers complied received sheet into one file and sent it to CEO.

5. Company CEO collated all the sheets and calculated total effort spent.

Each individually broke its task to smaller tasks (Mapped its input task to smaller tasks), Program Manager was required to provide effort spent, he mapped his task to smaller tasks, this is MAP.

Program Manager's on receiving data from their project managers compiled it back to single output, this is REDUCE.

Now lets zoom out and summarize how Map Reduce applies to distributed and parallel computing. Each node distributes its task to smaller tasks(Maps its given task). Each node receive results, combine them(REDUCE) to generate required output.


Sunday, January 27, 2013

Apache Roller Getting Started

If you are thinking to setup a blog website then apache roller is the quickest and simplest thing to do so, below are steps for getting started, though these instructions are present in documentation provided by apache, but hopefully following steps will make it very easy for you.

Here you Go !!

1. Download apache roller

http://roller.apache.org/download.cgi#roller50

2. Install Tomcat

http://tomcat.apache.org/download-70.cgi

3. Download mysql

http://dev.mysql.com/downloads/

4. Unzip apache roller zip file and copy roller-5.0.1-tomcat.war from
\roller-weblogger-5.0.1-for-tomcat.zip\roller-weblogger-5.0.1-tomcat\webapp to tomcat webapps folder

5. Create mysql database, dont worry about schema, that will be automatically created later.

mysql -u root -p
password: *****

mysql> create database rollerdb;
mysql> grant all on rollerdb.* to scott@'%' identified by 'tiger';
mysql> grant all on rollerdb.* to scott@localhost identified by 'tiger';

Otherwise you will get following error on webpage:

[com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'rollerdb'


6. Create a file roller-custom.properties in tomcat lib folder, here is default content:

installation.type=auto
mediafiles.storage.dir=/usr/local/rollerdata/mediafiles
search.index.dir=/usr/local/rollerdata/searchindex
log4j.appender.roller.File=/usr/local/rollerdata/roller.log
database.configurationType=jdbc
database.jdbc.driverClass=com.mysql.jdbc.Driver
database.jdbc.connectionURL=jdbc:mysql://localhost:3306/rollerdb?
autoReconnect=true&useUnicode=true&characterEncoding=utf-8&mysqlEncoding=utf8
database.jdbc.username=scott
database.jdbc.password=tiger
mail.configurationType=properties
mail.hostname=smtp-server.example.com
mail.username=scott
mail.password=tiger

If you dont add this file then you will get following error on startup:

Roller Weblogger: No customer properties found in classpath

7. Please download and copy jars for mail api and mysql jdbc drivers to lib directory of tomcat

mail.jar
mysql-connector-java-5.1.22-bin.jar

I came across following error and copying mail.jar to lib directory fixed it.

SEVERE: Error listenerStart
Jan 28, 2013 11:03:29 AM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/roller-5.0.1-tomcat] startup failed due to previous errors
Jan 28, 2013 11:03:29 AM org.apache.catalina.loader.WebappClassLoader clearRefer
encesThreads
SEVERE: The web application [/roller-5.0.1-tomcat] appears to have started a thr
ead named [Abandoned connection cleanup thread] but has failed to stop it. This
is very likely to create a memory leak.

8. Start tomcat and open following url

http://localhost:8080/roller-5.0.1-tomcat/

9. If you have followed all the steps, you should see following screen:


10. Click on "Yes - create tables now", once the tables are created you will see following confirmation message.

Database tables were created successfully as you can see below.
Database tables are present and up-to-date. Click here to complete the installation process and start using Roller.

Click on "Here"

11. You should now see welcome message

"Welcome to Roller"

12. Follow on screen steps to create a user and blog.

Hopefully you will find it very simple to setup and have fun setting up your blog site in few minutes.