Friday, July 31, 2009

GeoIP module installation with Apache from source

Here we will talk about installing and configuring free GeoIP from source for your Apache2.x and on RedHat servers. I normally install both GeoIP API as well as GeoIP module for Apache to make sure that, it works with Apache as well as i can query GeoIP database from command prompt also. Here are those two steps.

First - To install GeoIP API you need to download source and compile it, as well as you need to install GeoIP database file. The GeoIP data file will be used by Apache module also. Here are few commonds which you need to run to install the GeoIP API.

as you know already that I use to install everything in /usr/local so

$ cd /usr/local
$ wget http://www.maxmind.com/download/geoip/api/c/GeoIP.tar.gz
$ tar -zxvf GeoIP.tar.gz
$ cd GeoIP-1.4.4/
$ ./configure
$ make clean
$ make
$ make install

Trust me this will install the GeoIP API in one shot. :)

Now its time to install free GeoIP database file. So

$ cd /usr/local/share/
$ mkdir GeoIP
$ cd GeoIP
$ wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
$ tar -zxvf GeoLiteCity.dat.gz

Second - To install GeoIP module for the Apache you need to download the module source and compile it. Here are few commands which you need to run. Note that there is pre-requisit of Apache already installed on the server. Maxmind has Apache modules availible for both Apache 1.x and 2.x. You can refer http://geolite.maxmind.com/download/geoip/api/mod_geoip/ for Apache 1.x or http://geolite.maxmind.com/download/geoip/api/mod_geoip2/ for Apache 2.x module source. As I have Apache 2.x installed on my server so I have used second link.

$ cd ..
$ wget http://geolite.maxmind.com/download/geoip/api/mod_geoip2/mod_geoip2_1.2.5.tar.gz
$ tar -zxvf mod_geoip2_1.2.5.tar.gz
$ cd mod_geoip2_1.2.5/
$ /usr/local/apache2/bin/apxs -i -a -L /usr/local/GeoIP-1.4.4/libGeoIP -I /usr/local/GeoIP-1.4.4/libGeoIP -l GeoIP -c mod_geoip.c

Rewrite various paths in last line as per you apache insallation.

Now you need to check and configure your Apache. So open your httpd.conf file and find LoadModule string in it, most likely you get some thing like

LoadModule geoip_module modules/mod_geoip.so

if above line is not written in your apache configuration, paste it after others LoadModule lines.

Now paste follwing line also

GeoIPEnable On
GeoIPDBFile /usr/local/share/GeoIP/GeoLiteCity.dat


Now just restart your apache and your are done.

If you want to test that whether GeoIP module is install or not. Write a php script as



when you run this, it will output the array as

Array
(
[GEOIP_CONTINENT_CODE] => AS
[GEOIP_COUNTRY_CODE] => SG
[GEOIP_COUNTRY_NAME] => Singapore
[GEOIP_REGION] => 00
[GEOIP_CITY] => Singapore
[GEOIP_DMA_CODE] => 0
[GEOIP_AREA_CODE] => 0
[GEOIP_LATITUDE] => 1.293100
[GEOIP_LONGITUDE] => 103.855797
.
.
.
[REQUEST_TIME] => 1249023542
[argv] => Array
(
)

[argc] => 0
)

Note that now it has few " GEOIP_ " variables in the $_SERVER array.

One last and most trickest thing, that how to keep on updating this GeoIP database file. I will teach you how to do with free and non-free versions also.

For free you can simply write a shell script as

$ vim /usr/local/share/GeoIP/geoipupdate.sh

paste following lines into it

#!/bin/sh
cd /usr/local/share/GeoIP
wget -c http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
tar -zxvf GeoLiteCity.dat.gz

Change mode of file as

$ chmod 777 geoipupdate.sh

Now add the script into cron, do

$ crontab -e

and paste following line into it. This will update your geodatabase file every night at 1 AM.

1 0 * * * /usr/local/share/GeoIP/geoipupdate.sh

Here you are done with free GeoIP setup and you have already written a cron to update the free GeoIP database.

For paid version it almost same except update cron and different database files. In case of paid version you will have file name GeoIP.dat rather than GeoLiteCity.dat. This is how I am updating my GeoIP database files for paid version.

After Installtion you have to edit your GeoIP.conf file -

$ vim /usr/local/etc/GeoIP.conf

# If you purchase a subscription to the GeoIP database,
# then you will obtain a license key which you can
# use to automatically obtain updates.
# for more details, please go to
# http://www.maxmind.com/app/products

# see https://www.maxmind.com/app/license_key_login to obtain License Key,
# UserId, and available ProductIds

# Enter your license key here
LicenseKey [Enter your license key here]

# Enter your User ID here
UserId [Enter your User ID here]

# Enter the Product ID(s) of the database(s) you would like to update
# By default 106 (MaxMind GeoIP Country) is listed below
ProductIds 106

Now create a directory logs, this will store all update logs.

$ mkdir /usr/local/GeoIP-1.4.4/logs/

now we need a shell script geoipupdate.sh to update database file.

$ vim /user/local/share/GeoIP/geoipupdate.sh

#!/bin/bash
yyyymmdd=` date +%Y%m%d`
HOME_DIR=/usr/local/GeoIP-1.4.4
LOG_DIR=$HOME_DIR/logs
start=` date +'%Y%m%d %H:%M:%S'`
echo Start update geoip [$start] >> $LOG_DIR/update$yyyymmdd.log
$HOME_DIR/apps/geoipupdate >> $LOG_DIR/update$yyyymmdd.log
end=` date +'%Y%m%d %H:%M:%S'`
echo End update geoip [$end] >> $LOG_DIR/update$yyyymmdd.log
echo "***********" >> $LOG_DIR/update$yyyymmdd.log

You can setup the cron, as we did for free verison to run at 1 AM every night.

1 0 * * * /usr/local/share/GeoIP/geoipupdate.sh

Monday, July 27, 2009

Quick MySQL installation tutorial

Updated as per log date and for redhat 32/64 bit installation.

1. Login as root and change directory to /usr/local or where you want to install the mysql. I normally install mysql on /usr/local.

$ cd /usr/local

2. Download the mysql. This is the example of mysql version 5.1.36. You can always navigate in mysql site to find the latest version and paste it after wget.

$ wget http://dev.mysql.com/get/Downloads/MySQL-5.1/mysql-5.1.36.tar.gz/from/ftp://ftp.oss.eznetsols.org/mysql/

3. Untar the archive.

$ tar -zxvf mysql-5.1.36.tar.gz

4. Go to the directory

$ cd mysql-5.1.36

5. Run configure command

$ ./configure --prefix=/usr/local/mysql --with-unix-sock-path=/tmp/mysql.sock --with-charset=utf8

6. Make and install

$ make
$ make install

7. Copy configuration to /etc folder.

$ cp support-files/my-medium.cnf /etc/my.cnf

8. Setup auto startup.

$ cp support-files/mysql.server /etc/init.d/mysql
$ chmod 755 /etc/init.d/mysql
$ chkconfig --add mysql
$ chkconfig --level 35 mysql on

9. Add mysql user and group for permission.

$ groupadd mysql
$ useradd -g mysql mysql
$ cd /usr/local/mysql/bin/

10. Install default database.

$./mysql_install_db

11. Change var directory ownership to mysql, so that mysql can write to database.

$ cd ..
$ chown -R root .
$ chown -R mysql var
$ chgrp -R mysql .

12. Starting mysql and testing startup.

$ cd bin/
$./mysqld_safe &
$ ps -ef | grep mysql
$ service mysql stop
$ service mysql start

MySQL (my.cnf) which I am using is as following. My servers are quite higher end servers (quad core, 16Gigs, redhat 64bit installation). You can always change settings as per your server configuration to optimize.

[client]
port = 3306
socket = /tmp/mysql.sock

[mysqld]
safe-show-database
old_passwords
back_log = 75
skip-innodb
key_buffer = 192M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 2000
thread_cache_size = 384
tmp_table_size = 64M
max_heap_table_size = 64M
max_allowed_packet = 64M
max_connect_errors = 10
read_rnd_buffer_size = 512K
bulk_insert_buffer_size = 8M
query_cache_limit = 10M
query_cache_size = 100M
query_cache_type = 1
query_prealloc_size = 163840
query_alloc_block_size = 32768
default-storage-engine = MyISAM
port = 3306
socket = /tmp/mysql.sock
skip-locking
max_connections=8000
max_user_connections=8000

skip-networking
log-bin=mysql-bin

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M

[mysqlhotcopy]
interactive-timeout

[mysqld_safe]
nice=-20
open_files_limit = 8192

Monday, July 20, 2009

Few useful Linux commands

Just to let you know, that I mostly use Redhat.

Zipping a folder

$ tar czvf folder_name.tar.gz folder_name


Finding string in all files of folder

$ find . | xargs grep 'your string'


Removing files if rm -fR does not work

$ find . -name '*filename_pattern*' | xargs rm


Killing Linux process by name

$ kill `ps -ef | grep process_name_pattern | grep -v grep | awk '{print $2}'`


Linux command for find and replace

$ sed 's#FIND_STRING#REPLACE_STRING#g' File_Name > New_File_Name


Datetime update cron

Type crontab –e as su (super user) and paste following, replace your_datetime_server with IP or name of your server.

5 0 * * * /usr/sbin/ntpdate your_datetime_server > /dev/null 2>&1


Disk space usage in a folder

Change directory to your server and type

$ du -ks -h *


Find and remove files 14 days old

$ find /folder_name/ *file_name_pattern* -atime +14 -print | xargs rm

LAMP Stack Blog

Hi,

Today I realized that I must contribute my part to open source community. To start I will post what little I have learned in my past and later post more of real life problems I have faced and their solutions.

Cheers,
Pawan