CentOS / Redhat: Install nginx As Reverse Proxy Load Balancer
nginx is a Web and Reverse proxy server. Nginx used in front of Apache Web servers. All connections coming from the Internet addressed to one of the Web servers are routed through the nginx proxy server, which may either deal with the request itself or pass the request wholly or partially to the main web servers.
Our Sample Setup
Internet--
|
============= |---- apache1 (192.168.1.15)
| ISP Router| |
============= |---- apache2 (192.168.1.16)
| |
| |---- db1 (192.168.1.17)
| |eth0 -> 192.168.1.11 ----------/
|-lb0==| /
| |eth1 -> 202.54.1.1 ----/
|
| |eth0 -> 192.168.1.10 ----------\
|-lb1==| / |---- apache1 (192.168.1.15)
|eth1 -> 202.54.1.1 ----/ |
|---- apache2 (192.168.1.16)
|
|---- db1 (192.168.1.17)Where,
- lb0 - Linux box directly connected to the Internet via eth1. This is master load balancer.
- lb1 - Linux box directly connected to the Internet via eth1. This is backup load balancer. This will become active if master networking failed.
- 202.54.1.1 - This ip moves between lb0 and lb1 server. It is called virtual IP address and it is managed by keepalived.
- eth0 is connected to LAN and all other backend software servers are connected via eth0.
- nginx is installed on both lb0 and lb1. It will listen on 202.54.1.1. You need to configure nginx as reverse proxy server. It will connects to Apache1 and Apache2.
- Install httpd server on Apache#1 and Apache#2 server. Configure them to listen on 192.168.1.15:80 and 192.168.1.16:80. Do not assign public IP to this box. Only activate eth0 via LAN.
- Install MySQL / Oracle / PgSQL server on Db#1. Configure db server to listen on 192.168.1.17:$db_server_port. Do not assign public IP to this box. Only activate eth0 via LAN.
In short you need the following hardware:
- 2 load balancer reverse proxy servers (250GB SATA, 2GB RAM, Single Intel P-D930 or AMD 170s with RHEL 64 bit+keepalived+nginx)
- 2 Apache web servers (Software RAID-1, SCSI-73GBx2 15k disk, 6GB RAM, Dual Intel Xeon or AMD 64 bit CPU with RHEL 64 bit+Apache 2)
- 1 backup Apache web servers (Software RAID-1, SCSI-73GBx2 15k disk, 6GB RAM, Dual Intel Xeon or AMD 64 bit CPU with RHEL 64 bit+Apache 2)
- 1 database server (RAID-10, SCSI-73GBx4 15k disk, 16GB RAM, Dual Intel Xeon or AMD 64 bit CPU with RHEL 64 bit+MySQL 5)
- 1 Caching server (RAID-1, SCSI-73GBx2 15k disk, 8GB RAM, Dual Intel Xeon or AMD 64 bit CPU with RHEL 64 bit)
- 1 offsite backup server (RAID-6, 1TB SATAx4, 4GB RAM, Single Intel/AMD CPU with RHEL 64bit)
- Slave database, storage, pop3 and SMTP server as per requirements.
- Internet uplink 100Mbps+ or as per requirements.
Remove Unwanted Software From lb0 and lb1
Type the following commands:
# yum -y groupremove "X Window System"
# x=$(yum list installed | egrep -i 'php|httpd|mysql|bind|dhclient|tftp|inetd|xinetd|ypserv|telnet-server|rsh-server|vsftpd|tcsh' | awk '{ print $1}')
# yum -y remove $x
# yum -y install bind-utils sysstat openssl-devel.x86_64 pcre-devel.x86_64 openssl097a.x86_64
# /usr/sbin/authconfig --passalgo=sha512 --update
# passwd root
The above will remove X windows and other unwanted software from both lb0 and lb1.Install Nginx On Both lb0 and lb1
Type the following commands to download nginx, enter:
# cd /opt
# wget http://sysoev.ru/nginx/nginx-0.8.33.tar.gz
Untar nginx, enter:
# tar -zxvf nginx-0.8.33.tar.gz
# cd nginx-0.8.33
Configure nginx for 64 bit RHEL / CentOS Linux:
# ./configure --without-http_autoindex_module --without-http_ssi_module --without-http_userid_module --without-http_auth_basic_module --without-http_geo_module --without-http_fastcgi_module --without-http_empty_gif_module --with-openssl=/lib64
Sample outputs:....
nginx path prefix: "/usr/local/nginx"
nginx binary file: "/usr/local/nginx/sbin/nginx"
nginx configuration prefix: "/usr/local/nginx/conf"
nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
nginx pid file: "/usr/local/nginx/logs/nginx.pid"
nginx error log file: "/usr/local/nginx/logs/error.log"
nginx http access log file: "/usr/local/nginx/logs/access.log"
nginx http client request body temporary files: "client_body_temp"
nginx http proxy temporary files: "proxy_temp"
nginx http fastcgi temporary files: "fastcgi_temp"
...Install the same:
# make
# make installCreate nginx User Account
Type the following commands to create a user account:
# useradd -s /sbin/nologin -d /usr/local/nginx/html -M nginx
# passwd -l nginxConfigure nginx As Reverse Proxy Load Balancer On Both lb0 and lb1
Edit /usr/local/nginx/conf/nginx.conf, enter:
# vi /usr/local/nginx/conf/nginx.conf
Update it as follows:
pid logs/nginx.pid;
user nginx nginx;
worker_processes 10;
events {
worker_connections 1024;
}
http {
default_type application/octet-stream;
## Common options ##
include options.conf;
## Proxy settings ##
include proxy.conf;
## lb domains ##
include nixcraft.in.conf;
}Edit /usr/local/nginx/conf/options.conf, enter:
# vi /usr/local/nginx/conf/options.conf
Update it as follows:
## Size Limits
client_body_buffer_size 128K;
client_header_buffer_size 1M;
client_max_body_size 1M;
large_client_header_buffers 8 8k;
## Timeouts
client_body_timeout 60;
client_header_timeout 60;
expires 24h;
keepalive_timeout 60 60;
send_timeout 60;
## General Options
ignore_invalid_headers on;
keepalive_requests 100;
limit_zone gulag $binary_remote_addr 5m;
recursive_error_pages on;
sendfile on;
server_name_in_redirect off;
server_tokens off;
## TCP options
tcp_nodelay on;
tcp_nopush on;
## Compression
gzip on;
gzip_buffers 16 8k;
gzip_comp_level 6;
gzip_http_version 1.0;
gzip_min_length 0;
gzip_types text/plain text/css image/x-icon application/x-perl application/x-httpd-cgi;
gzip_vary on;
## Log Format
log_format main '$remote_addr $host $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" "$http_user_agent" '
'"$gzip_ratio"';
Edit /usr/local/nginx/conf/proxy.conf, enter:
## Proxy caching options
proxy_buffering on;
proxy_cache_min_uses 3;
proxy_cache_path /usr/local/nginx/proxy_temp/ levels=1:2 keys_zone=cache:10m inactive=10m max_size=1000M;
proxy_cache_valid any 10m;
proxy_ignore_client_abort off;
proxy_intercept_errors on;
proxy_next_upstream error timeout invalid_header;
proxy_redirect off;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_connect_timeout 60;
proxy_send_timeout 60;
proxy_read_timeout 60;
Edit /usr/local/nginx/conf/nixcraft.in.conf, enter:
## Connect to backend servers via LAN ##
## Reverse Proxy Load Balancer Logic ##
upstream nixcraft {
server 192.168.1.15 weight=10 max_fails=3 fail_timeout=30s;
server 192.168.1.16 weight=10 max_fails=3 fail_timeout=30s;
# only comes alive when above two fails
server 192.168.1.23 weight=1 backup;
}
server {
access_log logs/access.log main;
error_log logs/error.log;
index index.html;
root /usr/local/nginx/html;
server_name nixcraft.in www.nixcraft.in subdomain.nixcraft.in;
## Only requests to our Host are allowed
if ($host !~ ^(nixcraft.in|www.nixcraft.in|subdomain.nixcraft.in)$ ) {
return 444;
}
## redirect www to nowww
# if ($host = 'www.nixcraft.in' ) {
# rewrite ^/(.*)$ http://nixcraft.in/$1 permanent;
# }
## Only allow these request methods
if ($request_method !~ ^(GET|HEAD|POST)$ ) {
return 444;
}
## PROXY - Web
location / {
proxy_pass http://nixcraft;
proxy_cache cache;
proxy_cache_valid 200 24h;
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
proxy_ignore_headers Expires Cache-Control;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# redirect server error pages to the static page /50x.html
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}Start nginx web server:
# /usr/local/nginx/sbin/nginx
# netstat -tulpn | grep :80
# echo ' /usr/local/nginx/sbin/nginx' >> /etc/rc.local