发布于 由shang

lvs keepalived实现负载均衡及高可用

说明:
LVS全称Linux Virtual Server，即Linux虚拟服务器，是一个虚拟的服务器集群系统。该项目在1998年5月由章文嵩博士成立，是中国国内最早出现的自由软件项目之一。目前有三种IP负载均衡技术（VS/NAT、VS/TUN和VS/DR）；十种调度算法（rrr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq）

Keepalvied
Keepalived在这里主要用作web-RealServer健康状态检查及lvs-dr-master和lvs-dr-backup之间failover自动切换的实现

环境：

lvs-dr-master        61.164.122.6        
lvs-dr-backup        61.164.122.7        
LVS-DR-VIP           61.164.122.8        
WEB1-Realserver      61.164.122.9        
WEB2-Realserver      61.164.122.10

lvs-dr-master，lvs-dr-backup上安装lvs，keepalived软件。lvs-dr-master向局域网内广播61.164.122.8说这个ip是自己，从而实现负载调度，调配到web服务器的请求。(之所以会有2台，是为了当lvs-dr-master挂掉，lvs-dr-backup能够接替lvs-dr-maser继续实现负载调度。此时lvs-dr-slave会接替lvs-dr-master广播61.164.122.8)
WEB1-Realserver,WEB2-Realserver只需要运行realserver.sh，将61.164.122.8这个vip绑定到自己的lo:0上。
我们向61.164.122.8发出web请求时，其实是lvs-dr-master接收到。他通过keepalived.conf里的配置，再决定将web请求转发给哪台web服务器。

实现：
一、lvs-dr-master，lvs-dr-backup安装LVS和Keepalvied软件包

# mkdir /usr/local/src/lvs
# cd /usr/local/src/lvs
# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
# wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz

# lsmod |grep ip_vs  //查看是否开启lvs，安装完ipvsadm后可以再看看
# uname -r
2.6.18-53.el5PAE
# ln -s /usr/src/kernels/2.6.18-53.el5PAE-i686/  /usr/src/linux

# tar zxvf ipvsadm-1.24.tar.gz
# cd ipvsadm-1.24
# make && make install
# find / -name ipvsadm  # 查看ipvsadm的位置

# tar zxvf keepalived-1.1.15.tar.gz
# cd keepalived-1.1.15
# ./configure  && make && make install
# find / -name keepalived  # 查看keepalived位置    

# cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
# mkdir /etc/keepalived
# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
# cp /usr/local/sbin/keepalived /usr/sbin/
# service keepalived start|stop     #做成系统启动服务方便管理.

 
二、web服务器WEB1-Realserver，WEB2-Realserver配置Realserver脚本，将61.164.122.8这个vip绑定到自己的lo:0上。
# vi /usr/local/sbin/realserver.sh //内容如下

#!/bin/bash
# description: Config realserver lo and apply noarp 
# Written by :NetSeek http://www.linuxtone.org
        
SNS_VIP=192.168.102.127
         
/etc/rc.d/init.d/functions
         
case "$1" in
        start)
               ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
               /sbin/route add -host $SNS_VIP dev lo:0
               echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
               echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
               echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
               echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
               sysctl -p >/dev/null 2>&1
               echo "RealServer Start OK"
         
               ;;
        stop)
               ifconfig lo:0 down
               route del $SNS_VIP >/dev/null 2>&1
               echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
               echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
               echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
               echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
               echo "RealServer Stoped"
               ;;
        *)
               echo "Usage: $0 {start|stop}"
               exit 1
        esac
         
        exit 0

 
或者采用secondary ip address方式配置
# vi /etc/sysctl.conf

net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

# sysctl -p   //写入修改
# ip addr add 61.164.122.8/32 dev lo
# ip add list  //查看是否绑定

 
三、通过lvs实现负载均衡
1．lvs-dr-master，配置LVS脚本实现负载均衡。此脚本只是为了演示，方便大家理解lvs,在keepalived方案中不要启动此脚本。关于LVS的keepalvied的HA方案，完全由keepalived.conf一个文件搞定，不需要开启这个。
# vi /usr/local/sbin/lvs-dr.sh //内容如下

#!/bin/bash
# description: start LVS of DirectorServer
        
GW=192.168.102.2
# website director vip.
SNS_VIP=192.168.102.127
SNS_RIP1=192.168.102.128
SNS_RIP2=192.168.102.129

/etc/rc.d/init.d/functions

logger $0 called with $1

case "$1" in
start)
        # set squid vip
        /sbin/ipvsadm --set 30 5 60
        /sbin/ifconfig eth0:0 $SNS_VIP broadcast $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP up
        /sbin/route add -host $SNS_VIP dev eth0:0
        /sbin/ipvsadm -A -t $SNS_VIP:80 -s nq -p 3
        /sbin/ipvsadm -a -t $SNS_VIP:80 -r $SNS_RIP1:80 -g -w 1
        /sbin/ipvsadm -a -t $SNS_VIP:80 -r $SNS_RIP2:80 -g -w 1
        touch /var/lock/subsys/ipvsadm >/dev/null 2>&1

       ;;
stop)
        /sbin/ipvsadm -C
        /sbin/ipvsadm -Z
        ifconfig eth0:0 down
        route del $SNS_VIP
        rm -rf /var/lock/subsys/ipvsadm >/dev/null 2>&1
        echo "ipvsadm stoped"
       ;;

status)

        if [ ! -e /var/lock/subsys/ipvsadm ];then
                echo "ipvsadm stoped"
                exit 1
        else
                echo "ipvsadm OK"
        fi
      ;;

*)
        echo "Usage: $0 {start|stop|status}"
        exit 1
esac

exit 0

启动lvs-dr脚本和realserver启本后，在lvs-dr-master上可以查看LVS当前的状态了:
# watch ipvsadm -ln
这时候，如果你测试（web访问61.164.122.8），多访问几次，你会发现它会分别显示192.168.102.128，192.168.102.129的web内容。说明他已经有实现调度了。但如果关闭1台的web话，如这里的129，再多次web访问61.164.122.8就会现在，来自129的web是显示不出来的，而128的都正常。这里，我能理解是lvs只实现负载均衡，并没有办法自动剔除有问题的web服务器？

四、keepalvied实现负载均衡和和高可用性
1.lvs-dr-master主负载均衡服务器上配置keepalived.conf
# vi /etc/keepalived/keepalived.conf //内容如下

! Configuration File for keepalived

global_defs {
   notification_email {
         cnseek@gmail.com
   }
   notification_email_from sns-lvs@gmail.com
   smtp_server 127.0.0.1
  # smtp_connect_timeout 30
   router_id LVS_DEVEL
}

# VIP1
vrrp_instance VI_1 {
    state MASTER             #备份服务器上将MASTER改为BACKUP   
    interface eth0
    virtual_router_id 51
    priority 100    # 备份服务上将100改为99。注意Master数值要要大。
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        61.164.122.8   
        #(如果有多个VIP，继续换行填写.)
    }
}

virtual_server 61.164.122.8 80 {
    delay_loop 6                  #(每隔10秒查询realserver状态)
    lb_algo wrr                  #(lvs 算法)
    lb_kind DR                  #(Direct Route)
    persistence_timeout 60        #(同一IP的连接60秒内被分配到同一台realserver)
    protocol TCP                #(用TCP协议检查realserver状态)

    real_server 61.164.122.9 80 {
        weight 3               #(权重)
        TCP_CHECK {
        connect_timeout 10       #(10秒无响应超时)
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }
    real_server 61.164.122.10 80 {
        weight 3
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
     }
}

 
2. lvs-dr-backup备服务器同上配置，需先安装lvs、keepalived然后配置/etc/keepalived/keepalived.conf，只需修改state BACKUP priority 99这2处即可。

3. 关闭lvs_dr.sh演示，开启keepalived实现负载均衡及高可用

# /usr/local/sbin/lvs-dr.sh stop  //停止lvs-dr脚本
# /etc/init.d/keepalived start  //启动keepalived 服务，keepalived利用keepalived.conf配置文件实现负载均衡和高可用。整个LVS负均衡HA方案，只需keepalived.conf一个文件即可搞定！

 
4. 查看lvs服务是否正常

# watch ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddressort Scheduler Flags
  -> RemoteAddressort           Forward Weight ActiveConn InActConn
TCP  61.164.122.8:80 wrr persistent 60
  -> 61.164.122.10:80            Route   3      0          0
  -> 61.164.122.9:80             Route   3      0          0

 

# tail -f /var/log/message  监听日志，查看状态，测试LVS负载均衡及高可用性是否有效。

5．停Master服务器的keepalived服务，查看BAKCUP服务器是否能正常接管服务。

附录：
暂无！

原文参考：
http://bbs.linuxtone.org/thread-1077-1-1.html
http://www.keepalived.org/pdf/sery-lvs-cluster.pdf

Post Views: 203

发表评论

邮箱地址不会被公开。 必填项已用*标注