Linux/Python学习论坛-京峰教育

 找回密码
 立即注册

一键登录:

搜索
热搜: 活动 交友 discuz
查看: 579|回复: 0

高性能缓存服务器Varnish架构配置

[复制链接]

238

主题

288

帖子

1925

积分

超级版主

Rank: 8Rank: 8

积分
1925
QQ
发表于 2015-3-20 16:11:41 | 显示全部楼层 |阅读模式
Varnish跟Squid都是一款内容加速缓存服务器,我们可以使用它们来对我们的网页内容进行缓存,以此来从某个方面提高用户体验度,提升网站整体的抗压能力。
    目前自建的CDN中,有很多都是基于Squid、Varnish等相关缓存软件,经过内部的二次开发实现了更好的内容加速及管理。
    那今天我们一起来学习一下Varnish简单的搭建及日常的维护,深入的东西后期分享,跟大家一起来交流。这里直接上Shell脚本自动初始化并安装:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
#!/bin/sh
#auto install varnish
#2014-07-28 by wugk
DIR=/usr/src
CODE=$?
VER=$1

if  [ -z $1 ];then
    echo "Usage:{$0 'Version' install |config |start|help } , Example $0 2.1.5 install}"
    exit
fi

URL=https://repo.varnish-cache.org/source/varnish-${VER}.tar.gz

function install()
{

if  [ ! -d /usr/local/varnish -o ! -f /etc/init.d/varnishd ];then
    cd $DIR ;wget -c $URL
    if  [ $CODE == '0' ];then
        echo "This varnish Files is Download Successed!"

    else
        echo "This varnish Files is Download Failed!"
    fi

    useradd  -s /sbin/noin varnish
    mkdir -p /data/varnish/{cache,}
    chown -R varnish.varnish /data/varnish/

    tar xzf varnish-${VER}.tar.gz ;cd varnish-${VER} ;/bin/sh autogen.sh  ;./configure --prefix=/usr/local/varnish --enable-dependency-tracking --enable-debugging-symbols --enable-developer-warnings -enable-extra-warnings &&make &&make install
else

    echo "This Varnish is exists,Please exit..."
    sleep 1
    exit 0
fi
}

function config()
{

    if  [ ! -d /usr/local/varnish/ -o -f /etc/init.d/varnishd ];then
     
        echo "You can't config varnish ,Please ensure you varnish dir is or not exist..."
        exit 0
    else
         
        echo "Varnish Already Success Install ,Please Config varnish ......"
    fi
    mv /usr/local/varnish/etc/varnish/default.vcl /usr/local/varnish/etc/varnish/default.vcl.bak
    cat >>/usr/local/varnish/etc/varnish/default.vcl <<EOF

backend server_1
{
.host ="192.168.149.128";
.port = "8080";
.probe = {
.timeout = 5s;
.interval = 2s;
.window = 8;
.threshold = 5;
}
}
backend server_2
{
.host ="192.168.149.129";
.port = "8080";
.probe = {
.timeout = 5s;   
.interval = 2s;   
.window = 8;     
.threshold = 5;
}
}
director rsver random {
{
.backend = server_1;
.weight = 6;
}
{
.backend = server_2;
.weight = 6;
}
}
acl purge {
"localhost";
"127.0.0.1";
}
sub vcl_recv
{
  if (req.http.host ~"^(.*).tdt.com")
  {     
     set req.backend =rsver;
  }  
     else
     {     
       error 200 "Nocahce for this domain";
     }           
       if (req.request =="PURGE")
         {        
           if (!client.ip ~purge)
             {           
                error 405"Not allowed.";        
             }
          else
             {
                return (pipe);
             }
}
if(req.http.x-forwarded-for)
{         
set req.http.X-Forwarded-For =        
req.http.X-Forwarded-For "," client.ip;
}
else
{           
set req.http.X-Forwarded-For =client.ip;      
}
if (req.request !="GET" && req.request != "HEAD")
{        
return (pipe);
}
if (req.http.Expect)
{      
return (pipe);
}
if (req.http.Authenticate|| req.http.Cookie)
{        
return (pass);
}
if (req.http.Cache-Control~ "no-cache")
{      
return (pass);
}
if(req.url ~"\.jsp" || req.url ~ "\.php" )
{        
return (pass);
}
else
{
return (lookup);
}
}sub vcl_pipe
{
return (pipe);
}sub vcl_pass
{
return (pass);
}sub vcl_hash
{
set req.hash += req.url;
if (req.http.host)
{  
set req.hash +=req.http.host;
}
else
{
set req.hash +=server.ip;
}
  return (hash);
}sub vcl_hit
{
if (req.request =="PURGE")
{
set obj.ttl = 0s;      
error 200"Purged.";
}
if (!obj.cacheable)
{  
return (pass);
}
return (deliver);
}sub vcl_miss
{
if (req.request =="PURGE")
{  
error 404 "Not incache.";
}
if (req.http.user-agent ~"spider")
{   
error 503 "Notpresently in cache";
}
     return (fetch);
}
sub vcl_fetch
{
if (req.request =="GET" && req.url ~ "\.(txt|js)$")
{  
set beresp.ttl = 3600s;
}
else
{  
set beresp.ttl = 30d;
}
if (!beresp.cacheable)
{  
return (pass);
}
if (beresp.http.Set-Cookie)
{
return (pass);
}
return (deliver);
}
sub vcl_deliver {
if (obj.hits > 0) {
   set resp.http.X-Cache= "HIT FROM TDTWS Cache Center";
} else {
   set resp.http.X-Cache= "MISS FROM TDTWS Cache Center";
}
return (deliver);
}

EOF

if [ $? == 0 ];then
    echo '----------------------------------'
    sleep 2
    echo "This Varinsh Config Success !!!"
else
    echo "This Varinsh Config Failed,Please Check Conf!"
fi

}

function start()
{
if  [ ! -d /usr/local/varnish -o -f /etc/init.d/varnishd ];then
    echo "You can't config varnish ,Please ensure you varnish dir is or not exist..."
    exit 0

else
    count=`ps -ef |grep varnishd|grep -v grep |wc -l`
    if [ $count -eq 0 ];then
        echo "Varnish Already Success Install ,Now start varnish...."
        cat <<EOF
        ------------------------------------------------------
        Start Varnish to listen 0.0.0.0:80.
        The Varnish load balancer server1(192.168.149.128 8080).
        The Varnish load balancer server1(192.168.149.129 8080).
        The Varnish Mgr address to listen 0.0.0.0:8001.
        The Varnish Cache DIR /data/varnish/cache .
EOF
        /usr/local/varnish/sbin/varnishd -n /data/varnish/cache -f /usr/local/varnish/etc/varnish/default.vcl -a 0.0.0.0:80 -s file,/data/varnish/varnish_cache.data,16G  -p user=varnish -p group=varnish -p default_ttl=14400 -p thread_pool_max=8000 -p send_timeout=20 -w 5,51200,30 -T 0.0.0.0:8001  -P /usr/local/varnish/var/varnish.pid
        if [ $? == 0 ];then
            echo "Start varnish ...OK"
        fi
    else
        echo "Warning,This Varnish Service is exist."
    fi
fi
}

case $2 in

    install)
    install
    ;;
    config)
    config
    ;;

    start )
    start
    ;;

    *    )
    echo "Usage:{ Usage $0 version install |config |start|help }"
    ;;
esac
Varnish简单测试如下图:
1.第一张图,第一次访问没有命中,去后端服务器取数据,同时在本地缓存一份:(MISS)

2.第二张图,第二次访问,缓存服务器存在,则直接从缓存中返回:(HIT)

实际线上环境中,如果用户访问我们的网站,使用Ctrl+F5刷新,我们的缓存就会失效,因为我们配置文件里面是这么配置的,匹配浏览器头信息:
Pragma    no-cache
Cache-Control    no-cachea
if (req.http.Cache-Control~ "no-cache")
{
return (pass);
}
    只有注释掉这段代码或者设置只允许某个特定的IP,用户通过浏览器按Crtl+F5才不会把缓存给清除。
    下面是针对某个特定的IP地址允许刷新:
acl   local {
      "192.168.149.128"
}
sub vcl_hit {
      if (!obj.cacheable) {
         return (pass);
     }

     if (client.ip ~ local && req.http.Pragma ~ "no-cache") {
         set obj.ttl = 0s;
         return (pass);
     }
      return (deliver);
}
    由于时间的原因,文章就暂时写到这里,更多深入的东西继续分享,文章引用煮酒哥的配置,非常感谢。欢迎大家一起讨论。
http://yuhongchun.blog.51cto.com/1604432/1293169
http://zyan.cc/post/313/

回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

QQ|小黑屋|京峰教育,只为有梦想的人 ( 京ICP备15013173号 )

GMT+8, 2019-10-19 00:59 , Processed in 0.027781 second(s), 12 queries , Redis On.

快速回复 返回顶部 返回列表