内容简介：分布式文件系统之MogileFS
MogileFS是一个开源的分布式文件存储系统，由LiveJournal旗下的Danga Interactive公司开发。Danga团队开发了包括 Memcached 、MogileFS、Perlbal 等多个知名的开源项目。目前使用MogileFS 的公司非常多，如日本排名先前的几个互联公司及国内的yupoo(又拍)、digg、豆瓣、1号店、大众点评、搜狗和安居客等，分别为所在的组织或公司管理着海量的图片。
MogileFS由3个部分组成：
(1)server：主要包括mogilefsd和mogstored两个应用程序。mogilefsd实现的是tracker，它通过数据库来保存元数据信息，包括站点domain、class、host等；mogstored是存储节点(store node)，它其实是个WebDAV服务，默认监听在7500端口，接受客户端的文件存储请求。在MogileFS安装完后，要运行mogadm工具将所有的store node注册到mogilefsd的数据库里，mogilefsd会对这些节点进行管理和监控。
(2) utils（工具集）：主要是MogileFS的一些管理工具，例如mogadm等。
(3)客户端API：MogileFS的客户端API很多，例如 Perl 、 PHP 、 Java 、 Python 等，用这个模块可以编写客户端程序，实现文件的备份管理功能等。
现在我们在做一个mogilefs集群分布式存储,节点设计如下:
192.168.42.150 node1 [Nginx,Tracker, Storage,DataBase]
192.168.42.151 node2 [Tracker, Storage]
192.168.42.152 node3 [Tracker, Storage]
1.准备这三台机器,初始化工作,时间同步,分别在这三台机器上安装mogilefs,我们这里用rpm安装
cd /usr/local/src #下载RPM安装包 wget http://192.168.42.26/install_package/down/mogilefs/MogileFS-Server-2.46-2.el6.noarch.rpm wget http://192.168.42.26/install_package/down/mogilefs/MogileFS-Server-mogilefsd-2.46-2.el6.noarch.rpm wget http://192.168.42.26/install_package/down/mogilefs/MogileFS-Server-mogstored-2.46-2.el6.noarch.rpm wget http://192.168.42.26/install_package/down/mogilefs/MogileFS-Utils-2.19-1.el6.noarch.rpm wget http://192.168.42.26/install_package/down/mogilefs/perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm wget http://192.168.42.26/install_package/down/mogilefs/perl-MogileFS-Client-1.14-1.el6.noarch.rpm wget http://192.168.42.26/install_package/down/mogilefs/perl-Perlbal-1.78-1.el6.noarch.rpm #安装依赖 yum install perl-Net-Netmask perl-IO-stringy perl-Sys-Syslog perl-IO-AIO -y #安装rpm yum install *.rpm -y
2.在node1节点上部署mariadb
yum install mariadb-server -y vim /etc/my.cnf.d/server.cnf [mysqld] skip_name_resolve=1 innodb_file_per_table = 1 #启动mysql,做以下操作 systemctl start mariadb mysql MariaDB [(none)]> GRANT ALL ON *.* TO 'root'@'192.168.42.%' IDENTIFIED BY 'mageedu'; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'moguser'@'192.168.42.%' IDENTIFIED BY 'mogpass' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> FLUSH PRIVILEGES; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> CREATE DATABASE mogilefs; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> exit Bye #设定数据库,生成数据表 mogdbsetup --dbhost=192.168.42.150 --dbrootpass='mageedu' --dbuser='moguser' --dbpass='mogpass' --dbname=mogilefs MariaDB [(none)]> use mogilefs; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed MariaDB [mogilefs]> show tables; +----------------------+ | Tables_in_mogilefs | +----------------------+ | checksum | | class | | device | | domain | | file | | file_on | | file_on_corrupt | | file_to_delete | | file_to_delete2 | | file_to_delete_later | | file_to_queue | | file_to_replicate | | fsck_log | | host | | server_settings | | tempfile | | unreachable_fids | +----------------------+ 17 rows in set (0.00 sec)
3.配置mogilefsd
vim /etc/mogilefs/mogilefsd.conf db_dsn = DBI:mysql:mogilefs:host=192.168.42.150 db_user = moguser db_pass = mogpass listen = 0.0.0.0:7001 conf_port = 7001
4.启动mogilefsd
service mogilefsd start [root@node1 my.cnf.d]# service mogilefsd start Starting mogilefsd (via systemctl): [ OK ] [root@node1 my.cnf.d]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:22 *:* LISTEN 0 1024 *:7001 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 50 *:3306 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::*
5.配置storage(可以是所有节点)
mkdir -pv /data/mogilefs/dev1 #创建设备目录
chown -R mogilefs.mogilefs /data/mogilefs/dev1 #权限
vim /etc/mogilefs/mogstored.conf
maxconns = 10000
httplisten = 0.0.0.0:7500
mgmtlisten = 0.0.0.0:7501
docroot = /data/mogilefs/
修改脚本
lockfile下添加
pidfile=/var/run/mogilefsd/mogstored.pid
start()修改
[ $RETVAL = 0 ] && success && touch ${lockfile} && echo $(pidof mogstored) > ${pidfile} || failure
stop()修改
[ $RETVAL = 0 ] && success && rm -f ${lockfile} ${pidfile} || failure
#启动mogstored
[root@node1 init.d]# service mogstored start
Starting mogstored (via systemctl): [ OK ]
[root@node1 init.d]# ss -tnl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 1024 *:7500 *:*
LISTEN 0 1024 *:7501 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 1024 *:7001 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 50 *:3306 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
6.node2,node3上分别创建设备目录和分发配置文件,脚本文件
node2上操作: mkdir -pv /data/mogilefs/dev2 chown -R mogilefs.mogilefs /data/mogilefs/dev2 node3上操作: mkdir -pv /data/mogilefs/dev3 chown -R mogilefs.mogilefs /data/mogilefs/dev3 #分发脚本和配置文件 scp /etc/rc.d/init.d/mogstored root@192.168.42.151:/etc/rc.d/init.d/ scp /etc/mogilefs/mogstored.conf root@192.168.42.151:/etc/mogilefs/ scp /etc/rc.d/init.d/mogstored root@192.168.42.152:/etc/rc.d/init.d/ scp /etc/mogilefs/mogstored.conf root@192.168.42.152:/etc/mogilefs/ 在node2,和node3启动mogstored service mogstored start
7.node1节点上需要添加
#查看帮助 mogadm -help [root@node1 init.d]# mogadm check Checking trackers... 127.0.0.1:7001 ... OK Checking hosts... No devices found on tracker(s). 我们还可以使用mogadm --trackers=192.168.42.150:7001 check 得到同样的结果 #添加主机 mogadm --trackers=192.168.42.150:7001 host add 192.168.42.150 --ip=192.168.42.150 --status=alive mogadm --trackers=192.168.42.150:7001 host add 192.168.42.151 --ip=192.168.42.151 --status=alive mogadm --trackers=192.168.42.150:7001 host add 192.168.42.152 --ip=192.168.42.152 --status=alive [root@node1 init.d]# mogadm host list 192.168.42.150 [1]: alive IP: 192.168.42.150:7500 192.168.42.151 [2]: alive IP: 192.168.42.151:7500 192.168.42.152 [3]: alive IP: 192.168.42.152:7500
添加设备
(1).没有添加设备之前查看,是没有结果的
[root@node1 init.d]# mogadm device list
192.168.42.150 [1]: alive
used(G) free(G) total(G) weight(%)
192.168.42.151 [2]: alive
used(G) free(G) total(G) weight(%)
192.168.42.152 [3]: alive
used(G) free(G) total(G) weight(%)
(2)添加设备的操作命令
mogadm device add 192.168.42.150 1 mogadm device add 192.168.42.151 2 mogadm device add 192.168.42.152 3
(3)此时查看有结果了
[root@node1 init.d]# mogadm device list
192.168.42.150 [1]: alive
used(G) free(G) total(G) weight(%)
dev1: alive 1.688 15.298 16.986 100
192.168.42.151 [2]: alive
used(G) free(G) total(G) weight(%)
dev2: alive 1.557 15.430 16.986 100
192.168.42.152 [3]: alive
used(G) free(G) total(G) weight(%)
dev3: alive 1.557 15.430 16.986 100
(4)此时查看mogadm check
[root@node1 init.d]# mogadm check
Checking trackers...
127.0.0.1:7001 ... OK
Checking hosts...
[ 1] 192.168.42.150 ... OK
[ 2] 192.168.42.151 ... OK
[ 3] 192.168.42.152 ... OK
Checking devices...
host device size(G) used(G) free(G) use% ob state I/O%
---- ------------ ---------- ---------- ---------- ------ ---------- -----
[ 1] dev1 16.986 1.689 15.297 9.95% writeable 0.0
[ 2] dev2 16.986 1.557 15.429 9.17% writeable 0.0
[ 3] dev3 16.986 1.557 15.429 9.17% writeable 0.0
---- ------------ ---------- ---------- ---------- ------
total: 50.959 4.803 46.156 9.43%
(1)查看没有添加域的状态 [root@node1 init.d]# mogadm domain list domain class mindevcount replpolicy hashtype -------------------- -------------------- ------------- ------------ ------- (2)添加域操作 mogadm domain add files mogadm domain add images (3)再次查看列出 [root@node1 init.d]# mogadm domain list domain class mindevcount replpolicy hashtype -------------------- -------------------- ------------- ------------ ------- files default 2 MultipleHosts() NONE images default 2 MultipleHosts() NONE
添加class
(1)查看一下class列表 [root@node1 init.d]# mogadm class list domain class mindevcount replpolicy hashtype -------------------- -------------------- ------------- ------------ ------- files default 2 MultipleHosts() NONE images default 2 MultipleHosts() NONE (2).添加class操作(不知为什么这一步会导致只有一个副本) 需要这样添加 mogadm class add images jpeg --mindevcount=2 --replpolicy="MultipleHosts(2)" mogadm class add images jpeg mogadm class add images png mogadm class add images gif (3).再次查看列表 [root@node1 init.d]# mogadm class list domain class mindevcount replpolicy hashtype -------------------- -------------------- ------------- ------------ ------- files default 2 MultipleHosts() NONE images default 2 MultipleHosts() NONE images gif 2 MultipleHosts() NONE images jpeg 2 MultipleHosts() NONE images png 2 MultipleHosts() NONE
上传图片测试
(1)上传图片
mogupload --tracker=192.168.42.150:7001 --domain=images --key='1.gif' --file='/tmp/83e.gif' --class=gif
mogupload --tracker=192.168.42.150:7001 --domain=images --key='1.jpg' --file='/tmp/zhuoku194.jpg' --class=jpeg
mogupload --tracker=192.168.42.150:7001 --domain=images --key='2.jpg' --file='/tmp/1.jpg' --class=jpeg
(2)查看图片信息
mogfileinfo --tracker=192.168.42.150:7001 --domain=images --key='1.gif'
[root@node1 init.d]# mogfileinfo --tracker=192.168.42.150:7001 --domain=images --key='1.gif'
- file: /1.gif
class: gif
devcount: 1
domain: images
fid: 2
key: /1.gif
length: 499461
- http://192.168.42.151:7500/dev2/0/000/000/0000000002.fid
mogfileinfo --tracker=192.168.42.150:7001 --domain=images --key='1.jpg'
[root@node1 init.d]# mogfileinfo --tracker=192.168.42.150:7001 --domain=images --key='1.jpg'
- file: /1.jpg
class: jpeg
devcount: 1
domain: images
fid: 3
key: /1.jpg
length: 374408
- http://192.168.42.150:7500/dev1/0/000/000/0000000003.fid
(3)在浏览器访问
http://192.168.42.151:7500/dev2/0/000/000/0000000002.fid
http://192.168.42.150:7500/dev1/0/000/000/0000000003.fid
都是ok的
删除文件:
mogdelete --tracker=192.168.42.150:7001 --domain=images --key='1.jpg'
mogdelete --tracker=192.168.42.150:7001 --domain=images --key='2.jpg'
mogdelete --tracker=192.168.42.150:7001 --domain=images --key='1.gif'
删除class:
mogadm class delete images jpeg
mogadm class delete images gitj
mogadm class delete images png
删除域:
mogadm domain delete files
mogadm domain delete images
#配置Nginx
nginx 的mogilefs模块的官方文档地址：
http://www.grid.net.ru/nginx/mogilefs.en.html
wget http://www.grid.net.ru/nginx/download/nginx_mogilefs_module-1.0.4.tar.gz
tar xvf nginx_mogilefs_module-1.0.4.tar.gz
mv nginx_mogilefs_module-1.0.4 /usr/local/
cd /usr/local/src
wget http://192.168.42.26/nginx-1.11.8.tar.gz
tar xvf nginx-1.11.8.tar.gz
cd nginx-1.11.8
yum groupinstall "Development Tools" "Server Platform Developments"
yum install pcre-devel openssl-devel zlib-devel -y
groupadd -r nginx
useradd -r -g nginx nginx
我的编译安装方法
./configure --with-pcre --user=nginx --group=nginx --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_realip_module --add-module=/usr/local/nginx_mogilefs_module-1.0.4
make && make install
编译会可能报错:
cc1: all warnings being treated as errors
make[1]: *** [objs/addon/nginx_mogilefs_module-1.0.4/ngx_http_mogilefs_module.o] Error 1
make[1]: Leaving directory `/usr/local/src/nginx-1.11.8'
make: *** [build] Error 2
解决办法:
我们这里打开
vim objs/Makefile
#去掉警告
-Werror
#编辑nginx配置文件
user nginx;
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream trackers {
server 192.168.42.150:7001 weight=1;
server 192.168.42.151:7001 weight=1;
server 192.168.42.152:7001 weight=1;
check interval=1000 rise=2 fall=5 timeout=1000;
}
server {
listen 80;
server_name localhost;
location / {
root html;
index index.html index.htm;
}
location /images/ {
mogilefs_tracker trackers;
mogilefs_domain images;
mogilefs_methods GET;
mogilefs_noverify on;
mogilefs_pass {
proxy_pass $mogilefs_path;
proxy_hide_header Content-Type;
proxy_buffering off;
}
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
浏览器访问:
http://192.168.42.150/images/2.jpg http://192.168.42.150/images/1.gif 都OK
此时我们的trackers其实只有一个,但我们在nginx配置文件中写了三个,所以我们需要把另外两个节点也要 安装tracker
scp /etc/mogilefs/mogilefsd.conf 192.168.42.151:/etc/mogilefs/ scp /etc/mogilefs/mogilefsd.conf 192.168.42.152:/etc/mogilefs/ 启动mogilefsd:(两台节点) service mogilefsd start
三个节点都能查看内容列表
[root@node1 conf]# moglistkeys --trackers=192.168.42.150:7001 --domain=images 1.gif 1.jpg 2.jpg [root@node1 conf]# moglistkeys --trackers=192.168.42.151:7001 --domain=images 1.gif 1.jpg 2.jpg [root@node1 conf]# moglistkeys --trackers=192.168.42.152:7001 --domain=images 1.gif 1.jpg 2.jpg
以上就是本文的全部内容，希望本文的内容对大家的学习或者工作能带来一定的帮助，也希望大家多多支持 码农网
猜你喜欢:
- 分布式文件系统 go-fastdfs v1.2.8 发布，增加文件列表接口，优化文件快速迁移功能
- 分布式文件系统架构对比
- MFS分布式文件系统
- 分布式文件系统-HDFS
- 分布式文件存储 FastDFS
- 微服务之分布式文件系统
本站部分资源来源于网络，本站转载出于传递更多信息之目的，版权归原作者或者来源机构所有，如转载稿涉及版权问题，请联系我们。
Just My Type
Simon Garfield / Profile Books / 2010-10-21 / GBP 14.99
What's your type? Suddenly everyone's obsessed with fonts. Whether you're enraged by Ikea's Verdanagate, want to know what the Beach Boys have in common with easy Jet or why it's okay to like Comic Sa......一起来看看 《Just My Type》 这本书的介绍吧!
