Home › High Performance › Files › Mogilefs – A Tool for Massive Small File Storage

Mogilefs – A Tool for Massive Small File Storage

Categories: Files 2014-05-30 root Comments (0)

 

  • Data Storage Trends and Big Data Challenges

  • Distributed Storage and CAP Theorem

  • file system

  • Mogilefs Basic Principles

  • Mogilefs Implementation

  • Nginx Reverse Proxy for Tracker Nodes

#######################################################################

 

Data Storage Trends and Big Data Challenges

we in a information , in data method , information method , data . in not not data , : data , data , data Search, data , data , data , data can Issues.

in for data not from already is not , : , , node file system .

in has Issues, is in for data , in and : node , data , data , , file system support Issuesin not and .

 

Distributed Storage and CAP Theorem

to is a system has to , respectively is :

Consistency: a operation is can operation .

Availability: operation is can in .

Partition Tolerance: in network () , can and can .

 

2007, all all in CAP , Eric.BrewerCAP, can according to and , and can , is has , Brewer CAP.

and not can , need system not can operation Failure , can all can to operation . data CA, data AP.

for , Availability and Partition Tolerance Consistency, not is , is via implement data , : user and Commentcan for not , not user , data , is 10 data not , can “”need perform .

(ACID): in , can data ; in , to , is to can , not in .

(): system not can , in to not .

: is , system if for has , in not will .

server : Nnode ; W need already node ; Rdata need node .

W + R > N —-> (N=3, W=R=2)

W=N, R=1 —->

W=1, R=N —->

W + R <= N —->

 

file system

Google Filesystem GFS+MapReducefile
Hadoop Distributed Filesystem GFS +MapReduce, file
ClusterFS file
Taobao Filesystem file
MogileFS file
Ceph is a Linux PB file system
MooseFS , can not
Lustre file system

 

Mogilefs Basic Principles

 

MogileFSis a file system , file , LiveJournalDangaInteractive, Danga Memcached, MogileFS, Perlbalnot : (: Perlbalis a Perl server ). MogileFSis a file system . to : , , file , has RAID can , RAIDsupport ……:

trackernode: data node file data information each all ,data simultaneously node , storagenode data , mogilefsd(7001).

databasenode : trackernode provide data service .

storagenode: will has file in file , file , storagenode for , storagenode use httpperform data , perlbal, storagenode can use nginxperform , need nginx-mogilefs-module-masterperform , mogstored(7501),perbal(7500).

 

       Domain: a is , a MogileFScan has , can not data .

       Host: a node a , a can has ( ), each all has ID, Domain+Fidfile .

       Class:,file manage , file in not .

:

 

1. GETto Nginx.

2. Nginxaccording to to Node one.

3. Node onefrom server .

4. from server .

5. Node onewill Nginx.

6. Nginxwill according to urlNode three.

7. Node Threewill file via httpNginx.

8. Nginxwill .

 

Mogilefs Implementation

and service all is , service no , can use service , function :

 

1. via Nginx+Keepalivedimplement can Load Balancing, via upstreamcan has trackernode .

2. DRBD+Corosync+Pacemaker Mysql can , configuration : DASmysqlcan Solution.

3. Mysql can from node operation , from node data can according to Scale, if use data perform , , need can , Trackernode support from node , can according to , Mysql and implement : Mysql.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
##### Databasenode
mariadb
GRANT ALL ON *.* TO 'root'@'%' IDENTIFIED BY 'mypass';
CREATE DATABASE mogilefs;
GRANT ALL ON mogilefs.* TO 'moguser'@'%' IDENTIFIED BY 'mogpass';
FLUSH PRIVILEGES;
##### Trackernode (can is all node )
Installation mogilefs
configuration file
vim /etc/mogilefs/mogilefsd.conf       
daemonize = 1
pidfile = /var/run/mogilefsd/mogilefsd.pid
db_dsn = DBI:mysql:mogilefs:host=192.168.1.241
db_user = moguser
db_pass = mogpass
listen = 0.0.0.0:7001
conf_port = 7001
query_jobs = 100
delete_jobs = 1
replicate_jobs = 5
reaper_jobs = 1
service mogilefsd start     #service
ss -tanlp (LISTEN      0      128       192.168.1.241:7001  )
##### storagenode (can is all node )
Installation mogilefs
mkdir /mogdata/dev1 -pv     #directory
chown -R mogilefs.mogilefs /mogdata/dev2/   #
vim /etc/mogilefs/mogstored.conf
maxconns = 10000
httplisten = 0.0.0.0:7500
mgmtlisten = 0.0.0.0:7501
docroot = /mogdata          #directory
service mogstored start
ss -tanlp (*:7500)
##### trackernode storagenode and command
mogadm check        #node
mogadm host list    #each node a host
mogadm host add 192.168.1.213 --ip=192.168.1.213 --ip=192.168.1.213 --status=alive      #a node
mogadm host add 192.168.1.242 --ip=192.168.1.242 --ip=192.168.1.242 --status=alive       #a node
mogadm host add 192.168.1.241 --ip=192.168.1.241 --ip=192.168.1.241 --status=alive      #a node
mogadm device add 192.168.1.213 1    #a , not can
mogadm device add 192.168.1.242 2   #
mogadm device add 192.168.1.241 3   #
mogadm check    #can view status
mogadm domain add files     #file
mogadm domain add images    #
mogadm domain list  #view all
mogupload --trackers=192.168.1.241 --domain=files --key='/fstab' --file='/etc/fstab'     #fstabfile , key'/fstab'
mogfileinfo --trackers=192.168.1.241 --domain=files --key='/fstab'    #according to keyview file information

: moguploadtool is , is in use mogilefs APIperform .

 

 

Nginx Reverse Proxy for Tracker Nodes

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
##### configuration Nginx
tng
yum install pcre-devel -y
yum groupinstall "Development Tools" "Server Platform Development"
yum install libxslt-devel gd-devel lua-devel geoip-devel
tengine-1.5.1.tar.gz
nginx-mogilefs-module-master.zip    #mogilefsneed
mkdir -pv /var/tmp/nginx/client     #need
unzip nginx-mogilefs-module-master.zip
useradd -r nginx
./configure /
  --prefix=/usr/local/nginx /
  --sbin-path=/usr/local/nginx/sbin/nginx /
  --conf-path=/etc/nginx/nginx.conf /
  --error-log-path=/var/log/nginx/error.log /
  --http-log-path=/var/log/nginx/access.log /
  --pid-path=/var/run/nginx/nginx.pid  /
  --lock-path=/var/lock/nginx.lock /
  --user=nginx /
  --group=nginx /
  --enable-mods-shared=all /
  --add-module=/nginx-mogilefs-module-master
make && make install
vim /etc/profile.d/nginx.sh
export PATH=/usr/local/nginx/sbin:$PATH
. !$
provide Script.....
configuration nginx
vim /etc/nginx/nginx.cfg
upstream trackers {
        server 192.168.1.242:7001 weight=1;
        server 192.168.1.213:7001 weight=1;
        server 192.168.1.241:7001 backup;
        check interval=3000 rise=2 fall=5 timeout=1000;
        check_http_send "GET / HTTP/1.0/r/n/r/n";
        check_http_expect_alive http_2xx http_3xx;
   }
location /jpg/ {
        mogilefs_tracker trackers;
        mogilefs_domain images;
        mogilefs_methods GET;
        mogilefs_pass {
            proxy_pass $mogilefs_path;
            proxy_hide_header Content-Type;
            proxy_buffering off;
                }
       }
##### configuration keepalived
keepalived
vim /etc/keepalived/keepalived.conf  # backup  priority 99
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from admin@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LTT
}
vrrp_script chk_nginx {
   script "killall -0 nginx"
   interval 1
   weight -2
   fall 2
   rise 1
}
vrrp_instance IN_1 {
    state MASTER
    interface eth0
    virtual_router_id 22
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass aaaa
    }
    virtual_ipaddress {
        192.168.1.222
    }
   track_script {
    chk_nginx
}
}

 

view node:

 

GETNginx_mogilefs:

This article is from “” Blog, and Contact!

Related Posts

Leave a Comment

Your email address will not be published.