Deep Dive Spider Engine

1
Deep Dive Spider Engine
날짜 : 2016.03.21
Ver. 1.0
작성 : DB실 전수진

 What is Spider Engine?
 Architecture
 How to Create Spider Engine
 Limitation of a Spider Engine
 Guideline
 Q&A
Agenda

What is Spider Engine ?
http://spiderformysql.com/
https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/

Image Source : http://barrymieny.deviantart.com/art/Database-104013446

Spider Node 를 통해 하나의 DB 로 보임
Spider Node 는 데이터를 갖지않음

Architecture
use mysql
show tables like ‘spider%’
spider_link_failed_log : HA에서 사용되는 테이블로 스파이더에 구성된 노드가 에러가 발생하면 로그 기록
spider_link_mon_servers : HA에서 사용되는 테이블로 스파이더의 HA 모니터링하는 서버의 계정을 관리
spider_tables : 스파이더 엔진으로 구성된 테이블 리스트 출력
spider_xa : 서로 다른 노드에 분산 트랜잭션 관리
spider_xa_member : 서로 다른 노드에 분산 트랜잭션을 원격 관리하는 서버 정보 저장

■ Step by Step
1. 서버 준비
Spider node Node 1
Node 2
Node 3
Node 4
※ 5대 DB Server 준비
① Spider Node
(mariaDB 10.0.9)
② Data Node
⑴ MySQL 5.6.24
⑵ MySQL 5.5.30
⑶ MySQL 5.7.9
⑷ mariaDB 10.0.4
How to Create Spider Engine

■ Step by Step
2-1. 아래 링크에서 스파이더 엔진 설치 파일을 받아 설치
2-2. 스파이더 엔진을 사용하기 위한 소스 적용
mysql -uroot -p < ../mariadb/share/install_spider.sql
2-3. 스파이더 엔진 설치 완료확인
SELECT engine, support, transactions, xa
FROM information_schema.engines;
https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
+--------------------+---------+--------------+------+
| engine | support | transactions | xa |
+--------------------+---------+--------------+------+
| SPIDER | YES | YES | YES |
| CSV | YES | NO | NO |
| MyISAM | YES | NO | NO |
| BLACKHOLE | YES | NO | NO |
| FEDERATED | YES | YES | NO |
| MRG_MyISAM | YES | NO | NO |
| ARCHIVE | YES | NO | NO |
| MEMORY | YES | NO | NO |
| PERFORMANCE_SCHEMA | YES | NO | NO |
| Aria | YES | NO | NO |
| InnoDB | DEFAULT | YES | YES |
+--------------------+---------+--------------+------+

Spider Engine
Node 1 (mysql 5.7.9)
Node 4 (mariadb 10.0.4)
Local Server
Spider Table
*.frm
*.par
File
Remote Server
Spider Table
*.frm
File
Data
(*.ibd)
192.168.124.134
192.168.124.137
192.168.124.138
192.168.124.139
192.168.124.140
Schema info (O)
Partition info (O)
Index data (X)
■ Step by Step II
1. 서버 준비

Spider Engine
1. 원격서버 등록
sheard_node1 : 192.168.124.137
sheard_node2 : 192.168.124.138
sheard_node3 : 192.168.124.139
sheard_node4 : 192.168.124.140
CREATE SERVER shard_node1 FOREIGN DATA WRAPPER mysql
OPTIONS(
HOST '192.168.124.137',
DATABASE 'log_db',
USER 'root',
PASSWORD '123qwe!@#',
PORT 3306
);
CREATE SERVER shard_node4 FOREIGN DATA WRAPPER mysql
OPTIONS(
HOST '192.168.124.140',
DATABASE 'log_db',
USER 'root',
PASSWORD '123qwe!@#',
PORT 3306
);
☞ Reference
https://dev.mysql.com/doc/refman/5.7/en/create-server.html
...
Spider node
Node 1
Node 2
Node 3
Node 4
■ Step by Step II

CREATE TABLE log_db.shardTest
(
id int unsigned NOT NULL AUTO_INCREMENT
, name char(120) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
)
ENGINE=spider
COMMENT='wrapper "mysql", table "shardTest"'
PARTITION BY KEY (id)
(
PARTITION shard1 COMMENT = 'srv "shard_node1"'
, PARTITION shard2 COMMENT = 'srv "shard_node2"'
) ;
2. Shard 테이블을 생성
1) 데이터가 저장될 실제 테이블 이름과
파티션 Key 값에 따라
저장될 원격 서버의 정보를 Comment 에 입력함
Spider node
Node 1
Node 2
Node 3
Node 4
■ Step by Step II
Spider Engine

Node 1 Node 2 Node 3 Node 4
(
id int unsigned NOT NULL AUTO_INCREMENT
, PRIMARY KEY (id)
)
ENGINE=innodb;
Spider node
Node 1
Node 2
Node 3
Node 4
■ Step by Step II
3. 각 노드에 Data 테이블 생성

insert into log_db.shardTest(name) values ('spider_01');
select * from shardtest;
+----+-----------+
| id | name |
+----+-----------+
| 1 | spider_01 |
| 4 | spider_01 |
| 3 | spider_01 |
| 2 | spider_01 |
+----+-----------+
Spider node
Node 1
Node 2
Node 3
Node 4
■ Step by Step II
4. Spider 노드에서 데이터 입력 및 확인
Spider Engine

Node 1 Node 2
Node 3 Node 4
+----+-----------+
| id | name |
+----+-----------+
| 1 | spider_01 |
+----+-----------+
+----+-----------+
| id | name |
+----+-----------+
| 3 | spider_01 |
+----+-----------+
+----+-----------+
| id | name |
+----+-----------+
| 4 | spider_01 |
+----+-----------+
+----+-----------+
| id | name |
+----+-----------+
| 2 | spider_01 |
+----+-----------+
Spider node
Node 1
Node 2
Node 3
Node 4
■ Step by Step II
5. 각 Data 노드에서 데이터 확인
1) 데이터가 잘 분산 됐는지 체크

Spider node
Node 1
Node 2
Node 3
Node 4
CREATE TABLE shardTest_node1(id int unsigned NOT NULL AUTO_INCREMENT,name char(120) NOT NULL DEFAULT '',PRIMARY KEY (id) )
ENGINE=FEDERATED CONNECTION='shard_node1/shardTest';
■ Step by Step II
Spider Engine
0. 테스트를 위해 FEDERATED Table 구성

[localhost] ((none)) 06:31> show storage engines;
+--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| Engine | Support | Comment | Transactions | XA | Savepoints |
+--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| SPIDER | YES | Spider storage engine | YES | YES | NO |
| MRG_MyISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO |
| MyISAM | YES | MyISAM storage engine | NO | NO | NO |
| InnoDB | DEFAULT | Percona-XtraDB, Supports transactions, row-level locking, and foreign keys | YES | YES | YES |
| ARCHIVE | YES | Archive storage engine | NO | NO | NO |
| FEDERATED | NO | FederatedX pluggable storage engine | NULL | NULL | NULL |
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
| Aria | YES | Crash-safe tables with MyISAM heritage | NO | NO | NO |
| CSV | YES | CSV storage engine | NO | NO | NO |
+--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
(
id int(10) unsigned NOT NULL AUTO_INCREMENT
, PRIMARY KEY (id)
)
ENGINE=spider
COMMENT='wrapper "mysql", table "shardTest"'
PARTITION BY KEY (id)
(
PARTITION shard1 COMMENT = 'srv "shard_node1"'
) ;
SELECT * FROM log_db.shardtest;
+----+-----------+
| id | name |
+----+-----------+
| 1 | SPIDER_01 |
| 5 | key1 |
| 4 | SPIDER_01 |
…
| 6 | key1 |
+----+-----------+
(원격데이터 정상추출)
결론 :
FEDERATED 와 SPIDER ENGINES
간의 의존성 없슴!
Q. Spider Engine 이 Federated Engine 과 의존성이 있는 것은 아닐까? (FEDERATED STORAGE ENGINES OFF?)

■ spider_internal_sql_log_off
shard된 서버에 spider 엔진에서 보낸 Query를
general log 기록합니다.
구문
(show table status from `log_db` like 'v_shardtest_group‘)
(show index from `log_db`.`v_shardtest_group`)
■ spider_remote_sql_log_off
shard된 서버에 sql_log_off 를 설정합니다.
구문
(set session sql_log_off = 1; )

Spider Engine
insert into shardtest(name) values ('spider_01');
Node 1
root@192.168.124.137 on using TCP/IP
set session transaction isolation level read committed;
set session autocommit = 1;
start transaction
SET NAMES utf8
log_db
select ìd`,`name` from `log_db`.`shardTestòrder by ìd` desc
limit 1 for update
insert into `log_db`.`shardTest`(ìd`,`name`)values(5,'spider_01')
commit
Node 2 Node 3 Node 4
select ìd`,`name` from `log_db`.`shardTest` order by ìd` desc
limit 1 for update
UPDATE Lock?
UPDATE Lock?
Limitation of a Spider Engine
1. INSERT (Auto Increment 발번)

Spider Engine Node 1 Node 2 Node 3 Node 4
Auto_increment = 1
select ìd`,`name` from `log_db`.`shardTestòrder by
ìd` desc limit 1 for update
각 노드 Auto_increment MAX 확인
Auto_increment = 2
노드 선택
insert into shardtest(id,name) values (1,'spider_01');
insert into shardtest(id,name) values
(1,'spider_01');
Auto_increment = 2
Node 1
select ìd`,`name` from `log_db`.`shardTestòrder by
ìd` desc limit 1 for update
각 노드 Auto_increment MAX 확인
Auto_increment = 3
노드 선택
insert into shardtest(id,name) values (2,'spider_01');
Node 3
insert into shardtest(id,name) values
(2,'spider_01');

■ spider_auto_increment_mode
-1 : 테이블 매개 변수를 사용한다.
0 : 일반모드
자동 증가 값을 원격 서버에서 얻은 카운터를 사용한다.
1 : Quick 모드
자동 증가 값을 Spider Node 내부 카운터를 사용한다.
2 : Set Zero Value
자동 증가 값을 Data Node 내부 카운터를 사용한다.
3 : null 입력시 자동 증가 값은 원격 서버에서 생성
0을 입력하면 로컬 서버에서 생성
☞ Reference
https://mariadb.com/kb/en/mariadb/spider-server-system-variables/

Node 3Node 2Node 1
Node 4
Spider Engine
set spider_auto_increment_mode = 1;
root@192.168.124.137 on using TCP/IP
set session transaction isolation level read committed;
set session autocommit = 1;
start transaction
SET NAMES utf8
log_db
insert into `log_db`.`shardTest`(`id`,`name`)values(6,'spider_01')
commit

1) spider_auto_increment_mode 설정별 성능
spider_auto_increment_mode QPS AVG Query sec
0 Avg : 45 0.021
1 Avg : 185 0.005
2 Avg : 184 0.005

2) spider_auto_increment_mode = 0 문제 발생
응답이 없음
Spider Engine
?

Commit 이 없음

general_log Commit
( X )

Spider Engine
Node 1
update shardtest set id = 9 where id = 8;
(select 0,ìd`,`name` from `log_db`.`shardTest` where ìd` = 8 for update)
delete from `log_db`.`shardTest` where ìd` = 8 and `name` = 'spider_01' limit 1
insert high_priority into `log_db`.`shardTest`(ìd`,`name`)values(9,'spider_01')
Data Move
Data Move
Node 1Node 4
Data move
CREATE TABLE `shardtest` (
...
) ENGINE=SPIDER AUTO_INCREMENT=10 DEFAULT CHARSET=utf8 COMMENT='wrapper
"mysql", table "shardTest" '
2. Update (Shard Key Update)
Change
Auto
increment
Node 4

Spider Engine
update shardtest set id = 9 where id = 8;
delete from `log_db`.`shardTest` where ìd` = 8 and `name` = 'spider_01' limit 1
insert high_priority into `log_db`.`shardTest`(ìd`,`name`)values(9,'spider_01')
update shardtest set id = 17 where id = 9; (select 0,ìd`,`name` from `log_db`.`shardTest` where ìd` = 9 for update)
update `shardTest` set ìd` = 17 where ìd` = 9 and `name` = 'spider_01' limit 1
Node 1Spider Engine
...
Node 1 Data move
...
Change
Auto
increment
2. Update (Shard Key Update)
Node 1Node 4
Data move
Change
Auto
increment
Node 1
Node 4
Data Move
Data Move

Node 1
update shardtest set name = 'spider_02' where id = 1;
update shardtest set name = 'spider_03' where name = 'spider_01';
Spider Engine
update `log_db`.`shardTest` set `name` = 'spider_02' where ìd` = 1 limit 1
update `log_db`.`shardTest` set `name` = 'spider_03' where ìd` = 2 and
`name` = 'spider_01' limit 1
(select 0,ìd`,`name` from `log_db`.`shardTest` where `name` = 'spider_01'
for update)
Shard key (X)
Node 1
2. Update (Shard Key 가 아닌 컬럼의 Update)
Spider Engine
Node 3

Spider Engine
for update)
Shard key (X)
1 건 업데이트
2. Update (Shard Key 가 아닌 컬럼의 Update)
Node 1
Spider Engine
for update)
Shard key (X)
Node 1
여러 건 업데이트
Node 1
Node 2 Node 3 Node 4Node 1
1건씩 처리됨
⇒ 운영시 binlog size 급증 이슈발생가능!

Truncate table shardTest; truncate table `log_db`.`shardTest`
3. Truncate
Spider Engine Node 2 Node 3 Node 4Node 1

Alter table shardTest ADD COLUMN TEST_Col1 int;
Alter table shardTest DROP COLUMN TEST_Col1 int;
CREATE INDEX idx_shardtest ON shardtest(name);
DROP INDEX idx_shardtest ON shardtest;
Shard Shema 변경 없음
4. Alter

Alter table shardtest ADD COLUMN (add_col1 int);
SELECT * FROM shardtest;
(ERROR 1054 (42S22): Unknown column 'add_col1' in 'field list‘)
SELECT id, name FROM shardtest;
+----+-----------+
| id | name |
+----+-----------+
| 1 | SPIDER_01 |
…
| 2 | SPIDER_01 |
+----+-----------+
4 rows in set (0.26 sec)
select `id`,`name` from `log_db`.`shardTest` order by `id`,`name`
General_log Query
4. Alter

4. Alter
INSERT INTO shardtest(name) VALUES('key2');
Query OK, 1 row affected (0.45 sec)
select id, name from shardtest;
+----+-----------+
| id | name |
+----+-----------+
| 1 | SPIDER_01 |
| 2 | SPIDER_01 |
| … |
| 5 | key2 |
+----+-----------+
UPDATE shardtest
set name = 'key3'
where id = 5;
ERROR 1054 (42S22): Unknown column 'add_col1' in 'field list'
insert into `log_db`.`shardTest`(ìd`,`name`) values (5,'key2')
(select 0,ìd`,`name`,àdd_col1` from `shardTest` where ìd` = 5 for update)
rollback
General_log Query

Alter table shardtest DROP COLUMN (add_col1 int);
4. Alter
General_log Query
+----+-----------+
| id | name |
+----+-----------+
| 1 | SPIDER_01 |
| 2 | SPIDER_01 |
| 3 | SPIDER_01 |
| 4 | SPIDER_01 |
| 5 | key1 |
| 6 | key1 |
| 7 | SPIDER_01 |
| 8 | SPIDER_01 |
+----+-----------+
Select OK
(각 Data 노드에 컬럼추가)
select `id`,`name` from `log_db`.`shardTest` order by `id`,`name`
※ Schema 수정 순서에 유의
1) Data node 수정
2) Spider node 수정

Spider Node
“Sort buffer size”
설정에 따른 성능테스트
약 100만 row TEST
+-----+--------------+-------------+
| int | varchar(120) | varchar(10) |
+-----+--------------+-------------+
| id | name | node_num |
+-----+--------------+-------------+
| 1 | aaa | node1 |
+-----+--------------+-------------+
TEST QUERY
Variables (sort_buffer_size) avg retun time
256K 16.612
512K 12.76
1M 10.562
2M 9.988
4M 7.812
8M 7.93
16M 7.508 0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
256K 512K 1M 2M 4M 8M 16M
avg retun time

File Sort Algorithm
1. Single Pass
 SELECT되는 모든 컬럼
버퍼에 담아 정렬하여
바로 리턴
2. Two Pass
 정렬 대상 + KEY로 정렬
하여 해당 정렬 기준으
로 KEY값 리턴
※ Two Pass는 정렬을 위하여
temp file을 사용한다.
Sort_merge_passes는 Merge
횟수를 카운트 한다.
약 100만 row TEST
+-----+--------------+-------------+
| int | varchar(120) | varchar(10) |
+-----+--------------+-------------+
| id | name | node_num |
+-----+--------------+-------------+
| 1 | aaa | node1 |
+-----+--------------+-------------+
TEST QUERY
Variables (sort_buffer_size) avg retun time
256K 16.612
512K 12.76
1M 10.562
2M 9.988
4M 7.812
8M 7.93
16M 7.508
상태 확인 : show status like ‘%sort%’

INSERT INTO shardtest(name) values('SPIDER_01');
…
INSERT INTO shardtest(name) values('SPIDER_04’);
INSERT INTO shardtest(name) values('key1');
select ìd`,`name` from `log_db`.`shardTest`SELECT * FROM shardtest where name = 'key1';
+----+------+
| id | name |
+----+------+
| 5 | key1 |
+----+------+
Spider Engine
Filtering
Shard DB Filter ? ?
5. SELECT (where)
ìd` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` char(120) NOT NULL DEFAULT '',
PRIMARY KEY (ìd`),
) ENGINE=SPIDER DEFAULT CHARSET=utf8 COMMENT='wrapper "mysql", table
"shardTest" '
/*!50100 PARTITION BY KEY (ìd`)
(PARTITION shard1 COMMENT = 'srv "shard_node1＂ ' ENGINE = SPIDER,
…
PARTITION shard4 COMMENT = 'srv "shard_node4＂ ' ENGINE = SPIDER) */

select ìd`,`name` from `log_db`.`shardTest` where `name` = 'key1'
Shard DB Filtering !!!
CREATE INDEX idx_shardtest_01 ON shardtest(name);
SELECT * FROM shardtest where name = 'key1';
+----+------+
| id | name |
+----+------+
| 5 | key1 |
+----+------+
PRIMARY KEY (ìd`),
KEY ìdx_shardtest_01` (`name`)
) ENGINE=SPIDER DEFAULT CHARSET=utf8 COMMENT='wrapper "mysql", table
"shardTest" '
(PARTITION shard1 COMMENT = 'srv "shard_node1＂ ' ENGINE = SPIDER,
PARTITION shard2 COMMENT = 'srv "shard_node2＂ ' ENGINE = SPIDER,
PARTITION shard3 COMMENT = 'srv "shard_node3＂ ' ENGINE = SPIDER,
PARTITION shard4 COMMENT = 'srv "shard_node4＂ ' ENGINE = SPIDER)
*/
5. SELECT (where)

select name from shardtest group by name;
KEY `idx_shardtest_01` (`name`) 있는 상태
select count(*) from `log_db`.`shardTest`
select `name` from `log_db`.`shardTest` order by `name` desc
select `name` from `log_db`.`shardTest` order by `name`
select `name` from `log_db`.`shardTest` where `name` > 'key1' order by `name`
select `name` from `log_db`.`shardTest` where `name` > 'SPIDER_01' order by `name`
+-----------+
| name |
+-----------+
| key1 |
| SPIDER_01 |
+-----------+
KEY `idx_shardtest_01` (`name`) 를 제거하면…
DROP INDEX idx_shardtest_01 ON shardtest;
select name from shardtest group by name;
+-----------+
| name |
+-----------+
| key1 |
| SPIDER_01 |
+-----------+
select `name` from `log_db`.`shardTest`
Shard DB Grouping??
5. SELECT (Group By)
Shard DB Full ScanShard DB Grouping

select id from shardtest group by name;
+----+
| id |
+----+
| 5 |
| 1 |
+----+
Shard DB Sorting
select `id`,`name` from `log_db`.`shardTest` order by `name`
Shard DB Grouping
select name, count(*) from shardtest group by name;
+-----------+----------+
| name | count(*) |
+-----------+----------+
| key1 | 2 |
| SPIDER_01 | 6 |
+-----------+----------+
select count(*) from `log_db`.`shardTest` ?

`node_num` varchar(10) DEFAULT NULL,
PRIMARY KEY (ìd`)
)
ENGINE=InnoDB AUTO_INCREMENT=1048576 DEFAULT CHARSET=utf8
5. SELECT (Group By MIN, MAX, COUNT(*) {INDEX (O)})
select max(id) from shardtest;
select min(id) from shardtest;
select count(id) from shardtest;
select count(*) from shardtest;
select name, min(id) from shardtest;
select name, max(id) from shardtest;
select name, count(*) from shardtest;
select name, count(id) from shardtest;
select ìd` from `log_db`.`shardTest` order by ìd` desc limit 1;
select ìd` from `log_db`.`shardTest` order by ìd` limit 1;
select count(*) from `log_db`.`shardTest`;
select ìd`,`name` from `log_db`.`shardTest` order by ìd`,`name`;
select `name` from `log_db`.`shardTest` order by `name`;
“Name”
index (X)
PRIMARY KEY (ìd`),
KEY ìdx_shardtest_01` (`name`),
KEY ìdx_shardtest_02` (ìd`,`name`)
) …

5. SELECT (Group By MIN, MAX, COUNT(*) {INDEX (X)})
select max(name) from shardtest;
select min(name) from shardtest;
select count(name) from shardtest;
select count(*) from shardtest;
select name, min(id) from shardtest;
select name, max(id) from shardtest;
select name, count(*) from shardtest;
select name, count(id) from shardtest;
select `name` from `log_db`.`shardTest`;
select `name` from `log_db`.`shardTest`;
select ìd`,`name` from `log_db`.`shardTest`;
PRIMARY KEY (ìd`)
)
ENGINE=InnoDB AUTO_INCREMENT=1048576 DEFAULT CHARSET=utf8
PRIMARY KEY (ìd`)
) …
Full Scan

CREATE VIEW v_shardtest_group
AS
select name, max(id) as id, count(*) as cnt
from shardtest
group by name;
CREATE TABLE `v_shardtest_group` (
ìd` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`name` CHAR(120) NOT NULL DEFAULT '',
`cnt` INT NOT NULL,
PRIMARY KEY (ìd`),
KEY ìdx_shardtest_01` (`name`)
)
ENGINE=SPIDER
DEFAULT CHARSET=utf8
COMMENT='wrapper "mysql", table "v_shardtest_group" '
(PARTITION shard1 COMMENT = 'srv "shard_node1"' ENGINE = SPIDER,
PARTITION shard2 COMMENT = 'srv "shard_node2"' ENGINE = SPIDER,
PARTITION shard4 COMMENT = 'srv "shard_node4"' ENGINE = SPIDER) */ ;
select name, sum(cnt) from v_shardtest_group group by name; select `name`,`cnt` from `log_db`.`v_shardtest_group` order by `name`

select * from shardtest order by name;
select id from shardtest order by id;
select name from shardtest order by id;
select id from shardtest order by name;
select name from shardtest order by name;
5. SELECT (Order By)
select `id` from `log_db`.`shardTest` order by `id`
Shard DB DataSpider DB Sorting

create table join_tb(
jt_no int auto_increment
, ct_code char(4)
, ct_cnt int
, primary key(jt_no)
);
create table common_tb(
ct_no int auto_increment
, ct_code char(4)
, ct_name varchar(30)
, primary key (ct_no)
, key (ct_code)
)
ENGINE=SPIDER DEFAULT CHARSET=utf8
COMMENT='wrapper "mysql", table "common_tb" '
/*!50100 PARTITION BY KEY (ct_no)
(PARTITION shard1 COMMENT = 'srv "shard_node1"' ENGINE = SPIDER) */
create table join_tb(
jt_no int auto_increment
, ct_ code char(4)
, ct_ cnt int
, primary key(jt_no)
)
COMMENT='wrapper "mysql", table "join_tb" '
/*!50100 PARTITION BY KEY (`jt_no`)
PARTITION shard4 COMMENT = 'srv "shard_node4"' ENGINE = SPIDER) */
create table common_tb(
ct_no int auto_increment
, ct_code char(4)
, primary key (ct_no)
);
5. SELECT (Join)
Spider Engine Node 1

insert into join_tb(ct_code, ct_cnt) values ('A001',1);
…
insert into common_tb(ct_code, ct_name) values ('A001','TEST1');
…
select a.ct_cnt, a.ct_code, b.ct_name
from join_tb a
, common_tb b
where a.ct_code = b.ct_code;
select `ct_code`,`ct_cnt` from `log_db`.`join_tb`
select `ct_code`,`ct_name` from `log_db`.`common_tb` where `ct_code` = 'A010'
…
…
5. SELECT (Join)
join_tb
common_tb
join_tb join_tb join_tb
1단계 : join_tb 데이터 스캔
2단계 : 1단계에서 읽은 데이터를 대상
으로 common_tb 데이터 확인

Spider Engine
Node 1
join
Node 2
join
Node 3
join
Node 4
join
Spider Engine
join
Node 1
Node 2
Node 3
Node 4
5. SELECT (Join)

Insert common_tb
common_tb
DATA
common_tb
DATA
common_tb
DATA
common_tb
DATA
?
Global table Join View
View ?
5. SELECT (Join)

alter table common_tb
COMMENT='wrapper "mysql", table "common_tb" '
(PARTITION shard1
COMMENT = 'srv "shard_node2 shard_node3 shard_node4 shard_node1"'
ENGINE = SPIDER)
*/
TRUNCATE TABLE common_tb;
…
insert into `common_tb`(`ct_no`,`ct_code`,`ct_name`)values(1,'A001','TEST01')
…
5. SELECT (Join)

CREATE ALGORITHM = MERGE VIEW v_join_tb
as
select a.jt_no, a.ct_cnt, a.ct_code a_ct_code, b.ct_code b_ct_code, b.ct_name
from join_tb a
, common_tb b
where a.ct_code = b.ct_code;
CREATE TABLE v_join_tb(
jt_no int
, ct_cnt int
, a_ct_code char(4)
, b_ct_code char(4)
, primary key (jt_no)
, key (a_ct_code, b_ct_code)
)
COMMENT='wrapper "mysql", table "v_join_tb" '
/*!50100 PARTITION BY KEY (`jt_no`)
..
PARTITION shard4 COMMENT = 'srv "shard_node4"' ENGINE = SPIDER) */;
select * from v_join_tb
where a_ct_code = 'A001' and b_ct_code = 'A001';
select `jt_no`,`ct_cnt`,`a_ct_code`,`b_ct_code`,`ct_name`
from `log_db`.`v_join_tb`
where `a_ct_code` = 'A001' and `b_ct_code` = 'A001'
Key Point !!
5. SELECT (Join)
CREATE VIEW ALGORITHM
1) MERGE
2) TEMPTABLE

select * from v_join_tb
where a_ct_code = 'A001’and b_ct_code = 'A001'; select `jt_no`,`ct_cnt`,`a_ct_code`,`b_ct_code`,`ct_name`
from `log_db`.`v_join_tb`
where `a_ct_code` = 'A001' and `b_ct_code` = 'A001'
5. SELECT (Join)
INDEX Access OK !

SELECT *
FROM shardtest
WHERE name IN ('key1','SPIDER_01');
+----+-----------+
| id | name |
+----+-----------+
| 1 | SPIDER_01 |
| 5 | key1 |
| 4 | SPIDER_01 |
| 8 | SPIDER_01 |
| 3 | SPIDER_01 |
| 7 | SPIDER_01 |
| 2 | SPIDER_01 |
| 6 | key1 |
+----+-----------+
Temporary Table ← IN KEY
?
5. SELECT ( IN( ) )
drop temporary table if exists log_db.tmp_spider_bka_0x7f5ba7f23420_shardTest;
create temporary table log_db.tmp_spider_bka_0x7f5ba7f23420_shardTest(
id bigint
, c0 char(120) collate utf8_general_ci
)
engine=memory default charset=utf8 collate utf8_general_ci;
insert into log_db.tmp_spider_bka_0x7f5ba7f23420_shardTest(id,c0)values(0,'key1'),(1,'SPIDER_01');
select a.id,b.`id`,b.`name`
from log_db.tmp_spider_bka_0x7f5ba7f23420_shardTest a,`log_db`.`shardTest` b
where a.c0 <=> b.`name`;
drop temporary table if exists log_db.tmp_spider_bka_0x7f5ba7f23420_shardTest;
OR equal

5. SELECT ( IN( ) )
SELECT * FROM shardtest WHERE name = 'key1'
UNION ALL
SELECT * FROM shardtest WHERE name = 'SPIDER_01';
+----+-----------+
| id | name |
+----+-----------+
| 5 | key1 |
| 6 | key1 |
| 1 | SPIDER_01 |
| 4 | SPIDER_01 |
| 8 | SPIDER_01 |
| 3 | SPIDER_01 |
| 7 | SPIDER_01 |
| 2 | SPIDER_01 |
+----+-----------+
select `id`,`name` from `log_db`.`shardTest` where `name` = 'SPIDER_01'

create table shardtest_key (
shard_key int
, shard_no int NOT NULL AUTO_INCREMENT
, val varchar(100)
, PRIMARY KEY (shard_no, shard_key)
)
COMMENT='wrapper "mysql", table "shardtest_key" '
/*!50100 PARTITION BY LIST (`shard_key`)
(PARTITION shard1 VALUES IN (1) COMMENT = 'srv "shard_node1"' ENGINE = SPIDER,
PARTITION shard3 VALUES IN (3) COMMENT = 'srv "shard_node3"' ENGINE = SPIDER,
PARTITION shard2 VALUES IN (2) COMMENT = 'srv "shard_node2"' ENGINE = SPIDER) */
create table shardtest_key (
shard_key int
, shard_no int NOT NULL AUTO_INCREMENT
, val varchar(100)
, PRIMARY KEY (shard_no)
);
insert into shardtest_key(shard_key, val) values (1,'node1');
…
5. SELECT ( IN( ) + shardkey)

SELECT *
FROM shardtest_key
WHERE shard_key in (1,2)
AND shard_no in (1,5,9,2,26);
+-----------+----------+-------+
| shard_key | shard_no | val |
+-----------+----------+-------+
| 1 | 1 | node1 |
| 1 | 5 | node1 |
| 1 | 9 | node1 |
| 2 | 2 | node2 |
| 2 | 26 | node2 |
+-----------+----------+-------+
Spider Engine Node 2
Node 3 Node 4
Node 1
5. SELECT ( IN( ) + shardkey )
drop temporary table if exists log_db.tmp_spider_bka_0x7f0c403d1820_shardtest_key;
create temporary table log_db.tmp_spider_bka_0x7f0c403d1820_shardtest_key(
id bigint
, c0 int(11)
, c1 int(11)
)
engine=memory default charset=utf8 collate utf8_general_ci;
insert into log_db.tmp_spider_bka_0x7f0c403d1820_shardtest_key(id,c0,c1)
values(0,1,1),(1,2,1),(2,5,1),(3,9,1),(4,26,1)
select a.id,b.`shard_key`,b.`shard_no`,b.`val`
from log_db.tmp_spider_bka_0x7f0c403d1820_shardtest_key a
,`log_db`.`shardtest_key` b
where a.c0 <=> b.`shard_no` and a.c1 <=> b.`shard_key`
drop temporary table if exists log_db.tmp_spider_bka_0x7f0c403d1820_shardtest_key
values(0,1,2),(1,2,2),(2,5,2),(3,9,2),(4,26,2)

id >= 1 and id <= 6
equal
5. SELECT ( BETWEEN )
SELECT * FROM shardtest where id between 1 and 6;
+----+-----------+
| id | name |
+----+-----------+
| 1 | SPIDER_01 |
| 5 | key1 |
| 4 | SPIDER_01 |
| 3 | SPIDER_01 |
| 2 | SPIDER_01 |
| 6 | key1 |
+----+-----------+
(select 0,ìd`,`name`
from `log_db`.`shardTest`
where ìd` >= 1 and ìd` <= 6
order by ìd`)
order by ìd`

explain SELECT * FROM shardtest where name = 'key1' and id >= 1 and id <= 6;
+------+-------------+-----------+-------+--------------------------+---------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------+-------+--------------------------+---------+---------+------+------+-------------+
| 1 | SIMPLE | shardtest | range | PRIMARY,idx_shardtest_01 | PRIMARY | 4 | NULL | 8 | Using where |
+------+-------------+-----------+-------+--------------------------+---------+---------+------+------+-------------+
SELECT *
FROM shardtest
WHERE name = 'key1‘
AND id >= 1 and id <= 6;
+----+------+
| id | name |
+----+------+
| 5 | key1 |
| 6 | key1 |
+----+------+
order by `id`)
order by `id`
Spider Engine
“name” filter
Shard DB Data
Index PRIMARY KEY

explain SELECT * FROM shardtest use index(idx_shardtest_01) where name = 'key1' and id >= 1 and id <= 6;
+------+-------------+-----------+------+------------------+------------------+---------+-------+------+-------------+
+------+-------------+-----------+------+------------------+------------------+---------+-------+------+-------------+
| 1 | SIMPLE | shardtest | ref | idx_shardtest_01 | idx_shardtest_01 | 360 | const | 9687 | Using where |
+------+-------------+-----------+------+------------------+------------------+---------+-------+------+-------------+
idx_shardtest_01 : name
SELECT *
FROM shardtest use index(idx_shardtest_01)
AND id >= 1 and id <= 6;
+----+------+
| id | name |
+----+------+
| 5 | key1 |
| 6 | key1 |
+----+------+
Spider Engine
“id” filter
Shard DB Data

create index idx_shardtest_03 on shardtest(name, id);
explain SELECT * FROM shardtest use index(idx_shardtest_03) where name = 'key1' and id between 1 and 6;
+------+-------------+-----------+-------+------------------+------------------+---------+------+------+--------------------------+
+------+-------------+-----------+-------+------------------+------------------+---------+------+------+--------------------------+
| 1 | SIMPLE | shardtest | range | idx_shardtest_03 | idx_shardtest_03 | 364 | NULL | 8 | Using where; Using index |
+------+-------------+-----------+-------+------------------+------------------+---------+------+------+--------------------------+
SELECT *
FROM shardtest use index(idx_shardtest_03)
AND id between 1 and 6;
+----+------+
| id | name |
+----+------+
| 5 | key1 |
| 6 | key1 |
+----+------+
and `name` = 'key1'
order by `id`,`name`
)
order by `id`,`name`

Schema
• Shard 별로 스키마 구성 (물리적으로 같은 위치에)
• 자동 증가값은 Spider Engine 에서 처리하도록 구성
• Global 스카마의 경우 HA 기능을 사용하여 각 서버에 복제본 구성
• Index 는, Data node 에는 전통적으로 필요한 index 생성
Spider node 에는 필요한 모든 컬럼 index 생성
Performance
• 각 Node에서 가장 성능이 나쁜 서버의 속도가 전체 성능을 결정한다.
(아쉽게도 각 Node에 Query를 parallel 하게 전달하지는 않는다.)
Query
• IN절과 ORDER BY 절은 최소화
• 조건절에 Filter 위치를 확실히 확인
• 설계 및 개발단계에 선반영 검토 (ex. 모든 쿼리에 Shard Key 조건을 추가하여 개발)
Guideline

Q & A
■ Spider_internal_limit
shard node에 레코드 LIMIT 제한을 준다.
■ Spider_internal_limit = 100
Query : SELECT * FROM shardtest limit 101
Node 2
Node 3
Node 4
Node 1

Q & A
Query : SELECT * FROM shardtest order by id limit 101
■ Spider_internal_limit = -1
Query : SELECT * FROM shardtest order by id limit 100

Q & A
Query : SELECT * FROM shardtest limit 1000
■ 결과

Q & A
■ 스키마 변경은 각 노트에 상태와 무관한가?
Node 4
Shutdown
Spider Engine

Q & A
Node4에 접속 불가

Q & A
ALL Node
Shutdown

Q & A
■ Spider node에 shard_key Partition + 각 노드 Partition 구조 및 인덱스 설계 고려 사항이 있는가?
CREATE TABLE `shardtest_key` (
`shard_key` int(11) DEFAULT NULL,
`shard_no` int(11) NOT NULL AUTO_INCREMENT,
Partkey_week INT,
`val` varchar(100) DEFAULT NULL,
PRIMARY KEY (`shard_no`,Partkey_week)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
/*!50100 PARTITION BY LIST (Partkey_week)
(PARTITION character_crud_log_P00 VALUES IN (0) ENGINE = InnoDB,
PARTITION character_crud_log_P01 VALUES IN (1) ENGINE = InnoDB,
PARTITION character_crud_log_P05 VALUES IN (5) ENGINE = InnoDB) */;
Spider Engine
CREATE TABLE `shardtest_key` (
`shard_key` int(11) NOT NULL DEFAULT '0',
`shard_no` int(11) NOT NULL AUTO_INCREMENT,
Partkey_week int,
`val` varchar(100) DEFAULT NULL,
PRIMARY KEY (`shard_no`,shard_key, Partkey_week),
key (shard_key, Partkey_week)
) ENGINE=SPIDER DEFAULT CHARSET=utf8 COMMENT='wrapper "mysql", table "shardtest_key" '
/*!50100 PARTITION BY LIST (`shard_key`)
(PARTITION shard1 VALUES IN (1) COMMENT = 'srv "shard_node1"' ENGINE = SPIDER,
PARTITION shard2 VALUES IN (2) COMMENT = 'srv "shard_node2"' ENGINE = SPIDER) */
■ 결론
위에서 이야기 했던 것 처럼 각 노드에 필요한 조건이 모두 전달되려면 Spider에 인덱스가 존재해야 한다.
결국 partition을 제대로 사용하려면 spider 쪽에 (shard_key + Partition_key + 필요한 인덱스) 구조로 인덱스를 만들어야 한다.

Q & A
Spider Engine
실행 계획
1,2,3
1,2,3
■ Spider node에 shard_key Partition + 각 노드 Partition 구조 및 인덱스 설계 고려 사항이 있는가?

Q & A
■ Spider 복제 구성에서 SELECT 되는 기준 노드는?
Spider Engine
Node 2
Node 3
Node 4
Node 1
Client
insert
insert
Client
select
select
Insert
부하
Insert & Select
부하

Q & A
Spider Engine
CREATE TABLE `common_tb` (
`ct_no` int(11) NOT NULL AUTO_INCREMENT,
`ct_code` char(4) DEFAULT NULL,
`ct_name` varchar(30) DEFAULT NULL,
`node_name` varchar(6),
PRIMARY KEY (`ct_no`),
KEY `ct_code` (`ct_code`)
) ENGINE=SPIDER DEFAULT CHARSET=utf8 COMMENT='wrapper "mysql", table "common_tb" '
(PARTITION shard1 COMMENT = 'srv "shard_node1 shard_node2 shard_node3 shard_node4"' ENGINE = SPIDER) */
CREATE TABLE `common_tb` (
`ct_no` int(11) NOT NULL AUTO_INCREMENT,
`ct_code` char(4) DEFAULT NULL,
`ct_name` varchar(30) DEFAULT NULL,
`node_name` varchar(6) DEFAULT 'node4',
PRIMARY KEY (`ct_no`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8
Node 1
Node 2
Node 3
Node 4
Node1, Node2, Node3, Node4

Q & A
Spider Engine
General_log

Deep Dive Spider Engine

More Related Content

What's hot (20)

Similar to Deep Dive Spider Engine (18)

More from I Goo Lee (20)

Recently uploaded (20)

Deep Dive Spider Engine