[Pgday.Seoul 2018] 이기종 DB에서 PostgreSQL로의 Migration을 위한 DB2PG
1 like2,143 views
This document discusses DB2PG, a tool for migrating data between different database management systems. It began as an internal project in 2016 and has expanded its supported migration paths over time. It can now migrate schemas, tables, data types and more between Oracle, SQL Server, DB2, MySQL and other databases. The tool uses Java and supports multi-threaded imports for faster migration. Configuration files allow customizing the data type mappings and queries used during migration. The tool is open source and available on GitHub under the GPL v3 license.
![[Pgday.Seoul 2018] 이기종 DB에서 PostgreSQL로의 Migration을 위한 DB2PG](https://image.slidesharecdn.com/04-pgdaydb2pgv1-181112042107/85/Pgday-Seoul-2018-DB-PostgreSQL-Migration-DB2PG-7-320.jpg)
![/db2pg/setting/
1 {
2 "comment": {
3 "postgres": "#",
4 "mysql": [
5 "--"
6 ]
7 },
8 "string": {
9 "postgres": "'",
10 "mysql": [
11 """
12 ]
13 },
14 "classify_string": {
15 "postgres": """,
16 "mysql": [
17 "`"
18 ]
19 },
20 "integer_case_1": {
21 "postgres": "SMALLINT",
22 "mysql": [
23 "^(?i)TINYINTs*(?[0-9]*)s*UNSIGNED?$",
24 "^(?i)SMALLINT$(?[0-9]*)?$",
25 "^(?i)TINYINTs*(?[0-9]*)?$"
26 ],
27 "mss": [
28 "^(?i)TINYINT$"
29 ]
각 DB별
Data Type
변환 방식을
설정](https://image.slidesharecdn.com/04-pgdaydb2pgv1-181112042107/85/Pgday-Seoul-2018-DB-PostgreSQL-Migration-DB2PG-11-320.jpg)
![[Pgday.Seoul 2018] 이기종 DB에서 PostgreSQL로의 Migration을 위한 DB2PG](https://image.slidesharecdn.com/04-pgdaydb2pgv1-181112042107/85/Pgday-Seoul-2018-DB-PostgreSQL-Migration-DB2PG-18-320.jpg)
[Pgday.Seoul 2018] 이기종 DB에서 PostgreSQL로의 Migration을 위한 DB2PG
- 1. DB2PG
- 2. - 비용절감? - 정책 이슈 - 성능, 기능이슈 - 사용상의 편의성 - Cloud등으로의 환경변화 - 기타 등등 . . .
- 3. - Oracle이외의 DB는 어떤 tool로? - ora2pg의 성능(대용량) - Perl engineer? NO, java engineer? Yes - 수작업 ㅠㅠ - 국가정보자원관리원 Project진행
- 4. - 2016. 05월 사내 Project로 시작 - 2017. 03월 Oracle Spatial data 지원 - 2017. 10월 MS-SQL 지원 - 2018. 05월 공개SW 기술개발 지원사업 참여 - 2018. 06월 OpenSource화 결정 및 Github - 2018. 10월 Oracle, DB2, MS-SQL, Sybase, Mysql지원
- 5. - 제품별 데이터 구조의 호환성 - GIS데이터와 같은 특수한 데이터 변환 - DB별 개성적인 DDL - Resource 부족(개발자, DBA)
- 6. - PostgreSQL 시장확대 - 공개SW 기술개발 지원사업 참여 - 개발자의 참여 유도(능력자필요) - K4M value up!
- 8. DBMS DDL Extract Data Migration Oracle Yes Yes MS-SQL Yes Yes DB2 Yes Sybase Yes Mysql Yes Yes cubrid, altibase Soon
- 9. - Schema - Tables - Primary key - Unique - Foreign Key - View - Sequence - Index
- 10. Github db2pg / / Images / / setting / convert_map.json / setting / mapper / MetaExtractMapper.xml / src/main/java : java source
- 11. /db2pg/setting/ 1 { 2 "comment": { 3 "postgres": "#", 4 "mysql": [ 5 "--" 6 ] 7 }, 8 "string": { 9 "postgres": "'", 10 "mysql": [ 11 """ 12 ] 13 }, 14 "classify_string": { 15 "postgres": """, 16 "mysql": [ 17 "`" 18 ] 19 }, 20 "integer_case_1": { 21 "postgres": "SMALLINT", 22 "mysql": [ 23 "^(?i)TINYINTs*(?[0-9]*)s*UNSIGNED?$", 24 "^(?i)SMALLINT$(?[0-9]*)?$", 25 "^(?i)TINYINTs*(?[0-9]*)?$" 26 ], 27 "mss": [ 28 "^(?i)TINYINT$" 29 ] 각 DB별 Data Type 변환 방식을 설정
- 12. /db2pg/setting/mapper/ 1 <?xml version="1.0" encoding="UTF-8"?> 2 <!DOCTYPE mapper 3 PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" 4 "http://mybatis.org/dtd/mybatis-3-mapper.dtd"> 5 6 <mapper namespace="com.k4m.experdb.db2pg.mapper.MetaExtractMapper"> 7 8 <resultMap id="getColumnListResult" type="com.k4m.experdb.db2pg.common.LowerKeyMap"> 9 <result property="ORDINAL_POSITION" column="ORDINAL_POSITION" /> 10 <result property="COLUMN_NAME" column="COLUMN_NAME" /> 11 <result property="COLUMN_TYPE" column="COLUMN_TYPE" /> 12 <result property="COLUMN_DEFAULT" column="COLUMN_DEFAULT" jdbcType="VARCHAR" javaType="java.lang.String" /> 13 <result property="IS_NULL" column="IS_NULL" /> 14 <result property="COLUMN_COMMENT" column="COLUMN_COMMENT" /> 15 <result property="NUMERIC_PRECISION" column="NUMERIC_PRECISION" /> 16 <result property="NUMERIC_SCALE" column="NUMERIC_SCALE" /> </resultMap> 17 18 <resultMap id="getViewInformResult" type="com.k4m.experdb.db2pg.common.LowerKeyMap"> 19 <result property="VIEW_NAME" column="VIEW_NAME" /> 20 <result property="TEXT" column="TEXT" jdbcType="VARCHAR" javaType="java.lang.String" /> 21 </resultMap> 22 23 <select id="getTableNames" parameterType="map" resultType="java.lang.String" databaseId="ORA"> 24 SELECT OBJECT_NAME as TABLE_NAME FROM ALL_OBJECTS 25 WHERE OWNER=#{TABLE_SCHEMA} 26 AND OBJECT_NAME NOT IN ('TOAD_PLAN_TABLE','PLAN_TABLE') 27 AND OBJECT_NAME NOT LIKE 'MDRT%' 28 AND OBJECT_NAME NOT LIKE 'MDXT%' 29 <choose> 각 DBMS별 Object 추출 Query
- 13. db2pg.config 위치 : /db2pg/src/main/java/com/k4m/experdb/db2pg/sample/ 1 SRC_EXPORT=FALSE 2 PG_CONSTRAINT_EXTRACT=FALSE 3 SRC_DDL_EXPORT=FALSE 4 5 SRC_HOST= 6 SRC_USER= 7 SRC_PASSWORD= 8 SRC_DATABASE= 9 SRC_SCHEMA= 10 SRC_DB_TYPE=ORA 11 SRC_PORT=1521 12 SRC_DB_CHARSET=UTF8 13 SRC_LOB_FETCH_SIZE=1024 14 SRC_STATEMENT_FETCH_SIZE=3000 15 SRC_TABLE_SELECT_PARALLEL=1 16 SRC_TABLE_COPY_SEGMENT_SIZE=3000 17 VERBOSE=TRUE 18 #SRC_WHERE= 19 TABLE_ONLY=TRUE 20 TRUNCATE=FALSE 21 #SRC_ALLOW_TABLES= 22 #SRC_EXCLUDE_TABLES= 23 #SRC_ROWNUM= 24 TAR_HOST= 25 TAR_USER= 26 TAR_PASSWORD= 27 TAR_DATABASE= 28 TAR_SCHEMA= 29 TAR_PORT= Source 및 Target의 Connection info
- 14. queries.xml 위치 : /db2pg/src/main/java/com/k4m/experdb/db2pg/sample/ 1 <QUERIES> 2 <QUERY> 3 <NAME></NAME> 4 <SELECT> 5 </SELECT> 6 </QUERY> 7 </QUERIES> Query를 이용한 Migration시 설정
- 15. ora2pg vs DB2PG ora2pg db2pg Language perl java Multi thread Import NO YES DB to DB Migration NO YES Data type Mapping file NO YES Support DBMS 1+1(mysql) 5 DB2PG fast than ora2pg at lease 2
- 16. GPL v3
- 17. https://github.com/experdb/eXperDB-DB2PG facebook : https://www.facebook.com/experdb naver cafe : http://cafe.naver.com/psqlmaster
- 19. - Support Partition table - cubrid data migration - altibase data migration - Everything you want!!(wish) 능력자분들의 많은 참여 부탁 드립니다.
- 20. Q & A