Dbdeployer
0 recomendaciones666 vistas
Este documento presenta dbdeployer, una herramienta desarrollada en Go por Giuseppe Maxia para instalar y administrar sandboxes de MySQL de manera sencilla. Dbdeployer mejora sobre la herramienta anterior MySQL-Sandbox de Maxia al eliminar dependencias y hacer más fácil la instalación. Dbdeployer permite instalar sandboxes individuales, múltiples o con replicación de varias topologías de manera automática. La herramienta es de código abierto y el autor invita contribuciones para seguir mejorándola.
![dbdeployer help
$ dbdeployer --help
dbdeployer makes MySQL server installation an easy task.
Runs single, multiple, and replicated sandboxes.
Usage:
dbdeployer [command]
Available Commands:
admin sandbox management tasks
defaults tasks related to dbdeployer defaults
delete delete an installed sandbox
deploy deploy sandboxes
global Runs a given command in every sandbox
help Help about any command
sandboxes List installed sandboxes
unpack unpack a tarball into the binary directory
usage Shows usage of installed sandboxes
versions List available versions](https://image.slidesharecdn.com/dbdeployeresp-190118161006/85/Dbdeployer-27-320.jpg)
![unpack
$ dbdeployer unpack -h
If you want to create a sandbox from a tarball, you first need to
unpack it into the sandbox-binary directory. This command carries out
that task, so that afterwards you can call 'deploy single', 'deploy
multiple', and 'deploy replication' commands with only the MySQL
version for that tarball.
If the version is not contained in the tarball name, it should be
supplied using --unpack-version.
If there is already an expanded tarball with the same version, a new
one can be differentiated with --prefix.
Usage:
dbdeployer unpack MySQL-tarball [flags]
Flags:
--prefix string Prefix for the final expanded directory
--unpack-version string which version is contained in the
tarball
--verbosity int Level of verbosity during unpack (0-2)](https://image.slidesharecdn.com/dbdeployeresp-190118161006/85/Dbdeployer-28-320.jpg)
![deploy
$ dbdeployer deploy -h
Deploys single, multiple, or replicated
sandboxes
Usage:
dbdeployer deploy [command]
Available Commands:
multiple create multiple sandbox
replication create replication sandbox
single deploys a single sandbox](https://image.slidesharecdn.com/dbdeployeresp-190118161006/85/Dbdeployer-29-320.jpg)
![deploy flags (2)
Flags:
--native-auth-plugin in 8.0.4+, uses the native auth plugin
--port int Overrides default port
--post-grants-sql strings SQL queries after loading grants
--post-grants-sql-file string SQL file after loading grants
--pre-grants-sql strings SQL queries before loading grants
--pre-grants-sql-file string SQL file to run before loading grants
--remote-access string defines the database access ("127.%")
--rpl-password string replication password ("rsandbox")
--rpl-user string replication user ("rsandbox")
--sandbox-directory string Changes the default sandbox directory
--skip-load-grants Does not load the grants
--skip-report-host Does not include report host
--skip-report-port Does not include report port
--skip-start Does not start the database server
--use-template strings [template_name:file_name] Replace
existing template with one from file](https://image.slidesharecdn.com/dbdeployeresp-190118161006/85/Dbdeployer-31-320.jpg)
![deploy single
$ dbdeployer deploy single -h
single installs a sandbox and creates useful scripts for its use.
MySQL-Version is in the format x.x.xx, and it refers to a
directory named after the version
containing an unpacked tarball. The place where these directories
are found is defined by
--sandbox-binary (default: $HOME/opt/mysql.)
For example:
dbdeployer deploy single 5.7.21
For this command to work, there must be a directory $HOME/opt/
mysql/5.7.21, containing
the binary files from mysql-5.7.21-$YOUR_OS-x86_64.tar.gz
Use the "unpack" command to get the tarball into the right
directory.
Usage:
dbdeployer deploy single MySQL-Version [flags]](https://image.slidesharecdn.com/dbdeployeresp-190118161006/85/Dbdeployer-32-320.jpg)
![deploy multiple
$ dbdeployer deploy multiple -h
Creates several sandboxes of the same version, without any
replication relationship.
For this command to work, there must be a directory $HOME/opt/
mysql/5.7.21, containing the binary files from mysql-5.7.21-
$YOUR_OS-x86_64.tar.gz
Use the "unpack" command to get the tarball into the right
directory.
Usage:
dbdeployer deploy multiple MySQL-Version [flags]
Examples:
$ dbdeployer deploy multiple 5.7.21
Flags:
-n, --nodes int How many nodes will be installed (default 3)](https://image.slidesharecdn.com/dbdeployeresp-190118161006/85/Dbdeployer-33-320.jpg)
![deploy replication (1)
$ dbdeployer deploy replication -h
The replication command allows you to deploy several nodes
in replication.
Allowed topologies are "master-slave" for all versions,
and "group", "all-masters", "fan-in"
for 5.7.17+.
For this command to work, there must be a directory $HOME/
opt/mysql/5.7.21, containing
the binary files from mysql-5.7.21-$YOUR_OS-x86_64.tar.gz
Use the "unpack" command to get the tarball into the right
directory.
Usage:
dbdeployer deploy replication MySQL-Version [flags]](https://image.slidesharecdn.com/dbdeployeresp-190118161006/85/Dbdeployer-34-320.jpg)
Dbdeployer
- 1. Giuseppe Maxia Software explorer dbdeployer la herramienta amigable que simplifica la vida de desarrolladores y DBA
- 2. ¿Quién es este tío? intro ‣ Giuseppe Maxia, a.k.a. "The Data Charmer" ‣ Explorador de Software en VMware ‣ Experiencia de muchos años desarrollando y manteniendo con bases de datos ‣ En la comunidad de MySQL desde hace mucho. ‣ Blog: http://datacharmer.blogspot.com ‣ Twitter: @datacharmer !2
- 3. Advertencia •Lo que voy a decir aquí no tiene nada que ver con mi empresa. •Y yo ni siquiera trabajo en Oracle.
- 4. 2006-2017: MySQL-Sandbox • https://github.com/datacharmer/mysql-sandbox • Una herramienta para instalar MySQL en el espacio del usuario • Obtuvo el premio "MySQL application of the year" en 2013 • Es una aplicación escrita con Perl. • Algo difícil de instalar si no eres un experto de Perl. • Difícil usarlo cuando no hay Perl en el servidor ... • La sintaxis ha cambiado mucho en el tiempo (desde 2006) y es un lío. !4
- 6. Tenía un sueño re-escribir MySQL-Sandbox en Python! en Ruby! en C++ en Java! Es broma ¡en Go! en Javascript! Ni de coña !6 ¿Porque no?
- 8. ¿Y porque no Perl (6), Python, Ruby, C++? • Se me ha ido la paciencia esperando Perl 6 ... • Cansado de pelear con administradores sobre Python 2.6/7 vs. 3.x ... • Ruby ha desaparecido ... • Todavía me gusta C++, pero tiene el mismo problema que los otros: • La responsabilidad de la instalación ¡cae sobre los usuarios! Las dependencias de bibliotecas son un embrollo
- 9. ¿Porque Go? • La responsabilidad de instalar cae sobre el desarrollador. • Los usuarios solo tienen que descargar el ejecutable. • No hay dependencias (excepto tener un sistema que pueda usar MySQL) • Desarrollo rápido • Hay bibliotecas de funciones para todo. • La comunidad es maravillosa. !9
- 10. dbdeployer • Una interfaz a la línea de comandos • No hay dependencias (un ejecutable por cada S.O.) • Interfaz parecida a git, docker • ¡Es rápido! • Crea bases de datos individuales, con replicación sencilla o de grupo, despliegues múltiplos, replicación de fuentes múltiplas. • Ayuda integrada • Fácil de añadir más funcionalidades. https://github.com/datacharmer/dbdeployer
- 11. ¿Qué puede hacer? (1) • Instalar un sandbox individual de MySQL • Separado de cualquier otro servidor MySQL • Todo en el espacio del usuario (no necesita root) • Razonablemente aislado • Fácil de usar y de administrar
- 12. ¿Qué puede hacer? (2) • Instalar un grupo de sandboxes • Cada una aislada de las demás • Sin relaciones entre ellas • Con el binary log activado
- 13. ¿Qué puede hacer? (3) • Instalar una replicación de maestro-esclavo • 1 maestro y 2 (o más) esclavos • Replicación lista • Opcionalmente, con GTID • Hay distintos scripts para ejecutar operaciones en todos los nodos al mismo tiempo
- 14. ¿Qué puede hacer? (4) • Instalar una replicación de grupo (MySQL 5.7 y 8.0) • 3 nodos (o más) • Con nodo primario o múltiplos primarios • Configura y activa la replicación de grupo. • Hay distintos scripts para ejecutar operaciones en todos los nodos al mismo tiempo
- 15. ¿Qué puede hacer? (5) • Instalar una replicación de fuentes múltiplas (MySQL 5.7-8.0) • 3 nodos (o más) • Topología fan-in o all-masters • Hay distintos scripts para ejecutar operaciones en todos los nodos al mismo tiempo
- 16. ¿Qué puede hacer? (6) • Administrar los sandboxes • Mostrar los binarios disponibles • Mostrar sandboxes ya instalados • start, restart (con opciones), stop • mostrar status • test • Eliminar (delete)
- 17. ¿Qué puede hacer? (7) • Personalizar sandboxes • Inicializar y arrancar la base de datos con opciones personales • Hay plantillas (templates) para cada script • Se pueden remplazar plantillas por una vez o permanentemente. • Se pueden cambiar casi todos las opciones por defecto • Se pueden hacer los sandboxes permanentes (= No se pueden eliminar)
- 18. ¿Qué puede hacer? (8) • Buscar puertos disponibles automaticamente • Evaluar el flujo de la replicación • Mostrar las tablas de diccionario de MySQL 8 • Usar la replicación semi-sincrónica • Ejecutar comandos SQL antes y después de cargar los privilegios (grants) • Iniciar / Terminar el uso del protocolo X
- 19. Instalación 1. Navegar a https://github.com/datacharmer/ dbdeployer 2. Buscar "releases" 3. Descargar los binarios para vuestro S.O. 4. Ponerlo en un directorio que esté en el $PATH 5. ¡Y listos!
- 22. Como se usa 1. Descarga un tarball de binarios de MySQL 2. Usa dbdeployer para descomprimirlo 3. Empeza a crear sandboxes
- 23. Elige lo que debes descargar
- 24. Elige lo que debes descargar
- 25. Principios https://dev.mysql.com/downloads/mysql/descarga un tarball de binarios de MySQL 1 mysql-5.7.22-yourOS.tar.gz dbdeployer unpack2 $HOME/opt/mysql/5.7.22 dbdeployer deploy single3 $HOME/sandboxes/msb_5_7_22 dbdeployer deploy multiple3 $HOME/sandboxes/multi_msb_5_7_22
- 26. Operaciones: "unpack" una vez y usa por siempre $ dbdeployer unpack mysql-8.0.11-linux-glibc2.12-x86_64.tar.gz Unpacking tarball mysql-8.0.11-linux-glibc2.12- x86_64.tar.gz to $HOME/opt/mysql/8.0.11 $ dbdeployer available 8.0.11 $ dbdeployer deploy single 8.0.11 Database installed in $HOME/sandboxes/msb_8_0_11
- 27. dbdeployer help $ dbdeployer --help dbdeployer makes MySQL server installation an easy task. Runs single, multiple, and replicated sandboxes. Usage: dbdeployer [command] Available Commands: admin sandbox management tasks defaults tasks related to dbdeployer defaults delete delete an installed sandbox deploy deploy sandboxes global Runs a given command in every sandbox help Help about any command sandboxes List installed sandboxes unpack unpack a tarball into the binary directory usage Shows usage of installed sandboxes versions List available versions
- 28. unpack $ dbdeployer unpack -h If you want to create a sandbox from a tarball, you first need to unpack it into the sandbox-binary directory. This command carries out that task, so that afterwards you can call 'deploy single', 'deploy multiple', and 'deploy replication' commands with only the MySQL version for that tarball. If the version is not contained in the tarball name, it should be supplied using --unpack-version. If there is already an expanded tarball with the same version, a new one can be differentiated with --prefix. Usage: dbdeployer unpack MySQL-tarball [flags] Flags: --prefix string Prefix for the final expanded directory --unpack-version string which version is contained in the tarball --verbosity int Level of verbosity during unpack (0-2)
- 29. deploy $ dbdeployer deploy -h Deploys single, multiple, or replicated sandboxes Usage: dbdeployer deploy [command] Available Commands: multiple create multiple sandbox replication create replication sandbox single deploys a single sandbox
- 30. deploy flags (1) Flags: (cont.) --base-port int Overrides default base-port (for multiple sandboxes) --binary-version string Specifies the version for basedir directory --bind-address string defines the database bind-address ("127.0.0.1") --concurrent Runs multiple sandbox deployments concurrently --custom-mysqld string Uses an alternative mysqld -p, --db-password string database password (default "msandbox") -u, --db-user string database user (default "msandbox") --defaults strings Change defaults on-the-fly (--defaults=label:value) --disable-mysqlx Disable MySQLX plugin (8.0.11+) --enable-general-log Enables general log for the sandbox (MySQL 5.1+) --enable-mysqlx Enables MySQLX plugin (5.7.12+) --expose-dd-tables In MySQL 8.0+ shows data dictionary tables --force Overwrite destination sandbox --gtid enables GTID --init-general-log uses general log during initialization (MySQL 5.1+) -i, --init-options strings mysqld options to run during initialization --keep-server-uuid Does not change the server UUID --my-cnf-file string Alternative source file for my.sandbox.cnf -c, --my-cnf-options strings mysqld options to add to my.sandbox.cnf
- 31. deploy flags (2) Flags: --native-auth-plugin in 8.0.4+, uses the native auth plugin --port int Overrides default port --post-grants-sql strings SQL queries after loading grants --post-grants-sql-file string SQL file after loading grants --pre-grants-sql strings SQL queries before loading grants --pre-grants-sql-file string SQL file to run before loading grants --remote-access string defines the database access ("127.%") --rpl-password string replication password ("rsandbox") --rpl-user string replication user ("rsandbox") --sandbox-directory string Changes the default sandbox directory --skip-load-grants Does not load the grants --skip-report-host Does not include report host --skip-report-port Does not include report port --skip-start Does not start the database server --use-template strings [template_name:file_name] Replace existing template with one from file
- 32. deploy single $ dbdeployer deploy single -h single installs a sandbox and creates useful scripts for its use. MySQL-Version is in the format x.x.xx, and it refers to a directory named after the version containing an unpacked tarball. The place where these directories are found is defined by --sandbox-binary (default: $HOME/opt/mysql.) For example: dbdeployer deploy single 5.7.21 For this command to work, there must be a directory $HOME/opt/ mysql/5.7.21, containing the binary files from mysql-5.7.21-$YOUR_OS-x86_64.tar.gz Use the "unpack" command to get the tarball into the right directory. Usage: dbdeployer deploy single MySQL-Version [flags]
- 33. deploy multiple $ dbdeployer deploy multiple -h Creates several sandboxes of the same version, without any replication relationship. For this command to work, there must be a directory $HOME/opt/ mysql/5.7.21, containing the binary files from mysql-5.7.21- $YOUR_OS-x86_64.tar.gz Use the "unpack" command to get the tarball into the right directory. Usage: dbdeployer deploy multiple MySQL-Version [flags] Examples: $ dbdeployer deploy multiple 5.7.21 Flags: -n, --nodes int How many nodes will be installed (default 3)
- 34. deploy replication (1) $ dbdeployer deploy replication -h The replication command allows you to deploy several nodes in replication. Allowed topologies are "master-slave" for all versions, and "group", "all-masters", "fan-in" for 5.7.17+. For this command to work, there must be a directory $HOME/ opt/mysql/5.7.21, containing the binary files from mysql-5.7.21-$YOUR_OS-x86_64.tar.gz Use the "unpack" command to get the tarball into the right directory. Usage: dbdeployer deploy replication MySQL-Version [flags]
- 35. deploy replication (2) $ dbdeployer deploy replication -h Examples: $ dbdeployer deploy replication 5.7.21 # (implies topology = master-slave) $ dbdeployer deploy --topology=master-slave replication 5.7.21 # (explicitly setting topology) $ dbdeployer deploy replication --topology=group 5.7.21 $ dbdeployer deploy replication --topology=group 8.0.4 --single-primary $ dbdeployer deploy --topology=all-masters replication 5.7.21 $ dbdeployer deploy --topology=fan-in replication 5.7.21
- 36. deploy replication (3) Flags: --master-ip string Which IP the slaves will connect to ("127.0.0.1") --master-list string Which nodes are masters in a multi-source deployment ("1,2") -n, --nodes int How many nodes (3) --semi-sync Use semi-synchronous --single-primary Using single primary --slave-list string Which nodes are slaves in a multi-source deployment (default "3") -t, --topology string Which topology will be installed
- 37. usage (single) "./start", "./status", "./restart", and "./stop" do what their name suggests. start and restart accept parameters that are eventually passed to the server. e.g.: ./start --server-id=1001 ./restart --event-scheduler=disabled "./use" calls the command line client with the appropriate parameters, Example: ./use -BN -e "select @@server_id" ./use -u root "./clear" stops the server and removes everything from the data directory, letting you ready to start from scratch. (Warning! It's irreversible!) ... and more
- 38. usage (multiple) On a replication sandbox, you have the same commands (run "dbdeployer usage single"), with an "_all" suffix, meaning that you propagate the command to all the members. Then you have "./m" as a shortcut to use the master, "./s1" and "./s2" to access the slaves (and "s3", "s4" ... if you define more). In group sandboxes without a master slave relationship (group replication and multiple sandboxes) the nodes can be accessed by ./n1, ./n2, ./n3, and so on. start_all / status_all / restart_all / stop_all / use_all clear_all / m / s1 / s2 / n1 / n2 The scripts "check_slaves" or "check_nodes" give the status of replication in the sandbox. ... and more
- 39. • Muchísimas cosas … • Mirad el documento ./docs/features.md en el sitio de dbdeployer Qué puede hacer dbdeployer que MySQL-Sandbox no puede
- 40. LIVE DEMO
- 41. dbdeployer usa el sistema de versiones semánticas (https:// semver.org) Versión actual: 1.17.0 componente ¿Cuándo cambia? Ejemplo major Cambios de API incompatibles con el pasado. (Incluso nuevas funcionalidades) 1.21.3 => 2.0.0 minor Cambios de API compatibles con el pasado. Nuevas funcionalidades 1.21.3 => 1.22.0 revision Arreglo de bugs. Mejorías de código. Sin cambio de API 1.21.3 => 1.21.4
- 42. En que más estoy pensando • Mejorías posibles: • Instalar un sandbox con un tarball remoto (¡Acabo de hacerlo!) • Comando unpack puede reducir los contenidos de un tarball. • Instalar Galera, Percona Xtrabackup Cluster, MySQL Cluster (?) • Instalar contenedores Docker como si fueran sandboxes.
- 43. Antes de despedirnos • dbdeployer es fuente abierta (Apache license) • Las contribuciones son bienvenidas! https://github.com/datacharmer/dbdeployer