Ex-8-hive.pptx
Download as PPTX, PDF0 likes38 views
This document provides steps to install and run Hive on Windows. It involves downloading Hive and Derby, extracting the files, configuring environment variables and system paths, editing the hive-site.xml file, starting Hadoop, starting the Derby server, and then starting Hive to execute commands. Key steps include extracting Hive and Derby to directories, adding directories to paths, configuring properties in hive-site.xml for the metastore database connection, and using commands like hive and jps to validate the setup and run sample queries.
![STEP - 1: Extract the Hive file
Extract file apache-hive-2.1.0-bin.tar.gz and place
under "D:Hive", you can use any preferred
location –
[1] You will get again a tar file post extraction –](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-5-320.jpg)
![[2] Go inside of apache-hive-2.1.0-
bin.tar folder and extract again –](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-6-320.jpg)
![[3] Copy the leaf folder “apache-hive-2.1.0-bin” and
move to the root folder "D:Hive" and removed all other
files and folders –](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-7-320.jpg)
![STEP - 4: Moving Derby libraries
Next, need to drop all derby library to hive library location –
[1] Move to library folder under derby location D:Derbydb-derby-
10.12.1.1-binlib.](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-10-320.jpg)
![[2] Select all and copy all libraries.
[3] Move to library folder under hive location D:Hiveapache-hive-2.1.0-binlib.](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-11-320.jpg)
![[4] Drop all selected libraries
here.](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-12-320.jpg)
![Now need to do a cross check with Hive configuration file for Derby details –
hive-site.xml
[1] Edit file D:/Hive/apache-hive-2.1.0-bin/conf/hive-site.xml, paste
below xml paragraph and save this file.](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-16-320.jpg)
![STEP - 10: Start the Hive
Derby server has been started and ready to accept connection so
open a new command prompt under administrator privileges and
move to hive directory as “D:Hiveapache-hive-2.1.0-binbin” –
[1] Type “jps -m” to check NetworkServerControl
[2] Type “hive” to execute hive server.](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-24-320.jpg)
![STEP-11: Some hands on activities
[1] Create Database in Hive -
CREATE DATABASE IF NOT EXISTS TRAINING;
[2] Show Database -
SHOW DATABASES;](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-25-320.jpg)
![[3] Creating Hive Tables -
CREATE TABLE IF NOT EXISTS testhive(col1 char(10), col2 char(20));](https://image.slidesharecdn.com/ex-8-hive-220330100820/85/Ex-8-hive-pptx-26-320.jpg)
Ex-8-hive.pptx
- 1. Ex-8 8.Install and Run Hive
- 2. Hive Installation Hive is a data warehouse software project built on top of Hadoop, that facilitate reading, writing, and managing large datasets residing in distributed storage using SQL
- 3. Download Hive 2.1.0 https://archive.apache.org/dist/hive/hive-2.1.0/ Download Derby Metastore 10.12.1.1 https://archive.apache.org/dist/db/derby/db-derby-10.12.1.1/
- 5. STEP - 1: Extract the Hive file Extract file apache-hive-2.1.0-bin.tar.gz and place under "D:Hive", you can use any preferred location – [1] You will get again a tar file post extraction –
- 6. [2] Go inside of apache-hive-2.1.0- bin.tar folder and extract again –
- 7. [3] Copy the leaf folder “apache-hive-2.1.0-bin” and move to the root folder "D:Hive" and removed all other files and folders –
- 8. STEP - 2: Extract the Derby file Similar to Hive, extract file db-derby-10.12.1.1-bin.tar.gz and place under "D:Derby", you can use any preferred location –
- 9. STEP - 3: Moving hive-site.xml file Drop the downloaded file “hive-site.xml” to hive configuration location “D:Hiveapache-hive-2.1.0- binconf”.
- 10. STEP - 4: Moving Derby libraries Next, need to drop all derby library to hive library location – [1] Move to library folder under derby location D:Derbydb-derby- 10.12.1.1-binlib.
- 11. [2] Select all and copy all libraries. [3] Move to library folder under hive location D:Hiveapache-hive-2.1.0-binlib.
- 12. [4] Drop all selected libraries here.
- 13. STEP - 5: Configure Environment variables Set the path for the following Environment variables (User Variables) on windows 10 – HIVE_HOME - D:Hiveapache-hive-2.1.0-bin HIVE_BIN - D:Hiveapache-hive-2.1.0-binbin HIVE_LIB - D:Hiveapache-hive-2.1.0-binlib DERBY_HOME - D:Derbydb-derby-10.12.1.1-bin HADOOP_USER_CLASSPATH_FIRST - true
- 14. This PC - > Right Click - > Properties - > Advanced System Settings - > Advanced - > Environment Variables
- 15. STEP - 6: Configure System variables Next onward need to set System variables, including Hive bin directory path – HADOOP_USER_CLASSPATH_FIRST - true Variable: Path Value: D:Hiveapache-hive-2.1.0-binbin D:Derbydb-derby-10.12.1.1-binbin
- 16. Now need to do a cross check with Hive configuration file for Derby details – hive-site.xml [1] Edit file D:/Hive/apache-hive-2.1.0-bin/conf/hive-site.xml, paste below xml paragraph and save this file.
- 17. hive-site.xml <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:derby://localhost:1527/metastore_db;create=true</value> <description>JDBC connect string for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>org.apache.derby.jdbc.ClientDriver</value> <description>Driver class name for a JDBC metastore</description> </property> <property> <name>hive.server2.enable.impersonation</name> <description>Enable user impersonation for HiveServer2</description> <value>true</value> </property> <property> <name>hive.server2.authentication</name> <value>NONE</value> <description> Client authentication types. NONE: no authentication check LDAP: LDAP/AD based authentication KERBEROS: Kerberos/GSSAPI authentication CUSTOM: Custom authentication provider (Use with property hive.server2.custom.authentication.class) </description> </property> <property> <name>datanucleus.autoCreateTables</name> <value>True</value> </property> </configuration>
- 18. Open command prompt and change directory to “D:Hadoophadoop-2.8.0sbin" and type "start-all.cmd" to start apache
- 19. It will open four instances of cmd for following tasks – Hadoop Datanaode Hadoop Namenode Yarn Nodemanager Yarn Resourcemanager
- 20. It can be verified via browser also as – Namenode (hdfs) - http://localhost:50070 Datanode - http://localhost:50075 All Applications (cluster) - http://localhost:8088 etc.
- 22. Since the ‘start-all.cmd’ command has been deprecated so you can use below command in order wise - “start-dfs.cmd” and “start-yarn.cmd”
- 23. STEP - 9: Start Derby server Post successful execution of Hadoop, change directory to “D:Derbydb-derby-10.12.1.1-binbin” and type “startNetworkServer -h 0.0.0.0” to start derby server.
- 24. STEP - 10: Start the Hive Derby server has been started and ready to accept connection so open a new command prompt under administrator privileges and move to hive directory as “D:Hiveapache-hive-2.1.0-binbin” – [1] Type “jps -m” to check NetworkServerControl [2] Type “hive” to execute hive server.
- 25. STEP-11: Some hands on activities [1] Create Database in Hive - CREATE DATABASE IF NOT EXISTS TRAINING; [2] Show Database - SHOW DATABASES;
- 26. [3] Creating Hive Tables - CREATE TABLE IF NOT EXISTS testhive(col1 char(10), col2 char(20));