-
-
Save giwa/ed13ac177c1e1a97fba0 to your computer and use it in GitHub Desktop.
| $ brew update | |
| $ brew install hive |
| # ~/.bashrc | |
| export HADOOP_HOME=/usr/local/Cellar/hadoop/2.7.0 | |
| export HIVE_HOME=/usr/local/Cellar/hive/1.1.1/libexec |
| $ tar zxvf mysql-connector-java-5.1.35.tar.gz | |
| $ sudo cp mysql-connector-java-5.1.35/mysql-connector-java-5.1.35-bin.jar /usr/local/Cellar/hive/1.1.1/libexec/lib/ |
| $ mysql | |
| mysql> CREATE DATABASE metastore; | |
| mysql> USE metastore; | |
| mysql> CREATE USER 'hiveuser'@'localhost' IDENTIFIED BY 'password'; | |
| mysql> GRANT SELECT,INSERT,UPDATE,DELETE,ALTER,CREATE ON metastore.* TO 'hiveuser'@'localhost'; |
| $ cd /usr/local/Cellar/hive/1.1.1/libexec/conf | |
| $ cp hive-default.xml.template hive-site.xml |
| <property> | |
| <name>javax.jdo.option.ConnectionURL</name> | |
| <value>jdbc:mysql://localhost/metastore</value> | |
| </property> | |
| <property> | |
| <name>javax.jdo.option.ConnectionDriverName</name> | |
| <value>com.mysql.jdbc.Driver</value> | |
| </property> | |
| <property> | |
| <name>javax.jdo.option.ConnectionUserName</name> | |
| <value>hiveuser</value> | |
| </property> | |
| <property> | |
| <name>javax.jdo.option.ConnectionPassword</name> | |
| <value>password</value> | |
| </property> | |
| <property> | |
| <name>datanucleus.fixedDatastore</name> | |
| <value>false</value> | |
| </property> | |
| <property> | |
| <name>hive.exec.local.scratchdir</name> | |
| <value>/tmp/hive</value> | |
| <description>Local scratch space for Hive jobs</description> | |
| </property> | |
| <property> | |
| <name>hive.downloaded.resources.dir</name> | |
| <value>/tmp/hive</value> | |
| <description>Temporary local directory for added resources in the remote file system.</description> | |
| </property> | |
| <property> | |
| <name>hive.querylog.location</name> | |
| <value>/tmp/hive</value> | |
| <description>Location of Hive run time structured log file</description> | |
| </property> |
| $hive | |
| hive > show tables; |
Just to keep things going:
In file1.txt:
Instead of referring to specific versions, I use the links in /usr/local/opt:
export HADOOP_HOME=/usr/local/opt/hadoop
export HADOOP_COMMON_HOME=/usr/local/opt/hadoop
export HIVE_HOME=/usr/local/opt/hive/libexec
export HCAT_HOME=/usr/local/opt/hive/libexec/hcatalog
I also added HADOOP_COMMON_HOME and HCAT_HOME per warning I saw from brew install and an error from schematool
I am getting the following error while running "hive" after following all the steps:
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
ERROR: Invalid HADOOP_COMMON_HOME
Unable to determine Hadoop version information.
'hadoop version' returned:
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
ERROR: Invalid HADOOP_COMMON_HOME
hive
Unable to determine Hadoop version information.
'hadoop version' returned:
hive
Unable to determine Hadoop version information.
'hadoop version' returned:
This worked for me:
export HADOOP_HOME=/usr/local/Cellar/hadoop/3.1.1/libexec
just kill every detail:
chmod 777 /tmp/hivecp $HADOOP_HOME/share/hadoop/common/lib/guava-27.0-jre.jar $HIVE_HOME/lib/ ; mv guava-19.0.jar guava-19.0.bak # fix version conflicts.
Changes as per the errors I faced:
Note:
wherever you see "hive/1.1.1" replace it with "hive/{version-number}"
in file2.txt
mysql to jdbc drive can be downloaded from:
in file3.txt use:
For the first time users of mysql:
install mysql by: [First Time Users]
brew install mysqlthen run the following [First Time Users]
brew services start mysql[For all users]
in Place of:
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,ALTER,CREATE ON metastore.* TO 'hiveuser'@'localhost';use:
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,ALTER,CREATE, REFERENCES, INDEX ON metastore.* TO 'hiveuser'@'localhost';in file5.txt:
Also update this property:
in file6.txt:
Also, before running final "hive" command we have to run
./schematool -initSchema -dbType mysql