tencent cloud

As earlier EMR versions don't come with the Timeline Server and Tomcat packages necessary for Tez UI integration, you need to complete the steps below.

Note：

As starting Tez UI requires running Timeline Server, which uses a lot of resources, you need to assess the impact on your business and proceed with caution.

This document describes how to integrate Knox with Tez, such as installing Tomcat and Tez UI, creating roles, configuring Timeline Server, configuring Tez, and starting services. 172.**.**.9 is the private IP of the master node, 159.**.**.70 is the public IP of the master node, and the Tez version is v0.9.2.

Installing Tomcat and Tez UI

cd  /usr/local/service
wget https://jaydihu-package-1258469122.cos.ap-guangzhou.myqcloud.com/apache-tomcat-9.0.46.tar.gz
tar -zxvf apache-tomcat-9.0.46.tar.gz
mv /usr/local/service/apache-tomcat-9.0.46 /usr/local/service/tomcat

Modify Tomcat port numbers:

vim /usr/local/service/tomcat/conf/server.xml

1. Change 8005 to 2020.
   
2. Change 8080 to 2000.
   

mkdir -p /usr/local/service/tomcat/webapps/tez-ui
cp /usr/local/service/tez/tez-ui-0.9.2.war /usr/local/service/tomcat/webapps/tez-ui/
cd /usr/local/service/tomcat/webapps/tez-ui
unzip tez-ui-0.9.2.war
vim ./config/configs.env

Change localhost to the private IP of the current server.

Creating Roles

vim /usr/local/service/knox/conf/topologies/emr.xml 

Modify the emr.xml file:

1. Add the following content:
   

   <param>
   <name>TEZUI</name>
   <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
   </param>
   <param>
    <name>APPLICATIONHISTORY</name>
   <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
   </param>
   

2. Add the following content:
   

   <service>
    <role>TEZUI</role>
    <url>http://172.**.**.9:2000/tez-ui</url>
    <version>0.9.2</version>
   </service>
   <service>
    <role>APPLICATIONHISTORY</role>
    <url>http://172.**.**.9:8188</url>
    <version>2.7.3</version>
   </service>
   

Configuring YARN Timeline Server

Modify the yarn-site.xml file in configuration management, save the changes, and restart the components whose configuration has been changed.

Modifying Tez Configuration

In the tez-site.xml file, add the following parameters, save the changes, and restart the components whose configuration has been changed.

Starting Services

1. Start Timeline Server.

   /usr/local/service/hadoop/sbin/yarn-daemon.sh  start timelineserver  
   

2. Start Tomcat.

   /usr/local/service/tomcat/bin/startup.sh  
   

3. Restart Tez.

   su hadoop
   rm -rf  /usr/local/service/knox/data/deployments/*
   /usr/local/service/knox/bin/ldap.sh stop
   /usr/local/service/knox/bin/ldap.sh start
   /usr/local/service/knox/bin/gateway.sh stop
   /usr/local/service/knox/bin/gateway.sh start
   

4. Access Tez UI at the following address.

   Note：

   The account and password are the same as the server login account and password.

   https://{public IP of the cluster}:30002/gateway/emr/tez/  
   

Stopping Services

If you find that the Timeline Server significantly affects your business after running it for a period of time, you can stop relevant services as follows:

1. Stop Tomcat.

   /usr/local/service/tomcat/bin/shutdown.sh
   

2. Stop Timeline Server.

   /usr/local/service/hadoop/sbin/yarn-daemon.sh  stop  timelineserver