tencent cloud

Elastic MapReduce

Release Notes and Announcements
Release Notes
Announcements
Security Announcements
Product Introduction
Overview
Strengths
Architecture
Features
Use Cases
Constraints and Limits
Technical Support Scope
Product release
Purchase Guide
EMR on CVM Billing Instructions
EMR on TKE Billing Instructions
EMR Serverless HBase Billing Instructions
Getting Started
EMR on CVM Quick Start
EMR on TKE Quick Start
EMR on CVM Operation Guide
Planning Cluster
Administrative rights
Configuring Cluster
Managing Cluster
Managing Service
Monitoring and Alarms
TCInsight
EMR on TKE Operation Guide
Introduction to EMR on TKE
Configuring Cluster
Cluster Management
Service Management
Monitoring and Ops
Application Analysis
EMR Serverless HBase Operation Guide
EMR Serverless HBase Product Introduction
Quotas and Limits
Planning an Instance
Managing an Instance
Monitoring and Alarms
Development Guide
EMR Development Guide
Hadoop Development Guide
Spark Development Guide
Hbase Development Guide
Phoenix on Hbase Development Guide
Hive Development Guide
Presto Development Guide
Sqoop Development Guide
Hue Development Guide
Oozie Development Guide
Flume Development Guide
Kerberos Development Guide
Knox Development Guide
Alluxio Development Guide
Kylin Development Guide
Livy Development Guide
Kyuubi Development Guide
Zeppelin Development Guide
Hudi Development Guide
Superset Development Guide
Impala Development Guide
Druid Development Guide
TensorFlow Development Guide
Kudu Development Guide
Ranger Development Guide
Kafka Development Guide
Iceberg Development Guide
StarRocks Development Guide
Flink Development Guide
JupyterLab Development Guide
MLflow Development Guide
Practical Tutorial
Practice of EMR on CVM Ops
Data Migration
Practical Tutorial on Custom Scaling
API Documentation
History
Introduction
API Category
Cluster Resource Management APIs
Cluster Services APIs
User Management APIs
Data Inquiry APIs
Scaling APIs
Configuration APIs
Other APIs
Serverless HBase APIs
YARN Resource Scheduling APIs
Making API Requests
Data Types
Error Codes
FAQs
EMR on CVM
Service Level Agreement
Contact Us

EMR on CVM Quick Start

PDF
フォーカスモード
フォントサイズ
最終更新日: 2024-10-30 09:59:58
This document introduces the process of quickly creating an EMR on CVM cluster through the EMR console, submitting a job, and viewing the results.

Preparations

1. Before using an EMR cluster, you need to register a Tencent Cloud account and complete identity verification. For detailed directions, see Enterprise Identity Verification Guide.
2. Grant the system default role EMR_QCSRole to the service account for EMR. For detailed directions, see Role Authorization.
3. For online account recharge, EMR on CVM offers two billing modes: pay-as-you-go and monthly subscription. Before creating a cluster, you need to recharge your account balance to ensure it is greater than or equal to the configuration fees required for cluster creation, excluding promo vouchers. For detailed directions, see Top-up.

Creating Clusters

Log in to the EMR Console, click Create Cluster on the EMR on CVM cluster list page, and complete the relevant configuration on the purchase page. When the cluster status shows Running, it indicates that the cluster has been successfully created.
Purchase Steps
Configuration Item
Configuration Items Description
Example
Software configuration
Region
The physical data center where the cluster is deployed.
Note: Once the cluster is created, the region cannot be changed, so choose carefully.
Beijing, Shanghai, Guangzhou, Nanjing, Chengdu, and Silicon Valley
Cluster type
EMR on CVM supports multiple cluster types, with Hadoop being the default cluster type.
Hadoop and StarRocks
Product version
The components and their versions bundled with different product versions vary.
EMR-V2.7.0 includes Hadoop 2.8.5 and Spark 3.2.1.
Deployment components
Optional components that can be customized and combined based on your needs.
Hive-2.3.9 and Impala-3.4.1.
Region and hardware configuration
Billing mode
Billing mode for cluster deployment
Pay-as-You-go
Availability zone (AZ) and network configuration
AZ and cluster network settings. Note: Once the cluster is created, the AZ cannot be directly changed, so choose carefully.
Guangzhou Zone 7.
Secure login
Network access control settings for nodes, with a security group firewall feature.
Create a security group.
Node configuration
Select the appropriate model configuration for different node types based on business requirements. For more details, see Business Evaluation.
Enable high availability for node deployment.
Basic configuration
Associated project
Assign the current cluster to different project groups.
The associated project cannot be modified once the cluster is created.
Cluster name
The name of the cluster, which is customizable.
EMR-7sx2aqmu
Login method
Custom password setup and key association. SSH keys are used only for quick access through the EMR-UI.
Password.
Confirm configuration
Configuration list
Confirm if there is any error in the deployment information.
Select the terms of service and click Buy Now.
Note
You can view the information of each node in the CVM console. To ensure the normal operation of the EMR cluster, do not change the node configuration in the CVM console.

Submitting Jobs and Viewing Results

After the cluster is successfully created, you can create and submit jobs on that cluster. This document uses a submitted Spark task as an example, with the following steps.
Note
When creating an EMR cluster, you need to select the Spark component in the software configuration interface.
1. Use SSH to log in and connect to the cluster (the local system is Linux/Mac OS). For more details, see Login to Cluster.
2. In the EMR command line, use the following commands to switch to the Hadoop user and navigate to the Spark installation directory /usr/local/service/spark:
[root@172 ~]# su hadoop
[hadoop@172 root]$ cd /usr/local/service/spark
3. Submit and run the task using the following command:
/usr/local/service/spark/bin/spark-submit \\
--class org.apache.spark.examples.SparkPi \\
--master yarn \\
--deploy-mode cluster \\
--proxy-user hadoop \\
--driver-memory 1g \\
--executor-memory 1g \\
--executor-cores 1 \\
/usr/local/service/spark/examples/jars/spark-examples*.jar \\
10
4. After the job is submitted, on the EMR on CVM page, click Cluster Services in the row of the target cluster, and then click the WebUI link in the row of YARN UI. After logging in, you will enter the YARN UI page. Click the ID of the target job to view the job’s detailed running information.

Terminating Clusters

When the created cluster is no longer needed, you can terminate the cluster and return the resources. Terminating the cluster will forcibly stop all services provided by the cluster and release the resources.
On the EMR on CVM page, select Terminate from the More options for the target cluster. In the pop-up dialog box, click Terminate Now.

ヘルプとサポート

この記事はお役に立ちましたか?

フィードバック