Scenarios
This applies to scenarios where Cloud Log Service (CLS) is shared by multiple users and teams. In these scenarios, different users frequently need to log in to the Tencent Cloud console to search and analyze logs, which leads to the following issues:
High account management cost: To enable console access over the public network, Ops personnel must repeatedly create Tencent Cloud accounts for requesters, and are responsible for promptly identifying and deactivating accounts of departed employees.
Deep feature access path: When developers view logs or Ops personnel check dashboards, they face repeated login verifications. Each time, they must start from the cloud products menu and navigate through several pages to access CLS. Users expect instant access without frequent logins and redirections.
To address the above issues, CLS has launched the independent DataSight console, which offers the following features:
Lowered account management cost:
Users can access CLS without a Tencent Cloud account.
Access over the private network is supported for higher security. Access is automatically revoked for departed employees.
Users can be granted different access permissions to CLS based on their roles.
Permission misconfiguration only affects CLS and will not impact other cloud products, thereby reducing permission security control risks.
Streamlined feature access path:
Multiple methods are supported for accessing CLS, such as account/password login, login-free access, and single sign-on (SSO).
Specific pages can be quickly opened, shared, or embedded via access addresses.
Note:
DataSight is currently free to use.
DataSight Trial Environment
Features
Lowered Account Management Cost
Users can access CLS without a Tencent Cloud account.
Reduced Permission Security Control Risk
Users can be granted different access permissions to CLS based on their roles.
Permission misconfiguration only affects CLS and will not impact other cloud products.
Access over the private network is supported.
Streamlined Feature Access Path
Multiple methods are supported for accessing CLS, such as account/password login, anonymous login, and third-party authentication login.
Specific pages can be quickly opened or shared via access addresses.
Operation Steps
Step 1: Create a DataSight Instance
1. Log in to the CLS console. On the Overview page, locate DataSight and click Enable now. 2. On the instance configuration page, select an Access mode. Two options are supported: public network and private network.
|
Public network | This method allows access to DataSight from the Internet for log viewing and analysis. Only password authentication login is supported. |
Intranet | Prerequisite: The user's office network must be interconnected with Tencent Cloud via a secure connection, such as a dedicated connection or a VPN tunnel. This method allows access to DataSight from the Tencent Cloud private network for log viewing and analysis. Access from the Internet is not available, thus providing enhanced security. Multiple login methods are supported, including password authentication login and anonymous login. |
Step 2: Create a Sub-User to Obtain SecretId and SecretKey
2. Select Custom to enter the creation workflow, then select Accessible resources and message reception, and click Next.
3. Configure the sub-user's information, check the box for Programming access, and click Next.
4. Set user permissions. Select Add to group to obtain group permissions. The CLS read-only access permission policy QcloudCLSReadOnlyAccess is recommended. You can also create a custom policy based on your requirements. For the policy configuration, see CLS Access Policy Templates. After the configuration is complete, click Next. 5. Configure user tags, review role information, and complete the sub-user creation.
6. Click Show to reveal the key and copy the key information.
Step 3: Select a Login Method
Password Authentication Login
Third-Party Authentication Login
Enter the username, password, and the key corresponding to the user's permissions, and save the configuration. Multiple users can be added.
|
Username | Username for logging in to DataSight. |
Password | Password for logging in to DataSight. The password must be 8–30 characters long, cannot start with a forward slash (/), and must include at least three of the following character types: lowercase letters (a–z), uppercase letters (A–Z), digits (0–9), or special characters ()!@#$%^&*|?><. |
SecretId, SecretKey | |
IP access restriction | Restricts the source IP addresses allowed to access DataSight. Enter IP addresses or CIDR blocks, separated by commas. Allowlist: Only IP addresses matching the allowlist can access DataSight. Blocklist: IP addresses matching the blocklist will be denied access to DataSight. |
Enter the SecretId and SecretKey, and save the configuration. Anonymous login is only supported for access over the private network and does not require authentication. It is recommended to use a key with read-only permissions to avoid security risks.
|
SecretId, SecretKey | |
IP access restriction | Restricts the source IP addresses allowed to access DataSight. Enter IP addresses or CIDR blocks, separated by commas. Allowlist: Only IP addresses matching the allowlist can access DataSight. Blocklist: IP addresses matching the blocklist will be denied access to DataSight. |
DataSight supports access via a reverse proxy that handles identity authentication, enabling integration with third-party identity authentication systems such as LDAP and OAuth. Third-party authentication login is only supported for access over the private network. The configuration process is as follows: 1. Create a DataSight role.
Enter a role name and the key information that defines the role's permissions. This role will be mapped to a user role in your third-party identity authentication system.
Configuration Item | Description |
Role name | Name of the DataSight role. |
SecretId, SecretKey | |
2. Configure a reverse proxy and set the mapping between the user and the DataSight role.
2.1 Prepare an Nginx environment as the reverse proxy.
The reverse proxy must have network access to the Virtual Private Cloud (VPC) where DataSight resides. Additionally, ensure the --with-http_auth_request_module is installed. You can verify its presence by running the nginx -V command. This module is included by default in Nginx version 1.12 and above. If it's missing, upgrade your Nginx version or refer to the ngx_http_auth_request_module documentation for installation. Integrate with LDAP
2.2.1 Copy the template file to your local system.
git clone https://github.com/TencentCloud/cls-datasight-demo.git
2.2.2 Open the nginx-ldap-auth.conf file in the cls-datasight-demo/auth-proxy/ldap directory, and modify it according to the instructions within the file.
2.2.3 Copy the modified nginx-ldap-auth.conf file to the Nginx configuration file, and restart the Nginx service.
cp nginx-ldap-auth.conf /etc/nginx/conf.d/nginx-ldap-auth.conf
## Modify the file path according to your actual situation.
systemctl restart nginx
## Restart the Nginx service.
2.2.4 Open the .env file in the cls-datasight-demo/auth-proxy/ldap directory, and generate and configure the ENCRYPTION_KEY environment variable as per the comments.
2.2.5 Run the LDAP authentication and login modules in the auth-proxy/ldap directory.
(Recommended) Run using docker:
docker build -f Dockerfile-auth -t nginx-ldap-auth .
docker build -f Dockerfile-login -t nginx-ldap-login .
docker run -d -p 8888:8888/tcp --env-file ".env" nginx-ldap-auth
docker run -d -p 9000:9000/tcp --env-file ".env" nginx-ldap-login
Run locally with Python. If you encounter dependency installation errors, please refer to the error messages and search the relevant dependency's documentation for solutions.
python3 -m pip install python-ldap python-dotenv setuptools_rust cryptography
nohup python3 nginx-ldap-login.py >login.log 2>&1 &
nohup python3 nginx-ldap-auth.py >auth.log 2>&1 &
Integrate with OAuth
Prerequisite: This example is implemented using Nginx + vouch-proxy. Ensure Nginx is installed and vouch-proxy is downloaded locally. You can also use other OAuth authentication software such as oauth2-proxy based on your actual situation. This example uses keycloak (common OIDC protocol) as the OAuth provider. If you use another OAuth provider, modify the configuration accordingly by following the official vouch-proxy guidance and referencing this example. 2.2.1 Copy the template file to your local system.
git clone https://github.com/TencentCloud/cls-datasight-demo.git
2.2.2 Open the cls-datasight-demo/auth-proxy/oauth/keycloak(oidc) directory. Here, config.yml is the configuration file for vouch-proxy, and nginx-oauth-keycloak.conf is the configuration file for Nginx.
2.2.3 Modify the nginx-oauth-keycloak.conf configuration file as per the instructions within the file, copy it to the Nginx configuration file, and restart the Nginx service.
cp nginx-oauth-keycloak.conf /etc/nginx/conf.d/nginx-oauth-keycloak.conf
## Modify the file path according to your actual situation.
systemctl restart nginx
## Restart the Nginx service.
2.2.4 Modify config.yml according to the instructions within the file and place it in your local vouch-proxy config folder.
cp config.yml /root/vouch-proxy/config/config.yml
## Modify the vouch-proxy config file path according to your actual situation.
2.2.5 Refer to the vouch-proxy guidance to run the vouch service. Running via docker is recommended. If vouch-proxy needs to access the OAuth service using the host's network, add --network=host to the docker run command. docker run -d \\
-p 9090:9090 \\
-v ${PWD}/config:/config \\
quay.io/vouch/vouch-proxy
3. Enter the private IP address of the reverse proxy.
Enter the Nginx private IP address or CIDR block. Multiple entries are supported and should be separated by commas.
Step 4: Log in to DataSight to View Logs
Password authentication login or anonymous login
On the Overview page of the cloud console, view the DataSight management list and obtain the access address. Open the access address in a browser and log in to DataSight to use CLS. For password authentication login, you need to enter the username and password on the login page.
Third-party authentication login
In a browser, access the custom domain name configured in the Nginx configuration file. After successful authentication, you will be logged in to the DataSight console.
Step 5: Set Dashboard Subscription and Alarm Notification Links to Open the DataSight Console by Default (Optional)
1. Create a dashboard subscription and configure the dashboard link to Use the DataSight domain name. Once saved, the link included in the sent dashboard subscription will open DataSight by default. 2. Create an alarm notification channel group and configure the query data link to Use the DataSight domain name. Once saved, the link included in the sent alarm notification will open DataSight by default. DataSight Use Cases
Case 1: Providing DataSight as an Independent Console to Third Parties
A company needs to grant multiple third-party partners access to its business logs for querying. The requirements are as follows:
Multiple third-party users exist, each requiring different levels of resource access.
Third-party users have no Tencent Cloud accounts and need independent account/password authentication.
Solution
Leverage DataSight's account/password login authentication capability. Create distinct roles (PartnerA and PartnerB), each associated with a sub-user key with specific permissions. This achieves resource isolation and permission control between roles.
Partners use the access link, username, and password provided by the administrator to log in to DataSight and fulfill their respective business needs.
Operation Steps
3. Select password authentication login as the DataSight login method. Create two users named PartnerA and PartnerB, configure their passwords, and enter the corresponding SecretId and SecretKey. For details, see Password Authentication Login. 4. Save the configuration to obtain the DataSight access link. Users can then log in to DataSight via this link using their username and password to fulfill their respective business needs.
Case 2: Using DataSight as an Independent Console for Login-Free Access Within an Enterprise Private Network
The Ops personnel of a company are responsible for managing business logs and need to grant internal personnel access to specific business logs. The requirements are as follows:
Internal users do not need to log in with a Tencent Cloud account.
Ops personnel can specify which business logs and dashboard resources are accessible.
Users access through the company's internal domain name, not the DataSight domain name.
Solution
Select to access DataSight over the private network, use anonymous login, and configure a sub-user key with read-only permissions for specified resources to achieve anonymous access over the private network.
Configure an Nginx reverse proxy to implement a custom domain name.
Users within the private network can log in to DataSight via an access link (with the custom domain name) provided by Ops personnel to fulfill their needs.
Operation Steps
1. Create a DataSight instance and select access over the private network. For details, see the steps for creating an instance. 3. Select anonymous login as the DataSight login method and enter the SecretId and SecretKey. For details, see Anonymous Login. Then, save the configuration. 4. Configure an Nginx reverse proxy to use an internal domain name instead of the DataSight domain name. (Optional. Skip this step if a custom domain name is not required.)
Nginx configuration template:
# Modify the <your-domain.com>, <your-domain-cert>, and <domain-appid> (DataSight domain name prefix, supports public/private network domain names) parts in this example based on your actual situation.
# It is highly recommended to enable HTTPS for your custom domain name and force HTTP to redirect to HTTPS to enhance browser request security and reduce queuing delays.
# To disable forced redirection to HTTPS, comment out this server configuration.
server {
listen 80;
server_name your-domain.com;
return 301 https://$host$request_uri;
}
server {
# To allow access via HTTP, uncomment the following line.
# listen 80;
listen 443 ssl http2;
server_name your-domain.com;
ssl_certificate your-domain-cert.pem;
ssl_certificate_key your-domain-cert.key;
location ~ ^/(.*) {
# You can add custom access control policies here, such as restricting access based on specific Referer values.
#set $match "$1::$http_referer";
#if ($match !~* ^(.+)::http[s]*://[www]*[.]*\\1.*$ ) {
# return 403;
#}
proxy_pass https://datasight-111111111.clsconsole.tencent-cloud.com;
proxy_set_header Host $proxy_host;
proxy_set_header Origin https://$proxy_host;
proxy_set_header Referer "https://$proxy_host/$1";
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Proxy true;
proxy_set_header X-Proxy-Host $host;
proxy_set_header X-Proxy-Real-IP $remote_addr;
proxy_set_header X-Proxy-Forwarded-Proto $scheme;
proxy_redirect ~^(.*)/login\\?s_url=https?%3A%2F%2F[a-z0-9\\-]+(.internal)?.clsconsole.tencent-cloud.com%2F(.*)$ $1/login?s_url=https%3A%2F%2Fyour-domain.com%2F$3;
# To hide or customize the Tencent Cloud icon (favicon) on the DataSight page, use the following two headers.
# proxy_set_header X-DATASIGHT-HIDE-FAVICON true;
# proxy_set_header X-DATASIGHT-FAVICON-URL https://github.githubassets.com/favicons/favicon.png;
}
}
5. Obtain the DataSight access link. Within the private network, users log in to DataSight directly via this link to fulfill their respective business needs.
Case 3: Embedding and Integrating DataSight Pages into an Enterprise's Internal Platform
The Ops team of a company has already configured DataSight, enabling developers to view application logs through DataSight. However, the company has developed its own unified Ops platform and wants to integrate DataSight into this platform with login based on the company's unified identity authentication system. The requirements are as follows:
DataSight pages must be embedded, with page elements such as navigation menus and operation buttons being hideable.
The company's self-built unified identity authentication system (LDAP service) must be used for user login and authentication.
User resource isolation must be automatically implemented based on business team attributes.
Solution
Leverage DataSight's third-party authentication login capability. Currently, integration with LDAP (including OpenLDAP and Active Directory) and OAuth (including multiple OAuth providers like keycloak) is supported. Create two DataSight roles named userA and userB, and configure sub-user keys with different permissions, corresponding to two distinct business teams A and B. Integrate the company's LDAP service with DataSight via a reverse proxy, mapping the DataSight roles to LDAP personnel groups (Posixgroup).
Users access DataSight via the reverse proxy and log in with the account and password from the unified identity authentication system for authentication.
Embed the URL, with page elements controlled (hidden or shown) via parameters.
Operation Steps
1. Create a DataSight instance and select access over the private network. For details, see the steps for creating an instance. 3. Select third-party authentication login as the DataSight login method. Create two roles named userA and userB in the first step and enter their corresponding SecretId and SecretKey. Then, follow the instructions to configure LDAP reverse proxy authentication and enter the reverse proxy's IP address. For details, see Third-Party Authentication Login. Save the configuration upon completion. 4. Embed the URL into the company platform using an iframe (note: use the reverse proxy's domain name, not DataSight's domain name). Users then log in with their LDAP account to view logs.
Embedding example:
function prepareSdkFrame(url) {
var ifrm = document.createElement("iframe");
ifrm.setAttribute("src", url);
ifrm.style.width = "1280px";
ifrm.style.height = "960px";
document.body.appendChild(ifrm);
}
const url = 'https://<domain-appid>.clsconsole.tencent-cloud.com/cls/search?region=${Region}&topic_id=${TopicId}&query=${Query}&time=now-h,now&hideWidget=true&hideTopNav=true&hideLeftNav=true'
prepareSdkFrame(url)
Search and analysis page
// Modify the <domain-appid> (DataSight domain name prefix) part based on your actual situation.
https://<domain-appid>.clsconsole.tencent-cloud.com/cls/search?region=<region>&topic_id=<topic_id>
Dashboard page
https://<domain-appid>.clsconsole.tencent-cloud.com/cls/dashboard/d?id=<dashboard_id>
Related Issues
If you need to pass parameters from the embedded CLS page to the URL of the outer main page for scenarios like sharing or redirection (for example, opening a specified topic within the embedded CLS page via a shared URL), please use the CLS Page SDK for custom integration. For the implementation method, see cls-console-sdk. DataSight Permission Configuration Templates