You can set origin-pull rules for buckets through the Cloud Object Storage (COS) console. When the object requested does not exist in the bucket or redirection of specific requests is needed, you can access the corresponding data from COS via origin-pull rules. Setting origin-pull is primarily used for hot migration of data, redirection of specific requests, and other scenarios. You can configure it according to your actual needs.
Note:
The success rate of data origin-pull depends on your network environment.Please give priority to using IP segments such as China Telecom, China Mobile, and China Unicom.
Buckets in the Financial Cloud region do not support origin-pull settings.
Public cloud regions only partially support synchronous origin-pull, including: Beijing, Chengdu, Guangzhou, Nanjing, Shanghai, Hong Kong (China), Singapore, Jakarta, Tokyo, Frankfurt, Virginia (US East), and Silicon Valley (US West).
Origin-Pull Rules
Trigger condition
In async and sync origin-pull modes, trigger origin-pull only when a GET request returns 404. If it is a HEAD request or other requests, it will not trigger origin-pull.
In redirect mode, you can customize HTTP status codes between 400 and 599 to trigger origin-pull.
Origin access
In async and sync origin-pull modes, you can set whether to pass through the QueryString and header information of the COS access request to the origin, and configure to carry additional header information when requesting the origin. In redirect mode, you can only set whether to pass through the QueryString.
If GET range is specified during the GET operation, COS will send an async request without range in addition to the original request to get the complete object data and store it in COS.
Response and storage
The origin can return data in chunked encoding.
If the origin returns a 404 status code, it will be passed through to COS and returned to the user. If 3xx Following Policy is enabled, when the origin returns a 3xx status code, COS will pull data from another origin. If the origin returns other non-3XX status codes, COS will return a 424 status code.
The file returned by origin-pull will be stored in COS with the filename used when the origin is requested. For example, if the file example.jpg requested by a user does not exist in the bucket, COS will trigger the origin-pull mechanism to pull the file from the configured origin-pull address http://origin.com/example.jpg and rename the file stored in the bucket example.jpg.
The new object stored in COS will contain the following metadata, with the data content following the values in the origin:
3. Click the name of the bucket for which you want to set origin-pull.
4. Select Basic Configurations > Origin-Pull on the left, and click Add Rule.
5. In the pop-up window, configure the following information and click Next:
Origin-Pull Mode: Select an origin-pull mode as needed.
Asynchronous Origin-Pull: If the requested file doesn't exist in COS, COS will search for it on the specified origin. In the Async Origin-Pull mode, COS returns 302 to the client after finding that the file does not exist. The client jumps to the specified origin, and then asynchronously uploads the file to the COS bucket.
Note:
Async origin-pull does not directly return the file; instead, it returns a 302 status code to the client first and then asynchronously uploads the file to COS.
We recommend you enable Follow 302 to pull data from the origin.
The file upload time is subject to multiple factors, and no SLA can be promised. We recommend you select sync origin-pull if your business is sensitive to delays.
Sync Origin-Pull: If the requested file doesn't exist in COS, COS will search for it on the specified origin, return it to the client, and upload it to the bucket.
Note:
When using synchronous origin-pull, the timeliness can only be guaranteed if the file is fully downloaded and a status code 200 is returned. If the client accesses the file via a range request, the timeliness cannot be guaranteed.
Redirect: If a specified error is reported when accessing the bucket, COS will return the redirection address to the client but will not save the file from the origin. The client can request the resource from the origin at the redirection address.
Origin-Pull Condition:Select the conditions to trigger origin-pull as needed. Origin-pull is triggered only when all configured origin-pull conditions are satisfied simultaneously.
HTTP Status Code 404: If you select Async Origin-Pull or Sync Origin-Pull, trigger origin-pull when a GET request returns an HTTP status code of 404. This item is required and cannot be canceled. If you select Redirect, you can enter an HTTP status code ranging from 400 to 599.
File name prefix: When the requested file matches the prefix, the origin-pull rule will be triggered. For example, if you set this option to prefix, make a GET request to https://examplebucket-1250000000.cos.ap-chengdu.myqcloud.com/prefix123.jpg and the returned HTTP status code is 404, the origin-pull rule will be triggered.
Origin-Pull Protocol: The protocol used by COS to access the specified origin. The options include Force HTTPS, Force HTTP, and Follow request protocol.
If you select Force HTTPS or Force HTTP, COS will access the origin using HTTPS or HTTP respectively.
If you select Follow request protocol, COS will access the origin with the protocol used in the request.
Request Parameter: Specify whether to pass through the queryString request parameters carried when accessing COS to the origin.
Request header: You can choose to pass through all request headers or pass through specified request headers. If you choose to pass through all request headers, it is recommended that you configure it to prohibit pass-through of the host header. One-click configuration is supported here.
Passthrough specifies the request header: When selecting pass through specified request header, you need to set this item. Here, you can add the header information that you want to be passed through to the origin server. If you select the redirection origin-pull type, do not set this item.
Prohibitpassing through specified request header: Here you can add the header information in the request that you do not want to be passed through to the origin server. If you select the redirection origin-pull type, do not set this item.
New request header: Additional request headers can be added here when returning to the origin server. If the redirect back to the origin type is selected, this item should not be set.
6. Configure the following information based on the selected origin-pull mode and click Next:
Async origin-pull
Sync origin-pull
Redirect
Origin-Pull Address: Enter the domain name or IP address without the http:// or https:// prefix. You can also add the port number after the domain name or IP address.
Example of a correct address:
abc.example.com
abc.example.com:8080
202.96.128.86
202.96.128.86:8080
Standby forwarding address: Support setting the backup source address. For input format, refer to the configuration item description of origin address. This configuration depends on the bucket location. If the region supports it, it can be configured; otherwise, it cannot be configured.
Address settings: You can choose whether to set a specific file or file path to jump to. The configuration items are described as follows:
Fixed filesettings: When the origin-pull rules are triggered, all requests will be redirected to the fixed file.
PrefixSuffixSettingsConfiguration: When the origin-pull rules are triggered, it will redirect to the file path with the specified prefix and suffix.
For example, if the prefix is set to test, accessing https://examplebucket-1250000000.cos.ap-chengdu.myqcloud.com/path/prefix123.jpg triggers the origin-pull rules, and it will redirect to <origin address>/test/path/prefix123.jpg.
For example, if the suffix is set to .jpg, accessing https://examplebucket-1250000000.cos.ap-chengdu.myqcloud.com/path/prefix123 triggers the origin-pull rules, and it will redirect to <origin address>/path/prefix123.jpg.
Note:
The configurations of specifying a prefix and specifying a suffix can take effect at the same time.
3xx following policy: Once enabled, when the origin server returns a 3XX redirection status code, COS's default handling policy is to follow the 3XX and fetch data from another origin server again. Selecting Disable will prevent resource retrieval.
Note:
When the origin server returns a 3XX redirection to an intranet address, COS will not follow it. In synchronous origin-pull mode, COS will not follow the redirected intranet address. In asynchronous origin-pull and redirection modes, after an intranet address is returned, business judgment is required to decide whether to follow or disable the redirection.
Origin-Pull Address: Enter the domain name or IP address without the http:// or https:// prefix. You can also add the port number after the domain name or IP address.
Example of a correct address:
abc.example.com
abc.example.com:8080
202.96.128.86
202.96.128.86:8080
Standby forwarding address: Support setting the backup source address. For input format, refer to the configuration item description of origin address. This configuration depends on the bucket location. If the region supports it, it can be configured; otherwise, it cannot be configured.
Address settings: You can choose whether to set a specific file or file path to jump to. The configuration items are described as follows:
Fixed filesettings: When the origin-pull rules are triggered, all requests will be redirected to the fixed file.
PrefixSuffixSettingsConfiguration: When the origin-pull rules are triggered, it will redirect to the file path with the specified prefix and suffix.
For example, if the prefix is set to test, accessing https://examplebucket-1250000000.cos.ap-chengdu.myqcloud.com/path/prefix123.jpg triggers the origin-pull rules, and it will redirect to <origin address>/test/path/prefix123.jpg.
For example, if the suffix is set to .jpg, accessing https://examplebucket-1250000000.cos.ap-chengdu.myqcloud.com/path/prefix123 triggers the origin-pull rules, and it will redirect to <origin address>/path/prefix123.jpg.
Note:
The configurations of specifying a prefix and specifying a suffix can take effect at the same time.
3xx following policy: Once enabled, when the origin server returns a 3XX redirection status code, COS's default handling policy is to follow the 3XX and fetch data from another origin server again. Selecting Disable will prevent resource retrieval.
Packet of origin site: After this feature is enabled, the packet from the origin will be directly returned, including the status code and other information.This configuration depends on the bucket location. If the region supports it, it can be configured; otherwise, it cannot be configured.
Origin-Pull Address: Enter the domain name or IP address without the http:// or https:// prefix. You can also add the port number after the domain name or IP address.
Example of a correct address:
abc.example.com
abc.example.com:8080
202.96.128.86
202.96.128.86:8080
Address settings: You can choose whether to set a specific file or file path to jump to. The configuration items are described as follows:
Fixed filesettings: When the origin-pull rules are triggered, all requests will be redirected to the fixed file.
PrefixSuffixSettingsConfiguration: When the origin-pull rules are triggered, it will redirect to the file path with the specified prefix and suffix.
For example, if the prefix is set to test, accessing https://examplebucket-1250000000.cos.ap-chengdu.myqcloud.com/path/prefix123.jpg triggers the origin-pull rules, and it will redirect to <origin address>/test/path/prefix123.jpg.
For example, if the suffix is set to .jpg, accessing https://examplebucket-1250000000.cos.ap-chengdu.myqcloud.com/path/prefix123 triggers the origin-pull rules, and it will redirect to <origin address>/path/prefix123.jpg.
Note:
The configurations of specifying a prefix and specifying a suffix can take effect at the same time.
Redirect Code: 301, 302, and 307 are optional. The default is 302.
7. Confirm that the configured origin-pull rule is correct and click OK.
8. By default, COS always gives the highest priority to the most recent rule, by which it performs origin-pull. To change the priority manually, you can click the Edit icon under the Priority column in the rule list.
Note：
After a high-priority source fails, it will not revert to a low-priority source address, but instead will revert to the backup source address of the original source.
Examples
Background
A user whose APPID is 1250000000 created a bucket named "examplebucket-1250000000", and enabled CDN acceleration endpoint domain name:
examplebucket-1250000000.file.myqcloud.com
Configure the origin-pull address of the bucket to be:
abc.example.com
Store the image picture.jpg on the origin http://abc.example.com.
First access by client (without sync origin-pull enabled):
When COS finds that the object cannot be hit, it returns the HTTP status code 200 to the client and redirects to the following address:
http://abc.example.com/picture.jpg
The origin then provides the object to the client to ensure access, and COS copies picture.jpg from the origin and saves it to the root directory of the bucket "examplebucket-1250000000".
