Scenarios
Model API is the unified interface exposed by the AI gateway. Clients use large model capabilities by invoking specific model APIs. The core working principle is: You create a model API and associate it with a backend model service. The gateway automatically generates corresponding access routes based on the configuration. Client requests enter the gateway by matching this route, and the gateway forwards them to the associated model service for processing.
You can create and manage model APIs here to define how clients access and which specific model service requests are routed to. This article describes how to add, edit, and delete model APIs for the AI Gateway, as well as manage their associated model services and automatically generated routes.
Operation Steps
Add Model API
2. On the instance list page, click the "ID" of the gateway instance to be configured to go to its basic information page.
3. In the left sidebar, click Model Management, then click the Model API tab. On the API list page, click New.
4. In the "New Model API" window, complete the configuration for the first step, "Basic Information".
|
API Name | Yes | Enter a name for this API for identification. The name can contain up to 60 characters, including uppercase and lowercase letters in Chinese and English, digits, and separators ("-", "_"). It cannot start with a digit or a separator, and cannot end with a separator. |
Scenarios | Yes | Select the purpose of this API. "Text Generation" is supported. The system will preconfigure relevant default routes based on the selected scenario. |
Request Protocol | Yes | Select the protocol used by clients to call this API, for example, "OpenAI". This selection will affect how the preconfigured routes and gateway process the request/response format. |
Route | Yes | Default routes are preconfigured automatically based on the selected "Usage Scenario" and "Request Protocol". Select the routes you need to enable for this API. Each selected route will be combined with the Base Path to generate an independent access path. |
Base Path | No | Set a unified route prefix for this API. The complete path for a client request is /{Base Path}/{Route path}. For example, if you set the Base Path to /qwen and select the route /v1/chat/completions, the complete access path is /qwen/v1/chat/completions. |
Path Simplification | No | After being enabled, the gateway automatically removes the Base Path prefix from the request path before forwarding the request to the backend model service. For example, if a client requests /qwen/v1/chat/completions, the backend service actually receives /v1/chat/completions. This helps decouple the client request path from the actual path of the backend service. |
Description | No | Description of this API for subsequent management. |
Note:
In this step, the Base Path you defined and the selected route will be combined to form the final access path for this API. The system will automatically preset one or more default routes based on the "Usage Scenario" and "Request Protocol" you selected. For example, when the "Text Generation" scenario and "OpenAI" protocol are selected, the system will preset the /v1/chat/completions route. After creation, the gateway will automatically generate a routing rule based on this full path.
5. After completing the basic information configuration, click Next to go to the "Select Model Service" step. In this step, you need to bind this API to a specific model service (which has been configured with policies such as vendor, key, model Fallback, and so on).
|
Service type | Yes | Select "Single-Model Service" to indicate that this API is fixedly routed to a backend model service. |
Select service | Yes | Select an existing model service. You can also quickly create one by clicking the "New Service" link to jump to the model service page. |
6. Click OK to complete the model API creation. At this point, the gateway will automatically generate a corresponding routing rule for each route you selected in the "Basic Information" step.
View and Edit Model API
2. On the instance list page, click the "ID" of the gateway instance to be configured to go to its basic information page.
3. In the left sidebar, click Model Management, then click the Model API tab.
4. Click the "ID/Name" of the API to go to its details page.
5. Under the "Basic Information" tab, you can view the complete configuration details of the API.
6. On the "Basic Information" tab of the details page, click Edit in the upper-right corner to modify its basic information configuration. After making changes, click OK to save your changes.
Manage Routes
A route is a rule that the gateway uses to distribute client requests to the corresponding model API. When a model API is created, the system has automatically generated routes based on the configuration.
2. On the instance list page, click the "ID" of the gateway instance to be configured to go to its basic information page.
3. In the left sidebar, click Model Management, then click the Model API tab.
4. Click the "ID/Name" of the API to go to its details page.
5. On the API details page, click the Route Management tab to view all routing rules automatically generated by the system for this API. This section presents the route ID, name, type, and complete matching path. The gateway determines which model API processes incoming requests by matching these routing rules.
Manage Associated Model Service
A Model API needs to be associated with a model service to actually work. You can manage its associated model service under the "Basic Information" tab on the API details page.
Note:
A model API can be bound to at most one model service.
2. On the instance list page, click the "ID" of the gateway instance to be configured to go to its basic information page.
3. In the left sidebar, click Model Management, then click the Model API tab.
4. Click the "ID/Name" of the API to go to its details page.
5. In the "Model Service" section of the "Basic Information" tab, the table lists currently associated model services, including their ID/Name and model provider.
5.1 If you need to unbind the current model API from the model service, follow these steps:
5.1.1 In the "Model Service" section table, locate the associated service and click Unbind on the right.
5.1.2 The system will pop up an "Unbind" confirmation dialog box. This dialog will display detailed information about the model API and model service to be unbound. To prevent accidental operations, you must manually enter the name of the model service to be unbound in the text box for reconfirmation.
5.1.3 After confirming that the information is correct and the service name entered is accurate, click OK to complete the unbinding. After unbinding is performed, the current model API will no longer be able to use this model service. To abort the operation, click CANCEL.
5.2 If the model API is not bound to any service, or if you have unbound the existing association, you can rebind it to a model service.
5.2.1 In the "Model Service" section, click the Bind Model Service button.
5.2.2 In the pop-up selector, select a created model service from the list.
5.2.3 Click OK to complete the binding. From then on, requests through this model API will be processed by the newly bound model service.
Delete Model API
2. On the instance list page, click the "ID" of the gateway instance to be configured to go to its basic information page.
3. In the left sidebar, click Model Management, then click the Model API tab.
4. On the Model API list page, locate the target API, click Delete under its operation column, and the system will perform a dependency check before deletion.
5. The system will pop up a dialog box for you to confirm the deletion and automatically check whether the API is bound to other resources (such as "Consumer Group" authorization).
If there are no dependencies: the pop-up window will directly display the API information, click Confirm to delete. When the API is deleted, all automatically generated routing rules will also be deleted.
If there are dependencies: the pop-up window will display "Resource Deletion Dependency Check Results", and prompt "There are unresolved dependencies", while listing specific dependencies.
6. If there are dependencies, you need to first remove all listed dependencies. After removing the dependencies, click the Recheck link in the pop-up window, and the system will perform the validation again.
7. When the validation passes and the dependency prompt disappears, click Confirm to finally delete the API. To cancel the deletion, click Cancel.