Curator is Elasticsearch's official tool for managing indices. It can perform a wide variety of index lifecycle management tasks, such as clearing indices created 7 days ago, backing up specified indices regularly every day, and migrating indices from a hot node to a warm node regularly. For more information on operations in Curator, please see Actions. Curator provides a CLI which allows you to configure the tasks to be executed using parameters. It also comes with a complete set of Python APIs that can be used together with SCF, such as automatically deleting expired data from Elasticsearch using Curator. SCF has a pre-defined template for Curator, which can be run after simple parameter configuration. For more information on how to use SCF, please see Serverless Cloud Function. Curator Usage Example
The following describes how to configure and run Curator to delete expired indices regularly.
Installation
Purchase a CVM instance in the VPC where your Elasticsearch cluster resides and install the Curator package via pip.
pip install elasticsearch-curator
Running as command line parameters
The following command filters out the indices named in the format of logstash-20xx-xx-xx and created 7 days ago and then deletes them.
Caution:
The sample code performs a deletion operation to clear your data. Make sure that the above statement has been tested in a non-production environment. You can add the --dry-run parameter for testing purpose to avoid actually deleting data.
curator_cli --host 10.0.0.2:9200 --http_auth 'user:passwd' delete-indices --filter_list '[{"filtertype": "pattern", "kind": "prefix", "value": "logstash-"}, {"filtertype": "age", "source": "name", "direction": "older", "timestring": "%Y.%m.%d", "unit": "days", unit_count: 7}]'
Running as configuration files
If your operations are complex, there are a lot of parameters, or you don't want to use command line parameters, you can place the parameter in configuration files.
In the specified config directory, you need to edit the config.yml and action.yml configuration files. curator_cli --config PATH
Scheduled run
If you need a scheduled run, you can configure the command to a crontab on Linux, or directly use the timer trigger feature of SCF mentioned above.
Using the API
For more information on how to use the Python APIs, please see here.