Celery Beat is a scheduler that kicks off tasks at regular intervals, which are then executed by Celery workers. It is commonly used with the Celery distributed task queue to run periodic or scheduled jobs in Python applications.
Celery Beat runs as a separate service that pushes tasks to the Celery message broker (like Redis or RabbitMQ) according to a predefined schedule. The Celery workers then consume these tasks from the broker and execute them.
Install Celery and a Message Broker
First, install Celery and choose a message broker. For example:
pip install celery redis
Configure Celery App
Create a Celery application and configure the broker. Example celery_app.py:
from celery import Celery
app = Celery('tasks', broker='redis://localhost:6379/0')
Define a Task
Define the task you want to run periodically. Example:
@app.task
def print_hello():
print("Hello, this is a scheduled task!")
Set Up Celery Beat Schedule
Create a configuration for the beat schedule. You can define it directly in the Celery app or in a separate config. Example:
from celery.schedules import crontab
app.conf.beat_schedule = {
'print-every-10-seconds': {
'task': 'tasks.print_hello',
'schedule': 10.0, # Run every 10 seconds
},
# Alternatively, using crontab for cron-like scheduling
'run-every-minute': {
'task': 'tasks.print_hello',
'schedule': crontab(minute='*/1'), # Run every minute
},
}
Start Celery Worker and Beat
Run the Celery worker in one terminal:
celery -A celery_app worker --loglevel=info
Run Celery Beat in another terminal to schedule the tasks:
celery -A celery_app beat --loglevel=info
Celery Beat will push the tasks to the broker based on the schedule, and the worker will execute them.
If you're deploying your application in a cloud environment, consider using Tencent Cloud's Redis Service as the message broker for better performance, reliability, and scalability. Tencent Cloud also provides managed server solutions where you can deploy your Celery workers and Beat service with ease. For production environments, using Tencent Cloud Container Service or Cloud Virtual Machines ensures high availability and easy scaling of your Celery infrastructure.
By combining Celery Beat with Tencent Cloud’s robust backend services, you can build reliable and scalable scheduled task systems.