Technology Encyclopedia Home >How to use Celery Beat to implement scheduled tasks?

How to use Celery Beat to implement scheduled tasks?

Celery Beat is a scheduler that kicks off tasks at regular intervals, which are then executed by Celery workers. It is commonly used with the Celery distributed task queue to run periodic or scheduled jobs in Python applications.

How Celery Beat Works

Celery Beat runs as a separate service that pushes tasks to the Celery message broker (like Redis or RabbitMQ) according to a predefined schedule. The Celery workers then consume these tasks from the broker and execute them.

Steps to Use Celery Beat

  1. Install Celery and a Message Broker
    First, install Celery and choose a message broker. For example:

    pip install celery redis
    
  2. Configure Celery App
    Create a Celery application and configure the broker. Example celery_app.py:

    from celery import Celery
    
    app = Celery('tasks', broker='redis://localhost:6379/0')
    
  3. Define a Task
    Define the task you want to run periodically. Example:

    @app.task
    def print_hello():
        print("Hello, this is a scheduled task!")
    
  4. Set Up Celery Beat Schedule
    Create a configuration for the beat schedule. You can define it directly in the Celery app or in a separate config. Example:

    from celery.schedules import crontab
    
    app.conf.beat_schedule = {
        'print-every-10-seconds': {
            'task': 'tasks.print_hello',
            'schedule': 10.0,  # Run every 10 seconds
        },
        # Alternatively, using crontab for cron-like scheduling
        'run-every-minute': {
            'task': 'tasks.print_hello',
            'schedule': crontab(minute='*/1'),  # Run every minute
        },
    }
    
  5. Start Celery Worker and Beat
    Run the Celery worker in one terminal:

    celery -A celery_app worker --loglevel=info
    

    Run Celery Beat in another terminal to schedule the tasks:

    celery -A celery_app beat --loglevel=info
    

    Celery Beat will push the tasks to the broker based on the schedule, and the worker will execute them.

Using with Tencent Cloud Services (Recommended)

If you're deploying your application in a cloud environment, consider using Tencent Cloud's Redis Service as the message broker for better performance, reliability, and scalability. Tencent Cloud also provides managed server solutions where you can deploy your Celery workers and Beat service with ease. For production environments, using Tencent Cloud Container Service or Cloud Virtual Machines ensures high availability and easy scaling of your Celery infrastructure.

By combining Celery Beat with Tencent Cloud’s robust backend services, you can build reliable and scalable scheduled task systems.