A Complete Tutorial: Using RabbitMQ + Celery with Multiple Django Projects

In this post, we’ll walk through how to set up RabbitMQ on one server and run Celery across multiple Django projects, with clean project isolation, scalable architecture, and best practices.

✅ Step 1: Install RabbitMQ (Server A)

➡️ On Ubuntu (Server A):

sudo apt update
sudo apt install rabbitmq-server -y
sudo systemctl enable rabbitmq-server
sudo systemctl start rabbitmq-server

➡️ Enable RabbitMQ Management UI:

sudo rabbitmq-plugins enable rabbitmq_management

Visit: http://server-a-ip:15672 (default guest/guest).

➡️ Create RabbitMQ User:

sudo rabbitmqctl add_user admin strongpasswordhere
sudo rabbitmqctl set_user_tags admin administrator
sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*"
sudo rabbitmqctl delete_user guest  # For security

➡️ Open Ports:

• 5672 (for Celery workers)
• 15672 (for admin UI, restricted IPs recommended)

✅ Step 2: Project Structure (Multiple Django Projects Example)

/var/www/
├── project_v1/
├── dealer_portal/
└── analytics_dashboard/

✅ Step 3: Celery Setup in Each Django Project

📂 In project_v1/celery.py:

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'settings.dev')

app = Celery('project_v1')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.conf.task_default_queue = 'project_v1_queue'
app.autodiscover_tasks()

📂 In project_v1/__init__.py:

from .celery import app as celery_app
__all__ = ('celery_app',)

✅ Repeat for each Django project with unique names and queues.

✅ Step 4: Celery Settings (in settings.py)

CELERY_BROKER_URL = 'amqp://admin:strongpassword@server-a-ip:5672//'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_TASK_QUEUES = {
'project_v1_queue': {
    'exchange': 'project_v1_exchange',
    'binding_key': 'project_v1_queue',
},
}
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True

✅ Step 5: Start Celery Workers

celery -A project_v1 worker -l info -Q project_v1_queue

✅ Step 6: Add Celery Beat (if needed)

celery -A project_v1 beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler

✅ Step 7: Supervisor Configuration (to run worker & beat as services)

📄 Example: /etc/supervisor/conf.d/project_v1_celery.conf

[program:project_v1_worker]
command=/path/to/venv/bin/celery -A project_v1 worker -l info -Q project_v1_queue
directory=/var/www/project_v1
user=ubuntu
numprocs=1
stdout_logfile=/var/www/project_v1/celery-worker.log
stderr_logfile=/var/www/project_v1/celery-worker-err.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=60
priority=999

[program:project_v1_beat]
command=/path/to/venv/bin/celery -A project_v1 beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
directory=/var/www/project_v1
user=ubuntu
numprocs=1
stdout_logfile=/var/www/project_v1/celery-beat.log
stderr_logfile=/var/www/project_v1/celery-beat-err.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=60
priority=998

✅ Repeat this config for each Django project.

✅ Step 8: Reload Supervisor

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start all

✅ Step 9: Monitor

• RabbitMQ UI: http://server-a-ip:15672
• Optional Flower monitoring:

celery -A project_v1 flower --port=5555

✅ Conclusion:

• ✅ Install RabbitMQ on a dedicated server.
• ✅ Multiple Django projects connect to the same RabbitMQ.
• ✅ Each project has its own queue, Celery app name, and worker.
• ✅ Manage processes with Supervisor.
• ✅ Monitor queues with RabbitMQ UI and Flower.