In this post, we’ll walk through how to set up RabbitMQ on one server and run
Celery across multiple Django projects, with clean project isolation, scalable architecture, and
best practices.
✅ Step 1: Install RabbitMQ (Server A)
➡️ On Ubuntu (Server A):
sudo apt update
sudo apt install rabbitmq-server -y
sudo systemctl enable rabbitmq-server
sudo systemctl start rabbitmq-server
➡️ Enable RabbitMQ Management UI:
sudo rabbitmq-plugins enable rabbitmq_management
Visit: http://server-a-ip:15672
(default guest/guest).
➡️ Create RabbitMQ User:
sudo rabbitmqctl add_user admin strongpasswordhere
sudo rabbitmqctl set_user_tags admin administrator
sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*"
sudo rabbitmqctl delete_user guest # For security
➡️ Open Ports:
- 5672 (for Celery workers)
- 15672 (for admin UI, restricted IPs recommended)
✅ Step 2: Project Structure (Multiple Django Projects Example)
/var/www/
├── project_v1/
├── dealer_portal/
└── analytics_dashboard/
✅ Step 3: Celery Setup in Each Django Project
📂 In project_v1/celery.py
:
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'settings.dev')
app = Celery('project_v1')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.conf.task_default_queue = 'project_v1_queue'
app.autodiscover_tasks()
📂 In project_v1/__init__.py
:
from .celery import app as celery_app
__all__ = ('celery_app',)
✅ Repeat for each Django project with unique names and queues.
✅ Step 4: Celery Settings (in settings.py
)
CELERY_BROKER_URL = 'amqp://admin:strongpassword@server-a-ip:5672//'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_TASK_QUEUES = {
'project_v1_queue': {
'exchange': 'project_v1_exchange',
'binding_key': 'project_v1_queue',
},
}
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True
✅ Step 5: Start Celery Workers
celery -A project_v1 worker -l info -Q project_v1_queue
✅ Step 6: Add Celery Beat (if needed)
celery -A project_v1 beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
✅ Step 7: Supervisor Configuration (to run worker & beat as services)
📄 Example: /etc/supervisor/conf.d/project_v1_celery.conf
[program:project_v1_worker]
command=/path/to/venv/bin/celery -A project_v1 worker -l info -Q project_v1_queue
directory=/var/www/project_v1
user=ubuntu
numprocs=1
stdout_logfile=/var/www/project_v1/celery-worker.log
stderr_logfile=/var/www/project_v1/celery-worker-err.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=60
priority=999
[program:project_v1_beat]
command=/path/to/venv/bin/celery -A project_v1 beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
directory=/var/www/project_v1
user=ubuntu
numprocs=1
stdout_logfile=/var/www/project_v1/celery-beat.log
stderr_logfile=/var/www/project_v1/celery-beat-err.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=60
priority=998
✅ Repeat this config for each Django project.
✅ Step 8: Reload Supervisor
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start all
✅ Step 9: Monitor
- RabbitMQ UI:
http://server-a-ip:15672
- Optional Flower monitoring:
celery -A project_v1 flower --port=5555
✅ Conclusion:
- ✅ Install RabbitMQ on a dedicated server.
- ✅ Multiple Django projects connect to the same RabbitMQ.
- ✅ Each project has its own queue, Celery app name, and worker.
- ✅ Manage processes with Supervisor.
- ✅ Monitor queues with RabbitMQ UI and Flower.
Comments
Post a Comment