A Complete Tutorial: Using RabbitMQ + Celery with Multiple Django Projects

In this post, we’ll walk through how to set up RabbitMQ on one server and run Celery across multiple Django projects, with clean project isolation, scalable architecture, and best practices. ✅ Step 1: Install RabbitMQ (Server A) ➡️ On Ubuntu (Server A): sudo apt update sudo apt install rabbitmq-server -y sudo systemctl enable rabbitmq-server sudo systemctl start rabbitmq-server ➡️ Enable RabbitMQ Management UI: sudo rabbitmq-plugins enable rabbitmq_management Visit: http://server-a-ip:15672 (default guest/guest). ➡️ Create RabbitMQ User: sudo rabbitmqctl add_user admin strongpasswordhere sudo rabbitmqctl set_user_tags admin administrator sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*" sudo rabbitmqctl delete_user guest # For security ➡️ Open Ports: 5672 (for Celery workers) 15672 (for admin UI, restricted IPs recommended) ✅ Step 2: Project Structure (Multiple Django Projects Example) /var/www/ ├── project_...
Post a Comment

Implementing Throttling in Django REST Framework.

Implementing Throttling in Django REST Framework to Manage High Traffic and Prevent Abuse

Introduction

In high-traffic applications, it’s essential to protect server resources and ensure fair usage among users. Throttling is a method used in Django REST Framework (DRF) to limit the rate of requests, helping prevent abuse and manage load. In this article, we’ll explore how to implement throttling in DRF, including ways to apply it to specific views and dynamically whitelist IPs.

What is Throttling?

Throttling controls the rate of requests an API endpoint can receive from a single client. It’s a valuable tool for:

  • Preventing abuse by limiting request rates for each user or IP address.
  • Managing server load by avoiding high request volumes that can overwhelm the application.

In Django REST Framework, throttling is easy to configure with built-in classes like UserRateThrottle and AnonRateThrottle.

Configuring Basic Throttling in Django REST Framework

To set up global throttling for all requests, define DEFAULT_THROTTLE_CLASSES and DEFAULT_THROTTLE_RATES in your settings.py file:


# settings.py
REST_FRAMEWORK = {
'DEFAULT_THROTTLE_CLASSES': [
    'rest_framework.throttling.UserRateThrottle',
    'rest_framework.throttling.AnonRateThrottle',
],
'DEFAULT_THROTTLE_RATES': {
    'user': '1000/day',   # Limits authenticated users to 1000 requests per day
    'anon': '100/hour',    # Limits anonymous users to 100 requests per hour
}
}

Applying Throttling to Specific Views

If you only need throttling for certain endpoints, set throttle_classes directly in the view:


from rest_framework.views import APIView
from rest_framework.throttling import ScopedRateThrottle

class LimitedAccessView(APIView):
throttle_classes = [ScopedRateThrottle]
throttle_scope = 'limited_access'  # Reference a custom scope in settings.py

def get(self, request):
    return Response({"message": "This endpoint is throttled to 10 requests per minute."})

In settings.py, define the custom scope:


# settings.py
REST_FRAMEWORK = {
'DEFAULT_THROTTLE_RATES': {
    'limited_access': '10/minute',   # Custom rate for specific endpoint
}
}

Whitelisting IPs Dynamically for Throttling

For larger applications, you may want to store whitelisted IPs in a database, such as MongoDB, to allow dynamic updates. Here’s an example of using MongoDB with a custom throttle class:

Define the Custom Throttle Class:


from pymongo import MongoClient
from rest_framework.throttling import SimpleRateThrottle

client = MongoClient('mongodb://localhost:27017/')
db = client['your_database_name']
whitelisted_ips_collection = db['whitelisted_ips']

class CustomIPThrottle(SimpleRateThrottle):
scope = 'custom_ip'

def get_cache_key(self, request, view):
    ip_addr = self.get_ident(request)
    if whitelisted_ips_collection.find_one({"ip_address": ip_addr}):
        return None  # Skip throttling for whitelisted IPs
    return ip_addr

Add Custom Throttle to settings.py:


# settings.py
REST_FRAMEWORK = {
'DEFAULT_THROTTLE_CLASSES': [
    'path.to.CustomIPThrottle',   # Use the custom throttle class
],
'DEFAULT_THROTTLE_RATES': {
    'custom_ip': '50/hour',
}
}

With this setup, requests from whitelisted IPs will bypass throttling, while other IPs will be limited based on the specified rate.

Conclusion

Throttling is a crucial part of building scalable and secure APIs. By implementing DRF’s built-in throttling classes, configuring custom throttling for specific views, and dynamically whitelisting IPs from a database, you can manage high traffic efficiently while preventing abuse. These strategies help ensure that your API remains responsive and available to legitimate users.

Frequently Asked Questions (FAQs)

1. What is throttling in Django REST Framework?

Throttling in Django REST Framework (DRF) is a technique used to limit the rate of requests a client can make to an API. This prevents abuse, manages server load, and ensures fair resource usage by applying rate limits on requests from individual users or IP addresses.

2. Why do we need throttling in an API?

Throttling is essential in high-traffic applications to prevent overloading server resources, maintain performance, and reduce the impact of potential malicious attacks. By controlling the number of requests, throttling helps in preserving the availability and responsiveness of the API for all users.

3. What are some built-in throttle classes in DRF?

Django REST Framework includes several throttle classes, such as:

  • UserRateThrottle: Limits the rate of requests for authenticated users.
  • AnonRateThrottle: Limits the rate of requests for anonymous users.
  • ScopedRateThrottle: Applies specific rate limits to particular views, allowing different rates per endpoint.

4. How do I set global throttling rates in DRF?

To set global throttling rates, define DEFAULT_THROTTLE_CLASSES and DEFAULT_THROTTLE_RATES in the settings.py file. For example:


# settings.py
REST_FRAMEWORK = {
    'DEFAULT_THROTTLE_CLASSES': [
        'rest_framework.throttling.UserRateThrottle',
        'rest_framework.throttling.AnonRateThrottle',
    ],
    'DEFAULT_THROTTLE_RATES': {
        'user': '1000/day',   # Authenticated users can make 1000 requests per day
        'anon': '100/hour',    # Anonymous users can make 100 requests per hour
    }
}

5. Can I apply throttling to specific views only?

Yes, you can apply throttling to specific views by using the throttle_classes attribute directly in the view. This allows you to set custom throttling limits on particular endpoints without affecting the rest of the API.

6. Why do we use self.get_ident(request) in custom throttling classes?

The self.get_ident(request) method is used to extract a unique identifier, typically the IP address, from the incoming request. This is useful in custom throttling classes where we want to apply rate limits based on the client’s IP address. If the IP address is found in the whitelist (e.g., stored in a database), throttling can be bypassed by returning None. Otherwise, the IP address is used as the cache key to track the request rate.


def get_cache_key(self, request, view):
    ip_addr = self.get_ident(request)
    # Check if IP is whitelisted
    if whitelisted_ips_collection.find_one({"ip_address": ip_addr}):
        return None  # Skip throttling for whitelisted IPs
    return ip_addr

7. How can I whitelist specific IP addresses for throttling?

To whitelist specific IP addresses, you can create a custom throttle class that checks if an IP is in a whitelist (e.g., a database collection). If the IP is whitelisted, the custom class can return None to skip throttling. This allows dynamic control over which IPs can bypass rate limits.

8. Can I dynamically update the whitelist for throttling in DRF?

Yes, by storing whitelisted IPs in a database (e.g., MongoDB or PostgreSQL), you can update the whitelist dynamically. Each time a request is received, the custom throttle class can query the database to check if the IP is whitelisted, allowing real-time adjustments without needing to modify the code.

9. What happens if a user exceeds their request limit?

When a user exceeds their request limit, DRF will automatically respond with a 429 "Too Many Requests" HTTP status code. This indicates that the client should wait before making further requests.

10. Can I display a custom message for throttled requests?

Yes, DRF allows you to customize the throttling response by overriding the throttle_wait method in your custom throttle class or by modifying the default exception handling. You can display custom messages or instructions on how to proceed if the request limit is reached.

Location: X6R8+9P4, NH503, Balsun, Himachal Pradesh 176029, India

Comments

Post a Comment

Popular posts from this blog

How to Reset All Migrations in Django: A Comprehensive Guide

Image
Migrations in Django are essential for managing changes to your database schema over time. However, during development or when migrations become too cumbersome, you might need to reset them entirely. This guide will walk you through the process of resetting all migrations in a Django project, ensuring a clean slate for your database schema. Table of Contents Why Reset Migrations? Prerequisites Step-by-Step Reset Guide Backup Your Database Identify Affected Apps Delete Migration Files Reset Migrations to Zero Recreate Initial Migrations Apply Migrations Thoroughly Test Your Application Understanding Django Migration Commands Handling Manual Schema Changes Conclusion Why Reset Migrations? Over time, as a project evolves, the number of migration files can grow significantly. This can make it challenging to manage migrations, especially if: You've made substantial changes to your models. Mi...
Read more

Managing Python Projects with Pipenv and Pyenv: A Comprehensive Guide

Image
In the ever-evolving world of Python development, managing project dependencies and Python versions can become a complex task. Tools like Pipenv and Pyenv have been created to simplify these processes, but understanding how they work individually and together is crucial for efficient project management. This comprehensive guide will delve into the functionalities of Pipenv and Pyenv, highlight their differences, and demonstrate how to effectively use them in your Python projects. Table of Contents Introduction to Pipenv Key Features of Pipenv Pipenv General Commands Pipfile vs requirements.txt Managing Dependencies in a Django Application Creating a Pipfile and Pipfile.lock Example Pipfile Explained Automatic Updating of Pipfile and Pipfile.lock Setting Up the Project with Pipenv Introduction to Pyenv Key...
Read more

Differences Between List, Dictionary, and Tuple in Python

In Python, lists, dictionaries, and tuples are all built-in data structures that store collections of items. While they share similarities, each has its distinct characteristics and use cases. This post provides a comprehensive comparison. Comparison Table Aspect List Dictionary Tuple Definition An ordered collection of elements. A collection of key-value pairs. An ordered and immutable collection of elements. Access Method Index-based access ( my_list[0] ). Key-based access ( my_dict['key'] ). Index-based access ( my_tuple[0] ). Mutability Mutable: Elements can be added, removed, or changed. Mutable (for values): Keys/values can be added, modified, or removed, but keys must be immutable. Immutable: Elements cannot be changed once assigned. Order Guarantee Maintains th...
Read more