Redis Alternatives Implementation Guide¶

Overview¶

This guide explains how to replace expensive Redis infrastructure with cost-effective alternatives while maintaining the same caching functionality.

Cost Comparison¶

Available Cache Implementations¶

1. File-Based Cache (Immediate 90% Cost Reduction)¶

Best for: Development, staging, small production deployments

Pros: - Zero infrastructure cost - Simple implementation - Persists across restarts - TTL support

Cons: - Single-server only - File I/O performance - No distributed caching

Configuration:

CACHE_TYPE=file
CACHE_ENABLED=true

2. PostgreSQL Cache (85% Cost Reduction + Better Performance)¶

Best for: Production environments, better performance than file cache

Pros: - Uses existing database infrastructure - Better performance than file cache - Supports transactions - Can be distributed (with read replicas) - TTL support with automatic cleanup

Cons: - Slightly more complex than file cache - Database overhead

Configuration:

CACHE_TYPE=postgres
CACHE_ENABLED=true

3. Auto-Detection (Recommended)¶

Best for: Flexible deployment

Configuration:

CACHE_TYPE=auto
CACHE_ENABLED=true

Auto-detection logic: 1. If Redis is available and configured → Use Redis 2. If PostgreSQL is available → Use PostgreSQL cache 3. Otherwise → Use File cache

Implementation Steps¶

Step 1: Deploy New Cache Implementations¶

1. Update Environment Variables:

   # For immediate cost savings (use file cache)
   echo "CACHE_TYPE=file" >> .env
   
   # For production (use PostgreSQL cache)
   echo "CACHE_TYPE=postgres" >> .env
   
   # For flexible deployment (auto-detection)
   echo "CACHE_TYPE=auto" >> .env
   

2. Run Database Migration (if using PostgreSQL cache):

   # Local development
   cd fastapi-backend
   docker compose exec backend alembic upgrade head
   
   # Production
   alembic upgrade head
   

Step 2: Update Infrastructure¶

1. Remove Redis from Docker Compose (optional):

   # Comment out or remove Redis service
   # redis:
   #   image: redis:7-alpine
   #   ports:
   #     - "6379:6379"
   

2. Update Environment Variables:

   # Make Redis optional
   REDIS_URL=redis://localhost:6379/0  # Keep for backward compatibility
   CACHE_TYPE=postgres  # Use PostgreSQL cache instead
   

Step 3: Update Cloud Infrastructure¶

For GCP (Remove Memorystore):¶

1. Update Terraform (infra/terraform_gcp/redis.tf):

   # Comment out or remove Redis instance
   # resource "google_redis_instance" "redis" {
   #   ...
   # }
   

2. Update Cloud Run Configuration (infra/terraform_gcp/cloud_run.tf):

   # Remove Redis environment variables
   # env {
   #   name  = "REDIS_URL"
   #   value = "redis://${google_redis_instance.redis.host}:${google_redis_instance.redis.port}/0"
   # }
   
   # Add cache type configuration
   env {
     name  = "CACHE_TYPE"
     value = "postgres"
   }
   

For AWS (Remove ElastiCache):¶

1. Update Terraform (infra/terraform_aws/elasticache.tf):

   # Comment out or remove ElastiCache cluster
   # resource "aws_elasticache_cluster" "redis" {
   #   ...
   # }
   

2. Update ECS Configuration (infra/terraform_aws/ecs.tf):

   # Remove Redis environment variables
   # { name = "REDIS_URL", value = "redis://..." }
   
   # Add cache type configuration
   { name = "CACHE_TYPE", value = "postgres" }
   

Step 4: Deploy and Test¶

1. Deploy the changes:

   # For GCP
   cd infra/terraform_gcp
   terraform plan
   terraform apply
   
   # For AWS
   cd infra/terraform_aws
   terraform plan
   terraform apply
   

2. Test the application:

   # Check cache health
   curl -X GET "https://your-api.com/api/v1/health"
   
   # Test caching functionality
   curl -X POST "https://your-api.com/api/v1/travel/recommendations" \
     -H "Content-Type: application/json" \
     -d '{"budget": 1000, "duration": 7, "departure_city": "Sydney"}'
   

Migration Strategies¶

Strategy 1: Gradual Migration (Recommended)¶

1. Phase 1: Deploy with CACHE_TYPE=auto (keeps Redis if available)
2. Phase 2: Switch to CACHE_TYPE=postgres in staging
3. Phase 3: Test performance and functionality
4. Phase 4: Switch production to CACHE_TYPE=postgres
5. Phase 5: Remove Redis infrastructure

Strategy 2: Immediate Migration (Maximum Cost Savings)¶

1. Deploy with CACHE_TYPE=file for immediate savings
2. Remove Redis infrastructure immediately
3. Upgrade to PostgreSQL cache when ready

Strategy 3: Hybrid Approach¶

1. Keep Redis for critical caching (AI responses)
2. Use PostgreSQL cache for less critical data
3. Implement cache routing based on data type

Performance Optimization¶

File Cache Optimization¶

# Use environment variables for file cache configuration
CACHE_DIR=/tmp/macumba_cache  # Use fast storage
CACHE_CLEANUP_INTERVAL=100    # Cleanup frequency

PostgreSQL Cache Optimization¶

-- Create indexes for better performance
CREATE INDEX CONCURRENTLY idx_cache_entries_expires_at_not_null 
ON cache_entries (expires_at) WHERE expires_at IS NOT NULL;

-- Optimize cleanup queries
CREATE INDEX CONCURRENTLY idx_cache_entries_cleanup 
ON cache_entries (expires_at) WHERE expires_at < NOW();

Monitoring and Observability¶

1. Add cache metrics:

   # Monitor cache hit rates
   CACHE_HIT_RATE = cache_hits / (cache_hits + cache_misses)
   
   # Monitor cache size
   CACHE_SIZE = count(cache_entries)
   
   # Monitor cache cleanup frequency
   CACHE_CLEANUP_RUNS = count(cleanup_operations)
   

2. Set up alerts:

3. Cache hit rate below 70%
4. Cache size growing too large
5. Cache cleanup failures

Troubleshooting¶

Common Issues¶

1. Cache not working:

   # Check cache type
   echo $CACHE_TYPE
   
   # Check cache health
   curl -X GET "https://your-api.com/api/v1/health"
   
   # Check logs
   docker compose logs backend | grep -i cache
   

2. Performance issues:

   # Check cache hit rate
   grep "Cache hit" logs/app.log | wc -l
   grep "Cache miss" logs/app.log | wc -l
   
   # Check cache size (PostgreSQL)
   psql -c "SELECT COUNT(*) FROM cache_entries;"
   
   # Check cache cleanup
   psql -c "SELECT COUNT(*) FROM cache_entries WHERE expires_at < NOW();"
   

3. Database migration issues:

   # Check migration status
   alembic current
   
   # Run migration manually
   alembic upgrade head
   
   # Check table exists
   psql -c "\dt cache_entries"
   

Recovery Procedures¶

1. Rollback to Redis:

   # Set environment variable
   CACHE_TYPE=redis
   
   # Restart application
   docker compose restart backend
   

2. Clear cache:

   # File cache
   rm -rf /tmp/macumba_cache/*
   
   # PostgreSQL cache
   psql -c "DELETE FROM cache_entries;"
   
   # Redis cache
   redis-cli FLUSHALL
   

Expected Cost Savings¶

Monthly Cost Reduction¶

Performance Comparison¶

Conclusion¶

Implementing Redis alternatives can save $90-180+ per month (100% Redis cost reduction) while maintaining equivalent functionality. The PostgreSQL cache provides the best balance of performance and cost savings for most production deployments.

Support¶

For questions or issues: 1. Check application logs for cache-related errors 2. Verify database connectivity (for PostgreSQL cache) 3. Test cache health endpoint 4. Review this guide for troubleshooting steps