Macumba Travel Backend - FastAPI AI Travel Recommendation Engine¶
๐ Backend Overview¶
The Macumba Travel backend is a powerful FastAPI application that serves as the brain of the travel recommendation platform. It processes user travel requests, integrates with multiple AI providers, and returns personalized destination recommendations enriched with real-time data.
๐ฏ What This Backend Does¶
- AI-Powered Recommendations: Integrates with Claude and Gemini AI to generate intelligent travel suggestions
- Smart Budget Processing: Calculates per-person budgets and filters recommendations accordingly
- Multi-Source Data Enrichment: Combines AI recommendations with weather, flight, and location data
- Intelligent Caching: Uses PostgreSQL-based caching for cost-effective storage of expensive AI responses and external API calls
- Rate Limiting: Manages API usage for anonymous vs authenticated users
- User Management: Handles authentication, registration, and trip saving
- Production-Safe Schema Management: Environment-aware database schema handling
- Prometheus Monitoring: Standard metrics format for monitoring and alerting
- Comprehensive Health Checks: Liveness, readiness, and health endpoints
๐ Current Technology Stack¶
- Python: 3.12.4
- FastAPI: 0.115.8 - Modern, fully async web framework for building APIs
- SQLAlchemy: 2.0.38 - Powerful async ORM for database operations
- Pydantic: 2.10.6 - Data validation using Python type annotations
- Alembic: Database migration tool
- PostgreSQL: 15-alpine - Primary database with async support
- Docker: Containerized deployment
- JWT Authentication: Secure user authentication
- Amadeus API Integration: Flight search and booking
- MailerSend Integration: Email notifications
- AI Integration: Claude (Anthropic) and Gemini (Google) providers with async support
- Monitoring: Prometheus metrics integration
- Health Checks: Kubernetes-compatible liveness/readiness probes
- Comprehensive Testing: Unit and integration tests with pytest and async support
๐๏ธ Architecture Deep Dive¶
High-Level Request Flow¶
HTTP Request โ FastAPI Router โ Dependency Injection โ Service Layer โ AI/External APIs โ Database/Cache โ Response
Detailed Component Architecture¶
โโโโโโโโโโโโโโโโโโโ
โ API Layer โ FastAPI routers with automatic docs
โ โ /api/v1/travel/recommendations
โโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโ
โ Dependencies โ Dependency injection for services
โ โ Rate limiting, authentication
โโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโ
โ Service Layer โ Business logic and AI integration
โ โ TravelService, AIService, etc.
โโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโ
โ Data Layer โ Models, schemas, database
โ โ SQLAlchemy ORM with PostgreSQL
โโโโโโโโโโโโโโโโโโโ
๐ Detailed Directory Structure¶
fastapi-backend/
โโโ app/
โ โโโ main.py # FastAPI application entry point
โ โ
โ โโโ api/ # API layer - HTTP endpoints
โ โ โโโ deps.py # Dependency injection (auth, rate limiting)
โ โ โโโ deps_services.py # Service dependency injection
โ โ โโโ v1/
โ โ โโโ api.py # Main API router configuration
โ โ โโโ endpoints/ # Individual endpoint modules
โ โ โโโ recommendations.py # Travel recommendations API
โ โ โโโ auth.py # Authentication endpoints
โ โ โโโ users.py # User management
โ โ โโโ saved_trips.py # Trip saving/retrieval
โ โ
โ โโโ core/ # Core configuration and setup
โ โ โโโ config.py # Environment configuration (Pydantic)
โ โ โโโ db.py # Database connection setup
โ โ โโโ security.py # JWT token handling
โ โ โโโ enums.py # Application enumerations
โ โ
โ โโโ models/ # Database models (SQLAlchemy)
โ โ โโโ user.py # User account model
โ โ โโโ saved_trip.py # Saved trip model with JSONB data
โ โ โโโ travel.py # Travel-related models
โ โ โโโ associations.py # Many-to-many relationships
โ โ
โ โโโ schemas/ # API request/response schemas (Pydantic)
โ โ โโโ user.py # User registration/profile schemas
โ โ โโโ token.py # Authentication token schemas
โ โ โโโ destination.py # Destination and recommendation schemas
โ โ โโโ saved_trip.py # Trip saving schemas
โ โ
โ โโโ services/ # Business logic layer
โ โ โโโ travel_services.py # Main travel orchestration service
โ โ โโโ recommendations.py # Legacy recommendation service
โ โ โ
โ โ โโโ ai/ # AI service abstraction layer
โ โ โ โโโ service.py # Main AI service with provider switching
โ โ โ โโโ base.py # Base AI provider interface
โ โ โ โโโ providers/ # Individual AI provider implementations
โ โ โ โโโ claude.py # Anthropic Claude integration
โ โ โ โโโ gemini.py # Google Gemini integration
โ โ โ
โ โ โโโ cache/ # Caching infrastructure
โ โ โ โโโ cache_factory.py # Cache abstraction with multiple backends
โ โ โ โโโ postgres_cache.py # PostgreSQL-based cache implementation
โ โ โ โโโ file_cache.py # File-based cache for development
โ โ โ โโโ redis_manager.py # Redis cache (legacy, optional)
โ โ โ โโโ decorators.py # @cached decorator for functions
โ โ โ โโโ interface.py # Cache interface definition
โ โ โ โโโ key_generator.py # Consistent cache key generation
โ โ โ
โ โ โโโ external/ # External API integrations
โ โ โ โโโ google_maps.py # Google Maps/Places API
โ โ โ โโโ amadeus.py # Flight data (Amadeus API)
โ โ โ โโโ weather.py # Weather API integration
โ โ โ โโโ exchange_rate_service.py # Currency conversion
โ โ โ
โ โ โโโ image_service.py # Destination image fetching
โ โ โโโ weather_service.py # Weather data service
โ โ โโโ exceptions.py # Custom exception classes
โ โ
โ โโโ utils/ # Utility functions
โ โโโ logging.py # Structured logging setup
โ โโโ middleware.py # Custom FastAPI middleware
โ โโโ exceptions.py # Global exception handlers
โ
โโโ migrations/ # Alembic database migrations
โ โโโ versions/ # Individual migration files
โ โโโ env.py # Alembic environment configuration
โ โโโ script.py.mako # Migration template
โ
โโโ tests/ # Test suite
โ โโโ api/ # API endpoint tests
โ โโโ services/ # Service layer tests
โ โโโ integration/ # Integration tests
โ โโโ conftest.py # Test configuration and fixtures
โ
โโโ scripts/ # Utility scripts
โ โโโ setup_db.py # Database initialization
โ โโโ populate_airports.py # Seed airport data
โ
โโโ docker-compose.yml # Local development environment
โโโ Dockerfile # Container configuration
โโโ pyproject.toml # Poetry dependencies and config
โโโ requirements.txt # Pip dependencies
โโโ alembic.ini # Alembic configuration
๐ง Key Components Explained¶
1. AI Service Architecture¶
The AI service uses a provider abstraction pattern with automatic failover:
# app/services/ai/service.py
class AIService:
def __init__(self):
self.primary_provider = ClaudeProvider() # Primary: Anthropic Claude
self.fallback_provider = GeminiProvider() # Fallback: Google Gemini
async def get_recommendations(self, **params):
try:
return await self.primary_provider.get_recommendations(**params)
except Exception:
# Automatic failover to Gemini if Claude fails
return await self.fallback_provider.get_recommendations(**params)
Why This Design? - Reliability: If one AI service is down, the other takes over - Cost Optimization: Can switch providers based on pricing - A/B Testing: Can compare response quality between providers
2. Travel Service Orchestration¶
The TravelService is the main orchestrator that coordinates all data sources:
# app/services/travel_services.py
class TravelService:
async def get_initial_recommendations(self, budget, duration, departure_city, **params):
# 1. Calculate traveler-specific budget
total_travelers = num_adults + num_children
per_person_budget = budget / total_travelers
# 2. Get AI recommendations with traveler count
raw_recs = await self.ai_service.get_recommendations(
budget=budget,
travelers=total_travelers, # Critical for proper budget calculation!
**params
)
# 3. Filter recommendations within budget
filtered_recs = self._filter_by_budget(raw_recs, budget, total_travelers)
# 4. Enrich with images and external data
await self._fetch_and_attach_images(filtered_recs)
return filtered_recs
Critical Budget Logic: - The AI needs to know the total number of travelers to calculate correct per-person costs - Backend validates that total cost for all travelers doesn't exceed the total budget - This was the bug we fixed earlier - the travelers parameter wasn't being passed to the AI!
3. Caching Strategy¶
The application uses intelligent caching with multiple backend options for performance and cost optimization:
# PostgreSQL caching decorator (default)
@cached(prefix="travel_recs", ttl=timedelta(hours=1), key_params=["budget", "departure_city"])
async def get_travel_recommendations(self, budget, departure_city, **params):
# Expensive AI call only happens if not cached
return await self.ai_service.get_recommendations(**params)
Cache Backend Options: - PostgreSQL Cache (Default): Cost-effective, persistent, uses existing database - File Cache: For development and testing - Redis Cache: Optional, for high-performance scenarios
Cache Hierarchy: 1. AI Responses: 1 hour TTL (most expensive) 2. External API Data: 30 minutes TTL (weather, flights) 3. Images: 24 hours TTL (rarely change) 4. Rate Limit Counters: 24 hours TTL (reset daily)
4. Database Design¶
Key Models and Relationships:
# User model with authentication
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True)
email = Column(String, unique=True, index=True)
username = Column(String, unique=True, index=True)
hashed_password = Column(String)
query_count = Column(Integer, default=0) # Track API usage
# Relationship to saved trips
saved_trips = relationship("SavedTrip", back_populates="user")
# Saved trip with flexible JSONB storage
class SavedTrip(Base):
__tablename__ = "saved_trips"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
user_id = Column(Integer, ForeignKey("users.id"))
name = Column(String)
# JSONB columns for flexible data storage
search_params = Column(JSONB) # Original search criteria
results = Column(JSONB) # Simple recommendations
enriched_results = Column(JSONB) # Full enriched data
user = relationship("User", back_populates="saved_trips")
Why JSONB? - Flexibility: Travel data has complex, nested structures - Performance: PostgreSQL JSONB is indexed and queryable - Evolution: Schema can evolve without migrations for data fields
5. Rate Limiting Implementation¶
# app/api/deps.py
def check_query_limit(request: Request, current_user: Optional[User] = None):
if current_user:
return # Authenticated users have unlimited access
client_id = get_client_identifier(request)
if client_id in ip_query_tracker:
if ip_query_tracker[client_id]["count"] >= settings.RATE_LIMIT_UNAUTHENTICATED:
raise HTTPException(status_code=429, detail="Rate limit exceeded")
Rate Limiting Rules: - Anonymous users: 3 requests per IP per day - Authenticated users: Unlimited - Cached responses: Don't count against limits - Failed requests: Don't count against limits
๐ Request Processing Flow¶
Example: Travel Recommendation Request¶
-
HTTP Request Arrives
-
FastAPI Router (
app/api/v1/endpoints/recommendations.py) -
Dependency Injection
- Rate limiting check for anonymous users
-
Service initialization with proper dependencies
-
Service Layer Processing (
app/services/travel_services.py)# Calculate travelers and budget total_travelers = 3 # 2 adults + 1 child per_person_budget = 1500 / 3 = $500 # Call AI service with correct parameters await self.ai_service.get_recommendations( budget=1500, travelers=3, # This was the missing piece! duration=8, # 7 nights = 8 days departure_city="Sydney, Australia", preferences=["beach", "culture"] ) -
AI Provider (
app/services/ai/providers/gemini.py) -
AI Response Processing
-
Data Enrichment
- Fetch destination images (Pexels/Unsplash)
- Add weather data for dates
-
Include exchange rates if international
-
Caching
- Store results in Redis for 1 hour
-
Cache key includes all search parameters
-
HTTP Response
๐ ๏ธ Development Setup¶
Local Development with Docker (Recommended)¶
# Start all services (PostgreSQL, FastAPI)
cd fastapi-backend
docker compose up
# The backend will be available at:
# - API: http://localhost:8000
# - Docs: http://localhost:8000/docs
# - Database: localhost:5432 (includes cache_entries table)
Running Commands in Docker¶
Since the backend runs in a containerized environment, all Python commands should be executed through Docker:
# Run database migrations
docker compose exec backend alembic upgrade head
# Initialize database with seed data
docker compose exec backend python scripts/setup_db.py
docker compose exec backend python scripts/populate_airports.py
# Run tests
docker compose exec backend pytest
# Access Python shell
docker compose exec backend python
# Check installed package versions
docker compose exec backend python -c "import fastapi; print(f'FastAPI: {fastapi.__version__}')"
# View logs
docker compose logs backend -f
Manual Development Setup¶
# Create virtual environment
python -m venv venv
source venv/bin/activate # Windows: .\venv\Scripts\activate
# Install dependencies
poetry install
# OR
pip install -r requirements.txt
# Set up environment variables
cp .env.example .env
# Edit .env with your API keys
# Run database migrations
alembic upgrade head
# Initialize database with seed data
python scripts/setup_db.py
python scripts/populate_airports.py
# Start development server
uvicorn app.main:app --reload
Environment Variables¶
Create a .env file with these required variables:
# Database
DATABASE_URL=postgresql://postgres:password@localhost:5432/traveldb
POSTGRES_USER=postgres
POSTGRES_PASSWORD=password
POSTGRES_DB=traveldb
# Cache Configuration
CACHE_TYPE=postgres # Options: postgres, file, redis
CACHE_ENABLED=true
# AI Services
AI_PROVIDER=gemini # or "claude"
ANTHROPIC_API_KEY=sk-ant-api03-...
GOOGLE_AI_API_KEY=AIzaSy...
GOOGLE_AI_MODEL=models/gemini-2.0-flash
# External APIs
GOOGLE_MAPS_API_KEY=AIzaSy...
WEATHER_API_KEY=b626a390f...
AMADEUS_API_KEY=3Bo4eVi40W...
AMADEUS_API_SECRET=Y48KIATb7n...
# JWT Authentication
JWT_SECRET=your-secret-key
ACCESS_TOKEN_EXPIRE_MINUTES=30
# Application
PROJECT_NAME=TravelSage API
API_V1_STR=/api/v1
APP_ENV=development
๐งช Testing¶
Running Tests¶
# Run all tests
pytest
# Run with coverage
pytest --cov=app tests/
# Run specific test file
pytest tests/api/test_recommendations.py
# Run specific test
pytest tests/api/test_recommendations.py::test_get_recommendations_success
Test Structure¶
tests/
โโโ conftest.py # Test configuration and fixtures
โโโ api/ # API endpoint tests
โ โโโ test_recommendations.py
โ โโโ test_auth.py
โ โโโ test_users.py
โโโ services/ # Service layer tests
โ โโโ test_ai_service.py
โ โโโ test_travel_service.py
โ โโโ test_cache.py
โโโ integration/ # End-to-end tests
โโโ test_user_journey.py
Key Test Patterns¶
# API endpoint test
def test_get_recommendations_success(client, sample_request):
response = client.post("/api/v1/travel/recommendations", json=sample_request)
assert response.status_code == 200
data = response.json()
assert len(data) > 0
assert "destination" in data[0]
# Service layer test with mocking
@pytest.mark.asyncio
async def test_ai_service_with_fallback(mock_claude_error, mock_gemini_success):
ai_service = AIService()
result = await ai_service.get_recommendations(budget=1000, travelers=2)
# Should use Gemini after Claude fails
assert result is not None
mock_gemini_success.assert_called_once()
๐ Database Migrations¶
Creating New Migrations¶
# Auto-generate migration from model changes
alembic revision --autogenerate -m "Add new field to user model"
# Create empty migration for custom SQL
alembic revision -m "Add custom index"
# Apply migrations
alembic upgrade head
# Rollback one migration
alembic downgrade -1
Migration Best Practices¶
- Always review auto-generated migrations before applying
- Test migrations on a copy of production data
- Use transactions for data migrations
- Add indexes concurrently for large tables
๐ Deployment¶
Production Environment Variables¶
# Production database (Cloud SQL)
DATABASE_URL=postgresql://user:pass@host:5432/prod_db
# Cache Configuration (PostgreSQL cache saves $110/month vs Redis)
CACHE_TYPE=postgres
# Production AI keys
ANTHROPIC_API_KEY=sk-ant-prod-...
GOOGLE_AI_API_KEY=AIzaSy-prod-...
# Security
JWT_SECRET=production-secret-key
APP_ENV=production
# External APIs (production keys)
GOOGLE_MAPS_API_KEY=AIzaSy-prod-maps-...
Cloud Run Deployment¶
The backend deploys to Google Cloud Run via GitHub Actions:
# .github/workflows/deploy.yml
- name: Deploy to Cloud Run
run: |
gcloud run deploy macumba-backend \
--image gcr.io/$PROJECT_ID/macumba-backend:$GITHUB_SHA \
--platform managed \
--region us-central1 \
--set-env-vars APP_ENV=production
๐ Monitoring and Debugging¶
Logging¶
The application uses structured logging:
# app/utils/logging.py
logger = get_logger("app.services.travel")
# Usage in services
logger.info("Processing recommendation request",
departure_city=departure_city,
budget=budget,
travelers=total_travelers)
logger.error("AI service failed",
provider="claude",
error=str(e),
request_id=request.headers.get("x-request-id"))
Health Checks¶
# Application health
curl http://localhost:8000/health
# AI service health
curl http://localhost:8000/api/v1/travel/rate-limit
# Database health (via logs)
docker-compose logs backend | grep "Database"
Common Debugging Commands¶
# Check logs
docker-compose logs backend
# Connect to database
docker-compose exec db psql -U postgres -d traveldb
# Check PostgreSQL cache
docker-compose exec backend python -c "
from app.services.cache.cache_factory import CacheFactory
import asyncio
cache = CacheFactory.create_cache()
keys = asyncio.run(cache.keys('travel_recs:*'))
print('Cache keys:', keys)
"
# Test AI service directly
docker-compose exec backend python -c "
from app.services.ai.service import AIService
import asyncio
ai = AIService()
result = asyncio.run(ai.get_recommendations(budget=1000, travelers=2, departure_city='Sydney'))
print(result)
"
๐ฏ Performance Optimization¶
Caching Strategy¶
- AI responses: 1 hour (most expensive)
- External APIs: 30 minutes
- Images: 24 hours
- Database queries: Use SQLAlchemy's built-in query cache
Database Optimization¶
- Indexes: All foreign keys and frequently queried fields
- JSONB: Use GIN indexes for JSONB columns
- Connection pooling: Configured in
core/db.py
API Optimization¶
- Async/await: All I/O operations are non-blocking
- Background tasks: Heavy operations run in background
- Response compression: Gzip enabled for large responses
๐ API Documentation¶
Interactive Documentation¶
- Swagger UI:
http://localhost:8000/docs - ReDoc:
http://localhost:8000/redoc
Key Endpoints¶
# Get travel recommendations
POST /api/v1/travel/recommendations
Content-Type: application/json
{
"budget": 1500,
"from_date": "2025-07-15",
"to_date": "2025-07-22",
"departure_city": "Sydney, Australia",
"preferences": ["beach", "culture"],
"num_adults": 2,
"num_children": 1,
"with_pets": false
}
# Enrich a recommendation with detailed data
GET /api/v1/travel/{recommendation_id}/enrich
# Save a recommendation as a trip (requires auth)
POST /api/v1/travel/{recommendation_id}/save
Authorization: Bearer <token>
{
"name": "My Gold Coast Trip",
"notes": "Family vacation",
"is_public": false
}
# User authentication
POST /api/v1/auth/login
POST /api/v1/auth/register
# Check rate limit status
GET /api/v1/travel/rate-limit
This backend documentation provides junior developers with a comprehensive understanding of how the FastAPI application works, from high-level architecture down to specific implementation details. The key is understanding the flow from HTTP request through the various service layers to AI providers and back to the user.