Scaling Codes Logo
Scaling Codes

Migrating from Monolith to Microservices

A comprehensive guide to breaking down monolithic applications into scalable, maintainable microservices.

25 min read
Maria Rodriguez
4.2k views
2024-01-25
migration
microservices
architecture

Problem Statement

Transform a monolithic e-commerce application into a scalable microservices architecture while maintaining business continuity and improving system performance.

Context: Large e-commerce platform with 500k+ users, experiencing deployment bottlenecks, scaling issues, and difficulty in implementing new features.

Current Monolithic Architecture

Understanding the existing system before migration

Current monolithic structure with tightly coupled components

Monolith Characteristics

Challenges
  • • Single deployment unit
  • • Technology lock-in
  • • Scaling limitations
  • • Long build times
  • • Difficult testing
Benefits
  • • Simple deployment
  • • Shared data model
  • • Easy debugging
  • • Consistent technology
  • • Lower operational overhead

Target Microservices Architecture

The desired end state with clear service boundaries

Target microservices architecture with API gateway and service mesh

Service Decomposition Strategy

Domain-Driven Design
  • • User Management Domain
  • • Order Management Domain
  • • Inventory Management Domain
  • • Payment Processing Domain
  • • Analytics & Reporting Domain
Technical Considerations
  • • Database per service
  • • Event-driven communication
  • • API versioning strategy
  • • Service discovery
  • • Circuit breaker patterns

Migration Strategy & Phases

Step-by-step approach using the Strangler Fig pattern

Four-phase migration strategy with gradual service extraction

Phase 1: Preparation (Weeks 1-4)

// Domain Analysis Example
const domainBoundaries = {
  userManagement: {
    entities: ['User', 'Profile', 'Preferences', 'Authentication'],
    boundedContext: 'Identity & Access Management',
    dataConsistency: 'Strong',
    team: 'Platform Team'
  },
  orderManagement: {
    entities: ['Order', 'OrderItem', 'OrderStatus', 'Fulfillment'],
    boundedContext: 'Order Processing',
    dataConsistency: 'Eventual',
    team: 'Order Team'
  },
  inventoryManagement: {
    entities: ['Product', 'Stock', 'Reservation', 'Supplier'],
    boundedContext: 'Inventory Control',
    dataConsistency: 'Strong',
    team: 'Inventory Team'
  }
};

Phase 2: Strangler Fig Implementation (Weeks 5-16)

// Strangler Fig Pattern Implementation
class StranglerFigRouter {
  constructor() {
    this.microservices = new Map();
    this.monolithFallback = true;
  }
  
  async routeRequest(request) {
    const service = this.identifyService(request.path);
    
    if (this.microservices.has(service)) {
      try {
        return await this.callMicroservice(service, request);
      } catch (error) {
        if (this.monolithFallback) {
          return await this.callMonolith(request);
        }
        throw error;
      }
    }
    
    return await this.callMonolith(request);
  }
  
  identifyService(path) {
    if (path.startsWith('/api/users')) return 'user-service';
    if (path.startsWith('/api/orders')) return 'order-service';
    if (path.startsWith('/api/inventory')) return 'inventory-service';
    return 'monolith';
  }
}

Phase 3: Data Migration (Weeks 17-24)

// Database Migration Strategy
class DatabaseMigration {
  async migrateUserData() {
    // Step 1: Create new user service database
    await this.createUserServiceDB();
    
    // Step 2: Set up data synchronization
    await this.setupDataSync({
      source: 'monolith_users',
      target: 'user_service_users',
      strategy: 'dual-write',
      validation: true
    });
    
    // Step 3: Gradually shift traffic
    await this.graduallyShiftTraffic({
      startPercentage: 0,
      endPercentage: 100,
      duration: '2 weeks',
      rollbackThreshold: 0.05
    });
    
    // Step 4: Verify data consistency
    await this.verifyDataConsistency();
    
    // Step 5: Remove old data
    await this.removeOldData();
  }
}

Service Communication Patterns

How services communicate and maintain data consistency

Service interaction flow with API gateway routing

Communication Strategies

Synchronous Communication
  • • REST APIs for CRUD operations
  • • gRPC for high-performance calls
  • • GraphQL for flexible data queries
  • • Circuit breaker for resilience
Asynchronous Communication
  • • Event-driven architecture
  • • Message queues (RabbitMQ, Kafka)
  • • Event sourcing for audit trails
  • • Saga pattern for distributed transactions
// Event-Driven Communication Example
class OrderService {
  async createOrder(orderData) {
    const order = await this.orderRepository.create(orderData);
    
    // Publish domain events
    await this.eventBus.publish('OrderCreated', {
      orderId: order.id,
      userId: order.userId,
      totalAmount: order.totalAmount,
      timestamp: new Date()
    });
    
    // Publish integration events
    await this.eventBus.publish('InventoryReservationRequired', {
      orderId: order.id,
      items: order.items,
      timestamp: new Date()
    });
    
    return order;
  }
}

// Event Handler in Inventory Service
class InventoryEventHandler {
  async handleInventoryReservationRequired(event) {
    const { orderId, items } = event;
    
    for (const item of items) {
      await this.reserveInventory(item.productId, item.quantity, orderId);
    }
    
    // Publish confirmation event
    await this.eventBus.publish('InventoryReserved', {
      orderId,
      items,
      timestamp: new Date()
    });
  }
}

Implementation Guide

Technical implementation details with code examples

1. API Gateway Configuration

// Kong API Gateway Configuration
{
  "name": "ecommerce-gateway",
  "upstream": {
    "name": "monolith",
    "targets": [
      { "target": "monolith:3000", "weight": 100 }
    ]
  },
  "routes": [
    {
      "name": "user-service",
      "paths": ["/api/users"],
      "upstream": "user-service:3001",
      "plugins": {
        "rate-limiting": {
          "minute": 1000,
          "hour": 10000
        },
        "cors": {
          "origins": ["*"],
          "methods": ["GET", "POST", "PUT", "DELETE"]
        }
      }
    },
    {
      "name": "order-service",
      "paths": ["/api/orders"],
      "upstream": "order-service:3002",
      "plugins": {
        "jwt": {
          "secret": "your-secret-key"
        }
      }
    }
  ]
}

2. Service Discovery & Health Checks

// Consul Service Registration
const consul = require('consul')();

class ServiceRegistry {
  async registerService(serviceConfig) {
    const registration = {
      name: serviceConfig.name,
      id: `${serviceConfig.name}-${serviceConfig.instanceId}`,
      address: serviceConfig.host,
      port: serviceConfig.port,
      tags: serviceConfig.tags || [],
      check: {
        http: `http://${serviceConfig.host}:${serviceConfig.port}/health`,
        interval: '10s',
        timeout: '5s',
        deregistercriticalserviceafter: '1m'
      }
    };
    
    await consul.agent.service.register(registration);
  }
  
  async discoverService(serviceName) {
    const services = await consul.catalog.service.nodes(serviceName);
    return services.map(service => ({
      host: service.ServiceAddress,
      port: service.ServicePort
    }));
  }
}

// Health Check Endpoint
app.get('/health', (req, res) => {
  const health = {
    status: 'healthy',
    timestamp: new Date().toISOString(),
    uptime: process.uptime(),
    memory: process.memoryUsage(),
    checks: {
      database: checkDatabaseConnection(),
      redis: checkRedisConnection(),
      externalAPI: checkExternalAPI()
    }
  };
  
  const isHealthy = Object.values(health.checks).every(check => check.status === 'healthy');
  res.status(isHealthy ? 200 : 503).json(health);
});

3. Circuit Breaker Implementation

// Circuit Breaker Pattern
class CircuitBreaker {
  constructor(failureThreshold = 5, timeout = 60000) {
    this.failureThreshold = failureThreshold;
    this.timeout = timeout;
    this.state = 'CLOSED';
    this.failureCount = 0;
    this.lastFailureTime = null;
  }
  
  async execute(operation) {
    if (this.state === 'OPEN') {
      if (this.shouldAttemptReset()) {
        this.state = 'HALF_OPEN';
      } else {
        throw new Error('Circuit breaker is OPEN');
      }
    }
    
    try {
      const result = await operation();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }
  
  onSuccess() {
    this.failureCount = 0;
    this.state = 'CLOSED';
  }
  
  onFailure() {
    this.failureCount++;
    this.lastFailureTime = Date.now();
    
    if (this.failureCount >= this.failureThreshold) {
      this.state = 'OPEN';
    }
  }
  
  shouldAttemptReset() {
    return Date.now() - this.lastFailureTime > this.timeout;
  }
}

// Usage Example
const circuitBreaker = new CircuitBreaker();
const userService = new UserService();

app.get('/api/users/:id', async (req, res) => {
  try {
    const user = await circuitBreaker.execute(() => 
      userService.getUser(req.params.id)
    );
    res.json(user);
  } catch (error) {
    res.status(503).json({ 
      error: 'Service temporarily unavailable',
      fallback: await getCachedUser(req.params.id)
    });
  }
});

Testing Strategy

Comprehensive testing approach for microservices

Testing Pyramid

  • Unit Tests (70%) - Service logic, utilities
  • Integration Tests (20%) - Service boundaries, databases
  • End-to-End Tests (10%) - User workflows, API contracts

Testing Tools

  • • Jest for unit testing
  • • Supertest for API testing
  • • Testcontainers for integration tests
  • • Pact for contract testing
  • • Cypress for E2E testing

Contract Testing Example

// Pact Contract Testing
const { Pact } = require('@pact-foundation/pact');
const { UserService } = require('./user-service');

describe('User Service Contract', () => {
  let provider;
  
  beforeAll(async () => {
    provider = new Pact({
      consumer: 'order-service',
      provider: 'user-service',
      port: 1234,
      log: path.resolve(process.cwd(), 'logs', 'pact.log'),
      dir: path.resolve(process.cwd(), 'pacts'),
      logLevel: 'INFO',
      spec: 2
    });
    
    await provider.setup();
  });
  
  afterAll(async () => {
    await provider.finalize();
  });
  
  describe('get user by id', () => {
    beforeAll(async () => {
      await provider.addInteraction({
        state: 'user exists',
        uponReceiving: 'a request for user details',
        withRequest: {
          method: 'GET',
          path: '/api/users/123',
          headers: { 'Accept': 'application/json' }
        },
        willRespondWith: {
          status: 200,
          headers: { 'Content-Type': 'application/json' },
          body: {
            id: '123',
            name: 'John Doe',
            email: 'john@example.com',
            status: 'active'
          }
        }
      });
    });
    
    it('should return user details', async () => {
      const userService = new UserService('http://localhost:1234');
      const user = await userService.getUser('123');
      
      expect(user).toEqual({
        id: '123',
        name: 'John Doe',
        email: 'john@example.com',
        status: 'active'
      });
    });
  });
});

Monitoring & Observability

Comprehensive monitoring strategy for microservices

Metrics
  • • Response times
  • • Throughput
  • • Error rates
  • • Resource usage
Logging
  • • Structured logs
  • • Correlation IDs
  • • Log aggregation
  • • Log retention
Tracing
  • • Distributed tracing
  • • Request flows
  • • Performance analysis
  • • Dependency mapping

Monitoring Implementation

// Prometheus Metrics
const prometheus = require('prom-client');
const register = new prometheus.Registry();

// Custom metrics
const httpRequestDuration = new prometheus.Histogram({
  name: 'http_request_duration_seconds',
  help: 'Duration of HTTP requests in seconds',
  labelNames: ['method', 'route', 'status_code'],
  buckets: [0.1, 0.5, 1, 2, 5]
});

const httpRequestsTotal = new prometheus.Counter({
  name: 'http_requests_total',
  help: 'Total number of HTTP requests',
  labelNames: ['method', 'route', 'status_code']
});

// Middleware to collect metrics
app.use((req, res, next) => {
  const start = Date.now();
  
  res.on('finish', () => {
    const duration = (Date.now() - start) / 1000;
    
    httpRequestDuration
      .labels(req.method, req.route?.path || req.path, res.statusCode)
      .observe(duration);
    
    httpRequestsTotal
      .labels(req.method, req.route?.path || req.path, res.statusCode)
      .inc();
  });
  
  next();
});

// Metrics endpoint
app.get('/metrics', async (req, res) => {
  res.set('Content-Type', register.contentType);
  res.end(await register.metrics());
});

Results & Success Metrics

Expected outcomes and measurable improvements

75%
Faster Deployments
3x
Development Velocity
99.9%
System Uptime

Before vs After Comparison

Before (Monolith)
  • • 45-minute build times
  • • 2-hour deployments
  • • 4-week feature cycles
  • • Single point of failure
  • • Technology lock-in
After (Microservices)
  • • 5-minute build times
  • • 15-minute deployments
  • • 1-week feature cycles
  • • Fault isolation
  • Technology flexibility