Tracking Thread Pool Implementation for Spring Boot/Scala

tracking-threadpool-implementation.md

Tracking Thread Pool Implementation

Context

This document describes the implementation of an isolated thread pool for HTTP tracking requests in a Spring Boot/Scala application. Tracking requests are non-critical, can take up to 30 seconds to timeout, and should not impact the main application's request handling.

Requirements

• Volume: Up to 1000 tracking requests/second at peak
• Timeout: 2 seconds for tracking requests (configured in RestClient)
• Failure handling: Log failures, no retry logic needed
• Rejection policy: Can drop requests during overload - tracking is best-effort only
• Isolation: Tracking requests must not block or impact main request threads

Solution Overview

We create a dedicated ExecutionContext backed by a bounded ThreadPoolExecutor with:

• Fixed maximum thread pool size (100 threads)
• Bounded queue (300 capacity)
• Custom rejection handler that logs dropped requests
• Named daemon threads for debugging

This configuration handles normal operation smoothly and gracefully degrades by dropping tracking requests when the tracking service is slow or unavailable.

Implementation

1. Create TrackingExecutionContext

Create a new Scala object that manages the thread pool:

import java.util.concurrent.{
  ThreadPoolExecutor,
  LinkedBlockingQueue,
  TimeUnit,
  ThreadFactory,
  RejectedExecutionHandler
}
import java.util.concurrent.atomic.AtomicInteger
import scala.concurrent.ExecutionContext
import org.slf4j.LoggerFactory

object TrackingExecutionContext {
  private val logger = LoggerFactory.getLogger(getClass)

  // Thread pool configuration
  private val corePoolSize = 50
  private val maxPoolSize = 100
  private val queueCapacity = 300
  private val keepAliveTime = 60L // seconds

  // Custom thread factory for better thread naming/debugging
  private val threadFactory: ThreadFactory = new ThreadFactory {
    private val counter = new AtomicInteger(0)
    override def newThread(r: Runnable): Thread = {
      val thread = new Thread(r, s"tracking-pool-${counter.incrementAndGet()}")
      thread.setDaemon(true) // Don't block JVM shutdown
      thread
    }
  }

  // Custom rejection handler that logs when requests are dropped
  private val rejectionHandler: RejectedExecutionHandler =
    (r: Runnable, executor: ThreadPoolExecutor) => {
      logger.warn(s"Tracking request rejected. Pool stats: " +
        s"active=${executor.getActiveCount}, " +
        s"poolSize=${executor.getPoolSize}, " +
        s"queueSize=${executor.getQueue.size()}, " +
        s"completedTasks=${executor.getCompletedTaskCount}")
      // TODO: Add metric counter here for monitoring (see Monitoring section)
    }

  private val threadPool = new ThreadPoolExecutor(
    corePoolSize,
    maxPoolSize,
    keepAliveTime,
    TimeUnit.SECONDS,
    new LinkedBlockingQueue[Runnable](queueCapacity),
    threadFactory,
    rejectionHandler
  )

  // Wrap in ExecutionContext for Scala Futures
  val trackingEC: ExecutionContext = ExecutionContext.fromExecutor(threadPool)

  // Optional: method to get pool stats for health checks
  def getPoolStats: Map[String, Any] = Map(
    "activeCount" -> threadPool.getActiveCount,
    "poolSize" -> threadPool.getPoolSize,
    "corePoolSize" -> threadPool.getCorePoolSize,
    "maxPoolSize" -> threadPool.getMaximumPoolSize,
    "queueSize" -> threadPool.getQueue.size(),
    "queueRemainingCapacity" -> threadPool.getQueue.remainingCapacity(),
    "completedTaskCount" -> threadPool.getCompletedTaskCount,
    "taskCount" -> threadPool.getTaskCount
  )
}

2. Update TrackingService

Modify your tracking service to use the isolated thread pool:

import scala.concurrent.Future
import TrackingExecutionContext.trackingEC

class TrackingService(restClient: RestClient) {
  private val logger = LoggerFactory.getLogger(getClass)

  def trackRequest(data: TrackingData): Future[Unit] = {
    Future {
      try {
        restClient.sendTracking(data) // This has 2s timeout configured
      } catch {
        case e: Exception =>
          logger.error(s"Tracking request failed: ${e.getMessage}", e)
          // TODO: Add metric counter for tracking failures
          throw e // Re-throw so Future fails (though caller ignores it)
      }
    }(trackingEC)
  }
}

3. Usage in Controllers

Use fire-and-forget pattern in your request handlers:

class OrderController(trackingService: TrackingService) {

  def handleOrder(order: Order): Response = {
    // Process order (main business logic)
    val result = processOrder(order)

    // Fire tracking request and forget (don't wait)
    trackingService.trackRequest(TrackingData(order.id, result.status))

    // Return response immediately
    Response.ok(result)
  }
}

Configuration Rationale

Parameter	Value	Reasoning
Core pool size	50	Handles normal operation when tracking service responds quickly (~100ms avg)
Max pool size	100	Handles degraded operation when some requests are slow
Queue capacity	300	Buffers ~300ms of requests at peak (1000/sec), allows short spikes
Keep-alive time	60s	Scales down threads during low traffic
Rejection policy	Custom (log + drop)	Protects main app by dropping tracking requests when pool saturated
Timeout	2s	Configured in RestClient - aggressive for non-critical tracking

Load Scenarios

Normal operation (tracking service healthy, ~100ms response time):

• Uses ~10-20 threads
• Queue mostly empty
• No rejections

Degraded operation (tracking service slow, ~500ms-1s response time):

• Uses 50-100 threads
• Queue starts filling
• Occasional rejections during spikes

Outage (tracking service down, 2s timeouts):

• All 100 threads busy
• Queue fills to capacity
• Aggressive rejection of new requests
• Main app continues normally

Monitoring & Operations

Metrics to Track

Add the following metrics (using Micrometer or your metrics system):

1. tracking.requests.rejected (counter) - Requests dropped due to pool saturation
2. tracking.requests.failed (counter) - Requests that failed with exceptions
3. tracking.requests.success (counter) - Successful tracking calls
4. tracking.pool.active (gauge) - Active threads in pool
5. tracking.pool.queue_size (gauge) - Current queue size

Health Check Endpoint

Optional: Add a health check that exposes pool stats:

@GetMapping("/actuator/health/tracking-pool")
def trackingPoolHealth(): Map[String, Any] = {
  TrackingExecutionContext.getPoolStats
}

Alerts

Consider setting up alerts for:

• High rejection rate (> 10% of requests)
• Sustained high queue size (> 250 for > 5 minutes)
• Thread pool saturation (active threads = 100 for > 5 minutes)

Thread Dump Analysis

In thread dumps, tracking threads will appear as:

"tracking-pool-1" daemon
"tracking-pool-2" daemon
...

This makes it easy to identify if tracking requests are causing issues.

Testing

Unit Test Example

import org.scalatest.concurrent.ScalaFutures
import org.scalatest.time.{Seconds, Span}

class TrackingServiceTest extends AnyFunSuite with ScalaFutures {

  implicit val patience: PatienceConfig = PatienceConfig(Span(3, Seconds))

  test("trackRequest executes on tracking thread pool") {
    val mockClient = mock[RestClient]
    val service = new TrackingService(mockClient)

    val future = service.trackRequest(TrackingData("test"))

    // Future completes without blocking test thread
    whenReady(future) { _ =>
      verify(mockClient).sendTracking(any())
    }
  }

  test("trackRequest handles failures gracefully") {
    val mockClient = mock[RestClient]
    when(mockClient.sendTracking(any())).thenThrow(new RuntimeException("Service down"))
    val service = new TrackingService(mockClient)

    val future = service.trackRequest(TrackingData("test"))

    // Future fails but doesn't throw in caller
    whenReady(future.failed) { ex =>
      assert(ex.getMessage == "Service down")
    }
  }
}

Load Testing

To verify the thread pool behaves correctly under load:

1. Start application
2. Generate high request volume (1000+/sec)
3. Introduce latency/failures in tracking service
4. Verify:
   • Main request handling remains fast
   • Tracking pool metrics show rejections
   • No thread pool exhaustion in main app

Tuning

If you observe issues, consider adjusting:

• Too many rejections during normal operation: Increase maxPoolSize or queueCapacity
• Memory pressure: Decrease queueCapacity
• Slow scale-down: Decrease keepAliveTime
• Not enough isolation: Verify RestClient timeout is actually 2s and working

Dependencies

Ensure you have:

• Scala standard library
• SLF4J for logging
• Your RestClient configured with 2s timeout for tracking endpoints

Notes

• The thread pool is not a Spring-managed bean - it's a Scala object. If you need Spring integration, wrap it in a @Configuration class.
• Threads are daemon threads and won't prevent JVM shutdown
• The Future pattern allows flexibility - callers can wait/handle if needed, but typical usage is fire-and-forget
• Consider adding JMX exposure of pool stats for runtime monitoring