Phase 23 of 30 · Topic 23.5

Channel Graceful Shutdown & Poison Message Handling

1Concept

Production channels must handle cancellation tokens, exception boundaries, and dead-letter channels for poison messages to prevent silent worker crashes.

2Architecture Diagram

Incoming Item ──> Parse / Process
                        │ (Exception thrown!)
                        ▼
           [ Dead-Letter Channel ] ──> Alerts DevOps / Writes to Audit DB (Worker survives!)

3Code Example

C# 13 & .NET 9
using System;
using System.Threading.Channels;
using System.Threading.Tasks;

public class PoisonMessageDemo
{
    public static async Task Main()
    {
        var mainChannel = Channel.CreateUnbounded<string>();
        var deadLetterChannel = Channel.CreateUnbounded<string>();

        _ = Task.Run(async () =>
        {
            await mainChannel.Writer.WriteAsync("VALID_PAYLOAD");
            await mainChannel.Writer.WriteAsync("CORRUPT_PAYLOAD");
            mainChannel.Writer.Complete();
        });

        await foreach (var item in mainChannel.Reader.ReadAllAsync())
        {
            if (item.StartsWith("CORRUPT"))
            {
                await deadLetterChannel.Writer.WriteAsync(item);
                Console.WriteLine($"[Quarantine] Routed '{item}' to Dead-Letter Channel.");
            }
            else
            {
                Console.WriteLine($"[Success] Processed: {item}");
            }
        }
    }
}

4Expected Output

[Success] Processed: VALID_PAYLOAD
[Quarantine] Routed 'CORRUPT_PAYLOAD' to Dead-Letter Channel.

5Key Takeaways

  • ✓Never let an unhandled exception crash a channel consumer loop.
  • ✓Route unprocessable messages to dead-letter channels.
  • ✓Monitor `channel.Reader.Count` in Prometheus/OpenTelemetry metrics.