A NATS consumer that acknowledged too early
The acknowledgement went out before the work had actually finished.
- status
- resolved
- duration
- 1h 45m
- impact
- 14 jobs requeued
- systems
- nats jetstream · go
summary
A worker acknowledged the message immediately upon receiving it rather than upon finishing the GPU workload. When the container crashed during model warm-up, the message was never retried.