How to Design a Reliable Background Job System with Celery and Redis
Retries, idempotency, acknowledgements, timeouts and queue layout. How to make Celery and Redis jobs behave when workers die and APIs time out.
Notes on Python, Django, data pipelines and software architecture.
Retries, idempotency, acknowledgements, timeouts and queue layout. How to make Celery and Redis jobs behave when workers die and APIs time out.
What to get right on day one, what to leave alone, and the boring shape I reach for when a founder has a deadline and a shrinking runway.