The problem
As production systems get more complex, reliable job scheduling is becoming more and more important. I built this project to improve my understanding of distributed systems and durable job scheduling.
Technical highlights
- I selected PostgreSQL to be the datasource for the implementation (although a different database can be brought in).
- Instead of advisory locks, I utilized a simple state machine that switches job state and prevents other worker nodes from accessing the job.
- The system is built in Rust for maximum performance.
- In order to define jobs, users create Rust binaries with the
workerlibrary and deploy them manually.