.png)
Tech we used to set-up the Hatch Exchange
Team update: A lot has happened since this blog - all great things! We’re now in the United States and our new product launched in November 2021, helping teams in fast-growing organizations find and hire their best-fit junior and mid-level talent in Sales, Marketing, Operations, and Customer Success. Try it here for free. This means some of our articles before this date may have product shots that look a little different. That’s all from us, enjoy the blog.
The team at Hatch spun up the Labour Exchange in a few days re-purposing our tech to help stood down workers find employment during the Covid-19 crisis.
In order to get the system up and running in such a short time frame we decided to use S3 as a flat-file data store to maintain our serverless batch job states and caches. After a very cursory search to satisfy ourselves S3 would guarantee read-after-write consistency, we flew on.
Amazon S3 provides read-after-write consistency for PUTS of new objects in your S3 bucket in all Regions with one caveat. The caveat is that if you make a HEAD or GET request to a key name before the object is created, then create the object shortly after that, a subsequent GET might not return the object due to eventual consistency.
Unfortunately, we missed the bolded section and the part further down the page that explicitly states our use case:
A process replaces an existing object and immediately tries to read it. Until the change is fully propagated, Amazon S3 might return the previous data.
As these processes run on a schedule (not as part of a user facing API) we could afford to spend some extra calls to S3 to roll our own read-after-update consistency. Knowing that S3 guarantees read-after-firstWrite, we can write a new file for every change, read the latest file and make sure we cleanup.
So every time we write a file we:
- Append a timestamp to the filename
- Remove older files
When we read a file we:
- List all files with the key prefix (S3 guarantees listing files will be ordered by ascending UTF-8 binary order)
- Get the newest file in the list
