Kinesis Firehose writes concatenated JSON objects to S3. Most python solutions to reading the data rely on pre-processing with a lambda, or struggle to deal with large files. A better solution is to use a streaming parser. Luckily, I have just the thing.

I've recently started a new role as an engineering coach, and I've been working through some katas to remember how to write code.

I'm a huge fan of Emily Bache's work, and was particularly interested in the Delivery Service refactoring kata because it neatly captures some of the challenges of working with legacy codebases.

I'm going to walk through the process I followed when solving the kata, and I'll include links to each commit in the Github repository so you can see the full code as we go.