On July 18th at approximately 15:42 BST, our alerting systems flagged an unusual spike in API errors concerning the audit PDF generation feature of our product. These errors impacted multiple customers across all environments.
An investigation revealed that the increase in errors was due to authentication issues within the internal systems responsible for PDF generation. This was traced back to a regression in the library used to automate the process. We found that the library version was not properly pinned, which allowed independent deployments of our internal systems to potentially alter the library version.
The engineering team has locked the version of the specified library and resolved the authentication issues. These measures will prevent any future regressions caused by unwanted changes to the library used for generating Audit PDFs.