File type detection might be incorrect for certain files
Incident Report for Passfort
Postmortem

Summary

On February 16th at 10:36 GMT, we received a high-priority alert from a customer regarding an issue with the PassFort portal. Specifically, some files were not displaying correctly when opened, with certain PDFs appearing as text. Following a brief investigation, we determined that the root cause of the problem was an incorrect file classification, which was related to a recent update to our file service. We promptly reverted the update, and by 16:19 GMT, the issue was resolved. However, we still needed to remediate all files created during the period between the release of the update and its reversal.

Analysis

The root cause of the issue has been identified as an update to the file type detection strategy, which introduced a flawed sequence of detection steps. Specifically, the update prioritized classifying files as text/plain before checking for other types of files, leading to incorrect classifications.

Actions

The issue concerned files with a very specific structure while our automated and manual tests use generic files that did not cover this specific test case; for this reason automated tests and our QA processes were not able to detect the issue. As a result, we have:

  • Fixed the wrong sequencing of the detection steps
  • Expanded the subset of tested files to include examples similar to the ones affected
Posted Feb 26, 2024 - 17:00 GMT

Resolved
Remediation for EU has been completed.
Posted Feb 26, 2024 - 16:51 GMT
Update
Remediation for UAE has been completed. We expect to start remediation for the EU environment next week
Posted Feb 23, 2024 - 17:13 GMT
Update
We are still continuing working on starting the remediation process for the affected files.
Posted Feb 19, 2024 - 14:29 GMT
Identified
The root cause of the issue has been identified and we have released a fix. Any new file uploaded after the 16th of February should not be affected by the issue. Note that only files classified by PassFort as text/plain might be affected. We are going to start a remediation process for the affected files.
Posted Feb 19, 2024 - 14:29 GMT
Investigating
We are aware that files uploaded to the platform and attached to profiles might be incorrectly classified leading to errors when opening them in the portal. The issue only affects files uploaded between the 7th of February and the 16th of February
Posted Feb 19, 2024 - 14:27 GMT
This incident affected: PassFort Environments (🇪🇺 EU - identity.passfort.com, 🇦🇪 UAE - identity.uae.passfort.com, 🇺🇸 USA - identity.us.passfort.com).