Postmortem: Release 98
Incident window: August 19–25, 2025
Release 98 was a major milestone for our company, as it delivered upon deploying a far more flexible and scalable infrastructure that will help us avoid downtime, rollout issues, scalability and more. Although the deployment was well-planned and was completed as designed, several issues arose afterward that affected customers in different ways.
We regret that this release resulted in system issues, latency, intermittency and outages in some cases. Our team wants to assure you that this release is an investment that is designed to reduce the risk of such results occurring again in future releases.
The summary below outlines what occurred and how we responded.
August 20 – Scaling Challenge in New Infrastructure
When we moved our form engines into new infrastructure we encountered memory limitations that caused instability. This was resolved later that same day and we have taken steps to refine how our application scales going forward.
August 20 – Temporary Print Function Routing
Some print requests were not correctly routed after the release, which caused disruptions for a subset of customers. We have corrected this with an interim solution to ensure print requests are fulfilled reliably. An optimal solution will be included in an upcoming update.
August 21 – Form Generation Error
Certain forms failed to generate due to our system requiring null values in specific cases. We corrected the code to handle missing null values.
August 21 – Compatibility Issue with New Feature Parameters
A change in one of our APIs caused a higher payload than was expected for some customers. We resolved the issue to prevent issues with the high payload.
August 21 – Data Processing Issue
An upstream error code cause an issue with a data processing. The issue was quickly identified and resolved.
August 22 – Form Group Instances and Duplication Bug
An internal issue caused problems with data filling on forms and created errors with Saving. This was related to a navigation issue that resolved the problem.
August 25 – Authentication ID Requirement
A new feature introduced an unexpected requirement for an additional field in form requests, which caused errors in certain cases. We disabled the requirement and will reintroduce it in the future only when customers are fully prepared to support it.
Moving Forward
The scope of this release was unprecedented, both in size and complexity. While each issue was resolved relatively quickly, we recognize the impact this had on our customers. To prevent similar situations, we are:
We deeply appreciate your patience and partnership as we evolve our platform. Our team remains committed to transparency and continuous improvement to ensure the reliability you expect from us.