A new batch of fixes have been deployed and the application is back to normal operations.
Posted Jan 15, 2020 - 15:37 CET
Update
We can confirm that the fixes we deployed friday has worked. We still see a few errors that we are trying to identify.
Posted Jan 14, 2020 - 09:30 CET
Update
The fixes we rolled out friday helped, however we are still working on identifying all of the issues.
Posted Jan 13, 2020 - 10:38 CET
Update
We are now rolling out updates for the already identified issues and will be monitoring these. We are also still investigating if there is any other issues that needs to be fixed.
Posted Jan 10, 2020 - 12:19 CET
Update
We are still working on identifying all the issues, however we are also testing fixes for the already identified issues. There will be an update later today if we are able to deploy them.
Posted Jan 10, 2020 - 09:52 CET
Update
We are still working hard on identifying why the errors har happening, we have enabled extra logging and created a tactical team to focus on the issue. We have already identified some of the underlying issue and working hard on resolving them. More info will come when we have it.
Posted Jan 09, 2020 - 15:24 CET
Update
We can see that adding more capacity haven't helped with the error rates, we are still investigating and will update when we know more.
Posted Jan 08, 2020 - 14:11 CET
Investigating
The 7th of January at 12:20 we started experiencing elevated error rates and server crashes. We have therefore tripled our SOAP capacity, which should keep the error rates in check while we figure out why this is happening.