A few days ago, I was trying to download a six-month bank statement using a mobile banking app. The app kept loading for a very long time, and in the end it only showed a message saying that the statement would be sent by email. Nothing ever arrived.
What I noticed was this: when I requested a one-month statement, it worked. I was able to download it from the app, and I also received the email. But for six months, there was no result at all.
As a backend engineer, I started thinking about what the current service design might look like and where the problem could be.
The current design problem
Based on the way the app behaved, I imagined the current statement service working in a simple request-response pattern.
The app sends a request, then waits until the server finishes generating the file. That means the connection stays open until the whole operation is done. This is a common pattern in many APIs, but it is not a good fit for operations that may take a long time.
The problem with this design is that it can easily lead to server overload if many users request the same API at the same time.
Another possible issue is that the backend might be pulling all the data in one bulk query from the database. That affects performance, especially when the dataset is large.
And finally, I think the main issue may be in the email step itself. Most email providers accept attachments up to about 25 MB. If the one-month statement was already more than 7 MB, then a six-month statement could easily exceed the allowed attachment size by a large margin.
A better design
This design can be improved with a few simple changes.
First, instead of using a request-response pattern, we can use a fire-and-forget pattern.
The idea is simple: the app sends the request, the server accepts it, and immediately responds that the request is under processing. The server does not keep the user waiting for the full file generation to finish.
Second, once the server receives the request, it pushes it into a queue. This allows the system to process requests one by one, or in a controlled way, without consuming too many server resources at the same time.
Third, instead of pulling all data at once, the system can process the statement data in batches and append each batch to the generated file. This helps avoid unnecessary memory usage.
Finally, instead of sending the statement as an email attachment, the system can send an email that contains only a download link. This avoids the attachment-size problem completely.
For better security, that link can be valid for a limited time or for one download only.
And of course, the mobile app can receive a push notification telling the user that the statement is ready, together with the download link.
One important note
When we design any system, it is almost impossible to get the perfect design from the first attempt.
What we really need is to provide a reasonable level of safety and performance, then keep improving the design whenever we notice a weakness.
We also need to remember that the more complex solution is not always the better one. Sometimes this level of complexity is not needed at all, especially if the service is rarely used or has a small number of users.
So the real question is:
Have you ever gone through a user experience that felt like it could be much better if the backend design was improved?