Performance testing a financial application before a major client goes live checks two things: how fast the system answers under load, and whether the money still adds up. Covalience ran this program for a fund-management platform preparing to onboard a larger client. We found the performance issues, and the development team fixed them before launch.
Why is performance testing different for financial applications?
Performance testing a financial application is harder than testing a content site because most of the important user journeys write money, and money has rules. A fund cannot pay out more than it received. A transfer out of one fund needs a matching transfer into another. A month that has been reconciled is closed for good.
That changes what a load test has to prove. A fast response that leaves a fund with a negative balance is a failed test, even if every request came back in under a second.
Regulators also expect this work. The EU's Digital Operational Resilience Act (DORA) names performance testing in Article 25 among the tests financial entities run on the systems behind their critical functions
What the mock data revealed
Mock data, synthetic records generated to match the size and shape of production, changed the results more than any other factor in this program. We ran the same scenario twice, eight days apart: once before the mock data was loaded and once after. Both runs used the same journeys and the same mix of users and admins.
Before the data load, the error rate stayed under the 1% target up to 180 users, although response times already missed the 2-second API target. After the data load, the environment could not sustain the plan's 100-user starting tier, so we tested at 10, 20 and 30 users instead. At 30 users, 92% of requests failed.
What the testing found
Our job was to find the performance problems and give the development team enough evidence to fix each one. Across the two test cycles, the reports flagged these issues.
Errors and stability
- Error rates. Errors stayed under 1% up to 180 users before the mock data, then reached 92% at 30 users after it. They also arrived in bursts: at 200 users, 194 of 271 server errors landed inside one 45-second window.
- Container restarts. Datadog recorded 12 container restarts in the hour covering the first cycle and 13 in the 12 minutes covering the second.
Speed and capacity
- Slow endpoints. The query that builds a fund's holdings breakdown averaged 103 seconds at 10 users and hit the 120-second gateway timeout. The requests that load a user's funds after login took 73 to 108 seconds at the 95th percentile.
- Lagging mutations. Write operations slowed too. The page that adds a transaction to a fund took up to 36 seconds at the 95th percentile at 200 users, and the update that records a user's last login took up to 79 seconds at 30 users.
- Slow SQL queries. One read of fund transaction details used 97.73 minutes of database time inside a 12-minute window. Inserts into a status log table averaged under a quarter of a millisecond per call, yet together spent 20 to 34 seconds on disk and up to 825 milliseconds waiting on locks.
- Server latency. Requests took 4.16 seconds at the 95th percentile inside the application and 17.9 seconds at the load balancer, which told the team to look beyond the application code as well.
- Bottlenecks. Each login-to-dashboard page fires 17 to 29 requests one after another, and these pages took 82% to 99.9% of page time across both cycles. Active database connections reached 726 against a planned limit of 250, while database CPU sat at 5.2%. A team watching only CPU would have seen a healthy database.
Each finding went to the development team with the run, the timing and the metric behind it, and they fixed the issues before the larger client went live.
Start with test data that obeys the ledger
The test data decides whether a financial load test means anything. Think of testing a bank branch on a day when every account is empty and nobody has a transaction history: the queues move fast, and you learn nothing about a real Monday morning.
For this platform we planned the data around the shape of production, at a scale large enough to stress it:
Volume alone is not enough. Real financial data is uneven, so we built the skew in on purpose. About 5% of funds carry 100 or more linked users and 10,000 or more transactions, and about 5% of batches hold 20,000 or more transactions. Those heavy records are where slow queries hide, and the load scripts can target them directly.
Rules the data generator enforces
Every rule the business follows had to hold in the generated data too:
- Outgoing transactions stay below incoming ones. The generator caps each fund's outgoing transactions at 80% of what the fund received, and the loader checks the same rule again at load time.
- Transfers come in pairs. Every transfer out has a matching transfer in, with the same amount, date and linking identifier.
- Dates respect reconciliation. Transactions are dated inside open reconciliation windows, and a date that has already been reconciled is never reused.
- Pool pricing avoids month-end. Net asset value (NAV) dates, the days on which an investment pool's unit price is set, skip the last day of the month, because reconciliation takes priority then.
Load through the API, never straight into the database
We seeded every record through the same GraphQL API the web application uses, rather than inserting rows into database tables. Direct inserts are faster, but they skip the server's own validation, so they can create records the real application never could. Going through the API means every seeded record is one a real user could have created.
The API also taught us things the schema did not. Several operations behaved differently on the live server from what the schema described, so we recorded each confirmed behavior in a shared notes file and treated a live call as the final word.
Make the seeding pipeline safe to re-run
Loading millions of financial records through an API takes a long time, and some runs will fail halfway. The pipeline has to survive that without creating duplicate money.
We split the work into two stages. The first stage runs offline: it generates the records from a fixed random seed with FakerJS and writes them to CSV files, so the same seed always produces the same data. The second stage runs online and sends those records to the API. It works like writing out a stack of checks at your desk and then depositing them at the bank. If the bank closes halfway through, you go back with the same checks, and you never have to write them again.
Every record gets a reference the pipeline can find again
Each generated record carries its own reference, such as F000123 for a fund. A small SQLite database records which server ID each reference received, and later steps use it to link records together, for example a transaction to its fund.
Before the loader marks a record as failed, it asks the server whether that record already exists. Records carry their reference in a name or description field, so the loader can search for it. If the record is there, the loader saves its ID and moves on. That check is what lets a crashed run restart without creating a second copy of a $5,000 transaction.
Some operations must never be retried
File uploads were the exception. Uploading a positions file to an investment pool adds rows rather than replacing them, so an automatic retry counts the same holdings twice. For upload steps we switched automatic retries off entirely and confirmed the outcome by reading the data back afterwards.
Watch for money-specific traps in the numbers
Small numeric details caused some of the hardest failures to spot.
Store amounts as whole cents. The ledger rejected amounts that were not whole cents. We generate every amount as an integer number of cents, in $50 steps, and convert it to dollars only when writing the file. A value is never drawn as a decimal and then rounded.
Watch floating-point percentages. A fund's investment allocation must total exactly 100%, and the server checks the total in floating point. An allocation of 70%, 8% and 22% looks correct, but 0.7 + 0.08 + 0.22 evaluates to 0.9999999999999999 in binary floating point, and the server rejected it. The Python documentation on floating-point arithmetic explains why most decimal fractions have no exact binary form. Our fix was to express every percentage as a multiple of 1/1024. Fractions with a power-of-two denominator are exact in binary, so any set of them that should total 1 does.
Design scenarios around real roles and real calendars
A financial platform is used by different people doing very different work at the same time. Our scenario catalog has 14 scenarios, and each one mixes roles. A typical scenario sends about 70% of virtual users through user journeys, such as adding a transaction or browsing recipients, while admins run heavy back-office work such as fee processing, payment-file generation or reconciliation uploads.
The calendar matters as much as the role mix. Submitting a month-end reconciliation is something one admin does, so that scenario has a single admin submitting while everyone else generates load elsewhere. Investment pool pricing, by contrast, is submitted before month-end by several admins at once, and the scenario models exactly that.
Model what the browser really does
Three habits kept the scenarios close to real usage:
- Every user starts where the app puts them. After login, the application opens the same user dashboard for every role, admins included. Each journey pays for that landing first, then moves to its own section, so the test does not under-report the real cost per user.
- Queries are copied from the browser. Each request uses the exact GraphQL query the web app sends, taken from the browser's network tab, so the server does the same work it does for a real user.
- Users pause to think. A random wait of about 1 to 6 seconds sits between actions, so 1,000 virtual users behave like 1,000 people rather than 1,000 scripts firing at once. These pauses are left out of the reported response times.
Check every response for hidden errors
A GraphQL API can return HTTP 200 and still report a failure inside the response body. The GraphQL over HTTP specification allows a successful status code with an errors list in the body. A load tool such as Apache JMeter judges success by the status code unless you tell it otherwise, so a failed transaction can show up as a fast, successful request.
Our standard for every new test plan is an assertion on each GraphQL request that fails the sample if the body contains errors. Values passed from one response to the next also get a visible default, such as NOT_FOUND, so a broken chain shows up in the results instead of failing silently three steps later.
Know which limits belong to other services
Login traffic hit a limit that had nothing to do with the application. The platform signs users in through Amazon Cognito, which applies request-rate quotas per category of operation. Our environment allowed about 120 sign-in requests per second, so we ramp users up over at least 20 seconds. Without that ramp, a 1,000-user test measures the sign-in service's throttling instead of the application.
Protect the environment and reset it the same way every time
Financial workflows trigger side effects in the real world. On this platform, posting a reconciliation match sends receipt emails to users automatically. Before any bulk seeding or load run, the test environment needs outbound email and SMS switched off, and external callbacks disabled or mocked. It also needs its own user directory, separate from production.
Resetting needs the same care. We reset by restoring a validated database snapshot, taken only after the seeded data passed its balance and reconciliation checks. Every run then starts from an identical state, which is what makes results from different runs comparable. The pipeline's own reference database is restored together with the snapshot, because the two only make sense as a pair.
Set targets before you raise the load
Agree on pass and fail thresholds before the first run, so nobody argues about the numbers afterwards. These were the targets for this program:
The load then rises in tiers: a single-user check, a small smoke run, then 100, 200, 300, 500, 750 and 1,000 concurrent users. After each tier the team compares the results with the targets and confirms the data is still consistent before moving up. The test stops at the first tier where targets degrade materially. The report records the highest stable tier, the first point of disruption and the bottleneck behind it.
A checklist for your next financial load test
- Size the test data to production, including the uneven distribution of heavy accounts.
- Write the business rules (balances, transfer pairs, reconciliation dates) into the data generator.
- Seed through the application's API so the server validates every record.
- Make loading restartable, and never auto-retry an upload that adds rather than replaces.
- Keep amounts in whole cents and test how percentages sum.
- Mix roles in every scenario and schedule month-end work the way the business does.
- Assert on response bodies, since an HTTP 200 can still carry an error.
- Ramp logins within your identity provider's rate limits.
- Disable outbound email and callbacks, and reset from a validated snapshot.
- Agree on thresholds first, then raise load one tier at a time.
Planning a load test for a platform that moves money?
If you are preparing a financial platform for growth or an audit, our Managed QA services team can help you design the test data, scenarios and thresholds before the first virtual user logs in.
