A/B Testing Processors

The Data

Launching new payment processors for your commerce business can be tricky. Whether you add a new processor to grow your business into new countries, to accept new payment method types, or just to expand your processing capabilities in existing markets, you end up with a whole new set of variables that impact customer experience. For example, does the new payment processor create more checkout friction? Does it have higher approval rates for certain types of transactions? How can you A/B test which processor performs better overall?

In this use case example, we’ll showcase how to compare one payment processor’s performance to another for the same transaction, market, and payment type. Such an analysis provides context on when and how to best route transactions to increase their probability of processing successfully. This can be especially helpful when A/B testing new payment processors.

Exploring the Data in Peacock

If you use one processor to accept transactions globally and introduce a second to accept transactions only on certain markets, it can be difficult to compare their performance in any meaningful way. That being said, such a comparison is vital to understanding if employing the new processor benefits your business’ bottom line.

For example, consider a situation in which you’re using Processor #1 to process transactions originating from countries around the globe, made with any number of payment methods. As a means of slowly introducing another processor option (Processor #2), you start by routing only some transaction volume from the United States made with Visa or Mastercard payment cards through it. Because you have such different payments traffic routing through these two processors, a comparison of the global volume from Processor #1 to the regionally specific volume from Processor #2 wouldn’t be like-for-like.

To conduct a clear and fair performance analysis in Peacock, use filters to create a saved View of only those transaction types that go through both Processor #1 and Processor #2; in this example, that includes only US-based transactions made using Visa or Mastercard branded cards. Then create a custom dashboard of charts displaying those metrics most valuable to your processor performance analysis, such as Processor Approval Rate or Processor Decline Codes. In this new dashboard, you can apply your new saved View to observe how often transactions routed through the new payment processor (Processor #2) succeed compared to those routed to the old processor (Processor #1) along with why transactions processed through each processor are declined; from there, you can determine which one performs better for your business.

Drawing Conclusions

Imagine you’ve set up a custom dashboard as described in the section about. After a few weeks of processing Visa and Mastercard transactions in the US through both Processor #1 and Processor #2, you review the data. Under the Processor Approval Rate chart, you choose the Processor Approval Rate and Transaction Value sub-chart and notice that on average, transactions routed through Processor #2 have a 0.63% higher approval rate. This might not seem like much, but say your business processes an average monthly transaction volume of $30 million; with only a 0.63% higher approval rate, you’d see nearly $2.3 million more in annual revenue. Armed with this data, you decide to route all of your US-based Visa and Mastercard transaction volume through Processor #2.

The possibilities for this sort of analysis stretch far beyond this use case. If you manage multiple sub-merchants, for example, you could also apply soft descriptor filters to your custom dashboard to see how the different processors perform for your individual sub-merchants. Then you may discover that you want to only route certain sub-merchant transaction data through certain processors to improve overall authorization rates. If you’ve successfully identified another way to A/B test payment processor performance using Peacock, share your use case with us!