Fun thought exercise, Spencer. I'd suggest that there's an extra dimension you may consider which is "time" - there's a temporal difference between signal/feedback on a False-Positive (Customer Blocked) and a False-Negative (Chargeback). The former is often in real-time, whereas the latter is weeks, if not at least a month later. Opportunity often exists to tightly tie CS feedback to model optimization with the caveat that criminals are also incentivized to complain when their transactions are stopped. Models optimized on Chargebacks alone tend to suffer from "the horse has already bolted, won the race and been put out to stud when the stable door is closed". As someone who's been in the trenches, I know you've felt the frustration when Senior management yo-yo's between loosening models (due to complaints) only to tighten them up (when the chargebacks roll in).
100%. Loss forecasting models are one tool that's really worked well for me here. The earlier you know what expected losses will be, the faster you can react, and the clearer picture you can give to Execs on the operational decisions you're making.
I understand that it can be defined as a chargeback or fraud alert in payments.
However, a very high percentage of chargebacks and fraud alerts are due to 'friendly fraud', which is difficult to detect as it is not 'real fraud' (e.g. card testing fraud, account takeover fraud, etc.).
Additionally, there are other reasons for chargebacks that are not necessarily caused by fraudsters, such as dissatisfaction with the product.
Conversely, not all card testing fraud results in a chargeback or fraud alert, which means we need to be able to label such cases as fraud in the models.
When I’m measuring systems like this, I usually try to narrow the definition of “fraud” to fraud-coded chargebacks. That’s not because it’s a perfect label, but because it’s the cleanest and most consistently measurable outcome tied to real economic loss.
Friendly fraud, in my experience, is a different problem that tends to require different solutions. I’ve had more success tackling it through better representment, better data capture, and upstream product or CX fixes than through fraud models or manual review. At least with the data I’ve worked with, I haven’t found a reliable way to distinguish friendly fraud from legitimate customers at decision time.
Card testing is a bit different. You can model it in a similar expected-value framework, but the detection signals and remedies are often distinct. I view it as almost another totally separate fraud-prevention system to be optimized.
Nice frame work overview on fraud strategy optimization
Fun thought exercise, Spencer. I'd suggest that there's an extra dimension you may consider which is "time" - there's a temporal difference between signal/feedback on a False-Positive (Customer Blocked) and a False-Negative (Chargeback). The former is often in real-time, whereas the latter is weeks, if not at least a month later. Opportunity often exists to tightly tie CS feedback to model optimization with the caveat that criminals are also incentivized to complain when their transactions are stopped. Models optimized on Chargebacks alone tend to suffer from "the horse has already bolted, won the race and been put out to stud when the stable door is closed". As someone who's been in the trenches, I know you've felt the frustration when Senior management yo-yo's between loosening models (due to complaints) only to tighten them up (when the chargebacks roll in).
100%. Loss forecasting models are one tool that's really worked well for me here. The earlier you know what expected losses will be, the faster you can react, and the clearer picture you can give to Execs on the operational decisions you're making.
Thanks for the article, Spencer!
How is 'fraud' defined in your models?
I understand that it can be defined as a chargeback or fraud alert in payments.
However, a very high percentage of chargebacks and fraud alerts are due to 'friendly fraud', which is difficult to detect as it is not 'real fraud' (e.g. card testing fraud, account takeover fraud, etc.).
Additionally, there are other reasons for chargebacks that are not necessarily caused by fraudsters, such as dissatisfaction with the product.
Conversely, not all card testing fraud results in a chargeback or fraud alert, which means we need to be able to label such cases as fraud in the models.
Glad you enjoyed it, Bogdan!
When I’m measuring systems like this, I usually try to narrow the definition of “fraud” to fraud-coded chargebacks. That’s not because it’s a perfect label, but because it’s the cleanest and most consistently measurable outcome tied to real economic loss.
Friendly fraud, in my experience, is a different problem that tends to require different solutions. I’ve had more success tackling it through better representment, better data capture, and upstream product or CX fixes than through fraud models or manual review. At least with the data I’ve worked with, I haven’t found a reliable way to distinguish friendly fraud from legitimate customers at decision time.
Card testing is a bit different. You can model it in a similar expected-value framework, but the detection signals and remedies are often distinct. I view it as almost another totally separate fraud-prevention system to be optimized.
Appreciate the thoughtful question.