Home › What the research says

How Accurate Is Your Spending Memory? What Measurement Studies Show

The experiment that pitted recall against a diary

Kathleen Beegle, Joachim De Weerdt, Jed Friedman, and John Gibson ran a field experiment in Tanzania, published in the Journal of Development Economics, testing eight alternative methods of measuring household consumption on a sample of about 4,000 households. Because the method was randomly assigned, differences in the measured total are differences in the instrument rather than differences in the households. They found significant differences between consumption reported by the benchmark personal diary and other diary and recall formats, and specifically that recall modules measure lower consumption than a personal diary, with larger gaps among poorer households and among households with more adult members. Survey design choices were large enough to move measured poverty rates.

A test against actual records

The strongest form of this question compares a person's estimate to an objective record of the same behavior. Michael Auer, Niklas Hopfgartner, Denis Helic, and Mark Griffiths did this in the Journal of Gambling Studies in 2023 using 639 online casino players who estimated their deposits over the previous 30 days, then compared those estimates to the operator's account data. The median estimate was close to the median actual, 230 euros against 226 euros, so the typical person was not wildly off. The distribution is where the useful finding is. Around 40 percent underestimated, 43 percent overestimated, and underestimation rose sharply with the amount deposited, from about 15 percent of low-deposit players to 56 percent of high-deposit players.

Even the official statistics have this problem

It would be convenient to say diaries are simply accurate and recall is simply wrong, and that is not what the record shows. Adam Bee, Bruce Meyer, and James Sullivan compared both components of the US Consumer Expenditure Survey to national accounts data. They found most of the largest consumption categories are measured well in the Interview Survey, with ratios close to one that have not declined appreciably, while several other large categories are reported at low rates or have declined over time. They also report that Diary respondents show much greater dispersion and are much more likely to report zero spending for a category, and conclude that using Diary data to assess distributional questions is likely to mislead.

What this actually means for your budget

Put those three together honestly. Every method of measuring spending is an instrument with its own error, and the error has a direction that depends on the method and on who is doing the reporting. Recall runs low against a carefully kept personal diary. Individual estimates get worse, and specifically more downward-biased, as the amount goes up. And diaries fail in their own way, most obviously through under-recording that shows up as zeros. None of this makes you bad with money. It makes your memory a measuring device with known error, which is a different and much more workable problem.

The envelope change to make

Log the purchase at the moment it happens rather than reconstructing the week on Sunday. That is the diary method, and the friction people complain about is the measurement instrument doing its job: recording at the point of sale is the only moment when the amount, the category, and your attention are all present at once. It is also why an app that does not link to your bank is making a deliberate tradeoff rather than missing a feature. Then resize envelopes from the logged actuals after thirty days, since the recall estimates you started with are the ones the research says run low.

Sources

Every source below was retrieved and checked. Findings are reported as the source states them.

• Methods of Household Consumption Measurement Through Surveys: Experimental Results from Tanzania — Journal of Development Economics, 2012
  A field experiment testing eight alternative methods of measuring household consumption on about 4,000 households found significant differences between the benchmark personal diary and other formats; recall modules measured lower consumption than a personal diary, with larger gaps among poorer households and households with more adult members, and under-reporting was particularly apparent for illiterate households and urban respondents completing household diaries.
• Self-reported Deposits Versus Actual Deposits in Online Gambling: An Empirical Study — Journal of Gambling Studies, 2023
  Among 639 online casino players comparing 30-day deposit estimates to operator account records, the median estimate (230 euros) was close to the median actual (226 euros), but roughly 40 percent underestimated and 43 percent overestimated, and underestimation rose from about 15 percent among low-deposit players to 56 percent among high-deposit players.
• The Validity of Consumption Data: Are the Consumer Expenditure Interview and Diary Surveys Informative? — NBER Working Paper No. 18308, 2012
  Most of the largest consumption categories are measured well in the Interview Survey relative to national accounts, while several other large categories are reported at a low rate or have declined over time; Diary respondents show much greater dispersion and are much more likely to report zero spending for a category.

Common questions

How far off is the average person about their own spending?

There is no single figure, and the honest answer depends on the method and the amount. In the online gambling study, the median estimate was close to the median actual, but roughly 40 percent underestimated and underestimation rose from about 15 percent among low-deposit players to 56 percent among high-deposit players. In the household survey experiment, recall modules came out lower than a personal diary, with bigger gaps for poorer and larger households. The pattern is that error grows with the size and frequency of spending.

Is manual tracking better than automatic bank sync?

No study here compares them, so anyone claiming a winner is guessing. What the measurement literature does say is that recording at the time of the transaction produces different, generally higher, totals than reconstructing later, and that has been tested with randomly assigned methods. Automatic import solves a different problem, which is completeness of the record, and it does nothing about whether you noticed the purchase. Those are separate goals and you should pick based on which one you actually have.

Why do small purchases matter if they are small?

Because they are the ones a recall estimate is most likely to skip, and they recur. The survey experiment found method choice moved measured totals enough to change estimated poverty rates, which is not a small discrepancy. A category made of many small frequent items is exactly where your memory has the fewest distinct events to retrieve. That is a reason to log them, not a reason to feel guilty about them.

Related