Friday, August 13, 2010

A case study of problematic analysis and data collection in Realtytrac's foreclosure report

In recent wire stories referencing Realtytrac's second quarter foreclosure statistics, it has been noted that "[f]oreclosure filings rose in the second quarter in Idaho, Illinois, Utah, and Colorado compared with a year earlier." [Link]

The reporting in the Bloomberg/BusinessWeek article is based on recently released foreclosure data from Realtytrac, but Realtytrac's data is incorrect. The year-over-year change during the second quarter in Colorado is actually negative, while Realtytrac reports growth. This is an example of incorrect and potentially misleading data coming out of a Realtytrac report.

In this article, I will examine how Realtytrac's reported numbers for the second quarter do not match up with the numbers reported by the public trustees in Colorado, and how Realtytrac's practice of combining foreclosure filings totals with foreclosure sales totals leads to errors in interpreting the data.

The Realtytrac Method

The details of Realtytrac's data collection methods remain a mystery. The press releases note that Realtytrac engages in a kind of sampling: "Data is collected from more than 2,200 counties nationwide, and those counties account for more than 90 percent of the U.S. population."

It is unknown from which Colorado counties Realtytrac samples its data. The quarterly foreclosure reports provided by the Division of Housing, however, collect complete data on foreclosure activity directly from public trustees in all counties in Colorado. The public trustee is the county office responsible for administering foreclosure filings and auctions in Colorado.

In spite of a lack of information about collection methods, we can nevertheless glean some information from the published press releases. Based on the data it does report publicly, the numbers put forward as the statewide number for Colorado is based on the number of "Notices of Trustee Sales" (NTS) added to the number of "Real Estate Owned" (REO) filings.

Here are the numbers as listed on Realtytrac's second quarter report:

Note that the "Totals" column is clearly the total of the NTS and REO numbers added together. In Colorado, what Realtytrac calls NTS is known as the Notice of Election and Demand (NED), and what Realtytrac calls REO is in Colorado commonly referred to as a foreclosure "sale at auction" or "certificate of purchase."

Here we see our first problem with Realtytrac's analysis. By adding together the NED total and the sale number, and then basing foreclosure rates and trends on the combined number, Realtytrac is obscuring essential information that is key to accurate analysis of foreclosure trends in the state.

This is because the NED number and the sale number are two completely independent statistics, and often trend in opposite directions. For example, between April 2009 and April 2010, foreclosure sales at auction showed a generally upward trend, and overall they increased by about 45 percent. During the same period, NEDs showed a generally downward trend and showed an overall decrease during the period of about 16 percent.

NEDs and sales are affected differently by changes in the economic environment and should not be combined. NEDs are a leading indicator of economic pressures that produce foreclosures, while foreclosure sales are a trailing indicator. NEDs represent the beginning of the foreclosure process, while foreclosure sales represent the end.

It is not difficult to see that if NEDs are rising, but foreclosure sales are falling, that, all things being equal, one would expect to see a future rise in foreclosure sales. On the other hand, if NEDs are falling, but sales are rising, we would expect to see a fall in sales activity a few months down the road. If we combine the two numbers before making our analysis, however, these insights into future foreclosure trends become impossible.

Also, foreclosure sales at auction are always sales of loans that were already counted as foreclosures when the NED was filed. So, by combining the two numbers, and then including both in the same category, Realtytrac is reporting a foreclosing loan a second time.

It is probably best to not consider the Realtytrac totals as actual counts of foreclosures at all, but as an index number constructed from two dissimilar statistics. However, as we have seen, this index number can often cause us to miss important information contained in the actual foreclosure totals.

Recent Measurements

The recent wire story noting that year-over-year foreclosure activity is trending upward highlights some of the problems in basing analysis on Realtytrac's combined totals. The State of Colorado's data, which does not sample but relies on full totals collected from all counties, shows that 2010 second-quarter NEDs fell 15.6 percent. Foreclosure sales increased 17.7 percent. Clearly, this shows us that two different trends are at work if we look at sales and NEDs separately.

Realtytrac reports that foreclosure activity increased during this period by 4.73 percent. So, did foreclosures increase or decrease during this period? Realtytrac says they increased, but we see in our own numbers that new properties entering foreclosure actually fell almost 16 percent.

However, to make an honest comparison, we need to add together our own NED and sale totals to re-create the Realtytrac method. After adding together the two totals, we still find that Realtytrac's number is confusing. Adding together the State's NEDs and the sales totals for 2010's second quarter, and then comparing to the second quarter of last year, we find that the combined total is down 5.9 percent, year-over-year. Again, Realtytrac, using the same combined total, shows an increase of 4.73 percent. So, using Realtytrac's own method of analysis for the State's totals, we find that Realtytrac's numbers show an increase when the real change, based on data from all counties direct from public trustees, is a decline of 5.9 percent.

Since using Realtytrac's method of analysis still yields a discrepancy between the state numbers and the Realtytrac numbers, we must therefore conclude that Realtytrac's method of data collection is faulty.

Data Collection

Let's now look at the actual totals reported by Realtytrac and compare them to our own numbers.

First Realtytrac's numbers:
For the second quarter of 2010, Realtytrac reports an NED total of 10,002 and a sale number of 5,232. The combined total is 15,287. Realtytrac does not provide historical stats in its reports, but if we look up the old 2nd Q 2009 report, we find that Realtytrac's totals for that period were 10,938 NEDs and 3,618 sales. The combined total is 14,579. The change from 14,579 to 15,287 is an increase of 4.7 percent, which is reported in the 2nd Q 2010 Realtytrac report.

The State of Colorado numbers:
For the second quarter of 2010, the state's public trustees reported an NED total of 10,233 and a sale number of 5,885. The combined total is 16,118. For the second quarter of 2009, the state's public trustees reported 12,135 NEDs and 4,999 sales. The combined total is 17,134. The change from 17,134 to 16,118 is a decrease of 5.9 percent, although this statistic was not published in the State's 2nd Q report since we believe it is not useful and is misleading to combine the NED total and the Sale total.

Looking at these totals, we find that Realtytrac undercounted considerably during the second quarter of 2009, and that this incorrectly low number, when compared with the 2nd Q 2010 total, showed a net positive increase in foreclosure activity between the second quarter of 2009 and the same period this year. It is likely that the change over this period was, in fact, negative.

Also, the fact that NEDs declined over this period while sales increased is key information that is left out of the Realtytrac analysis as packaged for the media.


Since it is unknown from which counties Realtytrac samples in Colorado, it is difficult to determine the reasons for Realtytrac's significantly large undercounting during the second quarter of 2009. However, the fact that small and medium-size counties in Colorado have driven much of the growth in Colorado foreclosures in recent quarters may be important if Realtytrac is not collecting data from these counties.

The Division of Colorado continues to provide the most accurate information on foreclosure trends in Colorado. The state's data is taken not from samples, but from complete counts of foreclosure events as provided by the public trustee in each county.

A case study of the 2009 and 2010 second-quarter data is produced above as an example of some of the problematic elements in the Realtytrac data. At this time it is unknown as to what other quarterly or monthly analyses might be affected by similar problems in data collection or analysis.

We believe it is important that media reports of foreclosure trends in Colorado reflect the most accurate data available.