3 Ways Clean Data Can Turn into Dirty Data
We’ve written extensively about dirty data, and for good reason. Dirty data is a scourge on the real estate asset management industry – firms with dirty data (and that’s most firms) have a higher risk, greater inefficiency, higher headcounts, and thinner margins.
In this post, we’ll identify three ways that data get dirty – and then we’ll talk about how to clean it and keep it clean.
No. 1 – Excel Hell
If you’re using spreadsheets as your primary data warehouse, then you’re practically begging for dirty data.
“Accounting in Excel is a consistent source of dirty data,” says Tony Birrittieri, a partner at Saxony Partners. “If you have people with spreadsheets linking to other spreadsheets, and someone doesn’t refresh one of the spreadsheets that you’re linked to, then the data is not going to be right. It’ll be a nightmare.”
Yes, Excel and other spreadsheet tools are cheap and easy to use. But are you willing to trade your security, data integrity, and your single source of truth for a little convenience? If you are, then you’re a sucker.
“If you’ve got billions in assets and are using Excel as your accounting system, that’s a problem,” says Derek Thornhill, Saxony’s Vice President of Real Estate. “If you have huge Excel files that you’re trying to use as a database – and you’re leveraging those for reporting purposes, then that’s a problem.
“We’ve found, as fixers of dirty data, that when you start poking on those areas, you’ll quickly find dirty data.”
No. 2 – Consistently Inconsistent
Lack of consistency, as it relates to business processes, is another common way to corrupt key data. Without standard operating procedures in place to govern how data is categorized, entered, and reported on, it invites individuals to create proprietary processes of their own.
Once again, spreadsheets can play a facilitating role.
“When you’re in Excel, you have a lot more flexibility to create what I would call data anomalies or one-offs.” Birrittieri says. “For example, instances where people are creating their own processes or rules based on a certain client or situation.”
According to Birrittieri, once clients transition to a data management solution with stricter processes and rules, their eyes are opened to how the previous system had undermined their own data and reporting. Such software solutions eliminates the ability to establish new rules on the fly and creates a standard way of entering, measuring, and analyzing data.
“People understand that they have literally been doing things wrong, from calculations standpoint, from measuring, because they haven’t been doing things consistently,” Birrittieri says. “You can’t aggregate and report on the data because it’s not the same.”
“A lot of these things are going to be brought to the forefront, and then it becomes ‘OK, what is the standard that we want to operate at?’ Then you’ve got to figure that out.”
No. 3 – Blame the (Lack Of) Government
“When I hear dirty data, one of the first things I think of is data governance,” Birrittieri says.
Data governance is, in short, the way that an organization manages, uses, reports, integrates, and secures its data. Often, it’s the small-scale data governance issues that can lead to big problems. For example, when departments within the same organization define common terms in different ways.
“Net Operating Income (NOI) for example,” Birrittieri says. “If your asset managers have defined NOI differently from your portfolio managers, then the final data is going to be corrupted one way or another. In the reporting, you’ll have a label that says ‘NOI,’ but that number may not be what you think really is because it may not be right by your definition.”
Standardization of both terminology and definitions is the key to solving this particular data governance issue. Standardization is preceded by the elimination of siloed departments, processes, and data. In other words, it’s preceded by integration.
What’s the Solution?
How do you get to integration and standardization – for your data warehouse, your business processes, and your terminology and reporting? How can you eliminate dirty data – and maintain the integrity of your data going forward? What you need is a solution that encapsulates the entire lifecycle of the assets you manage.