I really love this post. We've had a bunch of consulting clients in the past where we've given them a huge amount of high-quality data and they've made judgements which made very little sense. I like the emphasis on building the systems, interpretation, and truth-seeking systems.
This story reminded me of a situation at a former employer - a national statistical agency akin to the BLS or StatCan.
It was in our mandate to provide, in addition to the regular statistics we published, custom aggregates by request. Provided we got paid for the time spent and that privacy was maintained, of course.
This was all uncontroversial until a political party known for being skeptical of immigration requested some numbers on crime broken down by immigration status. The directors balked, arguing the request was in bad faith and was intended to misconstrue by cherry-picking data. They offered instead to write a more detailed report with more data and analysis - but still billed by the hour and guaranteed to cost 10x what the simple tabulations would.
This turned into quite the controversy both in the media and internally. We did not doubt that our numbers were accurate, but nobody felt good about making them when they believed it was in bad faith. But should we be the arbiters of this? What did it mean to say that the numbers are correct but refuse to publish them by themselves? Anyone was free to order other tabulations showing perhaps a different picture. And we certainly could not force anyone to pay for a huge analysis we would feel better about (and that some analysts would have loved to write).
I don’t remember how the story ended, and in a way it doesn’t matter. No consensus was reached. It was, perhaps, simply a difficult question.
I go back and forth on how relevant this story is to the situation inside a company. Aiming to do more than just cross-tabulation is always a good idea, but at the end of the day it might be hard to insist on ownership.
I really love this post. We've had a bunch of consulting clients in the past where we've given them a huge amount of high-quality data and they've made judgements which made very little sense. I like the emphasis on building the systems, interpretation, and truth-seeking systems.
Adding this to my recommended list 🙂
Thank you for this! :) It’s a great shift in mindset. I now understand how to be more useful to everyone with the data I handle. Thank you!
This story reminded me of a situation at a former employer - a national statistical agency akin to the BLS or StatCan.
It was in our mandate to provide, in addition to the regular statistics we published, custom aggregates by request. Provided we got paid for the time spent and that privacy was maintained, of course.
This was all uncontroversial until a political party known for being skeptical of immigration requested some numbers on crime broken down by immigration status. The directors balked, arguing the request was in bad faith and was intended to misconstrue by cherry-picking data. They offered instead to write a more detailed report with more data and analysis - but still billed by the hour and guaranteed to cost 10x what the simple tabulations would.
This turned into quite the controversy both in the media and internally. We did not doubt that our numbers were accurate, but nobody felt good about making them when they believed it was in bad faith. But should we be the arbiters of this? What did it mean to say that the numbers are correct but refuse to publish them by themselves? Anyone was free to order other tabulations showing perhaps a different picture. And we certainly could not force anyone to pay for a huge analysis we would feel better about (and that some analysts would have loved to write).
I don’t remember how the story ended, and in a way it doesn’t matter. No consensus was reached. It was, perhaps, simply a difficult question.
I go back and forth on how relevant this story is to the situation inside a company. Aiming to do more than just cross-tabulation is always a good idea, but at the end of the day it might be hard to insist on ownership.