We also see systems engineering and product design better when we slow down. Connections are easier to make we slow down, but speed up understanding of the underlying reasons and potential solutions from it. Especially when there's a roadmap that's dependent on it. Speed focus can definitely direct us on the path of building something, without laying the foundation.
"This, plus the fact that we generally don’t have rigorous testing procedures for data work, means it’s easy to find ourselves in a tightrope walk over our stakeholders’ trust."
I can't take any single word from this passage. It's so beautifully crafted.
I love how you mentioned the lack of rigorous testing procedures in data work. It's night and day compared to software work.
Worst case scenario if soft. engineer make mistake (in the typical tech company nowadays) is "Oops, I didn't pass the test coverage. Well, I can just delete this particular branch, re-clone and start all over".
In the data work, the worst case scenario is your stakeholder coming to you and said, "Hey, I think you use wrong calculation for our revenue number. It confuse all of us during the meeting. Have you double check it?".
Incidentally, this also the reason why I bet it's quite hard for AI to eat analytics work.
"I love how you mentioned the lack of rigorous testing procedures in data work. It's night and day compared to software work."
Yes! This feels like something we don't talk about enough. I feel like we resign ourselves to "oh it's not code so it's not *testable*", but as an industry, we should probably come up with some set of standard practice for conducting tests [manually] during analytics work.
"Incidentally, this also the reason why I bet it's quite hard for AI to eat analytics work."
Interesting, never thought of it that way, but I think I'd agree. The ways you can be wrong are too numerous, too domain-specific, too dependent on tribal knowledge. Without an AI extremely intimated connected to the business, it'd be tough to rely blindly on any of its outputs.
We also see systems engineering and product design better when we slow down. Connections are easier to make we slow down, but speed up understanding of the underlying reasons and potential solutions from it. Especially when there's a roadmap that's dependent on it. Speed focus can definitely direct us on the path of building something, without laying the foundation.
"This, plus the fact that we generally don’t have rigorous testing procedures for data work, means it’s easy to find ourselves in a tightrope walk over our stakeholders’ trust."
I can't take any single word from this passage. It's so beautifully crafted.
I love how you mentioned the lack of rigorous testing procedures in data work. It's night and day compared to software work.
Worst case scenario if soft. engineer make mistake (in the typical tech company nowadays) is "Oops, I didn't pass the test coverage. Well, I can just delete this particular branch, re-clone and start all over".
In the data work, the worst case scenario is your stakeholder coming to you and said, "Hey, I think you use wrong calculation for our revenue number. It confuse all of us during the meeting. Have you double check it?".
Incidentally, this also the reason why I bet it's quite hard for AI to eat analytics work.
Thank you, Erald!!
"I love how you mentioned the lack of rigorous testing procedures in data work. It's night and day compared to software work."
Yes! This feels like something we don't talk about enough. I feel like we resign ourselves to "oh it's not code so it's not *testable*", but as an industry, we should probably come up with some set of standard practice for conducting tests [manually] during analytics work.
"Incidentally, this also the reason why I bet it's quite hard for AI to eat analytics work."
Interesting, never thought of it that way, but I think I'd agree. The ways you can be wrong are too numerous, too domain-specific, too dependent on tribal knowledge. Without an AI extremely intimated connected to the business, it'd be tough to rely blindly on any of its outputs.