How about "not yet" as an answer? Solutions always start with generating value against some relatively narrow use case. As they grow, they tackle new use cases but inevitably need to start thinking about efficiency. They also tend to move "upward" in the stack. Where are the constraints that this blocking more adoption? What are the new user groups that are using the solution but struggling? What components of this new solution are already aging out.
I love this idea. Makes me see the MDS as more of an iterative ecosystem movement than a concrete set of tools, and that's something I'd like to see sustain long-term.
If we look at Hadoop->Cloud Data Warehouse, we see increases in efficiency and lower barriers to entry. You could argue that cloud data warehouses are the scalable version of hadoop based solutions
Interesting, and actually, that comparison makes the UX value of cdw blatantly obvious. Where do you think things will go next for the cdw? Everyone still seems to default to bq or sf, but I wonder if there's any room for movement there. Perhaps time to discover opportunity to reduce cost takes too long, and at that point cost of migration is generally too massive/not priority
its a great question and highly relevant with LLMs looming in the corner as junior analysts. The best solutions optimize for enabling the most workers with the most power. If the definition of worker is changing, that potentially disrupts all solutions.
Seems inevitable to me that margins around compute will shrink, potentially VERY fast. We've seen that in industry after industry. Running SQL at scale is a fungible activity now. Look at what MotherDuck is doing. Look at what Athena et al have done in the past. It really doesn't matter who has the fastest data warehouse. In a world where parallelism scales nearly linearly, its more about compute/$, given equivalent experiences.
For me, right now Snowflake is my first choice since enabling users still gives me more bang for the buck over cheap compute. But for big jobs or work that is no longer evolving a lot, I have no choice but to look more closely at compute costs
How about "not yet" as an answer? Solutions always start with generating value against some relatively narrow use case. As they grow, they tackle new use cases but inevitably need to start thinking about efficiency. They also tend to move "upward" in the stack. Where are the constraints that this blocking more adoption? What are the new user groups that are using the solution but struggling? What components of this new solution are already aging out.
I love this idea. Makes me see the MDS as more of an iterative ecosystem movement than a concrete set of tools, and that's something I'd like to see sustain long-term.
If we look at Hadoop->Cloud Data Warehouse, we see increases in efficiency and lower barriers to entry. You could argue that cloud data warehouses are the scalable version of hadoop based solutions
Interesting, and actually, that comparison makes the UX value of cdw blatantly obvious. Where do you think things will go next for the cdw? Everyone still seems to default to bq or sf, but I wonder if there's any room for movement there. Perhaps time to discover opportunity to reduce cost takes too long, and at that point cost of migration is generally too massive/not priority
its a great question and highly relevant with LLMs looming in the corner as junior analysts. The best solutions optimize for enabling the most workers with the most power. If the definition of worker is changing, that potentially disrupts all solutions.
Seems inevitable to me that margins around compute will shrink, potentially VERY fast. We've seen that in industry after industry. Running SQL at scale is a fungible activity now. Look at what MotherDuck is doing. Look at what Athena et al have done in the past. It really doesn't matter who has the fastest data warehouse. In a world where parallelism scales nearly linearly, its more about compute/$, given equivalent experiences.
For me, right now Snowflake is my first choice since enabling users still gives me more bang for the buck over cheap compute. But for big jobs or work that is no longer evolving a lot, I have no choice but to look more closely at compute costs