Great panel/intro on semantic models! One challenge that I don't feel like it was covered enough in this post is about the need to manually document the semantic model, and the amount of time it takes. I actually just shared something about it today (coincidentally)!
Hi Yoni, you are right! "Proper" documentation is kinda an essential part of semantic models and as you mentioned in your post people don't like to document 🤦🏻♂️ As a Data Engineer I felt like your post is spot on reflecting documentation issues we are going through!
This is a great read. I’m really hoping semantic modelling remains in the age of AI, because I feel without it, human understanding of the data will become more and more elusive. Proper modelling, to me at least, is a map to understanding our data.
1/ Yeah, what we find is that the hardest and slowest part is coralling the business into agreeing a standard definition. There's often little point doing it without consensus because if people don't trust the definitions.. they'll revert to using their own anyway. So it requires buy-in, trust, communication, sensitivity... It requires getting senior people in a room to talk about a subject that superficially seems simple, and at a detail level seems esoteric.
2/ I'm been a Microsoft BI guy for a long while. Power BI is built for sharing and reusing semantic models. Now with semantic-link in Fabric and the REST Api. Prior to that we had MS OLAP cubes and that was best of breed for semantic querying, but brittle and complex to use.
3/ I feel like we're both getting closer and also have a long way to go before a standard semantic query language emerges. Largely this is because analytics is computationally expensive... and so there's always a priorietary advantage to be won in building a query~storage engine that is faster / cheaper than the competition. This inevitably leads to a query language that is optimised for the priorietary engine - storage solution.
Great panel/intro on semantic models! One challenge that I don't feel like it was covered enough in this post is about the need to manually document the semantic model, and the amount of time it takes. I actually just shared something about it today (coincidentally)!
Curious what you think: https://journey.getsolid.ai/p/semantic-layer-for-ai-lets-not-make?r=5b9smj&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false
Hi Yoni, you are right! "Proper" documentation is kinda an essential part of semantic models and as you mentioned in your post people don't like to document 🤦🏻♂️ As a Data Engineer I felt like your post is spot on reflecting documentation issues we are going through!
Thank you!
This is a great read. I’m really hoping semantic modelling remains in the age of AI, because I feel without it, human understanding of the data will become more and more elusive. Proper modelling, to me at least, is a map to understanding our data.
Great article!
1/ Yeah, what we find is that the hardest and slowest part is coralling the business into agreeing a standard definition. There's often little point doing it without consensus because if people don't trust the definitions.. they'll revert to using their own anyway. So it requires buy-in, trust, communication, sensitivity... It requires getting senior people in a room to talk about a subject that superficially seems simple, and at a detail level seems esoteric.
2/ I'm been a Microsoft BI guy for a long while. Power BI is built for sharing and reusing semantic models. Now with semantic-link in Fabric and the REST Api. Prior to that we had MS OLAP cubes and that was best of breed for semantic querying, but brittle and complex to use.
3/ I feel like we're both getting closer and also have a long way to go before a standard semantic query language emerges. Largely this is because analytics is computationally expensive... and so there's always a priorietary advantage to be won in building a query~storage engine that is faster / cheaper than the competition. This inevitably leads to a query language that is optimised for the priorietary engine - storage solution.
Thank tou Reuben for sharing your experince and ideas to the community, great posts :)