Yes, but marketers don't really care about math. Only about the numbers!
You would expect a few hiccups and a steep learning curve when the product they are trying to buy is a bid for visibility on insanely complex networks of platforms using said rocket science to guess where one's mind is at.
Disclaimer: worked years in the space on the topic of performance, only one time ever have i seen an AB test correctly run and correctly understood by a client. Most requested false numbers because they couldn't understand the difference between branding, attribution and incremental sales. To be fair, neither could most anyone working in the field... Seems like that doesn't change too quick!
And sad. Same thing happens in data science now - people are happy to have any number regardless of correctness. They just need something to report on and keep the work going.
As long as data science remains subservient to Product, the inevitable result is cherry picking and post hoc rationalizing of some PM's intuition. To blame is the "embedded" model in which DS works within (and takes direction from) a cross functional team driven by a PM.
Hi, please don't do this. If the comments become just replies saying "I wrote about this on my blog!" with a link, they're unreadable. If you summarized your thoughts and posted it as its own comment, without a self-promotional link, I'm sure it would be very well received.
Sounds fair! Thanks for the feedback. The article did spur an interesting discussion previously on HN when I published it but seems relevant again given the ongoing discussion.
This is one of those things that isn't a big deal in isolated instances, but becomes an issue if it gets more prevalent, so it's always hard to know when/if to even say anything :)
The other problem is that the worst offenders are the ones least likely to respond to getting or seeing this sort of nudge, so there is some adverse selection going on.
It's probably self correcting over the long term, but meanwhile we lose the sort of links that would (or rather, might) enrich the conversation through overcorrection by posters.
It's the obverse side of the slippery slope, I guess.
You would expect a few hiccups and a steep learning curve when the product they are trying to buy is a bid for visibility on insanely complex networks of platforms using said rocket science to guess where one's mind is at.
Disclaimer: worked years in the space on the topic of performance, only one time ever have i seen an AB test correctly run and correctly understood by a client. Most requested false numbers because they couldn't understand the difference between branding, attribution and incremental sales. To be fair, neither could most anyone working in the field... Seems like that doesn't change too quick!