{"id":2044,"date":"2019-04-23T06:37:28","date_gmt":"2019-04-23T06:37:28","guid":{"rendered":"https:\/\/www.aiproblog.com\/index.php\/2019\/04\/23\/causality-the-next-most-important-thing-in-ai-ml\/"},"modified":"2019-04-23T06:37:28","modified_gmt":"2019-04-23T06:37:28","slug":"causality-the-next-most-important-thing-in-ai-ml","status":"publish","type":"post","link":"https:\/\/www.aiproblog.com\/index.php\/2019\/04\/23\/causality-the-next-most-important-thing-in-ai-ml\/","title":{"rendered":"Causality \u2013 The Next Most Important Thing in AI\/ML"},"content":{"rendered":"

Author: William Vorhies<\/p>\n

\n

Summary:<\/em><\/strong>\u00a0 Finally there are tools that let us transcend \u2018correlation is not causation\u2019 and identify true causal factors<\/strong> and their relative strengths in our models.\u00a0 This is what prescriptive analytics was meant to be.<\/em><\/p>\n

\u00a0<\/a><\/p>\n

Just when I thought we\u2019d figured it all out, something comes along to make me realize I was wrong.\u00a0 And that something in AI\/ML is as simple as realizing that everything we\u2019ve done so far is just curve-fitting.\u00a0 Whether it\u2019s a scoring model or a CNN to recognize cats, it\u2019s all about association; reducing the error between the distribution of two data sets.\u00a0<\/p>\n

What we should have had our eye on is CAUSATION.\u00a0 How many times have you repeated \u2018correlation is not causation\u2019.\u00a0 Well it seems we didn\u2019t stop to ask how AI\/ML can actually determine causality. And now it turns out it can.<\/p>\n

But to achieve an understanding of causality requires us to cast loose of many of the common tools and techniques we\u2019ve been trained to apply and to understand the data from a wholly new perspective.\u00a0 Fortunately the constant advance of research and ever increasing compute capability now makes it possible for us to use new relatively friendly tools to measure causality.\u00a0<\/p>\n

However, make no mistake, you\u2019ll need to master the concepts of causal data analysis or you will most likely misunderstand what these tools can do.<\/p>\n

\u00a0<\/p>\n

Why Causality<\/strong><\/span><\/p>\n

In the age when the call for transparency in our models is a constant social media and regulatory cry, causality offers the greatest promise.\u00a0 Causal data analysis also gives you tools well beyond what you currently have to guide you to exactly what to do to get the best outcomes. \u00a0This speaks to the heart of prescriptive analytics.<\/p>\n

It may seem that correlation and curve fitting have done just fine at answering important questions like next best offer, is it fraud, what\u2019s the value going to be, and even is it a cat.\u00a0 But there are a whole variety of questions that our users would like to have answered like:<\/p>\n

    \n
  1. Given that there are X factors that predict preference for a product, which ones should the business actually try to influence, and in what order of importance. (What actually causes change in the target variable)?<\/li>\n<\/ol>\n

    Just ranking the strength of different variables on their ability to predict the target is not the same as selecting those that are independently predictive and evaluating their relative contribution to the outcome.<\/em><\/p>\n

      \n
    1. What are the rankings of those key drivers that actually cause change and how do they compare to my competitors so that I can make smart marketing allocations.<\/li>\n<\/ol>\n

      Isolating a variable like \u2018manufacturer\u2019 within the same probability model doesn\u2019t allow for re-ranking variables, and doesn\u2019t answer the causality question to begin with.<\/em><\/p>\n

        \n
      1. Did that medicine actually cure the disease?<\/li>\n<\/ol>\n

        This problem would require having actually performed both options on the same person, an impossibility in the real world.\u00a0 Simply splitting the sample universe into two gives a probability but not a causally supportable answer.<\/em><\/p>\n

          \n
        1. Would I still have contracted cancer if I had not smoked for the last two years?<\/li>\n<\/ol>\n

          Similarly, there is no way to express the probability associated with two years of not smoking without having an apriori understanding of the strength of the causal relationship not answered by our probability models.<\/em><\/p>\n

          Some other examples of causal questions:<\/p>\n