Summary:<\/em><\/strong>\u00a0 True prescriptive analytics requires the use of real optimization techniques that very few applications actually use.\u00a0 Here\u2019s a refresher on optimization with examples of where and how they\u2019re best used.<\/em><\/p>\n \u00a0<\/p>\n Originally I felt strongly that this was a distinction without a difference<\/u><\/em><\/a> and only served to confuse our customers who were having a hard enough time five years ago understanding why they should even be doing predictive.\u00a0 Time passed.\u00a0 I was over ruled and prescriptive analytics became a fixed part of our nomenclature.<\/p>\n As our attention has been pulled increasingly to AI, the greater business value by far is still being generated by ML and predictive models.\u00a0 And although a great deal of lip service is being given to using models to determine what should be done, very little of this involves true optimization.<\/p>\n The typical business uses such as \u2018next best offer\u2019 or churn reduction or even fraud detection are much more likely to be used in the context of a rules based engine or even robotic process automation.\u00a0 It seems time to review where true optimization fits in this and to outline some of the problem types where optimization should be used.<\/p>\n \u00a0<\/p>\n Confusion over Optimization<\/strong><\/span><\/p>\n Any quick survey of the literature will find a definition for optimization similar to this:<\/p>\n The act of obtaining the best results under the given circumstances.\u00a0 The goal being to minimize the required effort or maximize the desired benefit.<\/em><\/p>\n The problem is that this definition is much too broad to be used in data science.\u00a0 Starting with differentiating between single variable and multiple variable optimization.<\/p>\n \u00a0<\/p>\n Single Variable versus Multiple Variable Optimization<\/strong><\/span><\/p>\n To clarify, we are talking about the dependent or output variable which is the goal of every predictive model.\u00a0 We apply an appropriate cost function and use multiple techniques to get a best fit model.\u00a0 In almost all cases this involves a gradient descent calculation like SGD which is itself the very definition of optimization.<\/p>\n So although every model we create is essentially an optimization, this is decidedly not what we\u2019re talking about when talking about optimization in the context of prescriptive analytics.<\/p>\n True optimization problems need to be expressed in terms of at least two (seemingly conflicting) functions.\u00a0 For example:<\/p>\n It should be evident that each of these examples contains conflicting goals.\u00a0 Maximizing unit sales for example could easily be achieved by allowing price to fall well below cost.\u00a0 However maximizing for both unit sales and profit requires looking at where these two different curves intersect, showing both the lowest price to earn maximum unit sales while at the same time earning the greatest profit.<\/p>\n \u00a0<\/p>\n Defining the Problem<\/strong><\/span><\/p>\n Optimization problems are said to require a four part definition:<\/p>\n Decision Variable:\u00a0<\/strong> The measure to be optimized (maximized or minimized, e.g. profit, travel time).<\/p>\n Objective:<\/strong>\u00a0 Measured by minimization or maximization.<\/p>\n Constraints:<\/strong>\u00a0 This is the most important element and describes the formulas by which the two conflicting forces behave.\u00a0 For example trash throughput versus CO2 production, or revenue versus profit.<\/p>\n Bounds:<\/strong>\u00a0 Bounds define the limits of the feasible solutions.\u00a0 It might be something seemingly obvious like prices cannot be a negative number or that the operating temperature of the waste incinerator cannot exceed some logical physical maximum temperature.\u00a0 Many optimization problems don\u2019t define bounds so their solutions are open to any feasible solution in the search space.<\/p>\n \u00a0<\/p>\n More Than Two Variables<\/strong><\/span><\/p>\n While it\u2019s theoretically possible to have three or more variable to be optimized simultaneously, typically this involves such computational complexity that it\u2019s not recommended.<\/p>\n For example, in the profitability example we might seek to select the promotion that 1.) maximizes sales, 2.) maximizes profit, and 3.) focuses only on a specific characteristic of merchandise such as inventory that\u2019s over 60 days old.\u00a0 It\u2019s almost always better to reduce the third variable to a bounds, and if necessary segment the problem by different bounds groupings.<\/p>\n \u00a0<\/p>\n Solution Techniques and Other Considerations<\/strong><\/span><\/p>\n The mathematics and techniques available for optimization are almost as complex and multi-faceted as the techniques of machine learning itself.\u00a0 Start with the issue of whether the challenge falls into the category of \u2018Convex Optimization Problems\u2019 or \u2018Non-Convex\u2019.<\/p>\n Convex optimization problems can be shown to have a single optimum answer.\u00a0 Imagine the case of simply charting the two opposing constraint functions to see where they intersect, representing the single optimum solution.<\/p>\n One is to bound the problem to a narrow group of data that is likely to have only a single local optima.\u00a0 Frequently however it\u2019s necessary to use a multiple-start technique that randomly begins the search process at different places in the solution space and collects a group of local optima that can subsequently be compared.\u00a0 Evolutionary genetic algorithms are favored in this group.\u00a0 Still no guarantee that the true optima has been located.\u00a0<\/p>\n Swarm techniques of which there are many with many clever names like Ant Colony, Firefly optimization, or Bee optimization are available.\u00a0 These are multiple start techniques but instead of sequential iterations, these are agents which start simultaneously.\u00a0 Despite the connotation of \u2018swarm\u2019, generally about 10 or 12 independent agents are all that\u2019s required.\u00a0 Here\u2019s a somewhat comprehensive list of optimization techniques.<\/p>\n This is not to say that you need to master more than a few of these, but you should be aware that selecting the right algorithm is not simply selecting whatever default optimizer might be available in your favorite package.<\/p>\n If you find an opportunity for true optimization, and not just an application of a simple rule based on model score, then you may be able to create a truly optimized outcome for your organization.<\/p>\n \u00a0<\/p>\n \u00a0<\/p>\n Other articles by Bill Vorhies<\/u><\/em><\/a><\/p>\n \u00a0<\/p>\n About the author:\u00a0 Bill is Contributing Editor for Data Science Central.\u00a0 Bill is also President & Chief Data Scientist at Data-Magnum and has practiced as a data scientist since 2001.\u00a0 \u00a0His articles have been read more than 2.0 million times. He can be reached at:<\/p>\n Bill@DataScienceCentral.com<\/a> or<\/span> Bill@Data-Magnum.com<\/a><\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Author: William Vorhies Summary:\u00a0 True prescriptive analytics requires the use of real optimization techniques that very few applications actually use.\u00a0 Here\u2019s a refresher on optimization […] Read More<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":466,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[26],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/2511"}],"collection":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=2511"}],"version-history":[{"count":0,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/2511\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media\/474"}],"wp:attachment":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=2511"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=2511"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=2511"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}<\/a>Predictive analytics and optimization have gone hand in hand since the very beginning.\u00a0 But in 2014 some erudite journal decided we needed another phrase for this combo and it became Prescriptive Analytics, theoretically differentiating what could happen (predictive) from what should happen (prescriptive) through the application of optimization.<\/p>\n
\n
<\/a>More common and certainly familiar to data scientists are the \u2018non-convex\u2019.\u00a0 It\u2019s easy to visualize these as the complex solution spaces we all encounter with multiple local minima and maxima.\u00a0 There are dozens of techniques to address these situations.<\/p>\n
\n