{"id":2783,"date":"2019-11-06T06:37:29","date_gmt":"2019-11-06T06:37:29","guid":{"rendered":"https:\/\/www.aiproblog.com\/index.php\/2019\/11\/06\/creating-autonomous-entities-optimizing-systems-of-subsystems\/"},"modified":"2019-11-06T06:37:29","modified_gmt":"2019-11-06T06:37:29","slug":"creating-autonomous-entities-optimizing-systems-of-subsystems","status":"publish","type":"post","link":"https:\/\/www.aiproblog.com\/index.php\/2019\/11\/06\/creating-autonomous-entities-optimizing-systems-of-subsystems\/","title":{"rendered":"Creating Autonomous Entities: Optimizing Systems of Subsystems"},"content":{"rendered":"<p>Author: Bill Schmarzo<\/p>\n<div>\n<p><span>I continue to be infatuated with the potential of autonomous entities that self-monitor, self-diagnose, self-fix and self-learn without human intervention.\u00a0 The vision of leveraging new digital technologies, growing reams of customer, product and operational data, and advanced analytics to create autonomous entities \u2013 farms, oil fields, factories, airports, theme parks, vehicles, ships, trains, etc. \u2013 seems like science fiction.\u00a0<\/span><\/p>\n<p><span>Fortunately, the \u201c3 Horizons of Digital Transformation\u201d framework provides a roadmap leveraging for 1) today\u2019s market share grabbing operational excellence initiatives to 2) advance to tomorrow\u2019s IoT-induced smart entities that seek to optimize across interrelated use case decisions that lead to 3) the future world of autonomous entities that optimize their operations across a holistic and comprehensive <strong>utility function<\/strong> without human intervention (see Figure 1).<\/span><\/p>\n<p><span><a href=\"https:\/\/storage.ning.com\/topology\/rest\/1.0\/file\/get\/3697118183?profile=original\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" src=\"https:\/\/storage.ning.com\/topology\/rest\/1.0\/file\/get\/3697118183?profile=RESIZE_710x\" class=\"align-full\"><\/a><\/span><\/p>\n<p><strong>Figure<\/strong> <strong><span>1<\/span><\/strong><strong>:<\/strong> <a href=\"https:\/\/www.linkedin.com\/pulse\/return-thriller-3-horizons-digital-transformation-bill-schmarzo\/\"><strong>3 Horizons of Digital Transformation<\/strong><\/a><\/p>\n<p><span>This blog will explore how organizations can build out their digital, data and analytic capabilities today in order to strive towards that Horizon 3 autonomous entity vision of the future.\u00a0 But it won\u2019t be a big bang approach to creating a Horizon 3 autonomous entity.\u00a0 Instead, <em>it\u2019ll require mastering the economic power of data and analytics.<\/em><\/span><\/p>\n<h1><strong>Laws of Autonomous Entities<\/strong><\/h1>\n<p><span>I first wrote about autonomous in the blog \u201c<\/span><a href=\"https:\/\/www.linkedin.com\/pulse\/transforming-from-autonomous-smart-reinforcement-basics-bill-schmarzo\/\"><span>Transforming from Autonomous to Smart: Reinforcement Learning Basics<\/span><\/a><span>\u201d.\u00a0 That\u2019s where I talked about the role of reinforcement learning in creating autonomous vehicles.\u00a0<\/span><\/p>\n<ul>\n<li><span>Autonomous Law #1: Autonomous entities need lots and lots of granular data against which to apply Deep Reinforcement Learning techniques from which to learn.\u00a0 Some of that data must be real-world data, but undoubtedly some of that data will need to be generated in video games or simulations because there may not be enough humans to generate enough data.<\/span><\/li>\n<\/ul>\n<p><span>I next wrote about autonomous in the blog \u201c<\/span><a href=\"https:\/\/www.linkedin.com\/pulse\/leverage-ai-create-autonomous-policies-adapts-without-bill-schmarzo\/\"><span>Leverage AI to Create Autonomous Policies that Adapts without Human Intervention<\/span><\/a><span>\u201d where I talked about recent developments with Deep Reinforcement Learning to create \u201cRational AI Agents.\u201d<\/span><\/p>\n<ul>\n<li><span>Autonomous Law #2: Autonomous entities will rely upon Rational AI Agents that power autonomous entities ability to learn and operate safely and effectively\u2026 within a \u201cfail safe\u201d environment in which to master its \u201ctrial and error\u201d learning approach because there are some scenarios that are really hard and undesirable to replicate in real-life (like crashing an airplane or driving a car off a cliff (see Figure 2).<\/span><\/li>\n<\/ul>\n<p><strong><a href=\"https:\/\/storage.ning.com\/topology\/rest\/1.0\/file\/get\/3697118424?profile=original\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" src=\"https:\/\/storage.ning.com\/topology\/rest\/1.0\/file\/get\/3697118424?profile=RESIZE_710x\" class=\"align-full\"><\/a><\/strong><\/p>\n<p><strong>Figure<\/strong> <strong>2<\/strong><strong>:<\/strong> <a href=\"http:\/\/www.incompleteideas.net\/book\/ebook\/node28.html\"><strong>The Agent-Environment Interface<\/strong><\/a><\/p>\n<h1><strong>Autonomous Law #3: Optimizing Across Interdependent Subsystems<\/strong><\/h1>\n<ul>\n<li>Autonomous Law #3: Autonomous entities are complex systems comprised of numerous, sometimes conflicting subsystems that must work in concert to optimize its utility function without human intervention.<\/li>\n<\/ul>\n<p><span>For example, consider all the different subsystems which an autonomous vehicle must orchestrate across in order to achieve its operational objectives (see Figure 3).<\/span><\/p>\n<p><span><a href=\"https:\/\/storage.ning.com\/topology\/rest\/1.0\/file\/get\/3697118591?profile=original\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" src=\"https:\/\/storage.ning.com\/topology\/rest\/1.0\/file\/get\/3697118591?profile=RESIZE_710x\" class=\"align-full\"><\/a><\/span><\/p>\n<p><strong>Figure<\/strong> <strong><span>3<\/span><\/strong><strong>:<\/strong> <a href=\"http:\/\/www.scienceclarified.com\/As-Bi\/Automobile.html#ixzz62veAvjrS\"><strong>Automobile Subsystems<\/strong><\/a><\/p>\n<p>The major subsystems of a traditional internal combustion engine automobile against which an autonomous vehicle must orchestrate in order to optimize its utility function could include:<\/p>\n<ul>\n<li>The engine operates on internal combustion, meaning the fuel used for its power is burned inside of the engine. This burning occurs inside cylinders, which contain pistons. The pistons are attached, via a connecting rod, to a crankshaft.<\/li>\n<li>Fuel system operates on gasoline which must be properly mixed with air before it can be introduced into the cylinder. The combination of gasoline and air creates a greater explosion. The fuel pump draws the gasoline from the gas tank.<\/li>\n<li>Exhaust system. After the fuel is burned in the pistons, the gases and heat created must be released from the cylinder to make room for the next intake of fuel. The exhaust system is also responsible for reducing the noise caused by the explosion of the fuel.<\/li>\n<li>Once the pistons are firing and the crankshaft is spinning, this energy must be converted, or transmitted, to drive the wheels.<\/li>\n<li>The steering system allows the front wheels to guide the automobile.<\/li>\n<li>The suspension system enables the automobile to absorb the bumps and variations in the road surface, keeping the automobile stable.<\/li>\n<\/ul>\n<p>I<span>t\u2019s not (very) hard to optimize a system that is only comprised of itself.\u00a0 The braking system by itself could easily optimize its braking decisions if it only had to consider braking.\u00a0 But a car must carefully coordinate the interaction and interrelationships between braking, acceleration and navigational systems in order to successfully navigate a route (where success is defined by the utility function).<\/span><\/p>\n<h1><strong>Autonomous Law #4:<span>\u00a0<\/span> Carefully Defining the Utility Function<\/strong><\/h1>\n<ul>\n<li><span>Autonomous Law #4: For an entity to truly be autonomous, that entity must integrate, interoperate and optimize across all of these subsystems to achieve its <strong>utility function<\/strong> (maximizing rewards while minimizing penalties) without human intervention.<\/span><\/li>\n<\/ul>\n<p><span>To create an autonomous entity that understands how to orchestrate the appropriate actions across numerous subsystems within a dynamically changing environment without human intervention, the autonomous entity must seek to optimize the utility function across a variety of often-conflicting value dimensions.\u00a0 For example, an autonomous vehicle must seek to reduce travel time while reducing operational costs while ensuring pedestrian and traffic safety while reducing vehicle wear-and-tear while protecting the passengers while reducing its environment impact and carbon footprint while operating within the laws of driving\u2026all without human intervention (see Figure 4).<\/span><\/p>\n<p><span><a href=\"https:\/\/storage.ning.com\/topology\/rest\/1.0\/file\/get\/3697118859?profile=original\" target=\"_blank\" rel=\"noopener noreferrer\"><img decoding=\"async\" src=\"https:\/\/storage.ning.com\/topology\/rest\/1.0\/file\/get\/3697118859?profile=RESIZE_710x\" class=\"align-full\"><\/a><\/span><\/p>\n<p><strong>Figure<\/strong> <strong><span>4<\/span><\/strong><strong>:<\/strong> <a href=\"https:\/\/www.linkedin.com\/pulse\/why-utility-determination-critical-defining-ai-success-bill-schmarzo\/\"><strong>Utility Function Optimizing Across Conflicting Priorities<\/strong><\/a><\/p>\n<p><span>In order to make a holistic utility determination, collaboration across a diverse set of internal and external stakeholders is required to identify those short-term and long-term metrics and KPI\u2019s against which autonomous success will be measured.\u00a0 The careful weighing of the short-term and long-term metrics associated with the financial\/economic, operational, customer, society, environmental and spiritual dimensions must be taken into consideration if we are to successfully creation autonomous entities for the betterment of society.<\/span><\/p>\n<p>See the blog \u201c<a href=\"https:\/\/www.linkedin.com\/pulse\/why-utility-determination-critical-defining-ai-success-bill-schmarzo\/\">Why Utility Determination Is Critical to Defining AI Success<\/a>\u201d for my thoughts on how best to create the <strong>Utility Function<\/strong>.<\/p>\n<h1><strong>Creating Autonomous Summary<\/strong><\/h1>\n<p><span>Google\u2019s AI efforts is losing lots of money, and I mean LOTS:\u00a0 $154 million in 2016, $341 million in 2017, $572 million in 2018.\u00a0 And there were fears expressed in the article \u201c<\/span><a href=\"https:\/\/www.wired.com\/story\/deepminds-losses-future-artificial-intelligence\/\"><span>DeepMind&#8217;s Losses and the Future of Artificial Intelligence<\/span><\/a>\u201d that continued losses could actually discourage other companies to make AI investments.<\/p>\n<p><span>There aren\u2019t too many organizations around that can afford to lose $1B on a technology bet, so us mere mortal companies need a different approach.\u00a0 That approach is outlined in the 3 Horizons of Digital Transformation and its simple three step process:<\/span><\/p>\n<ul>\n<li><span>Step 1 (Horizon 1): Combine data (Big Data, IoT) with advanced analytics (AI, Machine Learning, Deep Learning) to drive operational excellence one use case at a time that not only gradually builds out your digital, data and analytics capabilities, but also should provide a compelling Return on Investment (ROI) for each use case.\u00a0 See the blog \u201cvalue engineering\u201d for more details on how to embrace this use case-by-use case approach for building out your organization\u2019s digital, data and analytics capabilities.\u00a0 \u201c<\/span><a href=\"https:\/\/www.linkedin.com\/pulse\/value-engineering-secret-sauce-data-science-success-bill-schmarzo\/\"><span>Value Engineering: The Secret Sauce for Data Science Success<\/span><\/a><span>.\u201d<\/span><\/li>\n<li><span>Step 2 (Horizon 2): Apply your data and analytic assets and insights (asset model propensities) created in Horizon 1 across the use cases that comprise a smart entity \u2013 farm, factory, city, hospital, theme park, airport.\u00a0 Incrementally creating \u201csmart\u201d entities where the learnings and outcomes from each operational use case improve the performance and outcomes of the other use cases.\u00a0 See the blog \u201c<\/span><a href=\"https:\/\/www.linkedin.com\/pulse\/3-stages-creating-smart-bill-schmarzo\/\"><span>3 Stages of Creating Smart<\/span><\/a><span>.\u201d<\/span><\/li>\n<li><span>Step 3 (Horizon 3): Leverage the 4 Laws of Autonomous Entities (which will undoubtedly grow as we learn more) to provide a roadmap for leveraging 1) today\u2019s market share grabbing operational excellence initiatives to 2) advance to tomorrow\u2019s IoT-induced smart entities that seek to optimize interrelated use case decisions to 3) the future world of autonomous entities who optimize their operations across a diverse utility function without human intervention.<\/span><\/li>\n<\/ul>\n<p><span>See, it\u2019s not that hard.\u00a0 And we don\u2019t need to lose $1B hoping that something valuable spits out the end.\u00a0 You can make bank today by leveraging the re-usable and refine-able data and analytic assets that accelerate time-to-value and de-risking use cases as the organization towards their autonomous goals.<\/span><\/p>\n<p><span>\u00a0<\/span><\/p>\n<p><span>Key blog points:<\/span><\/p>\n<ul>\n<li>Creating an autonomous entity is an aspirational vision that starts today by building out the organization\u2019s digital, data and analytic assets and capabilities.<\/li>\n<li><span>The \u201c3 Horizons of Digital Transformation\u201d provides a roadmap from today\u2019s Horizon 1 \u201coperational excellence\u201d initiatives to tomorrow\u2019s Horizon 3 \u201cautonomous markets\u201d vision.<\/span><\/li>\n<li><span>Autonomous Law #1: Autonomous entities needs lots and lots of granular (big) data from a robust and diverse set of use cases against which to apply Deep Reinforcement Learning techniques from to learn..<\/span><\/li>\n<li><span>Autonomous Law #2: Autonomous entities leverage fail-safe environments (games, simulations) to create Rational AI Agents that operate and learn across a robust number of use cases within a dynamically changing environment.<\/span><\/li>\n<li>Autonomous Law #3: Autonomous entities are complex systems comprised of numerous, sometimes conflicting subsystems that must work in concert to optimize its <strong>utility function<\/strong> without human intervention.<\/li>\n<li><span>Autonomous Law #4: For an entity to truly be autonomous, that entity must integrate, interoperate and optimize across all of these subsystems to achieve its <strong>utility function<\/strong> (maximizing rewards while minimizing penalties) without human intervention.<\/span><\/li>\n<li><span>Avoid big bang approach for creating autonomous entitles (unless you are<\/span> <a href=\"https:\/\/www.wired.com\/story\/deepminds-losses-future-artificial-intelligence\/\"><span>Google<\/span><\/a> <span>with a few extra $B to throw around)! Your AI models learn iteration from iteration.\u00a0 Your autonomous entities vision must operate in the same way.<\/span><\/li>\n<\/ul>\n<p><span>\u00a0<\/span><\/p>\n<\/div>\n<p><a href=\"https:\/\/www.datasciencecentral.com\/xn\/detail\/6448529:BlogPost:905399\">Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Author: Bill Schmarzo I continue to be infatuated with the potential of autonomous entities that self-monitor, self-diagnose, self-fix and self-learn without human intervention.\u00a0 The vision [&hellip;] <span class=\"read-more-link\"><a class=\"read-more\" href=\"https:\/\/www.aiproblog.com\/index.php\/2019\/11\/06\/creating-autonomous-entities-optimizing-systems-of-subsystems\/\">Read More<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":457,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[26],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/2783"}],"collection":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=2783"}],"version-history":[{"count":0,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/2783\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media\/456"}],"wp:attachment":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=2783"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=2783"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=2783"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}