Summary:<\/em><\/strong>\u00a0 Here are our 6 predictions for data science, machine learning, and AI for 2018.\u00a0 Some are fast track and potentially disruptive, some take the hype off over blown claims and set realistic expectations for the coming year.<\/em><\/p>\n \u00a0<\/p>\n We\u2019ve been watching and reporting on these trends all year and we scoured the web and some of our professional contacts to find out what others are thinking.\u00a0 There are only a handful of trends and technologies that look to disrupt or speed ahead.\u00a0 These are probably the most interesting in any forecast.\u00a0 But it also valuable to discuss trends we think are a tad overblown and won\u2019t accelerate as fast as some others believe.\u00a0 So with a little of both, here\u2019s what we concluded.<\/p>\n \u00a0<\/p>\n Prediction 1:\u00a0 Both model production and data prep will become increasingly automated.\u00a0 Larger data science operations will converge on a single platform (of many available).\u00a0 Both of these trends are in response to the groundswell movement for efficiency and effectiveness.\u00a0 In a nutshell allowing fewer data scientists to do the work of many.<\/strong><\/span><\/p>\n The core challenge is that there remains a structural shortage of data scientists.\u00a0 Whenever a pain point like this emerges we expect the market to respond and these two elements are its response.\u00a0 Both come at this from slightly different angles.<\/p>\n The first is that although the great majority of fresh new data scientists have learned their trade in either R or Python that having a large team freelancing directly in code is extremely difficult to manage for consistency and accuracy, much less to debug.<\/p>\n All the way back in their 2016 Magic Quadrant for Advanced Analytic Platforms, Gartner called this out<\/u><\/em><\/a> and wouldn\u2019t even rate companies that failed to provide a Visual Composition Framework (drag-and-drop elements of code) as a critical requirement.\u00a0 Gartner is very explicit that working in code is incompatible with the large organization\u2019s need for quality, consistency, collaboration, speed, and ease of use.<\/p>\n Langley Eide, Chief Strategy Officer at Alteryx<\/a> offered this same prediction, that \u201cdata science will break free from code dependence.\u00a0 In<\/em><\/span> 2018, we\u2019ll see increased adoption of common frameworks for encoding, managing and deploying Machine Learning and analytic processes. The value of data science will become less about the code itself and more about the application of techniques.\u00a0<\/em> We\u2019ll see the need for a common, code-agnostic platform where LOB analysts and data scientists alike can preserve existing work and build new analytics going forward.\u201d<\/em><\/p>\n Gartner says that by 2020, more than 40% of data science tasks will be automated.\u00a0 Hardly a month goes by without a new platform contacting<\/u><\/em><\/a> me wanting to be recognized on this list.\u00a0 And if you look into the clients many have already acquired you will find a very impressive list of high volume data science shops in insurance, lending, telecoms, and the like.<\/p>\n Even large traditional platforms like SAS offer increasingly automated modules for high volume model creation and maintenance, and many of the smaller platforms like BigML have followed suite with greatly simplified if not fully automated user interfaces.<\/p>\n \u00a0<\/p>\n Prediction 2:\u00a0 Data Science continues to develop specialties that mean the mythical \u2018full stack\u2019 data scientist will disappear.<\/strong><\/span><\/p>\n First there is the split between specialists in deep learning and predictive analytics.\u00a0 It\u2019s possible now to devote your career to just CNNs or RNNs, work in Tensorflow, and never touch or understand a classical consumer preference model.<\/p>\n Similarly, the needs of different industries have so diverged in their special applications of predictive analytics that industry experience is just as important as data science skill.\u00a0 In telecoms and insurance it\u2019s about customer preference, retention, and rates.\u00a0 In ecommerce it\u2019s about recommenders, web logs, and click streams.\u00a0 In banking and credit you can make a career in anomaly detection for fraud and abuse.\u00a0 Whoever hires you is looking for these specific skills and experiences.<\/p>\n Separately there is the long overdue spinoff of the Data Engineer from the Data Scientist.\u00a0 This is identification of a separate skills path that only began to be recognized a little over a year ago.\u00a0 The skills the data engineer needs to set up an instance in AWS, or implement Spark Streaming, or simply to create a data lake are different from the analytical skills of the data scientist.\u00a0 Maybe 10 years ago there were data scientists who had these skills but that\u2019s akin to the early days of personal computers when some early computer geeks could actually assemble their own boxes.\u00a0 Not anymore.<\/p>\n \u00a0<\/p>\n Prediction 3:\u00a0 Non-Data Scientists will perform a greater volume of fairly sophisticated analytics than data scientists.<\/strong><\/span><\/p>\n As recently as a few years ago the idea of the Citizen Data Scientist was regarded as either humorous or dangerous.\u00a0 How could someone, no matter how motivated, without several years of training and experience be trusted to create predictive analytics on which the financial success of the company relies?<\/p>\n There is still a note of risk here.\u00a0 You certainly wouldn\u2019t want to assign a sensitive analytic project to someone just starting out with no training.\u00a0 But the reality is that advanced analytic platforms, blending platforms, and data viz platforms have simply become easier to use, specifically in response to the demands of this group of users.\u00a0 And why have platform developers paid so much attention?\u00a0 Because Gartner says this group will grow 5X as fast as the trained data scientist group, so that\u2019s where the money is.<\/p>\n There will always be a knowledge and experience gap between the two groups, but if you\u2019re managing the advanced analytics group for your company you know about the drive toward \u2018data democratization\u2019 which is a synonym for \u2018self-service\u2019.\u00a0 There will always be some risk here to be managed but a motivated LOB manager or experienced data analyst who has come up the learning curve can do some pretty sophisticated things on these new platforms.<\/p>\n Langley Eide, Chief Strategy Officer at Alteryx<\/a> suggests that we think of these users along a continuum from no-code to low-code to code-friendly.\u00a0 They are going to want a seat at our common analytic platforms.\u00a0 They will need supervision, but they will also produce a volume of good analytic work and at very least can leverage the time and skills of your data scientists.<\/span><\/p>\n \u00a0<\/p>\n Prediction 4:\u00a0 Deep learning is complicated and hard.\u00a0 Not many data scientists are skilled in this area and that will hold back the application of AI until the deep learning platforms are significantly simplified and productized.<\/strong><\/span><\/p>\n Gartner says that by 2018, deep neural networks will be a standard component of 80% of data scientists\u2019 tool boxes.\u00a0 I say, I\u2019ll take that bet, that\u2019s way too optimistic.<\/p>\n The folks trying to simplify deep learning are the major cloud and DL providers, Amazon, Microsoft, Google, Intel, NVDIA, and their friends.\u00a0 But as it stands today, first good luck finding a well-qualified data scientists with the skills to do this work (have you seen the salaries they have to pay to attract these folks?). \u00a0<\/p>\n Second, the platforms remain exceedingly complex and expensive to use.\u00a0 Training time for a model is measured in weeks unless you rent a large number of expensive GPU nodes, and still many of these models fail to train at all.\u00a0 The optimization of hyperparameters is poorly understood and I expect some are not even correctly recognized as yet.<\/p>\n We\u2019ll all look forward to using these DL tools when they become as reasonable to use as the other algorithms in our tool kit.\u00a0 The first provider to deliver that level of simplicity will be richly rewarded.\u00a0 It won\u2019t be in 2018.<\/p>\n \u00a0<\/p>\n Prediction 5:\u00a0 Despite the hype, penetration of AI and deep learning into the broader market will be relatively narrow and slower than you think.<\/strong><\/span><\/p>\n AI and deep learning seems to be headed everywhere at once and there are no shortages of articles on how or where to apply AI in every business.\u00a0 My sense is that these applications will come but much slower than most might expect.<\/p>\n First, what we understand as commercially ready deep learning driven AI is actually limited to two primary areas, text and speech processing, and image and video processing.\u00a0 Both these areas are sufficiently reliable to be commercially viable and are actively being adopted.<\/p>\n The primary appearance of AI outside of tech will continue to be NLP Chatbots, both as input and output to a variety of query systems ranging from customer service replacements to interfaces on our software and personal devices.\u00a0 As we wrote in our recent series on chatbots<\/u><\/em><\/a>, in 2015 only 25% of companies had even heard of chatbots.\u00a0 By 2017, 75% had plans to build one.\u00a0 Voice and text is rapidly becoming a user interface of choice in all our systems and 2018 will see a rapid implementation of that trend.<\/p>\n However, other aspects of deep learning AI like image and video recognition, outside of facial recognition is pretty limited.\u00a0 There will be some adoption of facial and gesture recognition but those aren\u2019t capabilities that are likely to delight customers at Macy\u2019s, Starbucks, or the grocery store.<\/p>\n There are some interesting emerging developments in using CNNs and RNNs to optimize software integration and other relatively obscure applications not likely to get much attention soon.\u00a0 And of course there are our self-driving cars based on reinforcement learning but I wouldn\u2019t camp out at your dealership in 2018.<\/p>\n \u00a0<\/p>\n Prediction 6:\u00a0 The public (and the government) will start to take a hard look at social and privacy implications of AI, both intended and unintended.<\/strong><\/span><\/p>\n In the US the good news is that the government hasn\u2019t yet stepped in to create regulations this draconian.\u00a0 Yes there have been restrictions placed on the algorithms and data we can use for some lending and health models in the name of transparency.\u00a0 This also makes these models less efficient and therefore more prone to error.\u00a0<\/p>\n Also, the public is rapidly realizing that AI is not currently able to identify rare events with sufficient accuracy to protect them.\u00a0 After touting their AI\u2019s ability to spot fake news, or to spot and delete hate speech or criminals trolling for underage children, Facebook, YouTube, Twitter, Instagram, and all the others have been rapidly fessing up that the only way to control this is with legions of human reviewers.\u00a0 This does need to be solved.<\/p>\n Still, IMHO on line tracking and even location tracking through our personal devices is worth the intrusion in terms of the efficiency and lower cost it creates.\u00a0 After all, the materials those algorithms present to you on line are more tailored to your tastes and since it reduces advertising cost, should also reduce the cost of what you buy.\u00a0 You can always opt out or turn off the device.\u00a0 However, this is small beer compared to what\u2019s coming.<\/p>\n Thanks largely to advances in deep learning applied to image recognition, researchers have recently demonstrated peer-reviewed and well-designed data science studies that show that they can determine criminals from non-criminals, and gays from straights<\/u><\/em><\/a> with remarkable levels of accuracy based only on facial recognition.<\/p>\n The principle issue is that while you can turn off your phone or opt out of on-line tracking that the proliferation of video cameras tracking and recording our faces makes it impossible to opt out of being placed in facial recognition databases.\u00a0 There have not yet been any widely publicized adverse impacts of these systems.\u00a0 But this is an unintended consequence waiting to happen.\u00a0 It could well happen in 2018.<\/p>\n \u00a0<\/p>\n \u00a0<\/p>\n About the author:\u00a0 Bill Vorhies is Editorial Director for Data Science Central and has practiced as a data scientist since 2001.\u00a0 He can be reached at:<\/p>\n Bill@DataScienceCentral.com<\/a><\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"![](\"http:\/\/api.ning.com\/files\/ycVWu*1f9fx28L*TFRnl*KKCEmKv0XgHD6gb*UIxkCarP1CSS39Ykad4ffHZj3VHYc7Ssa7XdGSbwr6SBIPKYMwsXQWwIMdp\/zoltar.jpg\")
<\/a>It\u2019s that time of year again when we do a look back in order to offer a look forward.\u00a0 What trends will speed up, what things will actually happen, and what things won\u2019t in the coming year for data science, machine learning, and AI.<\/p>\n<\/a>The second element of this prediction which I do believe is disruptive in its implications is the very rapid evolution of Automated Machine Learning.\u00a0 The first of these appeared just over a year ago and I\u2019ve written several times<\/u><\/em><\/a> about the now 7 or 8 competitors in this field such as DataRobot, Xpanse Analytics, and PurePredictive.\u00a0 These AML platforms have achieved one-click-data-in-model-out convenience with very good accuracy.\u00a0 Several of these vendors have also done a creditable job of automating data prep including feature creation and selection.<\/p>\n<\/a>This prediction may already have come true.\u00a0 There may be some smaller companies that haven\u2019t yet got the message but trying to find a single data scientist, regardless of degree or years of experience, who can do it all just isn\u2019t in the cards.\u00a0<\/p>\n<\/a>There\u2019s lots of talk about moving AI into the enterprise and certainly a lot of VC money backing AI startups.\u00a0 But almost exclusively these are companies looking to apply some capability of deep learning to a real world vertical or problem set, not looking to improve the tool.<\/p>\n<\/a>This hasn\u2019t been so much a tsunami as a steadily rising tide that started back with predictive analytics tracking our clicks, our locations, and even more.\u00a0 The EU has acted on its right to privacy and the right to be forgotten now documented in their new GDPR regs just now taking effect.<\/p>\n