{"id":1136,"date":"2018-10-09T06:37:35","date_gmt":"2018-10-09T06:37:35","guid":{"rendered":"https:\/\/www.aiproblog.com\/index.php\/2018\/10\/09\/automated-deep-learning-so-simple-anyone-can-do-it\/"},"modified":"2018-10-09T06:37:35","modified_gmt":"2018-10-09T06:37:35","slug":"automated-deep-learning-so-simple-anyone-can-do-it","status":"publish","type":"post","link":"https:\/\/www.aiproblog.com\/index.php\/2018\/10\/09\/automated-deep-learning-so-simple-anyone-can-do-it\/","title":{"rendered":"Automated Deep Learning \u2013 So Simple Anyone Can Do It"},"content":{"rendered":"

Author: William Vorhies<\/p>\n

\n

Summary:<\/em><\/strong>\u00a0 There are several things holding back our use of deep learning methods and chief among them is that they are complicated and hard.\u00a0 Now there are three platforms that offer Automated Deep Learning (ADL) so simple that almost anyone can do it.<\/em><\/p>\n

\u00a0<\/p>\n

<\/a>There are several things holding back our use of deep learning methods and chief among them is that they are complicated and hard.\u00a0<\/p>\n

A small percentage of our data science community has chosen the path of learning these new techniques, but it\u2019s a major departure both in problem type and technique from the predictive and prescriptive modeling that makes up 90% of what we get paid to do.<\/p>\n

Artificial intelligence, at least in the true sense of image, video, text, and speech recognition and processing is on everyone\u2019s lips but it\u2019s still hard to find a data scientist qualified to execute your project.<\/p>\n

Actually when I list image, video, text, and speech applications I\u2019m selling deep learning a little short.\u00a0 While these are the best known and perhaps most obvious applications, deep neural nets (DNNs) are also proving excellent at forecasting time series data, and also in complex traditional consumer propensity problems.<\/p>\n

Last December as I was listing my predictions for 2018<\/u><\/em><\/a>, I noted that Gartner had said that during 2018, DNNs would become a standard component in the toolbox of 80% of data scientists.\u00a0 My prediction was that while the first provider to accomplish this level of simplicity would certainly be richly rewarded, no way was it going to be 2018.\u00a0 It seems I was wrong.<\/p>\n

Here we are and it\u2019s only April and I\u2019ve recently been introduced to three different platforms that have the goal of making deep learning so easy, anyone (well at least any data scientist) can do it.<\/p>\n<\/p>\n

Minimum Requirements<\/strong><\/span><\/p>\n

All of the majors and several smaller companies offer greatly simplified tools for executing CNNs or RNN\/LSTMs, but these still require experimental hand tuning of the layer types and number, connectivity, nodes, and all the other hyperparameters that so often defeat initial success.\u00a0<\/p>\n

To be part of this group you need a truly one-click application that allows the average data scientists or even developer to build a successful image or text classifier.<\/p>\n

The quickest route to this goal is by transfer learning.\u00a0 In DL, transfer learning means taking a previously built successful, large, complex CNN or RNN\/LSTM model and using a new more limited data set to train against it.\u00a0<\/p>\n

Basically transfer learning, most used in image classification, summarizes the more complex model into fewer or previously trained categories.\u00a0 Transfer learning can\u2019t create classifications that weren\u2019t in the original model, but it can learn to create subsets or summary categories of what\u2019s there.<\/p>\n

The advantage is that the hyperparameter tuning has already been done so you know the model will train.\u00a0 More importantly, you can build a successful transfer model with just a few hundred labeled images in less than an hour.<\/p>\n

The real holy grail of AutoDL however, is fully automated hyperparameter tuning, not transfer learning.\u00a0 As you\u2019ll read below, some are on track, and others claim to already have succeeded.<\/p>\n<\/p>\n

Microsoft CustomVision.AI<\/strong><\/span><\/p>\n

Late in 2017 MS introduced a series of greatly simplified DL capabilities covering the full range of image, video, text, and speech under the banner of the Microsoft Cognitive Services.\u00a0 In January they introduced their fully automated platform, Microsoft Custom Vision Services (https:\/\/www.customvision.ai\/<\/a>).<\/span><\/p>\n

The platform is limited to image classifiers and promises to allow users to create robust CNN transfer models based on only a few images capitalizing on MS\u2019s huge existing library of large, complex, multi-image classifiers.<\/span><\/p>\n

Using the platform is extremely simple.\u00a0 You drag and drop your images onto the platform and press go.\u00a0 You\u2019ll need at least a pay-as-you-go Azure account and basic tech support runs $29\/mo.\u00a0 It\u2019s not clear how long the models take to train but since it\u2019s transfer learning it should be quite fast and therefore, we\u2019re guessing, inexpensive (but not free).<\/p>\n

During project setup you\u2019ll be asked to identify a general domain from which your image set will transfer learn and these currently are:<\/p>\n

    \n
  1. General<\/li>\n
  2. Food<\/li>\n
  3. Landmarks<\/li>\n
  4. Retail<\/li>\n
  5. Adult<\/li>\n
  6. General (compact)<\/li>\n
  7. Landmarks (compact)<\/li>\n
  8. Retail (compact)<\/li>\n<\/ol>\n

    While all these models will run from a restful API once trained, the last three categories (marked \u2018compact\u2019) can be exported to run off line on any iOS or Android edge device.\u00a0 Export is to the CoreML format for iOS 11 and to the TensorFlow format for Android. \u00a0This should entice a variety of app developers who may not be data scientist to add instant image classification to their device.<\/p>\n

    You can bet MS will be rolling out more complex features as fast as possible.<\/p>\n<\/p>\n

    Google Cloud AutoML<\/strong><\/span><\/p>\n

    Also in January, Google announced its similar entry Cloud AutoML<\/u><\/em><\/a>.\u00a0 The platform is in alpha and requires an invitation to participate.<\/p>\n

    Like Microsoft, the service utilizes transfer learning from Google\u2019s own prebuilt complex CNN classifiers.\u00a0 They recommend at least 100 images per label for transfer learning.\u00a0<\/p>\n

    <\/a>It\u2019s not clear at this point what categories of images will be allowed at launch, but user screens show guidance for general, face, logo, landmarks, and perhaps others.\u00a0 From screen shots shared by Google it appears these models train in the range of about 20 minutes to a few hours.<\/p>\n

    In the data we were able to find, use appears to be via API.\u00a0 There\u2019s no mention of export code for offline use.\u00a0 Early alpha users include Disney and Urban Outfitters.\u00a0<\/p>\n

    Anticipating that many new users won\u2019t have labeled data, Google offers access to its own human-labeling services for an additional fee.<\/p>\n

    Beyond transfer learning, all the majors including Google are pursuing automated ways of automating the optimal tuning of CNNs and RNNs.\u00a0 Handcrafted models are the norm today and are the reason so many often unsuccessful iterations are required.\u00a0<\/p>\n

    <\/a>Google calls this next technology Learn2Learn.\u00a0 Currently they are experimenting with RNNs to optimize layers, layer types, nodes, connections, and the other hyperparameters.\u00a0 Since this is basically very high speed random search the compute resources can be extreme.\u00a0<\/p>\n

    Next on the horizon is the use of evolutionary algorithms to do the same which are much more efficient in terms of time and compute.\u00a0 In a recent presentation, Google researchers showed good results from this approach but they were still taking 3 to 10 days to train just for the optimization.<\/p>\n<\/p>\n

    OneClick.AI<\/strong><\/span><\/p>\n

    OneClick.AI is an automated machine learning (AML) platform new in the market late in 2017 which includes both traditional algorithms and also deep learning algorithms.<\/p>\n

    OneClick.AI would be worth a look based on just its AML credentials which include blending, prep, feature engineering, and feature selection, followed by the traditional multi-models in parallel to identify a champion model.<\/p>\n

    However, what sets OneClick apart is that it includes both image and text DL algos with both transfer learning as well as fully automated hyperparameter tuning for de novo image or text deep learning models.<\/p>\n

    Unlike Google and Microsoft they are ready to deliver on both image and text.\u00a0 Beyond that, they blend DNNs with traditional algos in ensembles, and use DNNs for forecasting.\u00a0<\/p>\n

    Forecasting is a little explored area of use for DNNs but it\u2019s been shown to easily outperform other times series forecasters like ARIMA and ARIMAX.<\/p>\n

    For a platform with this complex offering of tools and techniques it maintains its claim to super easy one-click-data-in-model-out ease which I identify as the minimum requirement for Automated Machine Learning, but which also includes Automated Deep Learning.<\/p>\n

    The methods used for optimizing its deep learning models are proprietary, but Yuan Shen, Founder and CEO describes it as using AI to train AI, presumably a deep learning approach to optimization.<\/p>\n<\/p>\n

    Which is Better?<\/strong><\/span><\/p>\n

    It\u2019s much too early to expect much in the way of benchmarking but there is one example to offer, which comes from OneClick.AI.<\/p>\n

    In a hackathon earlier this year the group tested OneClick against Microsoft\u2019s CustomVision (Google AutoML wasn\u2019t available).\u00a0 Two image classification problems were tested.\u00a0 Tagging photos with:<\/p>\n

    Horses running or horses drinking water.<\/p>\n

    \u00a0<\/a><\/p>\n

    \u00a0Detecting photos with nudity.<\/p>\n

    \u00a0<\/a><\/p>\n

    The horse tagging task was multi-label classification, and the nudity detection task was binary classification. For each task they used 20 images for training, and another 20 for testing.<\/p>\n