{"id":1568,"date":"2019-01-13T18:00:43","date_gmt":"2019-01-13T18:00:43","guid":{"rendered":"https:\/\/www.aiproblog.com\/index.php\/2019\/01\/13\/three-must-own-books-for-deep-learning-practitioners\/"},"modified":"2019-01-13T18:00:43","modified_gmt":"2019-01-13T18:00:43","slug":"three-must-own-books-for-deep-learning-practitioners","status":"publish","type":"post","link":"https:\/\/www.aiproblog.com\/index.php\/2019\/01\/13\/three-must-own-books-for-deep-learning-practitioners\/","title":{"rendered":"Three Must-Own Books for Deep Learning Practitioners"},"content":{"rendered":"<p>Author: Jason Brownlee<\/p>\n<div>\n<p>Developing neural networks is often referred to as a dark art.<\/p>\n<p>The reason for this is that being skilled at developing neural network models comes from experience. There are no reliable methods to analytically calculate how to design a \u201cgood\u201d or \u201cbest\u201d model for your specific dataset. You must draw on experience and experiment in order to discover what works on your problem.<\/p>\n<p>A lot of this experience can come from actually developing neural networks on test problems.<\/p>\n<p>Nevertheless, many people have come before and recorded their discoveries, best practices, and preferred techniques. You can learn a lot about how to design and configure neural networks from some of the best books on the topic.<\/p>\n<p>In this post, you will discover the three books that I recommend reading and having next to you when developing neural networks for your datasets.<\/p>\n<p>Let\u2019s get started.<\/p>\n<h2>Three Recommended Books on Neural Networks<\/h2>\n<p>There are three books that I think you must own <em>physical copies<\/em> of if you are a neural network practitioner.<\/p>\n<p>They are:<\/p>\n<ol>\n<li><a href=\"https:\/\/amzn.to\/2CMttJI\">Neural Networks for Pattern Recognition<\/a>, 1995.<\/li>\n<li><a href=\"https:\/\/amzn.to\/2pW6hjI\">Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks<\/a>, 1999.<\/li>\n<li><a href=\"https:\/\/amzn.to\/2CjIBgH\">Deep Learning<\/a>, 2016.<\/li>\n<\/ol>\n<p>These books are references, not tutorials.<\/p>\n<p>You dip into them again and again before and during projects to ensure that you are getting everything you can out of your data and models.<\/p>\n<p>These are the books that I read and reference all the time. If you have books that you recommend when developing neural network models, please let me know in the comments below.<\/p>\n<p>Now, let\u2019s take a closer look at each book in turn.<\/p>\n<h2>Neural Networks for Pattern Recognition<\/h2>\n<p><a href=\"https:\/\/amzn.to\/2CMttJI\">Neural Networks for Pattern Recognition<\/a> by <a href=\"https:\/\/en.wikipedia.org\/wiki\/Christopher_Bishop\">Christopher Bishop<\/a> was released in 1995.<\/p>\n<div id=\"attachment_6836\" style=\"width: 349px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/amzn.to\/2CMttJI\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-6836\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2018\/10\/Neural-Networks-for-Pattern-Recognition.jpg\" alt=\"Neural Networks for Pattern Recognition\" width=\"339\" height=\"499\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2018\/10\/Neural-Networks-for-Pattern-Recognition.jpg 339w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2018\/10\/Neural-Networks-for-Pattern-Recognition-204x300.jpg 204w\" sizes=\"(max-width: 339px) 100vw, 339px\"><\/a><\/p>\n<p class=\"wp-caption-text\">Neural Networks for Pattern Recognition<\/p>\n<\/div>\n<p>This great book was followed about a decade later by the still classic textbook <a href=\"https:\/\/amzn.to\/2Elsy4k\">Pattern Recognition and Machine Learning<\/a> (fondly referred to as PRML). Christopher Bishop is both a professor at the University of Edinburgh and a director at <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/cmbishop\/\">Microsoft\u2019s Cambridge research lab<\/a>.<\/p>\n<p>This book is a classic in the field of neural networks. It is a handbook that handily captures both the state of theory at the time, and techniques that remain just as relevant today nearly 25 years later.<\/p>\n<p>Although reading the book cover to cover will provide you a robust foundation, I\u2019d instead encourage you to use it as a reference for getting the most out of your neural network models.<\/p>\n<p>I\u2019d recommend dipping into the following chapters as needed:<\/p>\n<ul>\n<li>Chapter 7: Parameter Optimization Algorithms<\/li>\n<li>Chapter 8: Pre-Processing and Feature Extraction<\/li>\n<li>Chapter 9: Learning and Generalization.<\/li>\n<\/ul>\n<p>Chapter 9 is worth the sticker price for the book alone, giving a laundry list of descriptions for regularization methods and ensemble methods you should be testing.<\/p>\n<p>I recommend this book because given the description of new methods almost daily, practitioners often forget the tried and true basics.<\/p>\n<p>I don\u2019t think this book is in print anymore, but you can find secondhand and international versions everywhere online.<\/p>\n<div class=\"woo-sc-hr\"><\/div>\n<p><center><\/p>\n<h3>Want Better Results with Deep Learning?<\/h3>\n<p>Take my free 7-day email crash course now (with sample code).<\/p>\n<p>Click to sign-up and also get a free PDF Ebook version of the course.<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.lpages.co\/leadbox\/1433e7773f72a2%3A164f8be4f346dc\/5764144745676800\/\" target=\"_blank\" style=\"background: rgb(255, 206, 10); color: rgb(255, 255, 255); text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-weight: bold; font-size: 16px; line-height: 20px; padding: 10px; display: inline-block; max-width: 300px; border-radius: 5px; text-shadow: rgba(0, 0, 0, 0.25) 0px -1px 1px; box-shadow: rgba(255, 255, 255, 0.5) 0px 1px 3px inset, rgba(0, 0, 0, 0.5) 0px 1px 3px;\">Download Your FREE Mini-Course<\/a><script data-leadbox=\"1433e7773f72a2:164f8be4f346dc\" data-url=\"https:\/\/machinelearningmastery.lpages.co\/leadbox\/1433e7773f72a2%3A164f8be4f346dc\/5764144745676800\/\" data-config=\"%7B%7D\" type=\"text\/javascript\" src=\"https:\/\/machinelearningmastery.lpages.co\/leadbox-1543333086.js\"><\/script><\/p>\n<p><\/center><\/p>\n<div class=\"woo-sc-hr\"><\/div>\n<h2>Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks<\/h2>\n<p><a href=\"https:\/\/amzn.to\/2pW6hjI\">Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks<\/a> by Russell Reed and <a href=\"https:\/\/robertmarks.org\/\">Robert Marks<\/a> was released in 1999.<\/p>\n<div id=\"attachment_6837\" style=\"width: 253px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/amzn.to\/2pW6hjI\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-6837 size-medium\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2018\/10\/Neural-Smithing-Supervised-Learning-in-Feedforward-Artificial-Neural-Networks-243x300.jpg\" alt=\"Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks\" width=\"243\" height=\"300\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2018\/10\/Neural-Smithing-Supervised-Learning-in-Feedforward-Artificial-Neural-Networks-243x300.jpg 243w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2018\/10\/Neural-Smithing-Supervised-Learning-in-Feedforward-Artificial-Neural-Networks-768x947.jpg 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2018\/10\/Neural-Smithing-Supervised-Learning-in-Feedforward-Artificial-Neural-Networks-830x1024.jpg 830w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2018\/10\/Neural-Smithing-Supervised-Learning-in-Feedforward-Artificial-Neural-Networks.jpg 1103w\" sizes=\"(max-width: 243px) 100vw, 243px\"><\/a><\/p>\n<p class=\"wp-caption-text\">Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks<\/p>\n<\/div>\n<p>I have a large soft spot for this book.<\/p>\n<p>I purchased it soon after it was released and used it as a reference for many of my own implementations of neural network algorithms through the 2000s.<\/p>\n<p>There are two things I like the most about this book:<\/p>\n<ul>\n<li>Code<\/li>\n<li>Plots<\/li>\n<\/ul>\n<p>The book uses mathematics and descriptions to explain concepts, but importantly they also use snippets of pseudocode or ANSI C to show how things work. This is invaluable the first time you\u2019re coding backpropagation of error or an activation functions.<\/p>\n<p>The book also uses plots of the decision surface models. This is invaluable to understand what the models are doing\/seeing during training under different learning algorithms and how things like regularization effect the model.<\/p>\n<p>There is perhaps an over focus on pruning methods given the authors interest in the area; nevertheless, I\u2019d recommend dipping into the following chapters when developing your own models:<\/p>\n<ul>\n<li>Chapter 14: Factors Influencing Generalization<\/li>\n<li>Chapter 15: Generalization Prediction and Assessment<\/li>\n<li>Chapter 16: Heuristics for Improving Generalization<\/li>\n<li>Chapter 17: Effects of Training with Noisy Inputs<\/li>\n<\/ul>\n<p>Although I recommend buying this book and having it next to you (always), Robert Marks has a reprint of the book on his website in HTML format:<\/p>\n<ul>\n<li><a href=\"https:\/\/robertmarks.org\/REPRINTS\/NS\/NS-html\/NSindex.htm\">Neural Smithing \u2013 Supervised Learning in Feedforward Artificial Neural Networks<\/a><\/li>\n<\/ul>\n<h2>Deep Learning<\/h2>\n<p><a href=\"https:\/\/amzn.to\/2CjIBgH\">Deep Learning<\/a> by Ian Goodfellow, Yoshua Bengio, and Aaron Courville was released in 2016.<\/p>\n<div id=\"attachment_6838\" style=\"width: 385px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/amzn.to\/2CjIBgH\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-6838\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2018\/10\/Deep-Learning.jpg\" alt=\"Deep Learning\" width=\"375\" height=\"499\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2018\/10\/Deep-Learning.jpg 375w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2018\/10\/Deep-Learning-225x300.jpg 225w\" sizes=\"(max-width: 375px) 100vw, 375px\"><\/a><\/p>\n<p class=\"wp-caption-text\">Deep Learning<\/p>\n<\/div>\n<p>This is the missing bridge between the classic books of the 1990s and modern deep learning.<\/p>\n<p>Importantly, neural networks are introduced with careful mention of the innovations and milestones that have made the field into what it is today. Specifically <em>Chapter 6: Deep Feedforward Networks<\/em> and <em>Section 6.6 Historical Notes<\/em>.<\/p>\n<p>There are three chapters that are must-reads for neural network practitioners; they are:<\/p>\n<ul>\n<li>Chapter 7: Regularization for Deep Learning<\/li>\n<li>Chapter 8: Optimization for Training Deep Models<\/li>\n<li>Chapter 11: Practical Methodology<\/li>\n<\/ul>\n<p>Chapter 11 especially is important as it ties together specific methods and how and when to use them in practice. It is by far worth the purchase price of the book alone.<\/p>\n<p>This is a must have. You need a physical copy of this book. Nevertheless, the entire text is available on the books website here:<\/p>\n<ul>\n<li><a href=\"https:\/\/www.deeplearningbook.org\/\">Deep Learning Book Website<\/a><\/li>\n<\/ul>\n<h2>Further Reading<\/h2>\n<p>This section provides more resources on the topic if you are looking to go deeper.<\/p>\n<h3>Books<\/h3>\n<ul>\n<li><a href=\"https:\/\/amzn.to\/2CMttJI\">Neural Networks for Pattern Recognition<\/a>, 1995.<\/li>\n<li><a href=\"https:\/\/amzn.to\/2pW6hjI\">Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks<\/a>, 1999.<\/li>\n<li><a href=\"https:\/\/amzn.to\/2CjIBgH\">Deep Learning<\/a>, 2016.<\/li>\n<\/ul>\n<h3>Additional Books<\/h3>\n<ul>\n<li><a href=\"https:\/\/amzn.to\/2Elsy4k\">Pattern Recognition and Machine Learning<\/a>, 2006.<\/li>\n<\/ul>\n<h3>Additional Links<\/h3>\n<ul>\n<li><a href=\"https:\/\/robertmarks.org\/REPRINTS\/NS\/NS-html\/NSindex.htm\">Neural Smithing Reprint<\/a><\/li>\n<li><a href=\"https:\/\/www.deeplearningbook.org\/\">Deep Learning Book Website<\/a><\/li>\n<\/ul>\n<h2>Summary<\/h2>\n<p>In this post, you discovered the three reference books that I think that every neural network practitioner must own.<\/p>\n<p>Do you use one or more of these books yourself?<br \/>\nWhat chapters do you reference heavily?<\/p>\n<p>Are there other books that you reference a lot?<br \/>\nLet me know below.<\/p>\n<p>The post <a rel=\"nofollow\" href=\"https:\/\/machinelearningmastery.com\/books-for-deep-learning-practitioners\/\">Three Must-Own Books for Deep Learning Practitioners<\/a> appeared first on <a rel=\"nofollow\" href=\"https:\/\/machinelearningmastery.com\/\">Machine Learning Mastery<\/a>.<\/p>\n<\/div>\n<p><a href=\"https:\/\/machinelearningmastery.com\/books-for-deep-learning-practitioners\/\">Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Author: Jason Brownlee Developing neural networks is often referred to as a dark art. The reason for this is that being skilled at developing neural [&hellip;] <span class=\"read-more-link\"><a class=\"read-more\" href=\"https:\/\/www.aiproblog.com\/index.php\/2019\/01\/13\/three-must-own-books-for-deep-learning-practitioners\/\">Read More<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":1569,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[24],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/1568"}],"collection":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=1568"}],"version-history":[{"count":0,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/1568\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media\/1569"}],"wp:attachment":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=1568"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=1568"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=1568"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}