Deep learning neural networks are challenging to configure and train.<\/p>\n
There are decades of tips and tricks spread across hundreds of research papers, source code, and in the heads of academics and practitioners.<\/p>\n
The book \u201cNeural Networks: Tricks of the Trade<\/a>\u201d originally published in 1998 and updated in 2012 at the cusp of the deep learning renaissance ties together the disparate tips and tricks into a single volume. It includes advice that is required reading for all deep learning neural network practitioners.<\/p>\n In this post, you will discover the book \u201cNeural Networks: Tricks of the Trade<\/em>\u201d that provides advice by neural network academics and practitioners on how to get the most out of your models.<\/p>\n After reading this post, you will know:<\/p>\n Let\u2019s get started.<\/p>\n Neural Networks \u2013 Tricks of the Trade<\/p>\n<\/div>\n Neural Networks: Tricks of the Trade is a collection of papers on techniques to get better performance from neural network models.<\/p>\n The first edition<\/a> was published in 1998 comprised of five parts and 17 chapters. The second edition<\/a> was published right on the cusp of the new deep learning renaissance in 2012 and includes three more parts and 13 new chapters.<\/p>\n If you are a deep learning practitioner, then it is a must read book.<\/p>\n I own and reference both editions.<\/p>\n The motivation for the book was to collate the empirical and theoretically grounded tips, tricks, and best practices used to get the best performance from neural network models in practice.<\/p>\n The author\u2019s concern is that many of the useful tips and tricks are tacit knowledge in the field, trapped in peoples heads, code bases, or at the end of conference papers and that beginners to the field should be aware of them.<\/p>\n It is our belief that researchers and practitioners acquire, through experience and word-of-mouth, techniques and heuristics that help them successfully apply neural networks to difficult real-world problems. [\u2026] they are usually hidden in people\u2019s heads or in the back pages of space-constrained conference papers.<\/p>\n<\/blockquote>\n The book is an effort to try to group the tricks together, after the success of a workshop at the 1996 NIPS conference with the same name.<\/p>\n This book is an outgrowth of a 1996 NIPS workshop called Tricks of the Trade whose goal was to begin the process of gathering and documenting these tricks. The interest that the workshop generated motivated us to expand our collection and compile it into this book.<\/p>\n<\/blockquote>\n \u2014 Page 1, Neural Networks: Tricks of the Trade<\/a>, Second Edition, 2012.<\/p>\n\n
![\"Neural](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2018\/11\/Neural-Networks-Tricks-of-the-Trade.jpg\")
<\/a><\/p>\nOverview<\/h2>\n
Motivation<\/h2>\n
\n
\n
Breakdown of First Edition<\/h2>\n