Function optimization<\/strong> involves finding the input that results in the optimal value from an objective function.<\/p>\n Optimization algorithms navigate the search space of input variables in order to locate the optima, and both the shape of the objective function and behavior of the algorithm in the search space are opaque on real-world problems.<\/p>\n As such, it is common to study optimization algorithms using simple low-dimensional functions that can be easily visualized directly. Additionally, the samples in the input space of these simple functions made by an optimization algorithm can be visualized with their appropriate context.<\/p>\n Visualization of lower-dimensional functions and algorithm behavior on those functions can help to develop the intuitions that can carry over to more complex higher-dimensional function optimization problems later.<\/p>\n In this tutorial, you will discover how to create visualizations for function optimization in Python.<\/p>\n After completing this tutorial, you will know:<\/p>\n Let’s get started.<\/p>\n Visualization for Function Optimization in Python This tutorial is divided into three parts; they are:<\/p>\n Function optimization is a field of mathematics concerned with finding the inputs to a function that result in the optimal output for the function, typically a minimum or maximum value.<\/p>\n Optimization may be straightforward for simple differential functions where the solution can be calculated analytically. However, most functions we’re interested in solving in applied machine learning may or may not be well behaved and may be complex, nonlinear, multivariate, and non-differentiable.<\/p>\n As such, it is important to have an understanding of a wide range of different algorithms that can be used to address function optimization problems.<\/p>\n An important aspect of studying function optimization is understanding the objective function that is being optimized and understanding the behavior of an optimization algorithm over time.<\/p>\n Visualization plays an important role when getting started with function optimization.<\/p>\n We can select simple and well-understood test functions to study optimization algorithms. These simple functions can be plotted to understand the relationship between the input to the objective function and the output of the objective function and highlighting hills, valleys, and optima.<\/p>\n In addition, the samples selected from the search space by an optimization algorithm can also be plotted on top of plots of the objective function. These plots of algorithm behavior can provide insight and intuition into how specific optimization algorithms work and navigate a search space that can generalize to new problems in the future.<\/p>\n Typically, one-dimensional or two-dimensional functions are chosen to study optimization algorithms as they are easy to visualize using standard plots, like line plots and surface plots. We will explore both in this tutorial.<\/p>\n First, let’s explore how we might visualize a one-dimensional function optimization.<\/p>\n A one-dimensional function takes a single input variable and outputs the evaluation of that input variable.<\/p>\n Input variables are typically continuous, represented by a real-valued floating-point value. Often, the input domain is unconstrained, although for test problems we impose a domain of interest.<\/p>\n In this case we will explore function visualization with a simple x^2 objective function:<\/p>\n This has an optimal value with an input of x=0.0, which equals 0.0.<\/p>\n The example below implements this objective function and evaluates a single input.<\/p>\n Running the example evaluates the value 4.0 with the objective function, which equals 16.0.<\/p>\n The first thing we might want to do with a new function is define an input range of interest and sample the domain of interest using a uniform grid.<\/p>\n This sample will provide the basis for generating a plot later.<\/p>\n In this case, we will define a domain of interest around the optima of x=0.0 from x=-5.0 to x=5.0 and sample a grid of values in this range with 0.1 increments, such as -5.0, -4.9, -4.8, etc.<\/p>\n We can then evaluate each of the x values in our sample.<\/p>\n Finally, we can check some of the input and their corresponding outputs.<\/p>\n Tying this together, the complete example of sampling the input space and evaluating all points in the sample is listed below.<\/p>\n Running the example first generates a uniform sample of input points as we expected.<\/p>\n The input points are then evaluated using the objective function and finally, we can see a simple mapping of inputs to outputs of the objective function.<\/p>\n Now that we have some confidence in generating a sample of inputs and evaluating them with the objective function, we can look at generating plots of the function.<\/p>\n We could sample the input space randomly, but the benefit of a uniform line or grid of points is that it can be used to generate a smooth plot.<\/p>\n It is smooth because the points in the input space are ordered from smallest to largest. This ordering is important as we expect (hope) that the output of the objective function has a similar smooth relationship between values, e.g. small changes in input result in locally consistent (smooth) changes in the output of the function.<\/p>\n In this case, we can use the samples to generate a line plot of the objective function with the input points (x) on the x-axis of the plot and the objective function output (results) on the y-axis of the plot.<\/p>\n Tying this together, the complete example is listed below.<\/p>\n Running the example creates a line plot of the objective function.<\/p>\n We can see that the function has a large U-shape, called a parabola<\/a>. This is a common shape when studying curves, e.g. the study of calculus<\/a>.<\/p>\n Line Plot of a One-Dimensional Function<\/p>\n<\/div>\n The line is a construct. It is not really the function, just a smooth summary of the function. Always keep this in mind.<\/p>\n Recall that we, in fact, generated a sample of points in the input space and corresponding evaluation of those points.<\/p>\n As such, it would be more accurate to create a scatter plot of points; for example:<\/p>\n Running the example creates a scatter plot of the objective function.<\/p>\n We can see the familiar shape of the function, but we don’t gain anything from plotting the points directly.<\/p>\n The line and the smooth interpolation between the points it provides are more useful as we can draw other points on top of the line, such as the location of the optima or the points sampled by an optimization algorithm.<\/p>\n Scatter Plot of a One-Dimensional Function<\/p>\n<\/div>\n Next, let’s draw the line plot again and this time draw a point where the known optima of the function is located.<\/p>\n This can be helpful when studying an optimization algorithm as we might want to see how close an optimization algorithm can get to the optima.<\/p>\n First, we must define the input for the optima, then evaluate that point to give the x-axis and y-axis values for plotting.<\/p>\n We can then plot this point with any shape or color we like, in this case, a red square.<\/p>\n Tying this together, the complete example of creating a line plot of the function with the optima highlighted by a point is listed below.<\/p>\n Running the example creates the familiar line plot of the function, and this time, the optima of the function, e.g. the input that results in the minimum output of the function, is marked with a red square.<\/p>\n Line Plot of a One-Dimensional Function With Optima Marked by a Red Square<\/p>\n<\/div>\n This is a very simple function and the red square for the optima is easy to see.<\/p>\n Sometimes the function might be more complex, with lots of hills and valleys, and we might want to make the optima more visible.<\/p>\n In this case, we can draw a vertical line across the whole plot.<\/p>\n Tying this together, the complete example is listed below.<\/p>\n Running the example creates the same plot and this time draws a red line clearly marking the point in the input space that marks the optima.<\/p>\n Line Plot of a One-Dimensional Function With Optima Marked by a Red Line<\/p>\n<\/div>\n Finally, we might want to draw the samples of the input space selected by an optimization algorithm.<\/p>\n We will simulate these samples with random points drawn from the input domain.<\/p>\n We can then plot this sample, in this case using small black circles.<\/p>\n The complete example of creating a line plot of a function with the optima marked by a red line and an algorithm sample drawn with small black dots is listed below.<\/p>\n Running the example creates the line plot of the domain and marks the optima with a red line as before.<\/p>\n This time, the sample from the domain selected by an algorithm (really a random sample of points) is drawn with black dots.<\/p>\n We can imagine that a real optimization algorithm will show points narrowing in on the domain as it searches down-hill from a starting point.<\/p>\n Line Plot of a One-Dimensional Function With Optima Marked by a Red Line and Samples Shown with Black Dots<\/p>\n<\/div>\n Next, let’s look at how we might perform similar visualizations for the optimization of a two-dimensional function.<\/p>\n A two-dimensional function is a function that takes two input variables, e.g. x<\/em> and y<\/em>.<\/p>\n We can use the same x^2<\/em> function and scale it up to be a two-dimensional function; for example:<\/p>\n This has an optimal value with an input of [x=0.0, y=0.0], which equals 0.0.<\/p>\n The example below implements this objective function and evaluates a single input.<\/p>\n Running the example evaluates the point [x=4, y=4], which equals 32.<\/p>\n Next, we need a way to sample the domain so that we can, in turn, sample the objective function.<\/p>\n A common way for sampling a two-dimensional function is to first generate a uniform sample along each variable, x<\/em> and y<\/em>, then use these two uniform samples to create a grid of samples, called a mesh grid<\/a>.<\/p>\n This is not a two-dimensional array across the input space; instead, it is two two-dimensional arrays that, when used together, define a grid across the two input variables.<\/p>\n This is achieved by duplicating the entire x<\/em> sample array for each y<\/em> sample point and similarly duplicating the entire y<\/em> sample array for each x<\/em> sample point.<\/p>\n This can be achieved using the meshgrid() NumPy function<\/a>; for example:<\/p>\n We can then evaluate each pair of points using our objective function.<\/p>\n Finally, we can review the mapping of some of the inputs to their corresponding output values.<\/p>\n The example below demonstrates how we can create a uniform sample grid across the two-dimensional input space and objective function.<\/p>\n Running the example first summarizes some points in the mesh grid, then the objective function evaluation for some points.<\/p>\n Finally, we enumerate coordinates in the two-dimensional input space and their corresponding function evaluation.<\/p>\n Now that we are familiar with how to sample the input space and evaluate points, let’s look at how we might plot the function.<\/p>\n A popular plot for two-dimensional functions is a contour plot<\/a>.<\/p>\n This plot creates a flat representation of the objective function outputs for each x and y coordinate where the color and contour lines indicate the relative value or height of the output of the objective function.<\/p>\n This is just like a contour map of a landscape where mountains can be distinguished from valleys.<\/p>\n This can be achieved using the contour() Matplotlib<\/a> function that takes the mesh grid and the evaluation of the mesh grid as input directly.<\/p>\n We can then specify the number of levels to draw on the contour and the color scheme to use. In this case, we will use 50 levels and a popular “jet<\/em>” color scheme where low-levels use a cold color scheme (blue) and high-levels use a hot color scheme (red).<\/p>\n Tying this together, the complete example of creating a contour plot of the two-dimensional objective function is listed below.<\/p>\n Running the example creates the contour plot.<\/p>\n We can see that the more curved parts of the surface around the edges have more contours to show the detail, and the less curved parts of the surface in the middle have fewer contours.<\/p>\n We can see that the lowest part of the domain is the middle, as expected.<\/p>\n Contour Plot of a Two-Dimensional Objective Function<\/p>\n<\/div>\n It is also helpful to color the plot between the contours to show a more complete surface.<\/p>\n Again, the colors are just a simple linear interpolation, not the true function evaluation. This must be kept in mind on more complex functions where fine detail will not be shown.<\/p>\n We can fill the contour plot using the contourf() version of the function<\/a> that takes the same arguments.<\/p>\n We can also show the optima on the plot, in this case as a white star that will stand out against the blue background color of the lowest part of the plot.<\/p>\n Tying this together, the complete example of a filled contour plot with the optima marked is listed below.<\/p>\n Running the example creates the filled contour plot that gives a better idea of the shape of the objective function.<\/p>\n The optima at [x=0, y=0] is then marked clearly with a white star.<\/p>\n Filled Contour Plot of a Two-Dimensional Objective Function With Optima Marked by a White Star<\/p>\n<\/div>\n We may want to show the progress of an optimization algorithm to get an idea of its behavior in the context of the shape of the objective function.<\/p>\n In this case, we can simulate the points chosen by an optimization algorithm with random coordinates in the input space.<\/p>\n These points can then be plotted directly as black circles and their context color can give an idea of their relative quality.<\/p>\n Tying this together, the complete example of a filled contour plot with optimal and input sample plotted is listed below.<\/p>\n Running the example, we can see the filled contour plot as before with the optima marked.<\/p>\n We can now see the sample drawn as black dots and their surrounding color and relative distance to the optima gives an idea of how close the algorithm (random points in this case) got to solving the problem.<\/p>\n Filled Contour Plot of a Two-Dimensional Objective Function With Optima and Input Sample Marked<\/p>\n<\/div>\n Finally, we may want to create a three-dimensional plot of the objective function to get a fuller idea of the curvature of the function.<\/p>\n This can be achieved using the plot_surface() Matplotlib function<\/a>, that, like the contour plot, takes the mesh grid and function evaluation directly.<\/p>\n The complete example of creating a surface plot is listed below.<\/p>\n Running the example creates a three-dimensional surface plot of the objective function.<\/p>\n Surface Plot of a Two-Dimensional Objective Function<\/p>\n<\/div>\n Additionally, the plot is interactive, meaning that you can use the mouse to drag the perspective on the surface around and view it from different angles.<\/p>\n Surface Plot From a Different Angle of a Two-Dimensional Objective Function<\/p>\n<\/div>\n This section provides more resources on the topic if you are looking to go deeper.<\/p>\n In this tutorial, you discovered how to create visualizations for function optimization in Python.<\/p>\n Specifically, you learned:<\/p>\n Do you have any questions?<\/strong> The post Visualization for Function Optimization in Python<\/a> appeared first on Machine Learning Mastery<\/a>.<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Author: Jason Brownlee Function optimization involves finding the input that results in the optimal value from an objective function. Optimization algorithms navigate the search space […] Read More<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":4299,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[24],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/4298"}],"collection":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=4298"}],"version-history":[{"count":0,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/4298\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media\/4299"}],"wp:attachment":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=4298"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=4298"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=4298"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
![\"Visualization](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/01\/Visualization-for-Function-Optimization-in-Python.jpg\")
<\/p>\n
Photo by Nathalie<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
\n
\n
\n
Visualization for Function Optimization<\/h2>\n
Visualize 1D Function Optimization<\/h2>\n
Test Function<\/h3>\n
\n
# example of a 1d objective function\r\n\r\n# objective function\r\ndef objective(x):\r\n\treturn x**2.0\r\n\r\n# evaluate inputs to the objective function\r\nx = 4.0\r\nresult = objective(x)\r\nprint('f(%.3f) = %.3f' % (x, result))<\/pre>\nf(4.000) = 16.000<\/pre>\n<\/p>\n
Sample the Test Function<\/h3>\n
...\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\ninputs = arange(r_min, r_max, 0.1)\r\n# summarize some of the input domain\r\nprint(inputs[:5])<\/pre>\n
...\r\n# compute targets\r\nresults = objective(inputs)\r\n# summarize some of the results\r\nprint(results[:5])<\/pre>\n
...\r\n# create a mapping of some inputs to some results\r\nfor i in range(5):\r\n\tprint('f(%.3f) = %.3f' % (inputs[i], results[i]))<\/pre>\n# sample 1d objective function\r\nfrom numpy import arange\r\n\r\n# objective function\r\ndef objective(x):\r\n\treturn x**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\ninputs = arange(r_min, r_max, 0.1)\r\n# summarize some of the input domain\r\nprint(inputs[:5])\r\n# compute targets\r\nresults = objective(inputs)\r\n# summarize some of the results\r\nprint(results[:5])\r\n# create a mapping of some inputs to some results\r\nfor i in range(5):\r\n\tprint('f(%.3f) = %.3f' % (inputs[i], results[i]))<\/pre>\n[-5. -4.9 -4.8 -4.7 -4.6]\r\n[25. 24.01 23.04 22.09 21.16]\r\nf(-5.000) = 25.000\r\nf(-4.900) = 24.010\r\nf(-4.800) = 23.040\r\nf(-4.700) = 22.090\r\nf(-4.600) = 21.160<\/pre>\n
Line Plot of Test Function<\/h3>\n
...\r\n# create a line plot of input vs result\r\npyplot.plot(inputs, results)\r\n# show the plot\r\npyplot.show()<\/pre>\n
# line plot of input vs result for a 1d objective function\r\nfrom numpy import arange\r\nfrom matplotlib import pyplot\r\n\r\n# objective function\r\ndef objective(x):\r\n\treturn x**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\ninputs = arange(r_min, r_max, 0.1)\r\n# compute targets\r\nresults = objective(inputs)\r\n# create a line plot of input vs result\r\npyplot.plot(inputs, results)\r\n# show the plot\r\npyplot.show()<\/pre>\n
![\"Line](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Line-Plot-of-a-One-Dimensional-Function.png\")
<\/p>\nScatter Plot of Test Function<\/h3>\n
# scatter plot of input vs result for a 1d objective function\r\nfrom numpy import arange\r\nfrom matplotlib import pyplot\r\n\r\n# objective function\r\ndef objective(x):\r\n\treturn x**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\ninputs = arange(r_min, r_max, 0.1)\r\n# compute targets\r\nresults = objective(inputs)\r\n# create a scatter plot of input vs result\r\npyplot.scatter(inputs, results)\r\n# show the plot\r\npyplot.show()<\/pre>\n
![\"Scatter](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Scatter-Plot-of-a-One-Dimensional-Function.png\")
<\/p>\nLine Plot with Marked Optima<\/h3>\n
...\r\n# define the known function optima\r\noptima_x = 0.0\r\noptima_y = objective(optima_x)<\/pre>\n
...\r\n# draw the function optima as a red square\r\npyplot.plot([optima_x], [optima_y], 's', color='r')<\/pre>\n
# line plot of input vs result for a 1d objective function and show optima\r\nfrom numpy import arange\r\nfrom matplotlib import pyplot\r\n\r\n# objective function\r\ndef objective(x):\r\n\treturn x**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\ninputs = arange(r_min, r_max, 0.1)\r\n# compute targets\r\nresults = objective(inputs)\r\n# create a line plot of input vs result\r\npyplot.plot(inputs, results)\r\n# define the known function optima\r\noptima_x = 0.0\r\noptima_y = objective(optima_x)\r\n# draw the function optima as a red square\r\npyplot.plot([optima_x], [optima_y], 's', color='r')\r\n# show the plot\r\npyplot.show()<\/pre>\n
![\"Line](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Line-Plot-of-a-One-Dimensional-Function-with-Optima-Marked-by-a-Red-Square.png\")
<\/p>\n...\r\n# draw a vertical line at the optimal input\r\npyplot.axvline(x=optima_x, ls='--', color='red')<\/pre>\n
# line plot of input vs result for a 1d objective function and show optima as line\r\nfrom numpy import arange\r\nfrom matplotlib import pyplot\r\n\r\n# objective function\r\ndef objective(x):\r\n\treturn x**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\ninputs = arange(r_min, r_max, 0.1)\r\n# compute targets\r\nresults = objective(inputs)\r\n# create a line plot of input vs result\r\npyplot.plot(inputs, results)\r\n# define the known function optima\r\noptima_x = 0.0\r\n# draw a vertical line at the optimal input\r\npyplot.axvline(x=optima_x, ls='--', color='red')\r\n# show the plot\r\npyplot.show()<\/pre>\n
![\"Line](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Line-Plot-of-a-One-Dimensional-Function-with-Optima-Marked-by-a-Red-Line.png\")
<\/p>\nLine Plot with Samples<\/h3>\n
...\r\n# simulate a sample made by an optimization algorithm\r\nseed(1)\r\nsample = r_min + rand(10) * (r_max - r_min)\r\n# evaluate the sample\r\nsample_eval = objective(sample)<\/pre>\n
...\r\n# plot the sample as black circles\r\npyplot.plot(sample, sample_eval, 'o', color='black')<\/pre>\n
# line plot of domain for a 1d function with optima and algorithm sample\r\nfrom numpy import arange\r\nfrom numpy.random import seed\r\nfrom numpy.random import rand\r\nfrom matplotlib import pyplot\r\n\r\n# objective function\r\ndef objective(x):\r\n\treturn x**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\ninputs = arange(r_min, r_max, 0.1)\r\n# compute targets\r\nresults = objective(inputs)\r\n# simulate a sample made by an optimization algorithm\r\nseed(1)\r\nsample = r_min + rand(10) * (r_max - r_min)\r\n# evaluate the sample\r\nsample_eval = objective(sample)\r\n# create a line plot of input vs result\r\npyplot.plot(inputs, results)\r\n# define the known function optima\r\noptima_x = 0.0\r\n# draw a vertical line at the optimal input\r\npyplot.axvline(x=optima_x, ls='--', color='red')\r\n# plot the sample as black circles\r\npyplot.plot(sample, sample_eval, 'o', color='black')\r\n# show the plot\r\npyplot.show()<\/pre>\n
![\"Line](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Line-Plot-of-a-One-Dimensional-Function-with-Optima-Marked-by-a-Red-Line-and-Samples-Shown-with-Black-Dots.png\")
<\/p>\nVisualize 2D Function Optimization<\/h2>\n
Test Function<\/h3>\n
\n
# example of a 2d objective function\r\n\r\n# objective function\r\ndef objective(x, y):\r\n\treturn x**2.0 + y**2.0\r\n\r\n# evaluate inputs to the objective function\r\nx = 4.0\r\ny = 4.0\r\nresult = objective(x, y)\r\nprint('f(%.3f, %.3f) = %.3f' % (x, y, result))<\/pre>\nf(4.000, 4.000) = 32.000<\/pre>\n
Sample Test Function<\/h3>\n
...\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\nxaxis = arange(r_min, r_max, 0.1)\r\nyaxis = arange(r_min, r_max, 0.1)\r\n# create a mesh from the axis\r\nx, y = meshgrid(xaxis, yaxis)\r\n# summarize some of the input domain\r\nprint(x[:5, :5])<\/pre>\n
...\r\n# compute targets\r\nresults = objective(x, y)\r\n# summarize some of the results\r\nprint(results[:5, :5])<\/pre>\n
...\r\n# create a mapping of some inputs to some results\r\nfor i in range(5):\r\n\tprint('f(%.3f, %.3f) = %.3f' % (x[i,0], y[i,0], results[i,0]))<\/pre>\n# sample 2d objective function\r\nfrom numpy import arange\r\nfrom numpy import meshgrid\r\n\r\n# objective function\r\ndef objective(x, y):\r\n\treturn x**2.0 + y**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\nxaxis = arange(r_min, r_max, 0.1)\r\nyaxis = arange(r_min, r_max, 0.1)\r\n# create a mesh from the axis\r\nx, y = meshgrid(xaxis, yaxis)\r\n# summarize some of the input domain\r\nprint(x[:5, :5])\r\n# compute targets\r\nresults = objective(x, y)\r\n# summarize some of the results\r\nprint(results[:5, :5])\r\n# create a mapping of some inputs to some results\r\nfor i in range(5):\r\n\tprint('f(%.3f, %.3f) = %.3f' % (x[i,0], y[i,0], results[i,0]))<\/pre>\n[[-5. -4.9 -4.8 -4.7 -4.6]\r\n [-5. -4.9 -4.8 -4.7 -4.6]\r\n [-5. -4.9 -4.8 -4.7 -4.6]\r\n [-5. -4.9 -4.8 -4.7 -4.6]\r\n [-5. -4.9 -4.8 -4.7 -4.6]]\r\n[[50. 49.01 48.04 47.09 46.16]\r\n [49.01 48.02 47.05 46.1 45.17]\r\n [48.04 47.05 46.08 45.13 44.2 ]\r\n [47.09 46.1 45.13 44.18 43.25]\r\n [46.16 45.17 44.2 43.25 42.32]]\r\nf(-5.000, -5.000) = 50.000\r\nf(-5.000, -4.900) = 49.010\r\nf(-5.000, -4.800) = 48.040\r\nf(-5.000, -4.700) = 47.090\r\nf(-5.000, -4.600) = 46.160<\/pre>\n
Contour Plot of Test Function<\/h3>\n
...\r\n# create a contour plot with 50 levels and jet color scheme\r\npyplot.contour(x, y, results, 50, alpha=1.0, cmap='jet')\r\n# show the plot\r\npyplot.show()<\/pre>\n
# create a contour plot with 50 levels and jet color scheme\r\npyplot.contour(x, y, results, 50, alpha=1.0, cmap='jet')\r\n# show the plot\r\npyplot.show()\r\n\r\nTying this together, the complete example of creating a contour plot of the two-dimensional objective function is listed below.\r\n\r\n# contour plot for 2d objective function\r\nfrom numpy import arange\r\nfrom numpy import meshgrid\r\nfrom matplotlib import pyplot\r\n\r\n# objective function\r\ndef objective(x, y):\r\n\treturn x**2.0 + y**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\nxaxis = arange(r_min, r_max, 0.1)\r\nyaxis = arange(r_min, r_max, 0.1)\r\n# create a mesh from the axis\r\nx, y = meshgrid(xaxis, yaxis)\r\n# compute targets\r\nresults = objective(x, y)\r\n# create a contour plot with 50 levels and jet color scheme\r\npyplot.contour(x, y, results, 50, alpha=1.0, cmap='jet')\r\n# show the plot\r\npyplot.show()<\/pre>\n
![\"Contour](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Contour-Plot-of-a-Two-Dimensional-Objective-Function.png\")
<\/p>\nFilled Contour Plot of Test Function<\/h3>\n
...\r\n# create a filled contour plot with 50 levels and jet color scheme\r\npyplot.contourf(x, y, results, levels=50, cmap='jet')<\/pre>\n
...\r\n# define the known function optima\r\noptima_x = [0.0, 0.0]\r\n# draw the function optima as a white star\r\npyplot.plot([optima_x[0]], [optima_x[1]], '*', color='white')<\/pre>\n
# filled contour plot for 2d objective function and show the optima\r\nfrom numpy import arange\r\nfrom numpy import meshgrid\r\nfrom matplotlib import pyplot\r\n\r\n# objective function\r\ndef objective(x, y):\r\n\treturn x**2.0 + y**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\nxaxis = arange(r_min, r_max, 0.1)\r\nyaxis = arange(r_min, r_max, 0.1)\r\n# create a mesh from the axis\r\nx, y = meshgrid(xaxis, yaxis)\r\n# compute targets\r\nresults = objective(x, y)\r\n# create a filled contour plot with 50 levels and jet color scheme\r\npyplot.contourf(x, y, results, levels=50, cmap='jet')\r\n# define the known function optima\r\noptima_x = [0.0, 0.0]\r\n# draw the function optima as a white star\r\npyplot.plot([optima_x[0]], [optima_x[1]], '*', color='white')\r\n# show the plot\r\npyplot.show()<\/pre>\n
![\"Filled](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Filled-Contour-Plot-of-a-Two-Dimensional-Objective-Function-With-Optima-Marked-by-a-White-Star.png\")
<\/p>\nFilled Contour Plot of Test Function with Samples<\/h3>\n
...\r\n# simulate a sample made by an optimization algorithm\r\nseed(1)\r\nsample_x = r_min + rand(10) * (r_max - r_min)\r\nsample_y = r_min + rand(10) * (r_max - r_min)<\/pre>\n
...\r\n# plot the sample as black circles\r\npyplot.plot(sample_x, sample_y, 'o', color='black')<\/pre>\n
# filled contour plot for 2d objective function and show the optima and sample\r\nfrom numpy import arange\r\nfrom numpy import meshgrid\r\nfrom numpy.random import seed\r\nfrom numpy.random import rand\r\nfrom matplotlib import pyplot\r\n\r\n# objective function\r\ndef objective(x, y):\r\n\treturn x**2.0 + y**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\nxaxis = arange(r_min, r_max, 0.1)\r\nyaxis = arange(r_min, r_max, 0.1)\r\n# create a mesh from the axis\r\nx, y = meshgrid(xaxis, yaxis)\r\n# compute targets\r\nresults = objective(x, y)\r\n# simulate a sample made by an optimization algorithm\r\nseed(1)\r\nsample_x = r_min + rand(10) * (r_max - r_min)\r\nsample_y = r_min + rand(10) * (r_max - r_min)\r\n# create a filled contour plot with 50 levels and jet color scheme\r\npyplot.contourf(x, y, results, levels=50, cmap='jet')\r\n# define the known function optima\r\noptima_x = [0.0, 0.0]\r\n# draw the function optima as a white star\r\npyplot.plot([optima_x[0]], [optima_x[1]], '*', color='white')\r\n# plot the sample as black circles\r\npyplot.plot(sample_x, sample_y, 'o', color='black')\r\n# show the plot\r\npyplot.show()<\/pre>\n
![\"Filled](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Filled-Contour-Plot-of-a-Two-Dimensional-Objective-Function-With-Optima-and-Input-Sample-Marked.png\")
<\/p>\nSurface Plot of Test Function<\/h3>\n
...\r\n# create a surface plot with the jet color scheme\r\nfigure = pyplot.figure()\r\naxis = figure.gca(projection='3d')\r\naxis.plot_surface(x, y, results, cmap='jet')<\/pre>\n
# surface plot for 2d objective function\r\nfrom numpy import arange\r\nfrom numpy import meshgrid\r\nfrom matplotlib import pyplot\r\nfrom mpl_toolkits.mplot3d import Axes3D\r\n\r\n# objective function\r\ndef objective(x, y):\r\n\treturn x**2.0 + y**2.0\r\n\r\n# define range for input\r\nr_min, r_max = -5.0, 5.0\r\n# sample input range uniformly at 0.1 increments\r\nxaxis = arange(r_min, r_max, 0.1)\r\nyaxis = arange(r_min, r_max, 0.1)\r\n# create a mesh from the axis\r\nx, y = meshgrid(xaxis, yaxis)\r\n# compute targets\r\nresults = objective(x, y)\r\n# create a surface plot with the jet color scheme\r\nfigure = pyplot.figure()\r\naxis = figure.gca(projection='3d')\r\naxis.plot_surface(x, y, results, cmap='jet')\r\n# show the plot\r\npyplot.show()<\/pre>\n
![\"Surface](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Surface-Plot-of-a-Two-Dimensional-Objective-Function.png\")
<\/p>\n![\"Surface](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/09\/Surface-Plot-From-a-Different-Angle-of-a-Two-Dimensional-Objective-Function.png\")
<\/p>\nFurther Reading<\/h2>\n
APIs<\/h3>\n
\n
Articles<\/h3>\n
\n
Summary<\/h2>\n
\n
\nAsk your questions in the comments below and I will do my best to answer.<\/p>\n