The gradient of a function is ,
1. The gradient points to direction is increasing.
By Taylor Theorem, for near ,
For maximizing , we can choose a good , which means should be moved to the direction is increasing. Note that is maximized when is maximized. As is the inner product of two vectors.
where is the angle between and . It is maximized when . In other words, when and have the same direction, it is maximized. Therefore, should be moved to direction to locally maximized . For example, consider and for , . Then their gradients are and .
Their gradient point to the direction each is increasing at the point . Moreover, points to the direction is decreasing.
2. The gradient is perpendicular to the tangent plane in terms of an implicit function.
The gradient has the different meaning for explicit and implicit functions
- The gradient of an explicit function means the tangent vector at .
- The gradient of an implicit function means the normal vector of the tangent plane at .
For instance, consider . Then its gradient is . The total derivative of is , so . Since is the tangent of , is perpendicular to this.
For another example, consider . Then its gradient is . The total derivative of is , so . Since is the tangent of , is perpendicular to this.
 Michael T. Heath, Scientific Computing: An Introductory Survey. 2nd Edition, McGraw-Hill Higher Education.