summaryrefslogtreecommitdiffstats
path: root/docs/user-guide/07-autodiff.md
diff options
context:
space:
mode:
authorYong He <yonghe@outlook.com>2023-02-08 13:35:43 -0800
committerGitHub <noreply@github.com>2023-02-08 13:35:43 -0800
commit6bbd67390e9693d6a60221a030270d5ab67edfb9 (patch)
treee60c1c9f524b6391849aae777e518ff518a3df75 /docs/user-guide/07-autodiff.md
parentf2e564c8b739174f1774c152a87ef73aa8188468 (diff)
Update 07-autodiff.md
Diffstat (limited to 'docs/user-guide/07-autodiff.md')
-rw-r--r--docs/user-guide/07-autodiff.md4
1 files changed, 2 insertions, 2 deletions
diff --git a/docs/user-guide/07-autodiff.md b/docs/user-guide/07-autodiff.md
index 7d0624cf3..d72299229 100644
--- a/docs/user-guide/07-autodiff.md
+++ b/docs/user-guide/07-autodiff.md
@@ -118,7 +118,7 @@ $$Y = f_1 \circ f_2 \circ \cdots \circ f_n(\mathbf{w}_0)$$
Where $$\mathbf{w}_0$$ is the first layer of parameters.
### Forward Propagation of Derivatives
-When developing and training such a system, we are typically interested in evaluating the partial derivative of the system output with regard to some parameter $$\omega$$. To do so we can utilize the forward and backward derivative propagation functions for each $$f_i$$. Where the forward derivative propagation function is defined by:
+When developing and training such a system, we often need to evaluate the partial derivative of a differentiable function with regard to some parameter $$\omega$$. The simpliest way to obtain a partial derivative is to call a forward derivative propagation function, which is defined by:
$$ \mathbb{F}[f_i] = f_i'(\mathbf{w}_i, \mathbf{w}_i') = \sum_{\omega_i\in\mathbf{w}_i} \frac{\partial f}{\partial \omega_i} \omega_i' $$
@@ -127,7 +127,7 @@ Where $$\omega' \in \mathbf{w}'$$ represents the partial derivative of $$\omega_
Given this definition, $$\mathbb{F}[f]$$ can be used as a forward propagation function that is able to compute $$\frac{\partial f_i}{\partial \omega_0}$$ from $$\frac{\partial \omega_{i-1}}{\partial \omega_0}$$.
### Backward Propagation of Derivatives
-When training a neural network, we are more interested in figuring out the partial derivative of the final system output with regard to a parameter $$\omega_i$$ in $$f_i$$. To do so, we generally utilize the backward derivative propagation function
+When the backpropagation algorithm to train a neural network, we are more interested in figuring out the partial derivative of the final system output with regard to a parameter $$\omega_i$$ in $$f_i$$. To do so, we generally utilize the backward derivative propagation function
$$\mathbb{B}[f_i] = f_i^{-1}(\frac{\partial Y}{\partial f_i}) = \frac{\partial Y}{\partial \mathbf{w}_i}$$