Analysis of Linear Functions
\[
\newcommand{\ra}{\rightarrow}
\newcommand{\kC}{\mathcal{C}}
\newcommand{\IR}{\mathbb{R}}
\newcommand{\AINF}{\mathbb{A}}
\newcommand{\CINF}{\kC^\infty}
\newcommand{\half}{\frac{1}{2}}
\newcommand{\IFF}{\Leftrightarrow}
\newcommand{\inf}{\infty}
\newcommand{\Ind}{\mathbb{1}}
\newcommand{\IR}{\mathbb{R}}
\newcommand{\IA}{\mathbb{A}}
\newcommand{\IB}{\mathbb{B}}
\newcommand{\IC}{\mathbb{C}}
\newcommand{\ID}{\mathbb{D}}
\newcommand{\IF}{\mathbb{F}}
\newcommand{\IH}{\mathbb{H}}
\newcommand{\II}{\mathbb{I}}
\newcommand{\IL}{\mathbb{L}}
\newcommand{\IN}{\mathbb{N}}
\newcommand{\IP}{\mathbb{P}}
\newcommand{\IQ}{\mathbb{Q}}
\newcommand{\IR}{\mathbb{R}}
\newcommand{\IS}{\mathbb{S}}
\newcommand{\IV}{\mathbb{V}}
\newcommand{\IZ}{\mathbb{Z}}
\newcommand{\floor}[1]{\lfloor #1 \rfloor}
\newcommand{\ceil}[1]{\lceil #1 \rceil}
\newcommand{\set}[2]{\{\, #1 \;\vert\; #2 \,\}}
\newcommand{\Set}[2]{\left\{\, #1 \;\vert\; #2 \,\right\}}
\newcommand{\C}{\,\#}
\newcommand{\CSet}[2]{\#\{\, #1 \;\vert\; #2 \,\}}
\newcommand{\qtext}[1]{\quad\text{#1}\quad}
\newcommand{\stext}[1]{\;\text{#1}\;}
\newcommand{\dbrackets}[1]{ [\![ #1 ]\!]}
\newcommand{\KR}{\matcal{R}}
\newcommand{\KA}{\matcal{A}}
\newcommand{\KB}{\matcal{B}}
\newcommand{\KC}{\matcal{C}}
\newcommand{\KD}{\matcal{D}}
\newcommand{\KF}{\matcal{F}}
\newcommand{\KH}{\matcal{H}}
\newcommand{\KI}{\matcal{I}}
\Newcommand{\KL}{\matcal{L}}
\newcommand{\KN}{\matcal{N}}
\newcommand{\KP}{\matcal{P}}
\newcommand{\KQ}{\matcal{Q}}
\newcommand{\KR}{\matcal{R}}
\newcommand{\KS}{\matcal{S}}
\newcommand{\KV}{\matcal{V}}
\newcommand{\KZ}{\matcal{Z}}
\newcommand{\gc}{\mathfrak{C}}
\newcommand{\gd}{\mathfrak{D}}
\newcommand{\gM}{\mathfrak{M}}
\newcommand{\gm}{\mathfrak{m}}
\newcommand{\gf}{\mathfrak{f}}
\newcommand{\gu}{\mathfrak{U}}
\newcommand{\fa}{\mathfrak{a}}
\newcommand{\fg}{\mathfrak{g}}
\newcommand{\fn}{\mathfrak{n}}
\newcommand{\fk}{\mathfrak{k}}
\newcommand{\fm}{\mathfrak{m}}
\newcommand{\fp}{\mathfrak{p}}
\newcommand{\curly}[1]{\mathcal{#1}}
\newcommand{\op}[1]{\mathrm{#1}}
\newcommand{\Cat}[1]{\mathfrak{#1}}
\newcommand{\cat}[1]{\mathbf{#1}}
\newcommand{\vphi}{\varphi}
\newcommand{\sphi}{\phi}
\newcommand{\eps}{\varepsilon}
\newcommand{\tensor}{\otimes}
\newcommand{\tensors}{\tensor\dots\tensor}
\newcommand{\Tensor}{\bigotimes}
\newcommand{\ra}{\rightarrow}
\newcommand{\lra}{\longrightarrow}
\newcommand{\la}{\leftarrow}
\newcommand{\lla}{\longleftarrow}
\newcommand{\isom}{\cong}
\newcommand{\epi}{\twoheadrightarrow}
\newcommand{\mono}{\hookrightarrow}
\newcommand{\del}{\partial}
\newcommand{\union}{\cup}
\newcommand{\dotcup}{\ensuremath{\mathaccent\cdot\cup}}
\newcommand{\dunion}{\dotcup}
\newcommand{\<}{\langle}
\renewcommand{\>}{\rangle}
\newcommand{\inpart}[1]{\in\text{\part}(#1)}
\newcommand{\Vsum}{\bigoplus}
\newcommand{\vsum}{\oplus}
\renewcommand{\S}{\mathfrak{S}}
\newcommand{\id}{\mathrm{id}}
\newcommand{\rk}{\mathrm{rk}}
\newcommand{\Diff}{\mathrm{Diff}}
\newcommand{\Hom}{\mathrm{Hom}}
\newcommand{\Pic}{\mathrm{Pic}}
\newcommand{\Spec}{\mathrm{Spec}}
\newcommand{\End}{\mathrm{End}}
\newcommand{\Ext}{\mathrm{Ext}}
\DeclareMathOperator{\Supp}{\mathrm{Supp}}
\DeclareMathOperator{\Sym}{Sym}
\DeclareMathOperator{\Alt}{\Lambda}
\DeclareMathOperator{\ad}{ad}
\DeclareMathOperator{\ch}{ch}
\DeclareMathOperator{\td}{td}
\DeclareMathOperator{\pr}{pr}
\newcommand{\Jac}{\mathcal{J}}
\]
Let \(\psi : \IR^K \ra \IR^M, \phi: \IR^M \ra \IR^N, a: \IR^N \ra \IR\) be continuously differentiable functions.
We have the following notions of derivatives:
- Jacobi Matrix - \(D_p \phi \in \IR[M,N]\) is a MxN matrix of partial derivatives: \(D_p \phi[m,n] = (\del_n \phi^m) (p)\).
- Nabla Operators. We define \(\nabla a (p) = (D_p a)^* \in \IR[N,1]=\IR^N\) as transpose of the Jacobi Matrix.
The following computational rules hold true:
Chain Rule
\[ D_p(\phi \circ \psi) = ( D_{\psi(p)} \phi ) \cdot (D_p \phi), \qquad
\nabla_p(a \circ \phi) = (D_p \phi)^* \cdot \nabla_{\phi(p)} a \]
Linear Maps
-
For \(v \in \IR^N\) and \(f(p)=\<v,p\>\) we have:
\[ D_p f = f = v^* = \< v,\_\>, \qquad \nabla f (p) = v. \]
Quadratic Forms
-
Let \(f(p) = \half \| p\|^2 = \half \< p, p \>\), then
\[ D_p f = \< p, \_\> = p^*, \qquad \nabla f (p) = p. \]
-
Let \(f(p) = \half <G p, p>\), then
\[ D_p f = \half \< (G+G^*)p, \_ \>, \qquad \nabla f(p) = \half (G+G^*)p \]
If \(G=G^*\), this simplifies to:
\[ D_p f = \<G p,\_ \> = p^* G^*, \qquad \nabla f (p) = G p. \]
-
Let \(f(p) = \half \| A p \|^2\), then
\[ D_p f = \<A^* Ap,\_ \> = p^* A^*A, \qquad \nabla f (p) = A^*A p. \]
-
Let \(f(p) = \half \| A^* p - t \|^2\), then
\[ D_p f = \< A^* Ap - A^* t,\_\> \qquad \nabla f(p) = A^* A p - A^* t. \]