注：你可以使用瀏覽器自帶的翻譯功能使用簡體字閱讀本文
上文提到無解的線性方程 A x ⃗ = b ⃗ A\vec{x}=\vec{b} A x = b A T A x ⃗ = A T b ⃗ A^TA\vec{x}=A^T\vec{b} A T A x = A T b x ⃗ \vec{x} x A T A A^TA A T A x ⃗ = ( A T A ) − 1 A T b ⃗ \vec{x}=\left( A^TA\right)^{-1}A^T\vec{b} x = ( A T A ) − 1 A T b
之後介紹一下一般線性模型的最小二乘算子和極大似然算子。最後介紹內積空間。我們要在複數域 上定義內積空間，它兼容實屬域上的內積空間，如果讀者對複數內容欠缺了解，可參考下一篇文章。
定理 現有矩陣 A m × n A_{m \times n} A m × n ∀ x ⃗ ∈ R n \forall \vec{x} \in \mathbb{R}^n ∀ x ∈ R n A x ⃗ = 0 ⃗ ⟺ A T A x ⃗ = 0 ⃗ A\vec{x}=\vec{0} \iff A^TA\vec{x}=\vec{0} A x = 0 ⟺ A T A x = 0
Proof.
若 A x ⃗ = 0 ⃗ A\vec{x}=\vec{0} A x = 0 A T ( A x ⃗ ) = A T 0 ⃗ = 0 ⃗ A^T\left(A\vec{x}\right)=A^T\vec{0}=\vec{0} A T ( A x ) = A T 0 = 0
若 A T A x ⃗ = 0 ⃗ A^TA\vec{x}=\vec{0} A T A x = 0 x ⃗ T A T A x ⃗ = x ⃗ T 0 ⃗ = 0 \vec{x}^TA^TA\vec{x}=\vec{x}^T\vec{0}=0 x T A T A x = x T 0 = 0 ( A x ⃗ ) T A x ⃗ = 0 \left( A\vec{x} \right)^TA\vec{x}=0 ( A x ) T A x = 0 ∥ A x ⃗ ∥ 2 = 0 \lVert A\vec{x} \rVert^2=0 ∥ A x ∥ 2 = 0 A x ⃗ = 0 ⃗ A\vec{x}=\vec{0} A x = 0
注：上述定理表明 Nul A = Nul A T A \text{Nul}A=\text{Nul}A^TA Nul A = Nul A T A
定理 現有矩陣 A m × n A_{m \times n} A m × n A T A A^TA A T A ⟺ \iff ⟺ A A A
Proof.
A T A x ⃗ = 0 ⃗ ⟺ A x ⃗ = 0 ⃗ A^TA\vec{x}=\vec{0} \iff A\vec{x}=\vec{0} A T A x = 0 ⟺ A x = 0 A T A A^TA A T A A T A x ⃗ = 0 ⃗ A^TA\vec{x}=\vec{0} A T A x = 0 A x ⃗ = 0 ⃗ A\vec{x}=\vec{0} A x = 0 A A A A A A A T A A^TA A T A ⟺ \iff ⟺ A T A A^TA A T A A T A A^TA A T A ⟺ A \iff A ⟺ A A A A A T A A^TA A T A rank ( A T A ) = rank A \text{rank}(A^TA) = \text{rank}A rank ( A T A ) = rank A
定理
對於任意矩陣 A m × n A_{m\times n} A m × n
rank ( A T A ) = rank A \text{rank}(A^TA) = \text{rank}A
rank ( A T A ) = rank A
Proof.
由秩-零化度定理可知
n = rank A T A + null A T A = rank A T A + dim Nul A T A = rank A T A + dim Nul A = rank A T A + null A ⇒ rank A T A = n − null A = rank A \begin{aligned}
n
&=\text{rank}A^TA+\text{null}A^TA \\
&=\text{rank}A^TA+\text{dim Nul } A^TA\\
&=\text{rank}A^TA+\text{dim Nul } A\\
&=\text{rank}A^TA+\text{null } A\\
\Rightarrow \\
&\text{rank}A^TA=n-\text{null}A = \text{rank}A \\
\end{aligned}
n ⇒ = rank A T A + null A T A = rank A T A + dim Nul A T A = rank A T A + dim Nul A = rank A T A + null A rank A T A = n − null A = rank A
再補充一個下面求解一般線性模型會用到的結論。
一般來說對於兩個矩陣 A m × n , B n × p A_{m\times n},B_{n\times p} A m × n , B n × p
( A B ) T = B T A T (AB)^T = B^TA^T
( A B ) T = B T A T
當遇到點乘的時候，設 b ⃗ , c ⃗ ∈ R n \vec{b},\vec{c} \in \mathbb{R}^n b , c ∈ R n
b ⃗ ⋅ c ⃗ = b ⃗ T c ⃗ \vec{b}\cdot\vec{c}=\vec{b}^T\vec{c}
b ⋅ c = b T c
求轉置：
( b ⃗ ⋅ c ⃗ ) T = ( b ⃗ T c ⃗ ) T = c ⃗ T b ⃗ = [ c 1 , ⋯ , c n ] [ b 1 ⋮ b n ] = c 1 b 1 + ⋯ + c n b n = b 1 c 1 + ⋯ + b n c n = [ b 1 , ⋯ , b n ] [ c 1 ⋮ c n ] = b ⃗ T c ⃗ \begin{aligned}
\left(\vec{b}\cdot\vec{c}\right)^T
&=\left(\vec{b}^T\vec{c}\right)^T \\
&=\vec{c}^T\vec{b} \\
&=
\begin{bmatrix}
c_1, \cdots, c_n
\end{bmatrix}
\begin{bmatrix}
b_1 \\ \vdots \\ b_n
\end{bmatrix} \\
&=c_1b_1 + \cdots + c_nb_n \\
&=b_1c_1 + \cdots + b_nc_n \\
&=\begin{bmatrix}
b_1, \cdots, b_n
\end{bmatrix}
\begin{bmatrix}
c_1 \\ \vdots \\ c_n
\end{bmatrix} \\
&=\vec{b}^T\vec{c}
\end{aligned}
( b ⋅ c ) T = ( b T c ) T = c T b = [ c 1 , ⋯ , c n ] ⎣ ⎢ ⎡ b 1 ⋮ b n ⎦ ⎥ ⎤ = c 1 b 1 + ⋯ + c n b n = b 1 c 1 + ⋯ + b n c n = [ b 1 , ⋯ , b n ] ⎣ ⎢ ⎡ c 1 ⋮ c n ⎦ ⎥ ⎤ = b T c
稍複雜時，設 a ⃗ ∈ R n , B ∈ R n × m , c ⃗ ∈ R m \vec{a} \in \mathbb{R}^n, B \in \mathbb{R}^{n\times m}, \vec{c} \in \mathbb{R}^m a ∈ R n , B ∈ R n × m , c ∈ R m
a ⃗ T B c ⃗ = a ⃗ T ( B c ⃗ ) 此時 B c ⃗ ∈ R n = ( B c ⃗ ) T a ⃗ = c ⃗ T B T a ⃗ \begin{aligned}
\vec{a}^TB\vec{c}
&=\vec{a}^T(B\vec{c}) \quad \text{此時 } B\vec{c} \in \mathbb{R}^n\\
&= (B\vec{c})^T\vec{a} \\
&= \vec{c}^TB^T \vec{a}
\end{aligned}
a T B c = a T ( B c ) 此時 B c ∈ R n = ( B c ) T a = c T B T a
統計上我們習慣用 X β = y X\boldsymbol{\beta}=\boldsymbol{y} X β = y A x ⃗ = b ⃗ A\vec{x} = \vec{b} A x = b X X X 設計矩陣 β \boldsymbol{\beta} β 參數向量 y \boldsymbol{y} y 觀測變量
y = β 0 + β 1 x y=\beta_0+\beta_1x
y = β 0 + β 1 x
試驗中觀察了 n n n n n n ( x i , y i ) (x_i,y_i) ( x i , y i ) β 0 \beta_0 β 0 β 1 \beta_1 β 1 y = β 0 + β 1 x y=\beta_0+\beta_1x y = β 0 + β 1 x β 0 \beta_0 β 0 β 1 \beta_1 β 1 j j j ( x j , y j ) (x_j,y_j) ( x j , y j ) 觀測值 ( x j , β 0 + β 1 x j ) (x_j, \beta_0+\beta_1x_j) ( x j , β 0 + β 1 x j ) 預測值 殘差
試驗數據的擬和
找到最合適的 β 0 \beta_0 β 0 β 1 \beta_1 β 1 回歸 回歸係數 最小二乘線
下面介紹多重回歸的模型。該模型假設观测变量 y y y x 1 , ⋯ , x k x_1,\cdots, x_k x 1 , ⋯ , x k k k k n n n ( x i 1 , x i 2 , ⋯ , x i k , y i ) , i = 1 , 2 , ⋯ , n \left(x_{i1}, x_{i2}, \cdots, x_{ik}, y_i\right), \,\, i=1,2,\cdots,n ( x i 1 , x i 2 , ⋯ , x i k , y i ) , i = 1 , 2 , ⋯ , n
y 1 = β 0 + β 1 x 11 + β 2 x 12 + ⋯ + β k x 1 k + ε 1 y 2 = β 0 + β 1 x 21 + β 2 x 22 + ⋯ + β k x 2 k + ε 2 ⋮ y n = β 0 + β 1 x n 1 + β 2 x n 2 + ⋯ + β k x n k + ε n \begin{aligned}
y_{1} &=\beta_{0}+\beta_{1} x_{11}+\beta_{2} x_{12}+\cdots+\beta_{k} x_{1 k}+\varepsilon_{1} \\
y_{2} &=\beta_{0}+\beta_{1} x_{21}+\beta_{2} x_{22}+\cdots+\beta_{k} x_{2 k}+\varepsilon_{2} \\
& \vdots \\
y_{n} &=\beta_{0}+\beta_{1} x_{n 1}+\beta_{2} x_{n 2}+\cdots+\beta_{k} x_{n k}+\varepsilon_{n}
\end{aligned}
y 1 y 2 y n = β 0 + β 1 x 1 1 + β 2 x 1 2 + ⋯ + β k x 1 k + ε 1 = β 0 + β 1 x 2 1 + β 2 x 2 2 + ⋯ + β k x 2 k + ε 2 ⋮ = β 0 + β 1 x n 1 + β 2 x n 2 + ⋯ + β k x n k + ε n
將這 n n n
[ y 1 y 2 ⋮ y n ] = [ 1 x 11 x 12 … x 1 k 1 x 21 x 22 … x 2 k ⋮ ⋮ ⋮ ⋮ 1 x n 1 x n 2 … x n k ] [ β 0 β 1 ⋮ β k ] + [ ε 1 ε 2 ⋮ ε n ] \begin{bmatrix}
y_{1} \\
y_{2} \\
\vdots \\
y_{n}
\end{bmatrix}
=\begin{bmatrix}
1 & x_{11} & x_{12} & \ldots & x_{1 k} \\
1 & x_{21} & x_{22} & \ldots & x_{2 k} \\
\vdots & \vdots & \vdots & & \vdots \\
1 & x_{n 1} & x_{n 2} & \ldots & x_{n k}
\end{bmatrix}
\begin{bmatrix}
\beta_{0} \\
\beta_{1} \\
\vdots \\
\beta_{k}
\end{bmatrix}+
\begin{bmatrix}
\varepsilon_{1} \\
\varepsilon_{2} \\
\vdots \\
\varepsilon_{n}
\end{bmatrix}
⎣ ⎢ ⎢ ⎢ ⎡ y 1 y 2 ⋮ y n ⎦ ⎥ ⎥ ⎥ ⎤ = ⎣ ⎢ ⎢ ⎢ ⎡ 1 1 ⋮ 1 x 1 1 x 2 1 ⋮ x n 1 x 1 2 x 2 2 ⋮ x n 2 … … … x 1 k x 2 k ⋮ x n k ⎦ ⎥ ⎥ ⎥ ⎤ ⎣ ⎢ ⎢ ⎢ ⎡ β 0 β 1 ⋮ β k ⎦ ⎥ ⎥ ⎥ ⎤ + ⎣ ⎢ ⎢ ⎢ ⎡ ε 1 ε 2 ⋮ ε n ⎦ ⎥ ⎥ ⎥ ⎤
或者：
y = X β + ε \boldsymbol{y}=X\boldsymbol{\beta} + \boldsymbol{\varepsilon}
y = X β + ε
y ∈ R n , X ∈ R n × ( k + 1 ) , β ∈ R k + 1 , ε ∈ R n \boldsymbol{y} \in \mathbb{R}^n,X \in \mathbb{R}^{n\times (k+1)},\boldsymbol{\beta}\in\mathbb{R}^{k+1}, \boldsymbol{\varepsilon} \in \mathbb{R}^n y ∈ R n , X ∈ R n × ( k + 1 ) , β ∈ R k + 1 , ε ∈ R n
E ( ε ) = 0 E(\boldsymbol{\varepsilon})=\boldsymbol{0} E ( ε ) = 0 E ( y ) = X β E(\boldsymbol{y})=X\boldsymbol{\beta } E ( y ) = X β c o v ( ε ) = σ 2 I cov(\boldsymbol{\varepsilon})=\sigma^2I c o v ( ε ) = σ 2 I rank ( X ) = k + 1 < n \text{rank}(X)=k+1<n rank ( X ) = k + 1 < n
最小二乘期望找到 β \boldsymbol{\beta} β ε T ε \boldsymbol{\varepsilon}^T\boldsymbol{\varepsilon} ε T ε β \boldsymbol{\beta} β
ε ^ T ε ^ = ( y − X β ^ ) T ( y − X β ^ ) = ( y T − β ^ T X T ) ( y − X β ^ ) = y T y − 2 y T X β ^ + 2 X T X β ^ \begin{aligned}
\hat{\boldsymbol{\varepsilon}}^T\hat{\boldsymbol{\varepsilon}}
&=(\boldsymbol{y}-X\hat{\boldsymbol{\beta}})^T(\boldsymbol{y}-X\hat{\boldsymbol{\beta}}) \\
&=(\boldsymbol{y}^T-\hat{\boldsymbol{\beta}}^TX^T)(\boldsymbol{y}-X\hat{\boldsymbol{\beta}}) \\
&= \boldsymbol{y}^T\boldsymbol{y} -2\boldsymbol{y}^TX\hat{\boldsymbol{\beta}}+2X^TX\hat{\boldsymbol{\beta}}
\end{aligned}
ε ^ T ε ^ = ( y − X β ^ ) T ( y − X β ^ ) = ( y T − β ^ T X T ) ( y − X β ^ ) = y T y − 2 y T X β ^ + 2 X T X β ^
∂ ε ^ T ε ^ ∂ β ^ = 0 − 2 X T y + 2 X T X β ^ = 0 \frac{\partial\hat{\boldsymbol{\varepsilon}}^T\hat{\boldsymbol{\varepsilon}}}{\partial\hat{\boldsymbol{\beta}}}
=0-2X^T\boldsymbol{y}+2X^TX\hat{\boldsymbol{\beta}} =0
∂ β ^ ∂ ε ^ T ε ^ = 0 − 2 X T y + 2 X T X β ^ = 0
即：
X T X β ^ = X T y X^TX\hat{\boldsymbol{\beta}} = X^T\boldsymbol{y}
X T X β ^ = X T y
因 X X X X T X X^TX X T X
β ^ = ( X T X ) − 1 X T y \hat{\boldsymbol{\beta}} = (X^TX)^{-1}X^T\boldsymbol{y}
β ^ = ( X T X ) − 1 X T y
有的時候自變量和因變量之間不必是線性關係，只要非線性關係可以轉換成線性關係即可，例如 y = β 0 + β 1 x 1 + β 2 x 1 2 + β 3 sin ( x 3 ) + ε y=\beta_0+\beta_1x_1 + \beta_2x_1^2 + \beta _3\text{sin}{(x_3)}+\varepsilon y = β 0 + β 1 x 1 + β 2 x 1 2 + β 3 sin ( x 3 ) + ε
然後一樣求解。
假設 1 表示 y i y_i y i x 1 , ⋯ , x k x_1,\cdots,x_k x 1 , ⋯ , x k y i y_i y i
假設 2 表示
var ( ε i ) = σ 2 \text{var}(\varepsilon_i) = \sigma^2 var ( ε i ) = σ 2 ε \varepsilon ε x i x_i x i cov ( ε i , ε j ) = 0 \text{cov}(\varepsilon_i,\varepsilon_j)=0 cov ( ε i , ε j ) = 0 ε \varepsilon ε y y y
假設 3 表示
X X X X T X X^TX X T X ( X T X ) β = X T y (X^TX)\boldsymbol{\beta}=X^T\boldsymbol{y} ( X T X ) β = X T y k + 1 < n k+1 <n k + 1 < n X β = y X\boldsymbol{\beta} = \boldsymbol{y} X β = y
k + 1 = n k+1=n k + 1 = n k + 1 > n k+1>n k + 1 > n
有些情況我們會增加正態性假設。
極大似然還有正態假設：y ∼ N n ( X β , σ 2 I ) \boldsymbol{y} \sim N_n\left(X\boldsymbol{\beta}, \sigma^2I\right) y ∼ N n ( X β , σ 2 I ) ε ∼ N n ( 0 , σ 2 I ) \boldsymbol{\varepsilon} \sim N_n\left(\boldsymbol{0}, \sigma^2I\right) ε ∼ N n ( 0 , σ 2 I ) σ i j = 0 \sigma_{ij}=0 σ i j = 0 y y y ε \varepsilon ε
有了正態性假設，可以得到極大似然算子。似然函數是 y y y L ( β , σ 2 ) L(\boldsymbol{\beta}, \sigma^2) L ( β , σ 2 ) β \boldsymbol{\beta} β σ 2 \sigma^2 σ 2 y \boldsymbol{y} y X X X L ( β , σ 2 ) L(\boldsymbol{\beta}, \sigma^2) L ( β , σ 2 )
L ( β , σ 2 ) = f ( y ; β , σ 2 ) = 1 ( 2 π ) n / 2 ∣ σ 2 I ∣ 1 / 2 e − ( y − X β ) T ( σ 2 I ) − 1 ( y − X β ) / 2 = 1 ( 2 π σ 2 ) n / 2 e − ( y − X β ) T ( y − X β ) / 2 σ 2 \begin{aligned}
L(\boldsymbol{\beta}, \sigma^2)
=f(\boldsymbol{y}; \boldsymbol{\beta}, \sigma^2)
&=\frac{1}{(2\pi)^{n/2}\lvert\sigma^2I\rvert^{1/2}}e^{-(\boldsymbol{y}-X\boldsymbol{\beta})^T(\sigma^2I)^{-1}(\boldsymbol{y}-X\boldsymbol{\beta})/2} \\
&= \frac{1}{(2\pi\sigma^2)^{n/2}}e^{-(\boldsymbol{y}-X\boldsymbol{\beta})^T(\boldsymbol{y}-X\boldsymbol{\beta})/2\sigma^2}
\end{aligned}
L ( β , σ 2 ) = f ( y ; β , σ 2 ) = ( 2 π ) n / 2 ∣ σ 2 I ∣ 1 / 2 1 e − ( y − X β ) T ( σ 2 I ) − 1 ( y − X β ) / 2 = ( 2 π σ 2 ) n / 2 1 e − ( y − X β ) T ( y − X β ) / 2 σ 2
因為 y i y_i y i L ( β , σ 2 ) L(\boldsymbol{\beta}, \sigma^2) L ( β , σ 2 ) Π i = 1 n f ( y i ; x i T β , σ 2 ) \Pi_{i=1}^{n}f(y_i; \boldsymbol{x_i}^T\boldsymbol{\beta}, \sigma^2) Π i = 1 n f ( y i ; x i T β , σ 2 )
ln L ( β , σ 2 ) = − n 2 ln ( 2 π ) − n 2 ln ( σ 2 ) − 1 2 σ 2 ( y − X β ) T ( y − X β ) \text{ln}L(\boldsymbol{\beta}, \sigma^2) = -\frac{n}{2}\text{ln}(2\pi) - \frac{n}{2}\text{ln}(\sigma^2) - \frac{1}{2\sigma^2}(\boldsymbol{y}-X\boldsymbol{\beta})^T(\boldsymbol{y}-X\boldsymbol{\beta})
ln L ( β , σ 2 ) = − 2 n ln ( 2 π ) − 2 n ln ( σ 2 ) − 2 σ 2 1 ( y − X β ) T ( y − X β )
求偏導
∂ ln L ( β , σ 2 ) ∂ β = 1 2 σ 2 ( 2 X T X β − 2 X T y ) = 0 ⇒ X T X β = X T y ⇒ β = ( X T X ) − 1 X T y \frac{\partial\, \text{ln}L(\boldsymbol{\beta}, \sigma^2)}{\partial\boldsymbol{\beta}}
= \frac{1}{2\sigma^2}\left(2X^TX\boldsymbol{\beta}-2X^T\boldsymbol{y}\right) = 0 \\
\Rightarrow X^TX\boldsymbol{\beta} = X^T\boldsymbol{y} \\
\Rightarrow \boldsymbol{\beta} = (X^TX)^{-1}X^T\boldsymbol{y}
∂ β ∂ ln L ( β , σ 2 ) = 2 σ 2 1 ( 2 X T X β − 2 X T y ) = 0 ⇒ X T X β = X T y ⇒ β = ( X T X ) − 1 X T y
然後發現極大似然和最小二乘對 β \boldsymbol{\beta} β
探討一 回歸線過均值點
令 x ˉ = 1 n ( x 1 + ⋯ x n ) , y ˉ = 1 n ( y 1 + ⋯ y n ) \bar{x}=\frac{1}{n}(x_1+\cdots x_n), \bar{y}=\frac{1}{n}(y_1+\cdots y_n) x ˉ = n 1 ( x 1 + ⋯ x n ) , y ˉ = n 1 ( y 1 + ⋯ y n ) ( x 1 , y 1 ) , , ⋯ , ( x n , y n ) (x_1,y_1), ,\cdots,(x_n,y_n) ( x 1 , y 1 ) , , ⋯ , ( x n , y n ) ( x ˉ , y ˉ ) (\bar{x}, \bar{y}) ( x ˉ , y ˉ ) ( x ˉ , y ˉ ) (\bar{x}, \bar{y}) ( x ˉ , y ˉ ) y ˉ = β ^ 0 + β ^ 1 x ˉ \bar{y}=\hat{\beta}_0+\hat{\beta}_1\bar{x} y ˉ = β ^ 0 + β ^ 1 x ˉ
Proof.
把設計矩陣寫成 X = [ 1 , x ] X=\begin{bmatrix} \boldsymbol{1}, \boldsymbol{x} \end{bmatrix} X = [ 1 , x ] ε = y − X β ^ \boldsymbol{\varepsilon}=\boldsymbol{y}-X\hat{\boldsymbol{\beta}} ε = y − X β ^ Col X \text{Col}\,X Col X 1 \boldsymbol{1} 1 Col X \text{Col}\,X Col X X X X
0 = 1 ⋅ ε = 1 ⋅ ( y − X β ^ ) = 1 T y − ( 1 T X ) β ^ = ∑ y − [ n , ∑ x ] [ β 0 ^ β 1 ^ ] = ∑ y − n β 0 ^ − β 1 ^ ∑ x = n y ˉ − n β 0 ^ − n β 1 ^ x ˉ ( ∑ y = n y ˉ , ∑ x = n x ˉ ) ⇒ y ˉ = β 0 ^ + β 1 ^ x ˉ \begin{aligned}
0
&=\boldsymbol{1}\cdot\boldsymbol{\varepsilon} \\
&= \boldsymbol{1}\cdot\left(\boldsymbol{y}-X\hat{\boldsymbol{\beta}}\right)\\
&=\boldsymbol{1}^T\boldsymbol{y} - \left(\boldsymbol{1}^T
X\right)\hat{\boldsymbol{\beta}} \\
&=\sum y-
\begin{bmatrix} n , \sum x \end{bmatrix}
\begin{bmatrix} \hat{\beta_0} \\ \hat{\beta_1} \end{bmatrix} \\
&= \sum y - n\hat{\beta_0} - \hat{\beta_1}\sum x \\
&= n\bar{y} - n\hat{\beta_0} - n\hat{\beta_1}\bar{x} \qquad (\sum y =n\bar{y}, \sum x=n\bar{x}) \\
\Rightarrow \\
&\bar{y} =\hat{\beta_0} + \hat{\beta_1}\bar{x}
\end{aligned}
0 ⇒ = 1 ⋅ ε = 1 ⋅ ( y − X β ^ ) = 1 T y − ( 1 T X ) β ^ = ∑ y − [ n , ∑ x ] [ β 0 ^ β 1 ^ ] = ∑ y − n β 0 ^ − β 1 ^ ∑ x = n y ˉ − n β 0 ^ − n β 1 ^ x ˉ ( ∑ y = n y ˉ , ∑ x = n x ˉ ) y ˉ = β 0 ^ + β 1 ^ x ˉ
若自變量有多個，同理可得：
0 = 1 ⋅ ε = 1 ⋅ ( y − X β ^ ) = 1 T y − ( 1 T X ) β ^ = ∑ y − [ n , ∑ x 1 , ⋯ , ∑ x k ] [ β 0 ^ β 1 ^ ⋮ β k ^ ] = ∑ y − n β 0 ^ − β 1 ^ ∑ x 1 − ⋯ − β k ^ ∑ x k = n y ˉ − n β 0 ^ − n β 1 ^ x ˉ 1 − ⋯ − n β k ^ x ˉ k ⇒ y ˉ = β 0 ^ + β 1 ^ x ˉ 1 + c d o t s + β k ^ x ˉ k \begin{aligned}
0
&=\boldsymbol{1}\cdot\boldsymbol{\varepsilon} \\
&= \boldsymbol{1}\cdot\left(\boldsymbol{y}-X\hat{\boldsymbol{\beta}}\right)\\
&=\boldsymbol{1}^T\boldsymbol{y} - \left(\boldsymbol{1}^T
X\right)\hat{\boldsymbol{\beta}} \\
&=\sum y-
\begin{bmatrix} n , \sum x_1, \cdots, \sum x_k \end{bmatrix}
\begin{bmatrix} \hat{\beta_0} \\ \hat{\beta_1} \\ \vdots \\ \hat{\beta_k} \end{bmatrix} \\
&= \sum y - n\hat{\beta_0} - \hat{\beta_1}\sum x_1 - \cdots - \hat{\beta_k}\sum x_k \\
&= n\bar{y} - n\hat{\beta_0} - n\hat{\beta_1}\bar{x}_1 - \cdots - n\hat{\beta_k}\bar{x}_k\\
\Rightarrow \\
&\bar{y} =\hat{\beta_0} + \hat{\beta_1}\bar{x}_1 + cdots + \hat{\beta_k}\bar{x}_k
\end{aligned}
0 ⇒ = 1 ⋅ ε = 1 ⋅ ( y − X β ^ ) = 1 T y − ( 1 T X ) β ^ = ∑ y − [ n , ∑ x 1 , ⋯ , ∑ x k ] ⎣ ⎢ ⎢ ⎢ ⎡ β 0 ^ β 1 ^ ⋮ β k ^ ⎦ ⎥ ⎥ ⎥ ⎤ = ∑ y − n β 0 ^ − β 1 ^ ∑ x 1 − ⋯ − β k ^ ∑ x k = n y ˉ − n β 0 ^ − n β 1 ^ x ˉ 1 − ⋯ − n β k ^ x ˉ k y ˉ = β 0 ^ + β 1 ^ x ˉ 1 + c d o t s + β k ^ x ˉ k
探討二 平均偏差形式
平均偏差形式指將自變量的均值規劃為 0 ，即 ∑ x i = 0 , for ( x 1 , y 1 ) , ⋯ , ( x n , y n ) \sum x_i=0, \text{for} (x_1,y_1), \cdots, (x_n,y_n) ∑ x i = 0 , for ( x 1 , y 1 ) , ⋯ , ( x n , y n ) X T X X^TX X T X
Proof.
X T X = [ 1 1 ⋯ 1 x 1 x 2 ⋯ x k ] [ 1 x 1 1 x 2 ⋮ ⋮ 1 x k ] = [ n ∑ x i ∑ x i ∑ ( x i 2 ) ] = [ n 0 0 ∑ ( x i 2 ) ] \begin{aligned}
X^TX
&=
\begin{bmatrix}
1 & 1 & \cdots & 1 \\
x_1 & x_2 & \cdots & x_k
\end{bmatrix}
\begin{bmatrix}
1 & x_1 \\
1 & x_2 \\
\vdots & \vdots \\
1 & x_k
\end{bmatrix} \\
&=
\begin{bmatrix}
n & \sum x_i \\
\sum x_i & \sum(x_i^2)
\end{bmatrix} \\
&=\begin{bmatrix}
n & 0 \\
0 & \sum(x_i^2)
\end{bmatrix}
\end{aligned}
X T X = [ 1 x 1 1 x 2 ⋯ ⋯ 1 x k ] ⎣ ⎢ ⎢ ⎢ ⎡ 1 1 ⋮ 1 x 1 x 2 ⋮ x k ⎦ ⎥ ⎥ ⎥ ⎤ = [ n ∑ x i ∑ x i ∑ ( x i 2 ) ] = [ n 0 0 ∑ ( x i 2 ) ]
定義 SST SSR SSE
設線性方程 X β = y X\boldsymbol{\beta}=\boldsymbol{y} X β = y β ^ \hat{\boldsymbol{\beta}} β ^
S S R = ∥ X β ^ ∥ 2 SSR=\lVert X\hat{\boldsymbol{\beta}}\rVert^2 S S R = ∥ X β ^ ∥ 2 S S E = ∥ y − X β ^ ∥ 2 SSE=\lVert \boldsymbol{y} - X\hat{\boldsymbol{\beta}}\rVert^2 S S E = ∥ y − X β ^ ∥ 2 S S T = ∥ y ∥ 2 SST=\lVert \boldsymbol{y}\rVert^2 S S T = ∥ y ∥ 2
討論三 回歸方程平方和的關係
證明： S S T = S S R + S S E SST=SSR+SSE S S T = S S R + S S E
Proof.
S S T = ∥ y ∥ 2 = ∥ y ^ + ε ∥ 2 = ∥ y ^ ∥ 2 + ∥ ε ∥ 2 （因 y ^ 與 ε 正交） = ∥ X β ^ ∥ 2 + ∥ y − X β ^ ∥ 2 = S S R + S S E \begin{aligned}
SST=\lVert \boldsymbol{y} \rVert^2
&= \lVert \hat{\boldsymbol{y}} + \boldsymbol{\varepsilon} \rVert^2 \\
&= \lVert \hat{\boldsymbol{y}} \rVert^2 + \lVert\boldsymbol{\varepsilon} \rVert^2 \quad \text{（因 } \hat{\boldsymbol{y}} \text{ 與}\boldsymbol{\varepsilon}\text{ 正交）} \\
&= \lVert X\hat{\boldsymbol{\beta}}\rVert^2 + \lVert \boldsymbol{y} - X\hat{\boldsymbol{\beta}}\rVert^2 \\
&= SSR + SSE
\end{aligned}
S S T = ∥ y ∥ 2 = ∥ y ^ + ε ∥ 2 = ∥ y ^ ∥ 2 + ∥ ε ∥ 2 （因 y ^ 與 ε 正交） = ∥ X β ^ ∥ 2 + ∥ y − X β ^ ∥ 2 = S S R + S S E
在統計中，該等式對於回歸理論和方差分析都非常重要。
討論四 S S E SSE S S E
證明：
∥ X β ^ ∥ 2 = β ^ X T y \lVert X\hat{\boldsymbol{\beta}} \rVert^2=\hat{\boldsymbol{\beta}}X^T\boldsymbol{y}
∥ X β ^ ∥ 2 = β ^ X T y
Proof.
β ^ \hat{\boldsymbol{\beta}} β ^ X T X β ^ = X T y X^TX\hat{\boldsymbol{\beta}}=X^T\boldsymbol{y} X T X β ^ = X T y
S S R = ∥ X β ^ ∥ 2 = ( X β ^ ) T ( X β ^ ) = β ^ T X T X β ^ = β ^ T ( X T X β ^ ) = β ^ T X T y \begin{aligned}
SSR=\lVert X\hat{\boldsymbol{\beta}} \rVert^2
&= (X\hat{\boldsymbol{\beta}})^T(X\hat{\boldsymbol{\beta}}) \\
&= \hat{\boldsymbol{\beta}}^TX^TX\hat{\boldsymbol{\beta}} \\
&= \hat{\boldsymbol{\beta}}^T(X^TX\hat{\boldsymbol{\beta}}) \\
&= \hat{\boldsymbol{\beta}}^TX^T\boldsymbol{y}
\end{aligned}
S S R = ∥ X β ^ ∥ 2 = ( X β ^ ) T ( X β ^ ) = β ^ T X T X β ^ = β ^ T ( X T X β ^ ) = β ^ T X T y
這樣得到 S S E SSE S S E
S S E = y T y − β ^ T X T y SSE = \boldsymbol{y}^T\boldsymbol{y}-\hat{\boldsymbol{\beta}}^TX^T\boldsymbol{y}
S S E = y T y − β ^ T X T y
歐式內積即此前介紹過的點乘 ，現考慮複數的點乘。若 v ⃗ = [ v 1 , ⋯ , v n ] T ∈ C n \vec{v} = \begin{bmatrix} v_1,\cdots,v_{n} \end{bmatrix}^T \in \mathbb{C}^n v = [ v 1 , ⋯ , v n ] T ∈ C n
∥ v ⃗ ∥ 2 = v ⃗ T v ⃗ = v ⃗ ⋅ v ⃗ = ∣ v 1 ∣ 2 + ⋯ + ∣ v n ∣ 2 = v 1 v 1 ‾ + ⋯ + v n v n ‾ \begin{aligned}
\lVert\vec{v}\rVert^2
&= \vec{v}^T\vec{v} = \vec{v}\cdot\vec{v} \\
&= \lvert v_1\rvert^2+\cdots+\lvert v_n\rvert^2 \\
&= v_1\overline{v_1}+\cdots+v_n\overline{v_n}
\end{aligned}
∥ v ∥ 2 = v T v = v ⋅ v = ∣ v 1 ∣ 2 + ⋯ + ∣ v n ∣ 2 = v 1 v 1 + ⋯ + v n v n
可將 ∥ v ⃗ ∥ 2 \lVert \vec{v} \rVert^2 ∥ v ∥ 2 z z z w ⃗ = [ w 1 , ⋯ , w n ] T ∈ C n \vec{w} = \begin{bmatrix} w_1,\cdots,w_{n} \end{bmatrix}^T \in \mathbb{C}^n w = [ w 1 , ⋯ , w n ] T ∈ C n z z z
v ⃗ ⋅ w ⃗ = w 1 z 1 ‾ + ⋯ + w n z n ‾ \vec{v}\cdot\vec{w} = w_1\overline{z_1} + \cdots +w_n\overline{z_n}
v ⋅ w = w 1 z 1 + ⋯ + w n z n
所以 w ⃗ \vec{w} w z ⃗ \vec{z} z
w ⃗ ⋅ v ⃗ = v ⃗ ⋅ w ⃗ ‾ \vec{w}\cdot\vec{v} = \overline{\vec{v}\cdot\vec{w}}
w ⋅ v = v ⋅ w
定義 內積
F \mathbb{F} F V V V V V V ( u ⃗ , v ⃗ ) (\vec{u},\vec{v}) ( u , v ) ⟨ u ⃗ , v ⃗ ⟩ ∈ F \langle \vec{u},\vec{v} \rangle\in \mathbb{F} ⟨ u , v ⟩ ∈ F 內積
非負確定性：⟨ v ⃗ , v ⃗ ⟩ ≥ 0 , ⟨ v ⃗ , v ⃗ ⟩ = 0 ⟺ v ⃗ = 0 ⃗ , ∀ v ⃗ ∈ V \langle \vec{v},\vec{v} \rangle \geq 0 ,\,\, \langle \vec{v},\vec{v} \rangle = 0 \iff \vec{v}=\vec{0},\,\, \forall \vec{v} \in V ⟨ v , v ⟩ ≥ 0 , ⟨ v , v ⟩ = 0 ⟺ v = 0 , ∀ v ∈ V
首位可加性：⟨ u ⃗ + v ⃗ , w ⃗ ⟩ = ⟨ u ⃗ , w ⃗ ⟩ + ⟨ v ⃗ , w ⃗ ⟩ , ∀ u ⃗ , v ⃗ , w ⃗ ∈ V \langle \vec{u}+\vec{v}, \vec{w} \rangle = \langle \vec{u}, \vec{w} \rangle + \langle \vec{v}, \vec{w} \rangle, \,\, \forall \vec{u},\vec{v}, \vec{w}\in V ⟨ u + v , w ⟩ = ⟨ u , w ⟩ + ⟨ v , w ⟩ , ∀ u , v , w ∈ V
首位齊次性： ⟨ λ u ⃗ , v ⃗ ⟩ = λ ⟨ u ⃗ , v ⃗ ⟩ , ∀ u ⃗ , v ⃗ ∈ V \langle \lambda \vec{u}, \vec{v} \rangle=\lambda\langle \vec{u}, \vec{v} \rangle, \,\, \forall \vec{u},\vec{v}\in V ⟨ λ u , v ⟩ = λ ⟨ u , v ⟩ , ∀ u , v ∈ V
上面兩條可以合併為首位線性：⟨ α u ⃗ + β v ⃗ , w ⃗ ⟩ = α ⟨ u ⃗ , w ⃗ ⟩ + β ⟨ v ⃗ , w ⃗ ⟩ , ∀ u ⃗ , v ⃗ , w ⃗ ∈ V \langle \alpha\vec{u}+\beta\vec{v}, \vec{w} \rangle = \alpha\langle \vec{u}, \vec{w} \rangle + \beta\langle \vec{v}, \vec{w} \rangle, \,\, \forall \vec{u},\vec{v}, \vec{w}\in V ⟨ α u + β v , w ⟩ = α ⟨ u , w ⟩ + β ⟨ v , w ⟩ , ∀ u , v , w ∈ V
共軛對稱性：⟨ u ⃗ , v ⃗ ⟩ = ⟨ v ⃗ , u ⃗ ⟩ ‾ , ∀ u ⃗ , v ⃗ ∈ V \langle \vec{u},\vec{v} \rangle = \overline{\langle \vec{v},\vec{u} \rangle}, \,\, \forall \vec{u},\vec{v}\in V ⟨ u , v ⟩ = ⟨ v , u ⟩ , ∀ u , v ∈ V
實屬的共軛複數就是它自己，如果在實屬域上考慮內積，最後一條的共軛複數可簡化為： ⟨ u ⃗ , v ⃗ ⟩ = ⟨ v ⃗ , u ⃗ ⟩ \langle \vec{u},\vec{v} \rangle =\langle \vec{v},\vec{u} \rangle ⟨ u , v ⟩ = ⟨ v , u ⟩
例 內積
F n \mathbb{F}^n F n 歐式內積空間
⟨ ( w 1 , ⋯ , w n ) , ( z 1 , ⋯ , z n ) ⟩ = w 1 z 1 ‾ + ⋯ + w n z n ‾ . \langle
\left( w_1, \cdots, w_n \right),
\left( z_1, \cdots, z_n \right)
\rangle
=
w_1\overline{z_1} + \cdots + w_n\overline{z_n}.
⟨ ( w 1 , ⋯ , w n ) , ( z 1 , ⋯ , z n ) ⟩ = w 1 z 1 + ⋯ + w n z n .
若 c 1 , ⋯ , c n c_1, \cdots, c_n c 1 , ⋯ , c n F n \mathbb{F}^n F n
⟨ ( w 1 , ⋯ , w n ) , ( z 1 , ⋯ , z n ) ⟩ = c 1 w 1 z 1 ‾ + ⋯ c n w n z n ‾ . \langle
\left( w_1, \cdots, w_n \right),
\left( z_1, \cdots, z_n \right)
\rangle
=
c_1w_1\overline{z_1} + \cdots c_nw_n\overline{z_n}.
⟨ ( w 1 , ⋯ , w n ) , ( z 1 , ⋯ , z n ) ⟩ = c 1 w 1 z 1 + ⋯ c n w n z n .
在實屬域上，⟨ ( w 1 , ⋯ , w n ) , ( z 1 , ⋯ , z n ) ⟩ = c 1 w 1 z 1 + ⋯ c n w n z n \langle\left( w_1, \cdots, w_n \right),\left( z_1, \cdots, z_n \right)\rangle=c_1w_1 z_1 + \cdots c_nw_nz_n ⟨ ( w 1 , ⋯ , w n ) , ( z 1 , ⋯ , z n ) ⟩ = c 1 w 1 z 1 + ⋯ c n w n z n A = [ c 1 , 0 , ⋯ , 0 0 , c 2 , ⋯ , 0 0 , 0 , ⋯ , c n ] A=\begin{bmatrix} &c_1, &0, &\cdots, &0 &\\ &0, &c_2, &\cdots, &0 &\\ &0,&0, &\cdots, &c_n &\end{bmatrix} A = ⎣ ⎡ c 1 , 0 , 0 , 0 , c 2 , 0 , ⋯ , ⋯ , ⋯ , 0 0 c n ⎦ ⎤ R n \mathbb{R}^n R n w ⃗ = [ w 1 ⋮ w n ] , z ⃗ = [ z 1 ⋮ z n ] \vec{w}=\begin{bmatrix} w_1 \\ \vdots \\ w_n \end{bmatrix}, \vec{z}=\begin{bmatrix} z_1 \\ \vdots \\ z_n \end{bmatrix} w = ⎣ ⎢ ⎡ w 1 ⋮ w n ⎦ ⎥ ⎤ , z = ⎣ ⎢ ⎡ z 1 ⋮ z n ⎦ ⎥ ⎤
w ⃗ T A z ⃗ = [ w 1 ⋯ w n ] [ c 1 , 0 , ⋯ , 0 0 , c 2 , ⋯ , 0 0 , 0 , ⋯ , c n ] [ z 1 ⋮ z n ] = c 1 w 1 z 1 + ⋯ + c n w n z n \begin{aligned}
\vec{w}^TA\vec{z}
&= \begin{bmatrix} w_1 \cdots w_n \end{bmatrix}
\begin{bmatrix} &c_1, &0, &\cdots, &0 &\\ &0, &c_2, &\cdots, &0 &\\ &0,&0, &\cdots, &c_n &\end{bmatrix}
\begin{bmatrix} z_1 \\ \vdots \\ z_n \end{bmatrix}\\
&=c_1w_1z_1 + \cdots + c_nw_nz_n
\end{aligned}
w T A z = [ w 1 ⋯ w n ] ⎣ ⎡ c 1 , 0 , 0 , 0 , c 2 , 0 , ⋯ , ⋯ , ⋯ , 0 0 c n ⎦ ⎤ ⎣ ⎢ ⎡ z 1 ⋮ z n ⎦ ⎥ ⎤ = c 1 w 1 z 1 + ⋯ + c n w n z n
也可以在連續（實屬域上）函數的向量空間上定義內積空間，如假設 f , g f,g f , g [ a , b ] [a,b] [ a , b ]
⟨ f , g ⟩ = ∫ a b f ( t ) g ( t ) d t \langle f,g \rangle = \int_a^bf(t)g(t)dt
⟨ f , g ⟩ = ∫ a b f ( t ) g ( t ) d t
(1)檢查對稱性
⟨ f , g ⟩ = ∫ a b f ( t ) g ( t ) d t ⟨ g , f ⟩ = ∫ a b g ( t ) f ( t ) d t = ∫ a b f ( t ) g ( t ) d t \begin{aligned}
&\langle f,g \rangle = \int_a^bf(t)g(t)dt \\
&\langle g,f \rangle = \int_a^bg(t)f(t)dt = \int_a^bf(t)g(t)dt
\end{aligned}
⟨ f , g ⟩ = ∫ a b f ( t ) g ( t ) d t ⟨ g , f ⟩ = ∫ a b g ( t ) f ( t ) d t = ∫ a b f ( t ) g ( t ) d t
(2)檢查線性
⟨ α f + β g , h ⟩ = ∫ a b [ α f ( t ) + β g ( t ) ] h ( t ) d t = ∫ a b [ α f ( t ) h ( t ) + β g ( t ) h ( t ) ] d t = α ∫ a b f ( t ) h ( t ) d t + β ∫ a b g ( t ) h ( t ) d t = α ⟨ f , h ⟩ + β ⟨ g , h ⟩ \begin{aligned}
\langle \alpha f+\beta g,h \rangle
&= \int_a^b\left[\alpha f(t)+\beta g(t)\right]h(t)dt \\
&= \int_a^b\left[\alpha f(t)h(t)+\beta g(t)h(t)\right]dt\\
&= \alpha\int_a^bf(t)h(t)dt+\beta\int_a^bg(t)h(t)dt\\
&= \alpha\langle f,h \rangle + \beta\langle g,h \rangle
\end{aligned}
⟨ α f + β g , h ⟩ = ∫ a b [ α f ( t ) + β g ( t ) ] h ( t ) d t = ∫ a b [ α f ( t ) h ( t ) + β g ( t ) h ( t ) ] d t = α ∫ a b f ( t ) h ( t ) d t + β ∫ a b g ( t ) h ( t ) d t = α ⟨ f , h ⟩ + β ⟨ g , h ⟩
(3)檢查非負確定性
⟨ f , f ⟩ = ∫ a b f ( t ) f ( t ) d t = ∫ a b [ f ( t ) ] 2 d t ≥ 0 \begin{aligned}
\langle f,f \rangle
&= \int_a^bf(t)f(t)dt \\
&= \int_a^b\left[f(t)\right]^2dt \geq 0
\end{aligned}
⟨ f , f ⟩ = ∫ a b f ( t ) f ( t ) d t = ∫ a b [ f ( t ) ] 2 d t ≥ 0
函數 [ f ( t ) ] 2 \left[f(t)\right]^2 [ f ( t ) ] 2 [ a , b ] \left[a,b\right] [ a , b ] [ f ( t ) ] 2 \left[f(t)\right]^2 [ f ( t ) ] 2 [ f ( t ) ] 2 \left[f(t)\right]^2 [ f ( t ) ] 2 [ a , b ] \left[a,b\right] [ a , b ]
⟨ p , q ⟩ = ∫ 0 ∞ p ( x ) q ( x ) e − x d x \langle p,q \rangle = \int_0^{\infty}p(x)q(x)e^{-x}dx
⟨ p , q ⟩ = ∫ 0 ∞ p ( x ) q ( x ) e − x d x
定義 內積空間
一個 內積空間 V V V V V V
最常見的內積空間的例子是 F n \mathbb{F}^n F n
∀ u ⃗ ∈ V \forall \vec{u} \in V ∀ u ∈ V v ⃗ \vec{v} v ⟨ u ⃗ , v ⃗ ⟩ \langle \vec{u}, \vec{v} \rangle ⟨ u , v ⟩ V V V F \mathbb{F} F ⟨ 0 ⃗ , u ⃗ ⟩ = 0 , ∀ u ⃗ ∈ V \langle \vec{0},\vec{u} \rangle=0, \,\, \forall \vec{u} \in V ⟨ 0 , u ⟩ = 0 , ∀ u ∈ V ⟨ u ⃗ , 0 ⃗ ⟩ = 0 , ∀ u ⃗ ∈ V \langle \vec{u},\vec{0} \rangle=0, \,\, \forall \vec{u} \in V ⟨ u , 0 ⟩ = 0 , ∀ u ∈ V ⟨ u ⃗ , v ⃗ + w ⃗ ⟩ = ⟨ u ⃗ , v ⃗ ⟩ + ⟨ u ⃗ , w ⃗ ⟩ , ∀ u ⃗ , v ⃗ , w ⃗ ∈ V \langle \vec{u}, \vec{v}+\vec{w} \rangle = \langle \vec{u}, \vec{v} \rangle + \langle \vec{u}, \vec{w} \rangle, \,\, \forall \vec{u},\vec{v}, \vec{w} \in V ⟨ u , v + w ⟩ = ⟨ u , v ⟩ + ⟨ u , w ⟩ , ∀ u , v , w ∈ V ⟨ u ⃗ , λ v ⃗ ⟩ = λ ‾ ⟨ u ⃗ , v ⃗ ⟩ , ∀ λ ∈ F , ∀ u ⃗ , v ⃗ ∈ V \langle \vec{u}, \lambda\vec{v}\rangle = \overline{\lambda}\langle \vec{u}, \vec{v}\rangle, \,\, \forall \lambda\in\mathbb{F},\,\, \forall \vec{u}, \vec{v} \in V ⟨ u , λ v ⟩ = λ ⟨ u , v ⟩ , ∀ λ ∈ F , ∀ u , v ∈ V
Proof.
1. \, 疊加原理 。
⟨ c 1 u 1 ⃗ + c 2 u 2 ⃗ , v ⃗ ⟩ = ⟨ c 1 u 1 ⃗ , v ⃗ ⟩ + ⟨ c 2 u 2 ⃗ , v ⃗ ⟩ （首位可加性） = c 1 ⟨ u 1 ⃗ , v ⃗ ⟩ + c 2 ⟨ u 2 ⃗ , v ⃗ ⟩ （首位齊次性） \begin{aligned}
&\langle c_1\vec{u_1}+c_2\vec{u_2},\vec{v} \rangle \\
=& \langle c_1\vec{u_1},\vec{v} \rangle + \langle c_2\vec{u_2},\vec{v} \rangle \text{（首位可加性）} \\
=& c_1\langle \vec{u_1},\vec{v} \rangle + c_2\langle \vec{u_2},\vec{v} \rangle \text{（首位齊次性）}
\end{aligned}
= = ⟨ c 1 u 1 + c 2 u 2 , v ⟩ ⟨ c 1 u 1 , v ⟩ + ⟨ c 2 u 2 , v ⟩ （首位可加性） c 1 ⟨ u 1 , v ⟩ + c 2 ⟨ u 2 , v ⟩ （首位齊次性）
故滿足疊加性，為線性映射。
2. \, 0 ⃗ ↦ 0 \vec{0} \mapsto 0 0 ↦ 0
3. \, ⟨ u ⃗ , 0 ⃗ ⟩ = ⟨ 0 ⃗ , u ⃗ ⟩ ‾ = 0 ‾ = 0 \langle \vec{u},\vec{0} \rangle=\overline{\langle \vec{0},\vec{u} \rangle}=\overline{0}=0 ⟨ u , 0 ⟩ = ⟨ 0 , u ⟩ = 0 = 0
4. \, ∀ u ⃗ , v ⃗ , w ⃗ ∈ V \forall \vec{u},\vec{v}, \vec{w} \in V ∀ u , v , w ∈ V
⟨ u ⃗ , v ⃗ + w ⃗ ⟩ = ⟨ v ⃗ + w ⃗ , u ⃗ ⟩ ‾ = ⟨ v ⃗ , u ⃗ ⟩ + ⟨ w ⃗ , u ⃗ ⟩ ‾ = ⟨ v ⃗ , u ⃗ ⟩ ‾ + ⟨ w ⃗ , u ⃗ ⟩ ‾ = ⟨ u ⃗ , v ⃗ ⟩ + ⟨ u ⃗ , w ⃗ ⟩ \begin{aligned}
\langle \vec{u}, \vec{v}+\vec{w} \rangle
&= \overline{\langle \vec{v}+\vec{w},\vec{u}\rangle} \\
&= \overline{\langle \vec{v},\vec{u}\rangle + \langle \vec{w},\vec{u}\rangle} \\
&= \overline{\langle \vec{v},\vec{u}\rangle} + \overline{\langle \vec{w},\vec{u}\rangle} \\
&= \langle \vec{u},\vec{v}\rangle + \langle \vec{u},\vec{w}\rangle
\end{aligned}
⟨ u , v + w ⟩ = ⟨ v + w , u ⟩ = ⟨ v , u ⟩ + ⟨ w , u ⟩ = ⟨ v , u ⟩ + ⟨ w , u ⟩ = ⟨ u , v ⟩ + ⟨ u , w ⟩
5. \, ∀ λ ∈ F , ∀ u ⃗ , v ⃗ ∈ V \forall \lambda\in\mathbb{F},\,\, \forall \vec{u}, \vec{v} \in V ∀ λ ∈ F , ∀ u , v ∈ V
⟨ u ⃗ , λ v ⃗ ⟩ = ⟨ λ v ⃗ , u ⃗ ⟩ ‾ = λ ⟨ v ⃗ , u ⃗ ⟩ ‾ = λ ‾ ⟨ v ⃗ , u ⃗ ⟩ ‾ = λ ‾ ⟨ u ⃗ , v ⃗ ⟩ \begin{aligned}
\langle \vec{u}, \lambda\vec{v}\rangle
&= \overline{\langle \lambda\vec{v}, \vec{u}\rangle} \\
&= \overline{\lambda\langle \vec{v}, \vec{u}\rangle} \\
&= \overline{\lambda}\,\overline{\langle \vec{v}, \vec{u}\rangle} \\
&= \overline{\lambda}\,\langle \vec{u}, \vec{v}\rangle
\end{aligned}
⟨ u , λ v ⟩ = ⟨ λ v , u ⟩ = λ ⟨ v , u ⟩ = λ ⟨ v , u ⟩ = λ ⟨ u , v ⟩
定義 範數
對於 v ⃗ ∈ V \vec{v} \in V v ∈ V 範數 ∥ v ⃗ ∥ \lVert \vec{v} \rVert ∥ v ∥
∥ v ⃗ ∥ = ⟨ v ⃗ , v ⃗ ⟩ . \lVert \vec{v} \rVert = \sqrt{\langle \vec{v} , \vec{v} \rangle}
.
∥ v ∥ = ⟨ v , v ⟩ .
若 v ⃗ ∈ V \vec{v} \in V v ∈ V
∥ v ⃗ ∥ = 0 ⟺ v ⃗ = 0 ⃗ \lVert \vec{v} \rVert=0 \iff \vec{v} = \vec{0} ∥ v ∥ = 0 ⟺ v = 0 ∥ λ v ⃗ ∥ = ∣ λ ∣ ∥ v ⃗ ∥ , ∀ λ ∈ F \lVert \lambda\vec{v} \rVert= \lvert \lambda \rvert \, \lVert \vec{v} \rVert, \,\, \forall \lambda \in \mathbb{F} ∥ λ v ∥ = ∣ λ ∣ ∥ v ∥ , ∀ λ ∈ F
Proof.
1略
2中：
∥ λ v ⃗ ∥ 2 = ⟨ λ v ⃗ , λ v ⃗ ⟩ = λ λ ‾ ⟨ v ⃗ , v ⃗ ⟩ = ∣ λ ∣ 2 ∥ v ⃗ ∥ 2 \begin{aligned}
\lVert \lambda\vec{v} \rVert^2 &= \langle \lambda\vec{v},\lambda\vec{v} \rangle \\
&=\lambda\overline{\lambda} \langle \vec{v},\vec{v} \rangle \\
&=\lvert \lambda \rvert^2 \lVert \vec{v} \rVert^2
\end{aligned}
∥ λ v ∥ 2 = ⟨ λ v , λ v ⟩ = λ λ ⟨ v , v ⟩ = ∣ λ ∣ 2 ∥ v ∥ 2
