Keywords [1] The columns \(q_1,...,q_n\) are orthonormal if \(q_i^Tq_j = \begin{cases} 0\ \text{for}\ i \neq j \\\ 1 \text{for}\ i = j \end{cases}\) [2] If \(Q\) is also square, then \(QQ^T=I...

Keywords [1] Solving \(A^TA\hat{x}=A^Tb\) gives the projection \(p=A\hat{x}\) of \(b\) onto the column space of \(A\) [2] When \(Ax=b\) has no solution, \(\hat{x}\) is the least-squares solut...

Keywords [1] The projection of a vector \(b\) onto the line through \(a\) is the closest point \(p = a\frac{a^Tb}{a^Ta}\) [2] The error \(e = b - p\) is perpendicular to \(a\): Right triangle...

Keywords [1] Orthogonal vectors have \(v^Tw=0\). Then \(\lVert v \rVert^2 + \lVert w \rVert^2 = \lVert v + w \rVert^2 = \lVert v - w \rVert^2\) [2] Subspaces \(V\) and \(W\) are orthogonal wh...

Keywords [1] The column space \(C(A)\) and the row space \(C(A^T)\) both have dimension \(r\) (the rank of \(A\)) [2] The nullspace \(N(A)\) has dimension \(n-r\). The left nullspace \(N(A^T)...

Keywords [1] Independent columns of \(A\): The only solution to \(Ax=0\) is \(x=0\). The nullspace is \(Z\) [2] Independent vectors: The only zero combination \(c_1v_1 + \cdots + c_kv_k = 0\)...

Double Integral The double integral of \(f\) over the rectangle \(R\) is [\iintR f(x,y)dA = \lim_{m,n \rightarrow \infty} \sum_{i=1}^m \sum_{j=1}^n f(x{ij}^*, y_{ij}^*) \Delta A] If \(f(x,y)...

Lagrange Multiplier Suppose a function \(f\) has an extreme value at a point \(P(x_0,y_0,z_0)\) on the surface \(S\). Let \(C\) be a curve defined by a vector equation \(r(t) = \left\langle x...

Local maximum and Local minimum A function of two variables has a local maximum at \((a,b)\) if \(f(x,y) \leq f(a,b)\) when \((x,y)\) is near \((a,b)\). This means \(f(x,y) \leq f(a,b)\) for a...

Introduction Hello there! welcome to the new episode: Vision Transformer which you might’ve heard before. I highly recommend you to go through the breakthrough paper Attention is all you need ...