🟣 Vector Spaces: Span, Independence & Basis
Unlock the structure of space! Learn how vectors build dimensions through span, independence, and basis.
🏗️ Linear Combinations and Span
In Part 1, we introduced the abstract concept of vector spaces. Now, let's explore how to construct vectors within a vector space using a fundamental operation: linear combinations. The set of all possible vectors we can create in this way is called the span.
Definition: Linear Combination
A Linear Combination of vectors $\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_p$ is a vector of the form:
$c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \ldots + c_p\mathbf{v}_p$
where $c_1, c_2, \ldots, c_p$ are scalars (weights).
Definition: Span
The Span of a set of vectors is the set of all possible linear combinations of those vectors. It represents the "space" covered by the vectors.
$\text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_p\} = \{ c_1\mathbf{v}_1 + \ldots + c_p\mathbf{v}_p \mid c_i \in \mathbb{R} \}$
Example 1: Linear Combination in $\mathbb{R}^2$
Let $\mathbf{v}_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$ and $\mathbf{v}_2 = \begin{bmatrix} -1 \\ 1 \end{bmatrix}$. Find $3\mathbf{v}_1 - 2\mathbf{v}_2$.
$3\begin{bmatrix} 1 \\ 2 \end{bmatrix} - 2\begin{bmatrix} -1 \\ 1 \end{bmatrix} = \begin{bmatrix} 3 \\ 6 \end{bmatrix} + \begin{bmatrix} 2 \\ -2 \end{bmatrix} = \begin{bmatrix} 5 \\ 4 \end{bmatrix}$
So, $\begin{bmatrix} 5 \\ 4 \end{bmatrix}$ is a linear combination of $\mathbf{v}_1$ and $\mathbf{v}_2$.
Example 2: Span in $\mathbb{R}^2$
What is $\text{Span}\{ \begin{bmatrix} 1 \\ 0 \end{bmatrix} \}$?
It is the set of all vectors $c \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} c \\ 0 \end{bmatrix}$. This represents the entire x-axis.
Example 3: Span in $\mathbb{R}^3$
What is $\text{Span}\{ \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \}$?
It is the set of vectors $\begin{bmatrix} c_1 \\ c_2 \\ 0 \end{bmatrix}$. This represents the entire xy-plane in 3D space.
💡 Linear Independence
A set of vectors is linearly independent if none of them is redundant; that is, none can be written as a linear combination of the others.
Definition: Linear Independence
A set $\{\mathbf{v}_1, \ldots, \mathbf{v}_p\}$ is Linearly Independent if the equation:
$c_1\mathbf{v}_1 + \ldots + c_p\mathbf{v}_p = \mathbf{0}$
has only the trivial solution $c_1 = \ldots = c_p = 0$.
If there are non-zero weights that satisfy the equation, the set is Linearly Dependent.
Example 4: Independence Check
Are $\mathbf{v}_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$ and $\mathbf{v}_2 = \begin{bmatrix} -1 \\ 1 \end{bmatrix}$ independent?
Solve $c_1\begin{bmatrix} 1 \\ 2 \end{bmatrix} + c_2\begin{bmatrix} -1 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}$.
System: $c_1 - c_2 = 0$ and $2c_1 + c_2 = 0$. Adding gives $3c_1 = 0 \Rightarrow c_1=0, c_2=0$.
Result: Linearly Independent.
Example 5: Dependence Check
Vectors $\mathbf{v}_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \mathbf{v}_2 = \begin{bmatrix} 2 \\ 4 \end{bmatrix}$.
Notice $\mathbf{v}_2 = 2\mathbf{v}_1$. So $2\mathbf{v}_1 - \mathbf{v}_2 = \mathbf{0}$. Non-trivial weights exist.
Result: Linearly Dependent.
🔑 Basis and Dimension
A basis is the "gold standard" set of vectors: it spans the space and is independent. It's a minimal spanning set.
Definition: Basis & Dimension
A set $\mathcal{B}$ is a Basis for $V$ if:
- $\mathcal{B}$ spans $V$
- $\mathcal{B}$ is linearly independent
The Dimension of $V$, denoted $\dim(V)$, is the number of vectors in a basis.
Example 6: Standard Basis of $\mathbb{R}^2$
$\mathcal{B} = \{ \mathbf{e}_1, \mathbf{e}_2 \} = \{ \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \end{bmatrix} \}$
This set spans $\mathbb{R}^2$ and is independent. It is the standard basis. $\dim(\mathbb{R}^2) = 2$.
Example 7: Polynomial Basis
For $P_2$ (polynomials degree $\le 2$), the standard basis is $\{ 1, x, x^2 \}$.
Any polynomial $a + bx + cx^2$ is a linear combination of these three. $\dim(P_2) = 3$.
🌍 Real-World Applications
1️⃣ Digital Color Spaces (RGB)
Colors on a screen are vectors in a 3D space spanned by Red, Green, and Blue basis vectors. Any color is a linear combination: $C = rR + gG + bB$. This is a direct application of span and basis!
2️⃣ Image Compression (JPEG)
Images are broken down into basis blocks (like cosine waves). Compression works by keeping only the most important basis coefficients and discarding the rest (linear dependence/approximation).
3️⃣ Genetics (Genotype Space)
In population genetics, the state of a population can be represented as a vector in a space where the basis vectors represent different genotypes. Evolution is a transformation in this space.
4️⃣ Search Engines (Latent Semantic Analysis)
Documents are vectors in a "word space". Concepts are found by finding a new basis (using SVD) that captures the underlying meaning, reducing the dimension of the space.
🎯 Practice Questions
Master the building blocks of vector spaces!
🔥 Challenge Questions
Deep dive into basis and dimension!