Lemma.
If
V
is a subspace of
R
n
, then
(
V
⊥
)
⊥
=
V
. Further, if
P
V
is the orthogonal
projection of
R
n
onto
V
, then
I

P
V
is the orthogonal projection of
R
n
onto
V
⊥
.
Proof.
If
v
∈
V
, then
v
·
w
=
0
for all
w
∈
V
⊥
, so
v
∈
(
V
⊥
)
⊥
.
Thus
V
⊆
(
V
⊥
)
⊥
.
Let
x
∈
(
V
⊥
)
⊥
.
Write
x
=
v
+
w
, where
v
∈
V
and
w
∈
V
⊥
.
Since
x
⊥
V
⊥
, we have
0 =
x
·
w
=
v
·
w
+
w
·
w
=

w

2
. Hence
w
=
0
and
x
∈
V
. It follows that (
V
⊥
)
⊥
⊆
V
,
and consequently (
V
⊥
)
⊥
=
V
. Now
x
=
w
+
P
V
(
x
) where
w
∈
V
⊥
and
P
V
(
x
)
∈
(
V
⊥
)
⊥
.
Thus
w
=
P
V
⊥
(
x
). Since
x
=
P
V
(
x
) +
P
V
⊥
(
x
) for all
x
∈
R
n
,
P
V
+
P
V
⊥
is the identity
I
,
and
P
V
⊥
=
I

P
V
.
There is another useful characterization of the orthogonal projection, which follows from
the Pythagorean theorem.
Theorem 10.
Let
V
be a subspace of
R
n
, and let
P
V
be the orthogonal projection of
R
n
onto
V
. If
x
∈
R
n
, then
P
V
(
x
)
is the nearest point in
V
to
x
, that is,

x

P
V
(
x
)
 ≤ 
x

y

for all
y
∈
V
, with equality if and only if
y
=
P
V
(
x
)
.
Proof.
If
y
∈
V
, then
P
V
(
x
)

y
∈
V
, so
x

P
V
(
x
)
⊥
P
V
(
x
)

y
.
Since
x

y
=
(
x

P
V
(
x
)) + (
P
V
(
x
)

y
), we obtain from the Pythagorean theorem that

x

y

2
=

x

P
V
(
x
)

2
+

P
V
(
x
)

y

2
.
Thus

x

y

2
≥ 
x

P
V
(
x
)

2
, and equality holds if and only if

P
V
(
x
)

y

2
= 0, that is,
P
V
(
x
) =
y
.
4. Orthogonal Matrices
Definition: A square matrix
A
is
orthogonal
if
A
t
=
A

1
.
Theorem 11.
The following are equivalent, for an
n
×
n
matrix
A
:
(1)
A
is orthogonal, that is,
A
t
=
A

1
,
(2)
the columns of
A
form an orthonormal basis for
R
n
,
(3)
the rows of
A
form an orthonormal basis for
R
n
.
Proofsketch.
The matrix equation
AA
t
=
I
is equivalent to
n
2
scalar equations, which
are precisely the equations for orthonormality of the columns of
A
.
The matrix equation
A
t
A
=
I
is equivalent to the equations for the orthonormality of the rows of
A
.
Example: Permutation matrices are orthogonal. Each column of a permutation matrix
has one 1 and other entries 0, so each column vector has unit length. Different columns have
the 1’s in different places, so the column vectors are orthogonal.
Example: Rotation matrices are orthogonal. If a 2
×
2 matrix represents a rotation by
θ
,
the two columns represent the rotates of the unit vectors
e
1
and
e
1
by
θ
, and since
e
1
and
e
1
are orthogonal, their rotates by the same angle are orthogonal.
7
Lemma
.
Inverses of orthogonal matrices are orthogonal.
Proof
. If
A
is orthogonal, then
AA
t
=
I
, so (
A
t
)
t
=
A
= (
A
t
)

1
, and
A

1
is orthogonal.
Lemma
.
Products of orthogonal matrices are orthogonal.
Proof
. If
A
and
B
are orthogonal, then (
AB
)

1
=
B

1
A

1
=
B
t
A
t
= (
AB
)
t
, so
AB
is
orthogonal.
Lemma
.
The determinant of an orthogonal matrix is
±
1
.
Proof
.
From
I
=
AA
t
and the multiplicativity of the determinant, we obtain 1 =
det (
AA
t
) = det (
A
)det (
A
t
) = [det (
A
)]
2
. It follows that det (
A
) =
±
1.
Example: The determinant of a permutation matrix is the sign of the permutation.