What Is the Hint of a Matrix? – Nick Higham

January 24, 2023

96

The hint of an $ntimes n$ matrix is the sum of its diagonal components: $mathrm{trace}(A) = sum_{i=1}^n a_{ii}$ . The hint is linear, that’s, $mathrm{trace}(A+B) = mathrm{trace}(A) + mathrm{trace}(B)$ , and $mathrm{trace}(A) = mathrm{trace}(A^T)$ .

A key truth is that the hint can be the sum of the eigenvalues. The proof is by contemplating the attribute polynomial $p(t) = det(t I - A) = t^n + a_{n-1}t^{n-1} + dots + a_1 t + a_0$ . The roots of $p$ are the eigenvalues $lambda_1, lambda_2, dots, lambda_n$ of $A$ , so $p$ will be factorized

$notag p(t) = (t - lambda_1) (t - lambda_2) dots (t - lambda_n),$

and so $a_{n-1} = -(lambda_{11} + lambda_{22} + cdots + lambda_{nn})$ . The Laplace enlargement of $det(t I - A)$ reveals that the coefficient of $t^{n-1}$ is $-(a_{11} + a_{22} + cdots + a_{nn})$ . Equating these two expressions for $a_{n-1}$ offers

$notag mathrm{trace}(A) = displaystylesum_{i=1}^n lambda_i. qquadqquad(1)$

A consequence of (1) is that any transformation that preserves the eigenvalues preserves the hint. Due to this fact the hint is unchanged underneath similarity transformations: $mathrm{trace}(X^{-1}AX) = mathrm{trace}(A)$ for any nonsingular $X$ .

An an instance of how the hint will be helpful, suppose $A$ is a symmetric and orthogonal $ntimes n$ matrix, in order that its eigenvalues are $pm 1$ . If there are $p$ eigenvalues $1$ and $q$ eigenvalues $-1$ then $mathrm{trace}(A) = p - q$ and $n = p + q$ . Due to this fact $p = (n + mathrm{trace}(A))/2$ and $q = (n - mathrm{trace}(A))/2$ .

One other necessary property is that for an $mtimes n$ matrix $A$ and an $ntimes m$ matrix $B$ ,

$notag mathrm{trace}(AB) = mathrm{trace}(BA) qquadqquad(2)$

(although $AB ne BA$ basically). The proof is easy:

$notag begin{aligned} mathrm{trace}(AB) &= displaystylesum_{i=1}^m (AB)_{ii} = sum_{i=1}^m sum_{k=1}^n a_{ik} b_{ki} = sum_{k=1}^n sum_{i=1}^m b_{ki} a_{ik} & = sum_{k=1}^n (BA)_{kk} = mathrm{trace}(BA). end{aligned}$

This straightforward truth can have non-obvious penalties. For instance, take into account the equation $AX - XA = I$ in $ntimes n$ matrices. Taking the hint offers $0 = mathrm{trace}(AX) - mathrm{trace}(XA) = mathrm{trace}(AX - XA) = mathrm{trace}(I) = n$ , which is a contradiction. Due to this fact the equation has no resolution.

The relation (2) offers $mathrm{trace}(ABC) = mathrm{trace}((AB)C) = mathrm{trace}(C(AB)) = mathrm{trace}(CAB)$ for $ntimes n$ matrices $A$ , $B$ , and $C$ , that’s,

$notag mathrm{trace}(ABC) = mathrm{trace}(CAB). qquadqquad(3)$

So we will cyclically permute phrases in a matrix product with out altering the hint.

For instance of the usage of (2) and (3), if $x$ and $y$ are $n$ -vectors then $mathrm{trace}(xy^T) = mathrm{trace}(y^Tx) = y^Tx$ . If $A$ is an $ntimes n$ matrix then $mathrm{trace}(xy^TA)$ will be evaluated with out forming the matrix $xy^TA$ since, by (3), $mathrm{trace}(xy^TA) = mathrm{trace}(y^TAx) = y^TAx$ .

The hint is helpful in calculations with the Frobenius norm of an $mtimes n$ matrix:

$notag |A|_F = left(displaystylesum_{i=1}^m sum_{i=1}^n |a_{ij}|^2right)^{1/2} = bigr(mathrm{trace}(A^*A)bigr)^{1/2},$

the place $*$ denotes the conjugate transpose. For instance, we will generalize the formulation $|x+mathrm{i}y|^2 = x^2 + y^2$ for a fancy quantity to an $mtimes n$ matrix $A$ by splitting $A$ into its Hermitian and skew-Hermitian elements:

$notag A = frac{1}{2}(A+A^*) + frac{1}{2}(A-A^*) equiv B + C,$

the place $B = B^*$ and $C = -C^*$ . Then

$notag begin{aligned} |A|_F^2 = |B + C|_F^2 &= mathrm{trace}((B+C)^*(B+C)) &= mathrm{trace}(B^*B + C^*C) + mathrm{trace}(B^*C + C^*B) &= mathrm{trace}(B^*B + C^*C) + mathrm{trace}(BC - CB) &= mathrm{trace}(B^*B + C^*C) &= |B|_F^2 + |C|_F^2. end{aligned}$

If a matrix is just not explicitly identified however we will compute matrix–vector merchandise with it then the hint will be estimated by

$notag mathrm{trace}(A) approx x^TAx,$

the place the vector $x$ has components independently drawn from the usual regular distribution with imply $0$ and variance $1$ . The expectation of this estimate is

$notag begin{aligned} Ebigl( x^TAx bigr) &= Ebigl( mathrm{trace}(x^TAx) bigr) = Ebigl( mathrm{trace}(Axx^T) bigr) = mathrm{trace} bigl( E(Axx^T) bigr) &= mathrm{trace}bigl(A E(xx^T) bigr) = mathrm{trace}(A), end{aligned}$

since $E(x_ix_j) = 0$ for $ine j$ and $E(x_i^2) = 1$ for all $i$ , so $E(xx^T) = I$ . This stochastic estimate, which is because of Hutchinson, is subsequently unbiased.