In mathematics, particularly in linear algebra, the Schur product theorem states that the Hadamard product of two positive definite matrices is also a positive definite matrix. The result is named after Issai Schur[1] (Schur 1911, p. 14, Theorem VII) (note that Schur signed as J. Schur in Journal für die reine und angewandte Mathematik.[2][3])
We remark that the converse of the theorem holds in the following sense. If
is a symmetric matrix and the Hadamard product
is positive definite for all positive definite matrices
, then
itself is positive definite.
Proof
For any matrices
and
, the Hadamard product
considered as a bilinear form acts on vectors
as
![{\displaystyle a^{*}(M\circ N)b=\operatorname {tr} \left(M^{\textsf {T}}\operatorname {diag} \left(a^{*}\right)N\operatorname {diag} (b)\right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/de6c7ce5bf1a957f8a00e02034b1c86d56aea33b)
where
is the matrix trace and
is the diagonal matrix having as diagonal entries the elements of
.
Suppose
and
are positive definite, and so Hermitian. We can consider their square-roots
and
, which are also Hermitian, and write
![{\displaystyle \operatorname {tr} \left(M^{\textsf {T}}\operatorname {diag} \left(a^{*}\right)N\operatorname {diag} (b)\right)=\operatorname {tr} \left({\overline {M}}^{\frac {1}{2}}{\overline {M}}^{\frac {1}{2}}\operatorname {diag} \left(a^{*}\right)N^{\frac {1}{2}}N^{\frac {1}{2}}\operatorname {diag} (b)\right)=\operatorname {tr} \left({\overline {M}}^{\frac {1}{2}}\operatorname {diag} \left(a^{*}\right)N^{\frac {1}{2}}N^{\frac {1}{2}}\operatorname {diag} (b){\overline {M}}^{\frac {1}{2}}\right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/0c4f186c647530742e8226dc1889def097be02db)
Then, for
, this is written as
for
and thus is strictly positive for
, which occurs if and only if
. This shows that
is a positive definite matrix.
Proof using Gaussian integration
Case of M = N
Let
be an
-dimensional centered Gaussian random variable with covariance
. Then the covariance matrix of
and
is
![{\displaystyle \operatorname {Cov} \left(X_{i}^{2},X_{j}^{2}\right)=\left\langle X_{i}^{2}X_{j}^{2}\right\rangle -\left\langle X_{i}^{2}\right\rangle \left\langle X_{j}^{2}\right\rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/052f1a33465e1471f026f55ba04c174f1bb2c31d)
Using Wick's theorem to develop
we have
![{\displaystyle \operatorname {Cov} \left(X_{i}^{2},X_{j}^{2}\right)=2\left\langle X_{i}X_{j}\right\rangle ^{2}=2M_{ij}^{2}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/88c87b97de9ef5a5dd9f16606a611615de482623)
Since a covariance matrix is positive definite, this proves that the matrix with elements
is a positive definite matrix.
General case
Let
and
be
-dimensional centered Gaussian random variables with covariances
,
and independent from each other so that we have
for any ![{\displaystyle i,j}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f4cbf8bbc622154cda8208d6e339495fe16a1f9a)
Then the covariance matrix of
and
is
![{\displaystyle \operatorname {Cov} \left(X_{i}Y_{i},X_{j}Y_{j}\right)=\left\langle X_{i}Y_{i}X_{j}Y_{j}\right\rangle -\left\langle X_{i}Y_{i}\right\rangle \left\langle X_{j}Y_{j}\right\rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/bc76e7d809c87ae155f8c70d8051a1de7d6bc2bc)
Using Wick's theorem to develop
![{\displaystyle \left\langle X_{i}Y_{i}X_{j}Y_{j}\right\rangle =\left\langle X_{i}X_{j}\right\rangle \left\langle Y_{i}Y_{j}\right\rangle +\left\langle X_{i}Y_{i}\right\rangle \left\langle X_{j}Y_{j}\right\rangle +\left\langle X_{i}Y_{j}\right\rangle \left\langle X_{j}Y_{i}\right\rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/879a3eb72aea2a8f8f4263445310cfb236ee63bc)
and also using the independence of
and
, we have
![{\displaystyle \operatorname {Cov} \left(X_{i}Y_{i},X_{j}Y_{j}\right)=\left\langle X_{i}X_{j}\right\rangle \left\langle Y_{i}Y_{j}\right\rangle =M_{ij}N_{ij}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/3dc916d06c7909c93d0a8cfbe6f3474f276f7cef)
Since a covariance matrix is positive definite, this proves that the matrix with elements
is a positive definite matrix.
Proof using eigendecomposition
Proof of positive semidefiniteness
Let
and
. Then
![{\displaystyle M\circ N=\sum _{ij}\mu _{i}\nu _{j}\left(m_{i}m_{i}^{\textsf {T}}\right)\circ \left(n_{j}n_{j}^{\textsf {T}}\right)=\sum _{ij}\mu _{i}\nu _{j}\left(m_{i}\circ n_{j}\right)\left(m_{i}\circ n_{j}\right)^{\textsf {T}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/9c1f83d126a04f5e6e38ef3865179c85ce9063c6)
Each
is positive semidefinite (but, except in the 1-dimensional case, not positive definite, since they are rank 1 matrices). Also,
thus the sum
is also positive semidefinite.
Proof of definiteness
To show that the result is positive definite requires even further proof. We shall show that for any vector
, we have
. Continuing as above, each
, so it remains to show that there exist
and
for which corresponding term above is nonzero. For this we observe that
![{\displaystyle a^{\textsf {T}}(m_{i}\circ n_{j})(m_{i}\circ n_{j})^{\textsf {T}}a=\left(\sum _{k}m_{i,k}n_{j,k}a_{k}\right)^{2}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/294c2fc38c699281e78688a8f6469e6906d1e525)
Since
is positive definite, there is a
for which
(since otherwise
for all
), and likewise since
is positive definite there exists an
for which
However, this last sum is just
. Thus its square is positive. This completes the proof.
References
- ^ Schur, J. (1911). "Bemerkungen zur Theorie der beschränkten Bilinearformen mit unendlich vielen Veränderlichen". Journal für die reine und angewandte Mathematik. 1911 (140): 1–28. doi:10.1515/crll.1911.140.1. S2CID 120411177.
- ^ Zhang, Fuzhen, ed. (2005). The Schur Complement and Its Applications. Numerical Methods and Algorithms. Vol. 4. doi:10.1007/b105056. ISBN 0-387-24271-6., page 9, Ch. 0.6 Publication under J. Schur
- ^ Ledermann, W. (1983). "Issai Schur and His School in Berlin". Bulletin of the London Mathematical Society. 15 (2): 97–106. doi:10.1112/blms/15.2.97.
External links
- Bemerkungen zur Theorie der beschränkten Bilinearformen mit unendlich vielen Veränderlichen at EUDML