In theoretical mathematics a singular matrix is usually defined as an n by n matrix with rank less than n. For rectangular matrices it is natural to generalize this and to define an m by n matrix to be singular if its rank is less than min(m,n) . However, these definitions have difficulties in the presence of errors in the matrix elements. To quote from Matrix Computations, by Gene Golub and Charles Van Loan, third edition, page 72:

Numerous theorems in linear algebra have the form "if such-and-such a matrix has full rank, then such-and-such property holds." While neat and aesthetic, results of this flavor do not help us address the numerical difficulties frequently encountered in situations where near rank deficiency prevails. Rounding errors and fuzzy data make rank determination a nontrivial exercise. Indeed, for some small ε we may be interested in the ε rank of a matrix which we define byIn this quote Golub and Van Loan are using the letter u to indicate relative machine precision and the norm || || to indicate the Euclidean or two norm. Golub and Van Loan prove that rank(A, tol) can be calculated by determining the number of singular values greater than tol. Therefore our use of the term "numerically singular" is the same as Golub and Van Loan's use of "numerically rank deficient" except that our tolerance for the numerical rank is slightly different. We use a tolerance of max(m,n) eps( norm(A) ) which is also the default tolerance in Matlab's rank command.rank(A,ε) = min Thus if A is obtained in a laboratory with each a_{||A-B|| ≤ ε }rank(B)._{ij}correct to within ± .001 then it might make sense to look at rank(A, .001). Along these same lines, if A is an m by n floating point matrix then it is reasonable to regard A asnumerically rank deficientif rank(A,ε) < min(m,n) with ε = u ||A||.

The difference between exact rank and numerical rank can also be characterized in terms of matrix null spaces. An m by n matrix A has rank r if the dimension of the null space of A is n-r. This means that there exist a subspace S of dimension n-r such that

In practice, in a floating point number system, due to computer arithmetic errors the calculated value of A x in (1) will usually not be exactly zero for x ≠ 0. Therefore, in practice, one must be satisfied if A x in (1) is "almost zero" or "numerically zero." The concept of numerical rank captures this idea in a precise sense. Using the minimax characterization of singular values one can show that if A has numerical rank r with tolerance tol then there exists a subspace S of dimension n-r such that

For an example see this link .