Let (S-n)(n >= 0) denote an indeterminate Hamburger moment sequence and let H = {s(m+n)} be the corresponding positive definite Hankel matrix. We consider the question if there exists an infinite symmetric matrix A = {a(j, k)} which is an inverse of H in the sense that the matrix product AN is defined by absolutely convergent series and AN equals the identity matrix I, a property called ( aci). A candidate for A is the coefficient matrix of the reproducing kernel of the moment problem, considered as an entire function of two complex variables. We say that the moment problem has property (aci), if (aci) holds for this matrix A. We show that this is true for many classical indeterminate moment problems but not for the symmetrized version of a cubic birth-and-death process studied by Valent and co-authors. We consider mainly symmetric indeterminate moment problems and give a number of sufficient conditions for (aci) to hold in terms of the recurrence coefficients for the orthonormal polynomials. A sufficient condition is a rapid increase of the recurrence coefficients in the sense that the quotient between consecutive terms is uniformly bounded by a constant strictly smaller than one. We also give a simple example, where (aci) holds, but an inverse matrix of H is highly non-unique.