One of the major problems in the homotopy theory of finite loop spaces is the classification problem for p-compact groups. It has been proposed to use the maximal torus normalizer (which at an odd prime essentially means the Weyl group) as the distinguishing invariant. We show here that the maximal torus normalizer does indeed classify many p-compact groups up to isomorphism when p is an odd prime.