Inflationary spacetimes have been argued to be past geodesically incomplete in many situations. However, whether the geodesic incompleteness implies the existence of an initial spacetime curvature singularity or whether the spacetime may be extended (potentially into another phase of the universe) is generally unknown. Both questions have important physical implications. In this paper, we take a closer look at the geometrical structure of inflationary spacetimes and investigate these very questions. We first classify which past inflationary histories have a scalar curvature singularity and which might be extendible and/or non-singular in homogeneous and isotropic cosmology with flat spatial sections. Then, we derive rigorous extendibility criteria of various regularity classes for quasi-de Sitter spacetimes that evolve from infinite proper time in the past. Finally, we show that beyond homogeneity and isotropy, special continuous extensions respecting the Einstein field equations with a perfect fluid must have the equation of state of a de Sitter universe asymptotically. An interpretation of our results is that past-eternal inflationary scenarios are most likely physically singular, except in situations with very special initial conditions.