X-ray diffraction study of black phosphorus (BP) shows forbidden reflections, which imply a broken inversion symmetry with atomic distortion in the orthorhombic lattice. The unexpected distortion occurs throughout the entire bulk crystal, rather than being confined to just the top-surface layer. Thus our observation suggests that this distortion is an intrinsic bulk property, in contrast to earlier studies which attributed the distortion to surface effects or defects. Our diffusion quantum Monte Carlo calculation predicts the same asymmetric structure. Density functional theory calculation incorporating Hubbard correction attributes the symmetry breaking to intrinsic electronic interaction in sp hybridized orbitals. We propose that the asymmetric lattice distortion in BP could enable stable two-dimensional ferroelectricity.