Understanding the intrinsic phase stability and inherent band gap of formamidinium lead triiodide (FAPbI(3)) perovskites is crucial to further improve the performance of perovskite solar cells (PSCs). Herein, we explored the alpha- to delta-phase transition and band gap of FAPbI(3) single crystals grown by an inverse temperature solubility method. We found that the residual gamma-butyrolactone solvents in the inner empty space of the FAPbI(3) single crystal accelerate the phase transition at kinetics. By adopting 2-methoxyethanol as the solvent, over 2000 h of stable alpha-FAPbI(3) crystals could be acquired. This proves that although FAPbI(3) is regarded as unstable at thermodynamics, it could own excellent kinetic stability without any doping or additives because of the slow solid to solid phase transition instead of the fast phase transition assisted by the solvents. Furthermore, we revealed that the bulk FAPbI(3) single crystal with a size above 100 mu m can have an inherent band gap of 1.41 eV. Thus, our work provides key scientific guidance for high-performance FAPbI(3)-based PSCs.