The energy loss (E-loss), especially the nonradiative recombination loss and energetic disorder, needs to be minimized to improve the device performance with a small voltage (V-OC) loss. Urbach energy (E-U) of organic photovoltaic materials is related to energetic disorder, which can predict the E-loss of the corresponding device. Herein, a polymer donor (PBDS-TCl) with Si and Cl functional atoms for organic solar cells (OSCs) is synthesized. It can be found that the V-OC and E-loss can be well manipulated by regulation of the energy level of the polymer donor and E-U, which is dominated by the morphology. A low energetic disorder with an E-U of 23.7 meV, a low driving force of 0.08 eV, and a low E-loss of 0.41 eV are achieved for the PBDS-TCl:Y6-based OSCs. Consequently, an impressive open circuit voltage (V-OC) of 0.92 V is obtained. To the best of knowledge, the V-OC value and E-loss are both the record values for the Y6-based device. These results demonstrate that fine-tuning the polymer donor by functional atom modification on the side chain is a promising way to reduce E-U and energy loss, as well as obtain small driving force and high V-OC for highly efficient OSCs.