Unlike graphene and other 2D materials, borophene is 2D polymorphic with diverse hexagonal holes (HHs)-triangles ratios and the concentrations of HHs are highly substrate dependent. Here, we systematically explored the evolution of boron cluster on Ag(111) surface, B-N@Ag(111) (N=1 similar to 36), to understand the nucleation of 2D boron sheet on metal surface. Our calculation showed that, with the size increasing, the structures of most stable B-N clusters undergo an evolution from compact triangular lattice, such as double-chains or triple-chains, to the ones with mixed triangular-hexagonal lattices. The first single-HH appears at N=12 and the first double-HH appears at N=27. The stability of large B-N clusters with mixed structures is derived from the charge transfer between triangular lattice and the HHs, as well as between the substrates and the B-N clusters. Our results provide a deep understanding on the formation of small boron clusters in the initial nucleation stage of borophene growth.