In this work, we propose a fast and second-order numerical method for solving the Caputo variable-order(VO) time fractional diffusion equation based on a L2-1σ method [1] and “SBBP” algorithm [2]. For the Caputo VO function α(t), the sum of exponential (SOE) calculation of the kernel function that is an efficient algorithm to reduce computational complexity, requires approximating the points and weights at every time step. Here we employ the shifted binary block partition (SBBP) to decompose the integral in the derivative and approximate the scaled kernel function in each sub-intervals by polynomials of degree r whose computational complexity is O(rn log n). But it has a low convergence order of O(Δt2−¯α), where ¯α = ||α(t)||∞. We propose a new L2-1σ that has a convergence of O(Δt3−¯α) and apply it to approximate the Caputo variable-order(VO) time fractional diffusion equation. The stability and the error analysis has been proved. Several numerical results show the effectiveness of the proposed method and demonstrate the accuracy and performance of the theory.