Silicon is a common material of choice for semiconductor optics in the infrared spectral range, due to its low cost, well-developed high-volume manufacturing methods, high refractive index, and transparency. It is, however, typically ill-suited for applications in the visible range, due to its large absorption coefficient, especially for green and blue light. Counterintuitively, we demonstrate how ultra-thin crystalline meta-optics enable full-color imaging in the visible range. For this purpose, we employ an inverse design approach, which maximizes the volume under the broadband modulation transfer function of the meta-optics. Beyond that, we demonstrate polarization-multiplexed functionality in the visible. This is particularly important as polarization optics require high index materials, a characteristic often difficult to obtain in the visible