Protein O-GlcNAcylation is a post-translational modification critical for multiple cellular functions including transcription, translation, signal transduction, and protein homeostasis. While it has been shown that O-GlcNAcylation modulates neuronal functions through “on-demand” protein modification, it remains to be determined whether O-GlcNAcylation is essential for glial cells such as microglia. In this study, we generated microglia-specific O-GlcNAc transferase (OGT) knockout (KO) mice to reveal the physiological roles of microglial O-GlcNAcylation in the brain. We found that the absence of O-GlcNAcylation in microglia leads to alterations in microglial morphology, lysosomal contents, and innate electrophysiological properties. Notably, the potassium channel Kv1.3 was found to interact with OGT and undergo O-GlcNAcylation. In addition, Kv1.3 exhibited an elevated expression level and channel conductance in the microglia located in the hippocampus of the OGT conditional KO mice. Notably, Kv1.3 abundant microglia specifically modulated hippocampal GABAergic synapses and inhibitory tone, resulting in a shift in E/I balance. Collectively, these data demonstrate that microglia are important for fine-tuning inhibitory tone in the hippocampus through O-GlcNAcylation of the microglial proteins, including the Kv1.3 channel.