The remarkable reasoning abilities of large language models (LLMs) have opened new research opportunities in wireless networks. As demonstrated in [1], pretrained LLMs have been proven to handle various network optimization tasks universally without prior knowledge of systems, such as mathematical models, channel propagation, and scenario-specific fine-tuning processes. This knowledge-free ability promotes LLMs as powerful optimization agents that autonomously determine network management strategies. Such an LLM optimizer technology is still in its early stages and requires significant evolution for real-world implementation. In particular, existing works need centralized operations, which lack the flexibility with distributed devices for wireless networks. To address this challenge, this article presents a multi-agent LLM optimizer (MALO) framework where individual LLM agents make their own decisions for different wireless nodes in a decentralized manner. The effectiveness of the MALO approach is verified in decentralized wireless resource allocation problems. Numerical results confirm that the proposed decentralized MALO framework outperforms existing centralized LLM optimizer methods and achieves performance comparable to traditional optimization algorithms.