Mumble - How do I get the lowest latencies possible?
In Mumble unlike in Teamspeak, this is not done through the server but through every individual user. Please click on "advanced" in the settings menu. Under Audio Input, you will now find two controls summarized under "Compression".
A lower Quality setting means less quality of speech but improved latency, which means that what you say reaches your partners faster. The same applies to the Audio per packet value. Of course, low settings here mean lower voice quality so it is up to you to find the perfect setting suited for your needs.
To further improve the quality, you should try get everyone on the server to use the same settings.