The principle of law involved is called coming to the nuisance. If the several good suggestions presented above don't stop the complaints, perhaps a good lawyer might convince the powers that be that because the field was there first, those complaining must foot the bill to have you relocate. Then, Sir, your Ministry of Silly Walks could provide the complainants with a silly walk because of the pain you've inflicted to their backsides! ;-)