Kevin Coleville, 57, flew the Union Flag for many years before adding a second pole to display a flag in memory of his late sailor father who died earlier in the year.
But Bromley Council, in Kent, have said he must remove one of the flagpoles after complaints from neighbours.
Mr Coleville, who was a policeman for more than 30 years before retiring, has flown the Union Flag outside his house without any complaints for a number of years.
Only recently has he put up a second pole flying the Royal Navy flag in memory of his late father James ‘Dusty’ Maurice Miller, who received three medals for his Navy service during the Second World War.
Mr Coleville said: “I erected a flag pole years ago in my back garden, I checked with the council, they said there’s no problem flying a union jack.
“On the 5th February my father died so I put up another flag pole with the White Ensign, the official Navy flag.
“I’m flying it as a mark of respect.”
With both poles around 15 foot high each, he lowered his union flag to half mast after his father passed away.
But he later received a visit from the council telling him there had been complaints about his flag pole, and he has to take one down.
The married father-of-two, and grandfather-to-three, said: “I want to fly both flags. I don’t have a problem with having to remove one pole, but how can they be offended by a national flag?
“I’ll just have to fly both from the same pole, but I’m a bit sad they’re making me take it down.
“They’re in my back garden; they don’t block anyone’s view or anything like that. “I’ve never had a problem before. I don’t know why someone would complain.
“I’m a bit concerned that someone was ‘offended’ by it, will I continue to battle with who complained?”
A spokesman for the council said: “Properties are allowed to display one flagpole under permitted development rights, with any further poles requiring planning permission.
“Following a report we investigated and asked the resident to remove one flagpole and apply for planning permission if they would like any further poles.”
source: The Express