His Dad appears to have been born in 1925 so would have been 19 around the time of D Day so it's certainly very possible though his precise role appears to be currently obscure.
According to the
digging that went on here, there is no record of him in the RN but there does seem to have been a man of the same name in the Royal Pioneer Corps from 45-48 who served in Palestine.
Now members of the Pioneer Corps would certainly have been present at D Day attached to various units, though exactly why one would have been posted to a RN minesweeper, if it is the same man, is open to conjecture, but stranger things have happened in the chaos of war.
I see no reason to doubt that his Dad served in the armed forces at that time and for that alone his Dad should be respected, though I think perhaps there's been a slight embellishment of what he actually did occuring over the years when telling his son tales.
(All a bit like that Harry character who died the other year being somehow referred to as an RAF pilot by a lot of people when he wasn't air crew at all but ground crew.)