My husband reverts to his native tongue. He thinks in his native language, so it's natural for him to slip into it. Also, he knows all the nuances of the words so he can express himself exactly.
While I believe etiquette would say one should speak in the language that everyone understands, I sometimes speak in my husband's native tongue to him to clarify something or -- honestly -- if I am saying something I don't want others to understand, rude as that may be.
You may have said to the others, do you mind if I speak in _____ so my grandfather could better understand _____? That way, you are considering the other peoples' feelings but still able to help your grandfather understand