Why physics?
Why are you talking about physics? I don't remember that I have ever talked about physics with anyone (except perhaps briefly with a cousin who is a physicist). I suspect perhaps you (you plural, i.e. "you with them") shouldn't talk about that. If I were to guess why you (plural) are talking about it, I would guess that they are because of something to do with you (singular). Did you start that conversation somehow? When I was young and people used to ask me "What do you study?" and I replied "Maths", no-one would ever try to pretend to talk Maths with me, that would end the conversation and we'd talk about something else.
Why disputes?
Canonically, Buddhism warns that Māna is a cause of disputes:
Māna (Sanskrit, Pali; Tibetan: nga rgyal) is a Buddhist term that may be translated as "pride", "arrogance", or "conceit". It is defined as an inflated mind that makes whatever is suitable, such as wealth or learning, to be the foundation of pride. It creates the basis for disrespecting others and for the occurrence of suffering.
This answer too explains that an element is "comparison". IMO an example of comparison is, "I know more about physics than they do." Even when that comparison is "true" that's not enough to imply that it's "right".
I remember when I was about 14 I was scornful of my Mum at the dinner-table because I knew more about chemistry than she did (because I was learning it in school), which was of course incredibly rude of me and wrong (i.e. morally wrong even if not factually untrue), a mistake.
What is right?
There's a summary of what "right speech" means on this page: https://www.accesstoinsight.org/ptf/dhamma/sacca/sacca4/samma-vaca/index.html.
Can you teach?
There's some advice in the suttas about whether, when, and who can teach Buddhist doctrine -- maybe similar conditions apply to teaching physics too.
There are several conditions, I think one is that the audience must be willing and attentive, listening. I think another says that it's difficult to be a teacher because the teacher should do it without a feeling a superiority which is harmful to the teacher and the students (sorry but I'm not able to find the little sutta which says this so I'm paraphrasing from memory).
How do I deal with the kind of people who do not know THE first thing about a topic?
I'm not sure I can answer that, like I said I'm not sure I've met them. I can only guess at why they're doing that and my only, uninformed guess was that it's an ego thing, a reaction to a perception that their view is inferior. Why else would they talk about physics of all things?
For example perhaps your father deserves respect for some reason, instead you scorn his "high-school at best" assertions about physics.
Where I went to school we started to specialize at the beginning of high-school, and if I remember right I started/chose to study Maths and sciences -- instead of e.g. English and history -- partly because that was a different field from my own father's.
We didn't compete. He was very knowledgeable about a lot of subjects and I didn't try lecture him. Or maybe I did a bit because he kindly said sometimes that I was "good at explaining things". But to me "high-school-level physics" means Newtonian mechanics, and therefore calculus, and therefore doesn't belong outside the classroom! :-) In fact he did National Service (compulsory military service) and trained as a gunnery (artillery) officer, even for that I assume that in practice they used tables of pre-calculated range/elevation figures rather than calculus.
Other option is to stop hanging out with people like that?
That sounds kind of harsh to me, an over-reaction: i.e. not hanging out with your dad, because he "doesn't even inquire into" PhD-level Physics.
Maybe change the subject, find some common ground, discuss physics only elsewhere i.e. with colleagues?
Do you maybe feel belittled somehow, is displaying your superior knowledge of physics an ego-defense mechanism?
Maybe you might find the link (above) to descriptions of "Right speech" helpful -- good to know as an ideal.
Another useful set of descriptions is the brahmaviharas which are said to be the right attitudes to have towards people.
But then isn't that coming from aversion - we are aiming for harmony and remaining undisturbed in the face of things rather than avoiding them?
Yes at least one person on this site explains that this is a role of "ethics" -- including right speech, right action, and right livelihood -- to promote harmony and avoid inter-personal conflict (see for example this answer).
