This is obviously self preserving pragmatism
No. It's a Sila, Samadhi, Panna preserving and improving principle. It is very conducive to the practice of the noble eightfold path.
ex:
The right effort
(1) to prevent the arising of unarisen unwholesome states;
(2) to abandon unwholesome states that have already arisen;
(3) to arouse wholesome states that have not yet arisen;
(4) to maintain and perfect wholesome states already arisen.
A person who is your equal will help you maintain your wholesome qualities and alert you when you start to let your standards slip.
A person who is higher than you will help you abandon your unwholesome states and help you develop new wholesome qualities as well as perfect your current standards.
So such persons are worthy of being considered as friends. But a fool is not worthy of being considered as a friend as he would only hinder your right effort and drag you down.
For me to associate with someone better, that person would have to
accept me, his/her inferior. Ergo, one of us must break this rule...
Not necessarily. You can help someone to get better by giving him good advice. That doesn't make him your friend. That makes him your student or someone you practice compassion on. The other might consider you as a teacher or just a friend. But you don't follow him or participate in unworthy activities he might engage in. A monk might advise many lay people on a daily basis. That doesn't make them his friends.
This doesn't sound very loving and compassionate. Fools will remain
fools...
Someone need not be your friend for you to practice compassion towards him or her. The Buddha did not seek companionship with those whom he helped to become liberated. It was always an act of compassion. Kesi Sutta explains how the Buddha dealt with those who are too haughty to give up their unwholesome ways:
"But if a tamable person doesn't submit either to a mild training or to a harsh training or to a mild & harsh training, then the Tathagata doesn't regard him as being worth speaking to or admonishing. His knowledgeable fellows in the holy life don't regard him as being worth speaking to or admonishing."
Buddha himself tolerated Devadatta and others
Tolerating is not the same as making friends with someone. You can tolerate a dog barking while you try to listen to something. That doesn't make the dog your friend.
The accompanying story of the Thera who blindly trusts his disciples
is a little too simplistic
Venerable Maha Kassapa is not trusting the disciple. He's simply trying to make him better. But only the Buddha could definitely know if someone cannot be helped. Who would've thought that Angulimala and the demon Alavaka could be helped?
Did the Thera get undermined by a false sense of duty or compassion
No. Venerable Mahakassapa was an Arahanth. So his association with the student did not undermine him as he can never fall back. Lord Buddha probably used this as a pretext to give this sermon to the other good student. As you can see, he attained the state of Sotapatti after listening to it.