Mindfulness in daily activities can be very important to one's Buddhist practice. Meeting every and all social expectation isn't so important to one's practice. So to the extent possible, you may wish to remain mindfully absorbed in your task and not look for social cues that strangers want to talk to you. There is really no need to "be more present in the interaction with the passerby than I actually want to". You can let go of the feeling that you need to be available to everyone.
Of course this won't always work and sometimes it's a friend that's come to talk to you and you must stop what you're doing and respond. Noting your feelings ("annoyed" or "bored" or "anxious") without judging the feelings can help you remain mindful throughout the time you are talking.
If your friends drop by unexpectedly very often, they may not understand your need for time alone for your practice of mindfulness and meditation and the constant interruptions can make it hard for you to sustain your practice. Talking to them about this may be helpful especially since you are feeling "not as kind" as you'd like to be and that's not good for your mental state or the continuation of your friendship.
In other words, there is value to remaining mindfully absorbed at times and limiting the circumstances under which you even need to switch focus may be more beneficial than trying to learn a skillful way of switching focus quickly over and over through the day.