This typifies the Zen approach to koan practice, in which the master gives the student a kaonkoan which cannot be answered by conventional methods. This eventually causes the student to break down which in Zen is called The Great Doubt. This is seen as the breaking down of the intellect which can cause a myriad of unsettling symptoms - annoyance being one such symptom - but leads to a more refined and fluid version of mental processing.
In your particular case, the koan you unwittingly stumbled upon might be "what was your original face before your parents were born?"