"All of these things seem very unrealistic [...]"
Notice that a doctrine is not the same as a biography.
The buddhist literature is vast and the biographical accounts are small in comparison and really not the main point. So, for a practitioner, the Buddha's biography (though profoundly inspiring and worth of learning) is not the place to focus the most on and it's not the best place to find the "logic" that seduced you.
To better illustrate this point, if we consider only the four main collections of the Pāli canon, the biographical accounts of the Buddha not only are rare but are pretty down to earth -- I recall only one or two suttas mentioning the Buddha as a baby talking and walking and so forth.
Consider now that there are over twelve thousand discourses in these collections alone. So if these extraordinary events are hard to conciliate, it's easy to dismiss one or two passage and focus on the actual bulk of the teachings. Furthermore, one will eventually notice that these teachings are, in a sense, telling one to not waste energy dwelling with these conflicts anyway.
However, there are more texts describing "supernatural" powers, and a few more with spiritual beings participating. These increase in size if we also consider the Mahayana texts. So, these elements are really intrinsic to Buddhism, though there are buddhists who prefer to dismiss them, like secular buddhists. In some way, this is also a testament to the strength of the nucleus of the doctrine which, even when these elements are removed, remain valuable to people who dislike them.
"Are these stories, therefore, meant to be understood as metaphors, or are they believed as fact?"
One thing here is how buddhist should perceive this stories. Whereas commonly we understand religions in the west as "believing is seen", perhaps Buddhism is more aligned with "seen is believing". In the orthodoxy, I know of no part of Buddhism that states something like "you have to believe this, or else go away". Other sects, however, may put some extra value on faith about something.
So, some buddhists believe the stories, some believe them partially, and others don't. But, as per the Buddha's discourses, these stories are meant to inspire disciples to practice.
On the factuality of these stories, however, there's really not much evidence available to judge their veracity. Plus, there are different biographical stories -- so you better check your sources. Also, apart from his tours across the country, some events of his life have been argued more substantially that they are likely to have happened, like the period where the Buddha studied under the other two ascetic teachers.
But other biographies show some signs of creativity. For example, a popular story (I'm not entirely sure from which texts it's from) says that the Buddha-to-be saw a sick, an old, and a dead after leaving his palace for the first time and that these marked him deeply. However, it's known from an unrelated discourse of the Buddha that he referred to these three things as metaphors or signs that "wakes" a person to the tragedy of existence and inspires them to look for an escape from suffering. So it's not far fetched to imagine that the author of this particular biography decided to put these signs quite explicitly in the Buddha's own story.