Or is it more complicated than that? i.e random?
The Buddha connects rebirth with morality. Right view and shining morality leads to the heavens. Wrong view and hellish conduct leads to the hells.
It is a little bit difficult to translate the Buddhist term "hell-dweller" as "demon." They are not quite the same as "demons" which we know from Judeo-Christian folklore.
While it's certainly a bit more complicated that that, it definitely doesn't happen randomly. Refer to the 31 Planes of Existence. Notice the 3rd column on each table for each realm lists the various causes that lead to rebirth into that particular realm. It also provided backup sutta references.
In the Pali scriptures there are various teachings on 'rebirth', which give the impression The Buddha was inconsistent or, otherwise, scriptures were added after the Buddha passed away.
For example, the following stock phrase indicates two general 'rebirths' of 'heaven' & 'hell':
When my concentrated mind was thus purified, bright, unblemished, rid of imperfection, malleable, wieldy, steady, and attained to imperturbability, I directed it to knowledge of the passing away and reappearance of beings. With the divine eye, which is purified and surpasses the human, I saw beings passing away and reappearing, inferior and superior, fair and ugly, fortunate and unfortunate. I understood how beings pass on according to their actions thus: ‘These worthy beings who were ill conducted in body, speech, and mind, revilers of noble ones, wrong in their views, giving effect to wrong view in their actions, on the dissolution of the body, after death, have reappeared in a state of deprivation, in a bad destination, in perdition, even in hell; but these worthy beings who were well conducted in body, speech, and mind, not revilers of noble ones, right in their views, giving effect to right view in their actions, on the dissolution of the body, after death, have reappeared in a good destination, even in the heavenly world.’ Thus with the divine eye, which is purified and surpasses the human, I saw beings passing away and reappearing, inferior and superior, fair and ugly, fortunate and unfortunate, and I understood how beings pass on according to their actions.
Yet the following teaching indicates five potential 'rebirths':
And what is the diversity in kamma? There is kamma to be experienced in hell, kamma to be experienced in the realm of common animals, kamma to be experienced in the realm of the hungry shades [ghosts], kamma to be experienced in the human world, kamma to be experienced in the world of the devas. This is called the diversity in kamma.
The following teaching indicates six potential 'rebirths', which includes 'demons':
Unfree from the bonds of Māra They swell the ranks of hell, Existence in the animal realm, Asura-demons and the sphere of ghosts.
The following teaching indicates the virtuous can have 'rebirth' in 'hell' and unvirtuous can have 'rebirth' in 'heaven':
the evil-doer who goes to hell (or some other low state of birth)
the evil-doer who goes to heaven
the good man who goes to heaven, and
the good man who goes to hell (or other low birth).
The following teaching indicates all of the virtuous (puthujjana) who have not gained enlightenment will be 'reborn' in 'hell':
… the sentient beings who die as humans and are reborn as humans are few, while those who die as humans and are reborn in hell are many …
Why is that? It’s because they haven’t seen the four noble truths.