If Buddhists don't believe that we have a soul then why do some believe in ghosts or spirits? this seems contradictory.
Soul in the context of Buddhism is that was contemporary considers atman, which is permanent, pleasant and controllable. What is established in Buddhism is that there is nothing as such and everything constituting a being is subjected to the 3 Universal Characteristics.
Also there is not entity as a being. What is there is a causality arising process, which also is presented as the 4 Noble Truths, Dependent Origination, Conditional Relations. The continuity of the causality process can be taken as a trip in a vehicle. Each point of the journey is different from the other but still point of the same journey, hence there is a continuity but not also the entirely the same. Death is like changing vehicles.
In Buddhism Ghosts and Spirits are not tormented souls of past being. They themselves are life forms which inhabit the 31 planes of existence. When one dies and is reborn as a Ghost or Spirit this is like any other rebirth. This is a new existence. It is not the same being in the previous birth or nor a different being, but the continuation of the causality process.
In buddhism, nothing is permanent. One can be reborn as an animal, reborn as a ghost, reborn in hell, reborn as a god thus cycling through all 6 realms of birth. Therefore, there can't be one soul, otherwise there'll be animal soul when animal dies, human dies there will be human soul, a ghost dies there will be ghost soul, a god dies there will be god soul and thus 6 different souls - and not one soul. Thus there is no soul, just cycle through the 6 realms.