In short, sexual misconduct means performing any sexual act that will be harmful to yourself and/or others. Just because the act is consensual does not necessarily mean it won't hurt someone, as example in the case of adultery, the two people performing the act may consent to it, but it hurts another person and family. If the spouse consents and/or participates(like swinger couples for example), then it is not considered misconduct.
The Buddha advised against sexual activity with those who are "under the protection" of their family(minors), a promised or engaged person, dhamma(monks/nuns) etc.
"He avoids unlawful sexual intercourse, abstains from it. He has no intercourse with girls who are still under the protection of father or mother, brother, sister, or relative; nor with married women, nor female convicts; nor lastly with betrothed girls."
These are all situations that can cause much wide ranging trouble beyond the couple engaged in the act.
This link may be helpful for learning more about the 3rd precept, as there is much more to this precept then sexual misconduct.
"What precisely, then, does the Third Precept imply for the ordinary lay Buddhist? Firstly, in common with all the other precepts, it is a rule of training. It is not a "commandment" from God, the Buddha, or anyone else saying: "Thou shalt not..." There are no such commandments in Buddhism. It is an undertaking by you to yourself, to do your best to observe a certain type of restraint, because you understand that it is a good thing to do. This must be clearly understood. If you don't think it is a good thing to do, you should not undertake it. If you do think it is a good thing to do, but doubt your ability to keep it, you should do your best, and probably, you can get some help and instruction to make it easier. If you feel it is a good thing to attempt to tread the Buddhist path, you may undertake this and the other precepts, with sincerity, in this spirit.
Secondly, what is the scope and purpose of this precept? The word kama means in Pali "sensual desire," which is not exclusively sexual. It is here used in a plural form which comes close to what is meant by the Biblical expression "the lusts of the flesh." Greed for food and other sensual pleasure is also included. Most people who are strongly addicted to sexual indulgence are also much drawn to other sense-pleasures. Though we are here only concerned with the sexual aspect, this point should be noted. For those with any grasp at all of Buddhist principles, the basic reason for such an injunction should be immediately obvious. Our dukkha — our feeling, of frustration and dissatisfaction with life — is rooted in our desires and cravings. The more these can be brought under control, the less dukkha we shall experience. It is as simple as that. But of course, that which is simple is not necessarily easy."