Since karma isn't the only thing that determines what happens to you what are the other aspects? Is there a sutta referencing this?
Yes, the following discourse is what you are looking for:
On one occasion the Blessed One was dwelling near Rajagaha in the
Bamboo Grove Monastery, the Squirrel's Feeding Place. There
Moliyasivaka the wanderer went to the Blessed One and, on arrival,
exchanged courteous greetings with him. After an exchange of friendly
greetings & courtesies, he sat to one side. As he was sitting there,
he said to the Blessed One, "Master Gotama, there are some brahmans &
contemplatives who are of this doctrine, this view: Whatever an
individual feels — pleasure, pain, neither-pleasure-nor-pain — is
entirely caused by what was done before. Now what does Master Gotama
say to that?"
[The Buddha:] "There are cases where some feelings arise based on
bile. You yourself should know how some feelings arise based on bile.
Even the world is agreed on how some feelings arise based on bile. So
any brahmans & contemplatives who are of the doctrine & view that
whatever an individual feels — pleasure, pain,
neither-pleasure-nor-pain — is entirely caused by what was done before
— slip past what they themselves know, slip past what is agreed on by
the world. Therefore I say that those brahmans & contemplatives are
wrong."
"There are cases where some feelings arise based on phlegm... based on
internal winds... based on a combination of bodily humors... from the
change of the seasons... from uneven care of the body... from harsh
treatment... from the result of kamma. You yourself should know how
some feelings arise from the result of kamma. Even the world is agreed
on how some feelings arise from the result of kamma. So any brahmans &
contemplatives who are of the doctrine & view that whatever an
individual feels — pleasure, pain, neither pleasure-nor-pain — is
entirely caused by what was done before — slip past what they
themselves know, slip past what is agreed on by the world. Therefore I
say that those brahmans & contemplatives are wrong."
When this was said, Moliyasivaka the wanderer said to the Blessed One:
"Magnificent, lord! Magnificent! Just as if he were to place upright
what was overturned, to reveal what was hidden, to point out the way
to one who was lost, or to carry a lamp into the dark so that those
with eyes could see forms, in the same way has the Blessed One —
through many lines of reasoning — made the Dhamma clear. I go to the
Blessed One for refuge, to the Dhamma, & to the community of monks.
May the Blessed One remember me as a lay follower who has gone for
refuge from this day forward, for life."
-SN 36.21, To Sivaka