The answer to your question depends on how one understands the word "start." Etymologically, the English word means to jump, leap up, or move or spring suddenly, thus, to arise, to come into being. In accordance with the law of karma, the whole process of samsara itself must be self-starting.
This can be proved logically based on the law of karma, which requires that every cause have an effect and every effect a cause. Since samsara is time itself, the essence of time is the identity of cause and effect, thus, the moment, which is then extended in space as past and future to create the infinitely differentiated world that we experience.
However, as the illusory negation of reality samsara must inhere in the non-illusory, which is reality itself. Therefore, samsara also inheres in reality, and in that sense has a logical - but not a temporal - "origin." This "start" is the principle of karma itself, which is the inherent kinetic principle in transdual reality.
This is necessary to explain our experience of samsara. Otherwise, there would just be nothing at all. This point of identity of samsara and reality is, once again, the moment. Therefore, the answer to your question is that the start of samsara is the inherently timeless and therefore eternal moment, which paradoxically is the origin of time, space, and causality that endlessly differentiates in infinitely variegated patterns. This view of the cosmos remarkably presages certain theories in modern physics, especially the holographic theory of the universe.