I'm from the same viewpoint too. I often wondered about Zen/Chan and Mahayana versus Vajrayana. My guess is the crux shouldn't be rituals and forms, if that's the original non-aim.
Also Nubchen Sangye Yeshe who's Guru Rinpoche's disciple summarized Mahayana covering 4 paths.
- Sutrayana (gradual) - basically foundations of #2,3,4. Own learning/understanding and application.
- Sutrayana (sudden) -Zen/Chan etc. Includes Koans, direct pointing. Adatra t combination of 3 and 4 but without rituals/images etc.
- Maha-yoga (gradual) - Visualization, yogas, includes Mahamudra and usually Shamatha first, then Vipashyana.
- Ati-yoga/Dzogchen (sudden).formless and objectless meditation.
Nonetheless you will realize all paths are actually the same, combinations of Shamatha and Vipyashana (Vipasanna) in specified orders. Surangama as pointed earlier also works and fall under Sutrayana (gradual).
However, the above 4 bullet points are a bit of false options because one needs to prepare the mind before the instant paths are made available. That means only 2 essential paths of Sutrayana and the Yogas. Whichever picked is based on the cause and conditions your in. (Also expedient means provided by the earlier practioners)
So for example, usually need years of Shamatha to have mental stability (from defilements), or need guru to have a direct experience of true nature before you can do Dzogchen repeatedly in normal lay life. One could also do the same with development/completion visualizations with an appropriate Yidam etc. to lay the foundations leading up to Shamatha. Visualizations contain meditation components.
I haven't got a clear conclusion as yet but this is what I gathered so far. Thankfully I'm also physically based in the confluence of all major practices including the cultures and languages.