You can't go wrong with the Roy Thomas/Neal Adams run, which are among the best comics of the Silver Age and which - as Adams correctly states - basically created the X-Men template that all later X-creators cleave to.
Personally - but I am biased because Neal Adams is my favourite comics artist - I think their run is the best mutant run ever. Only Claremont/Byrne/Austin at their peak can compare.

Otherwise you can write off the X-Men's first series. Issues 11-19 have some highly entertaining stories and interesting concepts and some prime Kirby artwork so they're worthwhile too.

I hated the X-Factor concept from the very beginning - especially resurrecting Jean Grey - and saw it as the ultimate sell-out to the X-Men's then burgeoning popularity. Little did I suspect in those naieve days just how bad exploitation can get.
Compared with Marvel's policy today, Marvel under Shooter was a model of restraint and good taste which it came to exploiting popular properties.