So should I start on the core rule book or what do anyone recommend?

So I'm pretty adept at DnD and its getting bland up to the point that my created classes are dying horribly and even getting shot down from the DM so I'm just trying something out I've already played alot of strategy 40k mostly mark crusade Necrons Rule

I'm a little confused by what you're asking. Are you a player looking to suggest a new system, or a GM looking to start up a new campaign?

You absolutely need the Core Rulebook for Rogue Trader, because it has all of the core character creation rules, as well as Skill and Feat Descriptions. Unless you want to play as a Xenos, in which case Into The Storm has rules for Kroot/Ork Freebooter, Navis Primer has Ork Weirdboy and The Soul Reaver has Dark Eldar. Should you want a preview of what character creation is like, the online supplement The Dark Kin has the rules for creating a Dark Eldar Wych and how Alternate Ranks work, although it lacks all of the explanations of Talents and what things do.

For general feel, Rogue Trader for me has been the most enjoyable 40K setting when it comes to roleplaying and versatility. You inevitably roll characters who are bombastic minmaxers, and can support anything from serious, grimdark adventures to lighthearted tales where the galaxy is made your player's indentured servant one system at a time.

For combat and mechanics wise, this is by far and away the easiest of the RPGs to create characters who are flat out incredible at what they do, and in general I've found the opponents of the book are almost universally underrated in a straight-up fight with your PCs, and even if they employ trickery your players will usually come out ahead. It'll take a little bit to find a nice groove, but there is still the potential for memorable fights, you just need to never do what a published adventure says.

Some thoughts on the other RPGs.

Dark Heresy - this is a system where your players will be relentlessly in over their heads, outgunned, outmatched, and it's a desperate struggle for survival. Good times.

Deathwatch - Much more focused on the tactical action. You have the biggest fights with the biggest monsters, but you get the worst roleplaying out of every system.

Black Crusade - Sometimes your players just want to cut loose and be evil, or you want to play the slightly darker side of the 40K universe. It combines little bits from every system generally pretty well, though there are clear power imbalances between Space Marines/Psykers/Everyone else.

Only War - Flat out amazing character creation and the mechanics and rules are streamlined into something that is fun and enjoyable, but it is the least interesting setting for me to play. There is a Dark Heresy/Rogue Trader to Only War conversion on their respective forums for a reason however.

I was just looking to see which has the most options for classes because I make my own classes after I've played every class to an extent I'll probably be posting some after I play for a couple dozen sessions I am going to try and push every class to its limit and beyond its how I get my fix in life I guess

Well, Only War or Black Crusade have the most open character creation systems. You pick areas in which you specialise, areas in which you're weak, and then everything Skill/Talent wise is open to you, it's just based on your initial decisions some things are cheaper, and some things are more expensive. Thus with careful planning you can pull off some truly Hellish combinations.

Dark Heresy, Rogue Trader and Deathwatch have the most defined classes, but Dark Heresy for most of the campaign will feel low powered, but gradually increases until you surpass equivalent Rogue Trader characters. It also has the most alternate career ranks out of every character.

Rogue Trader has careers that really do feel different, but in general everyone breaks them the same way: Arch-Militants get ridiculous Ballistics, Explorators get ridiculous Int, Astropaths get ridiculous WP, Rogue Traders get ridiculous Fel, Navigators get ridiculous powers, Missionaries get ridiculous Faith, and Void Masters look for alternate ranks that make them feel special instead of the Red Mage of the setting.

Deathwatch has you be Space Marines, which lend themselves to really powerful builds naturally, but their enemies tend to echo that right back.

Depends what you mean really by building your own classes. I know Rogue Trader much better than I do the other systems, and there are quite a few people who have made their own Core/Alternate classes on these forums. As a GM though, no matter what sort of build my characters have come up with, there is always at least one thing that's been published that absolutely demolishes their core strengths.

Went to Hastings today and someone bought the only rogue trader core rules guess ill try and find a dark heresy core rule book if what you say is true not that I don't believe you