Love LCGs, I play Star Wars and Netrunner, but before I jump in I want to understand the order I purchase things in and what is necessary when buying expansions. Like do I need to buy the small expansions or can I just buy the large ones? Also I read each individual pack includes a set quest, so does this mean I do not need all the cards in a set?

Welcome to the game! You need the Core set first. The Shadows of Mirkwood cycle (6 small Adventure Packs) doesn't need an expansion, but the Dwarrowdelf cycle (another 6 packs) needs the Khazad-Dûm expansion. The Against The Shadow cycle needs the Heirs of Númenor expansion. The two Hobbit expansions and The Black Riders are Saga expansions, and you only need the core set to play them.

So:

-Core set

-Shadows of Mirkwood:

The Hunt for Gollum

Conflict at the Carrock

A Journey to Rhosgobel

The Hills of Emyn Muil

The Dead Marshes

Return to Mirkwood

-Khazad-Dûm Expansion

-The Dwarrowdelf:

The Redhorn Gate

The Road to Rivendell

The Watcher in the Water

The Long Dark

Foundations of Stone

Shadow and Flame

-The Heirs of Númenor Expansion

-Against the Shadow:

The Steward's Fear

The Drúadan Forest

Encounter at Amon Dín

Assault on Osgiliath

The Blood of Gondor

The Morgul Vale

... and then the independent expansions:

-The Hobbit: Over Hill and Under Hill

-The Hobbit: On the Doorstep

-The Black Riders

The expansions contain the encounter (enemy, location etc) cards used in the following cycles. So, for instance, to play The Steward's Fear you need cards from Heirs of Númenor. You are of course free to buy the expansions in any order you like. You don't need to play Khazad-Dûm in order to appreciate Heirs of Númenor - though Khazad-Dûm and Dwarrowdelf contain some great cards that make Númenor scenarios a lot easier. Hope this helps, and don't hesitate to keep asking!

Interesting, so does each pack make a complete encounter (if I have the matching large expansion/core set) or would I need the entire set of 6 to have a complete encounter?

You need, say, Khazad-Dûm to play any of the 6 following adventure packs, but the small packs are complete scenarios and independent of each other. So you can play Road to Rivendell and Foundations of Stone, and then get Watcher in the Water if you feel like it. (Though it is, of course, encouraged that you play them in order for the sake of the story.)

When you sit down to play, you choose a quest like "The Redhorn Gate," which is a scenario in and of itself. This sets the flavor of the game, victory conditions, and the composition of the encounter deck.

The Adventure Pack quests use some cards from the Deluxe Expansions, as Olorin93 pointed out, but those AP scenarios are unique quests and played separately from any other quest. In other words, if you want to play one particular AP, you don't need the other APs in that cycle. They are tied together thematically, like an overarching story, but not tied together in the sense that you can't play one without the other.

Awesome, so I can pick one up at a time and play that quest til I have succeeded then pick up the next one. Also can I only play 2 players with 1 core set or do I need more than one. Assuming I don't care about being tournament legal but want to play with 4 total players?

You can do 4 players with 1 core set. Your decks will be weaker because there aren't enough of the most powerful cards to make 4 strong decks, but you can still build 4 decks. With just the core, you will be under the 50-card minimum but sounds like that's not a problem.

The other thing is that you need to keep track of player threat, resources, damage etc. With 4 players you may run out of resource tokens, and you'll only have 2 threat trackers. But pen and paper (or 10-sided dice) work just as well.

You can play 3-4 players with just one core set, provided players 3 and 4 use, for instance, pen and paper to keep track of their threat. You will probably also need more resource, progress and damage tokens than those included in a single core set. Also, you may find it tricky to get along and distribute powerful cards evenly with such a limited card pool - but expansions should quickly fix that. So yes, it's possible - but the experience for a 3-4-player game is better with 2 core sets.

Awesome, just got my set yesterday, need to still read the rules but otherwise all set to play.