The rate determining step (RDS) in SN1 mechanism is dissociation of leaving group to form carbocation. Thus, 3o halides and tosylates react fast by this mechanism since they form relatively stable 3o carbocation intermediates. Imagine what would happen if methyl bromide (CH3Br) were to react by this mechanism. Do you think the cation formed would be very stable?
SN2 mechanism occurs in one step in which the nucleophile attacks the alkyl halide from the backside of the C-LG bond. When a nucleophile attacks a methyl or primary halide it does not encounter much steric hindrance (i.e. the Nu can easily attack the carbon without bumping into too many other groups).
On the contrary, if a nucleophile attempted to attack a tertiary halide it would encounter severe hindrance and be pushed away.
It's easy for methyl, primary (1o) and tertiary (3o) halides and tosylates. Methyl and 1o halides and tosylates will undoubtedly follow the SN2 pathway since they can not form stable carbocations required for SN1, and they have the least steric hindrance (favors SN2). Tertiary (3o) will follow SN1 pathway since they can form stable cations and have the most steric congestion which limits nucleophilic attack in SN2 like reactions. Secondary halides and tosylates are difficult to predict (in practice, they can be problematic as well). Bear in mind that elimination reactions (E1/E2) compete with substitution and often times elimination products form as well.