There are separate explanations for SN1 and SN2.
SN1: The SN1 mechanism involves a carbocation intermediate, and both allylic and benzylic carbocations have resonance, which increases the stability of their carbocations, and speeds up the rate of SN1 reaction.
SN2: This explanation is less obvious, and is probably only mentioned in passing in your orgo textbook, if at all.
The pi systems present in allylic and benzylic halides are able to overlap with the pi orbitals of the nucleophile and the leaving group in the 5-coordinate transition state of an SN2 reaction. This orbital overlap lowers the energy of the transition state (which stabilizes it), and so increases the rate of SN2 reaction.
There are other possible explanations as well (Which also vary by textbook). For one, an allylic halide is less sterically hindered than an alkyl halide (less hydrogens sticking out in 3D).
Also, the carbons on the double bond of allyl and benzyl compounds are sp2 hybridized, and so are more electronegative than sp3 carbons. So the sp2 carbons pull alway some electron density from the alpha carbon (the carbon attached to the leaving group), making it more electrophilic, and thus increasing SN2 reactivity.