We propose a new model for recognizing complex actions named Bag-of-Sequencelets. We represent a video as a sequence of primitive actions. We model a complex action as an ensemble of sub-sequences (sequencelets). We automatically learn sequencelets without any annotation about action structures. We achieve state-of-the-art results on the Olympic sports and UCF YouTube datasets.