This paper focuses on clustering the motion from dense optical flow fields. An unsupervised clustering methods are presented and a model selection is proposed. A comparison between the proposed technique with the K-means and EM is made. The experimental evaluations reveal that the proposed technique achieved the best F1 score. The proposed technique is superior in terms of robustness and computational demands.