Ye and Zhou [Appl. Soft. Comput. 22 (2014) 351–357] proposed an efficient chaotic based image encryption scheme which only employs diffusion, while usually both confusion and diffusion are used for encryption structures. We present both chosen-plaintext and chosen-ciphertext attacks against the scheme for any number of its rounds r by exploiting r-round differentials with probability 1. The result shows that the encryption structure proposed by Ye and Zhou does not fulfill basic requirements of a secure image encryption scheme.