We report on the crystallographic tilting of AlN/GaN layers grown on Si (111) substrates with different miscut angles toward [110] direction. The inclination angle of the Si (111), AlN (002) and GaN (002) planes to the sample surface was measured by high resolution X-ray diffraction. We found that the tilt angles of the AlN and GaN layers with respect to the Si (111) plane are significantly larger than that predicted by the classical Nagai model. We proposed that in-plane mismatch played an important role in the tilt epitaxy of AlN/GaN layers on miscut Si (111) substrate.