In the present study, mechanical behavior of two types of Al/Mg composite rods, Mg AZ31 core/soft Al 1100 sleeve and Mg AZ31 core/hard Al 7050 sleeve, under compression along the extrusion direction (ED) was systematically studied, with a great emphasis on the effect of different Al sleeves and Al fractions. The rule of mixtures for flow curve was also addressed. Our results show that the strength and fraction of Al sleeve greatly affect the shape of flow curves of Al/Mg rods. A plateau shape that often exists in flow curve of compression along the ED of a monolithic Mg extruded rod also appears in that of the Al/Mg rods. This plateau completely disappears with the fraction of Al 7050 up to 78.7%, but is still visible with Al 1100 86%. A fluctuation exists in the flow curves of composite with the soft Al 1100 sleeve, but is absent in those with the hard Al 7050 one. Different types of Al sleeve hardly affect the twinning fraction in Mg core during ED compression. Compared to the soft Al 1100 sleeve, a hard 7050 one contributes to a homogeneous deformation of the whole composite and, hence, reduces the propensity for cracking at Al/Mg interface. The measured yield strengths of Al/Mg rods slightly deviate from the predicted ones by the rule of mixtures (about 4–36 MPa), while the rule of mixtures for the compressive flow curves does not work. It is found for the first time that the quite different strain hardening behaviors between the Mg core and the Al sleeve result in the large deviation of the experimental curves from the predicted ones.