XPS is used to study bulk-like properties of GaAsBi crystals. Surface effects are removed from XPS signal by an epitaxial AlAs cap film. Local variation of Bi composition is found. The result is consistent with photoluminescence and theoretical results. Ga vacancies and Bi crystallites are suggested to be dominating defects.