A new method to calculate fatigue life of specimen with impact pit is proposed. The initial impact damage and elastic-plastic fatigue damage are considered. The residual stress relaxation is investigated in this method. The calculated results agree well with the experimental result.