An automatic video-mining method is developed to retrieve abundant highly-qualified web videos as the training data. A reliable SWM is generated to assist a surgical system with surgery understanding. This reliable and valuable knowledge is learned to support an intelligent surgical system in performing tasks such as abnormal surgery operation detection and instrument recommendation. It is feasible for an intelligent surgical system to learn reliable SWM knowledge in an economic way.