Common factors significantly affect the connectivity of the network among stocks. Common factors significantly affect the distribution of investment weight for stocks. The devised method substantially assist in constructing a more diversified portfolio. This diversified portfolio achieves better out-of-sample performance. These results are robust in both the Korean and the U.S. stock markets.