In this article we discuss the distribution of asset price movements by introducing a market potential function. From the principle of free energy minimization we analyze two different kinds of market potentials. We obtain a U-shaped potential when market reversion (i.e. contrarian investors) is dominant. On the other hand, if there are more trend followers, flat and logarithmic potentials appear. By using the cyclically adjusted price-to-earning ratio, which is a common valuation tool, we empirically investigate the market data. By studying long term data we observe the historical change of the market potential of the US stock market. Recent US data show that the market potential looks more like a trend-following potential. Next, we compare the market potentials for 12 different countries. Though some countries have similar market potentials, there are specific examples like Japan which exhibits a very flat potential.