On the mental side of things it has been shown that caffeine improves concentration for between 1 and 3 hours because it stimulates the central nervous system at high levels. The areas it normally effects are the medulla and the cortex and actually has the ability to reach the spinal cord in even larger amounts. Its the impact in the cortex that provides the effect of increased concentration. It was also shown that "for athletes competing in sports where quick thinking and rapid reactions are necessary, caffeine can provide a huge edge." This leads me to believe that in gymnastics and diving this is the area in which caffeine consumption would be an advantage.
There have been three main theories about how caffeine works. The first one, I think my favorite, is that caffeine actually causes the body to burn more fat and less carbohydrates. Muscles are primarily fueled by glycogen however fat is a much more abundant, though more difficult source the body can use for energy but caffeine can help the body utilize the fat.
Studies show that in the first fifteen minutes of exercise caffeine has the potential to reduce the loss of glycogen by fifty percent. When this happens, the saved glycogen can be used for the remainder of the workout where normally it would be entirely depleted. (http://www.rice.edu/~jenky/sports/caffeine.html)
The second theory is that caffeine acts as an ergogenic aid both before and during the workout. This was tested by performing a study involving a two hour cycling endurance test both with and without caffeine and it was found that the caffeine led to a higher work output during the ride by an average of 7%.
The last theory was regarding caffeines ability to increase the athletes mental focus because it is a stimulant to the central nervous system. It slows substances which are used to stop neuronal firing in order to increase both awareness and reaction times.
There are no precise answers to exactly how caffeine impacts the body, but my guess is that it is a combination of all 3 theories. While I can see why the ICO and NCAA have a ban on caffeine I am still surprised thats it is placed in a category along steroids and cocaine. I never realized just how much caffeine was able to impact the body and physical performance, particularly because it is so widely used on a daily basis.