NenaMark3 [iPhone]

Nenamark3 is a benchmark that measures the 3d performance of a system by gradually increasing the complexity of a scene until the frame rate drops below 60.

Purpose
Nenamark3 is run to compare different systems against each other from a 3d graphics perspective. A 3d graphics benchmark tries to show how well a system will perform in games or or apps that make use of 3d graphics.

3D Graphics Apps
A 3d app, such as a game, will typically have individual settings for different tiers of devices. A lo-end system will have less features and look less amazing than a hi-end system. If hi-end settings are used on a lo-end system it will not be able to maintain its frame rate of 60 fps, causing it to drop frames and “stutter”.

What a 3D benchmark should do
This means that a graphics benchmark should define a set of scenes of increasing complexity and find how far a system can go while maintaining 60 fps. The common way of running static scenes and measure the fps have several problems. First, it is not relevant to run at any other frame rate than 60. It is not certain that lower fps are comparable to each other. All we can say for certain is - did it make 60 or not? Second, once 60 is reached where do you go? While it is possible to do off screen render and let the fps go as high as it can this is also irrelevant. If you have more power you would probably want more advanced features rather than more of the lesser features (perhaps you’d want a more correct lighting model rather than say more geometry).

Go Slow
There is a point to scaling the complexity gradually over time. Especially embedded systems have quite sophisticated clock governors that speeds up the clocks when needed, even for the GPU. These sample the load at certain intervals and adapt so it makes sense to slowly build complexity. In a game there would be a few slow frames before the system could react which would be annoying but not critical.