Another year, another 3DMark. Futuremark is back at it again with 3DMark05, this time with the same company name! If 3DMark was a game, it would be a highly-successful game franchise. This is actually the 5th release of 3DMark, 6th if you include Final Reality. Over 24 million copies of 3DMark03 have been distributed worldwide! This amazing achievement shows both the growing popularity of 3DMark, as well as the rising awareness of graphics performance.
This popularity has been both a blessing and a curse for Futuremark. They took a battering during the 3DMark03 era, with the increased emphasis on benchmark optimization by Independent Hardware Vendors’s (IHV) in their drivers. This allowed for a greater analysis of the relevance of synthetic benchmarks in graphics card reviews.
The controversial issue of driver optimizations, particularly shader replacement, stirred up a storm that lead to a battle between Futuremark and Nvidia. This ultimately led to Nvidia’s withdrawal from the program. Simply put, this whole issue was messy and was not handled well. The credibility of Futuremark fell off significantly as 3DMark was questioned for how forward-thinking the tests were. People were no longer sure if they really were a good indicator of how 3D graphics were likely to be seen over the next year and a half.
All this, of course, got the whole community in a large discussion that involved both professional reviewers and their readers. The central theme in my view was, “Do gamers play benchmarks?” and therefore, “What use were they in helping to determine the value of a graphics card?”
In my opinion, this 3D-industry soap opera impacted the industry positively in the long term. The industry as a whole matured including the IHVs, the reviewers, the readers and Futuremark. IHVs have set down strict guidelines on optimizations. For example, Nvidia has the following three stipulations:
- An optimization must produce the correct image.
- An optimization must accelerate more than just a benchmark.
- An optimization must not contain a precomputed state.
ATI has similar rules in place with the release of Catalyst AI and specifically stated there would be no synthetic benchmark optimizations. The key is that all optimizations must produce the correct image and not something inferior. Reviewers were forced to review (pardon the pun) what in the past was judging the performance of video cards, towards a more balanced approach of testing actual game play via fraps, built-in game benchmarks and of course synthetic benchmarks. Hopefully, readers have gained more knowledge of the roles that these various tests play. Finally, Futuremark began a driver certification program and further defined their role in the 3D gaming world.
3DMark05 was developed during these turbulent times and I'm sure the developers worked hard to guarantee its relevance. Every effort was made to load the graphics subsystem as much as possible in order to scale with future generations of cards.