SWX 2003 Benchmark

What would you like to see in a benchmark? The spec and the Solid Solutions benchmarks are primarily testing the graphics card. In my experience the graphics card is not usually the bottleneck.

My thoughts to get things rolling are:

A benchmark should have two parts:

1. An exhaustive test of each command broken down into user interface and actual command performance.

1. An exhaustive test of real world modeling, perhaps broken down into part, assembly and drawing performance (and perhaps geometry import should be included too.)

In addition a benchmark should run without problem on pretty much any version of SolidWorks from SW99 to the present in order to be able to benchmark performance changes. The benchmark should report things like time taken to complete a task and also resource usage. The benchmark should be GPLed so that the source can be made available.

Navy Diver wrote:

Paul,

Sounds like a good plan. It won't give anyone much to cover up with.

So....when are you going to have it finished ????

Regards

Mark

ND,

Not exactly. I don't think you can test the relative performance of different versions (releases), on the same hardware. If you could you would see a declining curve, from SW99 to the present, that is grossly out of proportion to the added overhead of the extra features. SW doesn't want you to be able to assign a number to this. They're counting on faster hardware to cover this up.

Slash and burn programming at it's best

Mark

I've looked at the source code which is open source. There is one little section of code that tests performance and it is hardly exhaustive. And the algorithm they use to calculate the final result is heavily biased towards graphics. This is not surprising because the people with the most input are graphic card suppliers like 3D Labs.

Just because the test has a CPU report > the SPECapc benchmark tests more than just graphics.

Mark, I think you have a point here. I like the graphics check but would rather see the CPU tested more.

Certainly their software has slipped in performance; the drawing package has gone backwards

Snip

Paul,

I would love to see such a benchmark, and agree that a group effort is the only reasonable way to get there, but I do see a problem with your description. It can't work back to SW99 and test every command, since some of them were added after SW99.

I could see a set of benchmarks. One for the flat plates with holes people, one for the surface crazies, one for the lots of parts people...

The first and third probably would work fine back at SW99. The second would probably have to have slightly different parts for each release.

Jerry Steiger Tripod Data Systems

Jerry,

You are right about not every command being available that far back. And some commands change from release to release.....what is it, OpenDoc4 by now. One way to handle that is to use the old version API call and the other way is to test for the release of SW and execute the correct API calls for that version. Perhaps being somewhat limited is a good thing. After all, one purpose for a benchmark is to keep an eye on the core functionality that is already there and not be distracted by all the new gee whiz stuff.

Having benches that test various subsets is a good idea. Of course it will require people to think when interpreting results.

I have a couple of the prismatic part routines already built. One does holes in a plate and other creates a fractal in solids. The hole in a plate routine is capable of running almost any Windoze box out of memory.