benchmark results

Damn man, how'd you get the IO number so low... That's a real time sucking battle on my benchmarks. I know that these tests aren't the end all but they're a lot of fun. And they are honestly revealing.

As for the 1000 vs the 3000... I suspect that the benchmark assys just aren't enough to challenge the 3000. I went with 1000 because I felt it was the price/performance sweet spot. And the benchmark reveals that anything more might be overkill at this moment. Next year, I'll buy the 3000 for the same money as the 1000 today and still be way ahead.

Thanks for the post! AMD all the way bro!

- Eddy

I just happened to think.... could it be the 2Gb ram vs 1Gb in my setups? I believe that would adjust the IO number accordingly. Would you be willing to pull a gig out and rerun the benchmarks to confirm? That way, we would all know the exact effect of 1Gb vs 2Gb.

Thanks again!

- Eddy

Those numbers really look good - here are mine.

Dell Precision 350

2.8 mHz P4 1 GB ram nVidia Quadro 900XGL Win XP Seagate SCSI drive

Total = 164 Graphics = 27 CPU = 86 I/O = 51

WT

Sounds Good, but I would like to see more info on the hardware used. IE: make and model of the hard drives, do they have cache built in? Scsi drive speed, wide, U-wide, U-wide2..? Is the ram the same speed? Have you tried exchanging the graphic cards ect..? All these have a bearing on the results, what I am trying to see is what causes the speed increase, or is it a little bit of each new component. Many thanks Pete

There is life on Mars, but it is a little confused at the moment!

I won't be able to change the ram because we haven't confirmed that we are keeping the system yet. We have to convince our systems guy and/or owner to go away from the HP that they have been standardizing on. These numbers should help with that. I don't know if I trust myself pulling ram out either. I will post again though with whatever additional test results I come up with.

Thanks for your posts, Bob

Damn,. you're making some of us salivate!!

Alright AMD!!

.. ;^)

bszotko wrote:

Has anyone run this test using 2004? If so, what are your results, please.

Best Regards, Devon T. Sowell

Look back to ~1-9-04 for my thread "Solidworks 2004 - confirming what everyone knows"

I explain what I did and how...

- Eddy

No trouble. Interestingly, I went back after SP2.1 came out and I ran 2.0 and 2.1 benches again from the same machine. This time they took almost twenty more seconds to run. I can't figure out why. Since I'm still testing anyway I think I'll start over. The good news is that despite both versions taking longer than the last time, 2.1 *is* faster than 2.0. I haven't looked into it yet but I suspect the files are marginally smaller. I also can't figure out how Bszotko got his similar machine to bench twenty seconds faster than mine from 1-9-04, even with more memory and upgraded graphics. I'm running Raid-0 and supposedly one of the fastest Athlon64 motherboards you can buy and it takes very little time to load the files, etc. and his benchmarks absolutely smoke. Well, I'll pursue the differences as time permits. For now, the short story is that 2.1 seems better :)

- Eddy

Eddy,

The benchmark doesn't need 1GB to run in. In fact it would probably give the same numbers with 256MB.

The SpecAPC benchmark on which the SW benchmark was based was highly biased toward video card testing. For those of us who are regeneration time bound it is the cpu and i/o numbers that are important. The fact that the Opteron is 17% faster than the Xeon in cpu and almost 30% faster on i/o suggests that large assembly performance is going to be greatly helped. The ship in a bottle benchmark which is almost purely regeneration time based also shows this kind of improvement.

The type of hard drive in the system will probably have little to do with benchmark i/o times because after the first iteration most of the disk access is covered by cached files. You have to look at the difference between the first iteration i/o and the subsequent iteration i/o to see any difference hard drive i/o makes.

I think i/o to memory is going to be the big benefit of the 64 bit machines. If I don't miss my guess the Xeon has a bigger on chip cache and it still can't beat the Opteron on i/o.

Another th> I just happened to think.... could it be the 2Gb ram vs 1Gb in my setups? > I

I would have agreed before now but a faster video card shouldn't account for nearly twenty seconds. His IO numbers account for most of it while his video numbers were the same as mine. I'm having a hard time understanding exactly what would help the IO. 'Cause is it's possible to that extent, I want to do it too? Does the graphics card help the IO, independent of the graphics card performance number? (which was also his original question)

As for the Xeon, yes, it has more on chip cache. Unless a test depends greatly on core cpu frequency, the 64bit AMD chips run circles around them. Take a look at server comparisons and you'll see the same results. The tests that aren't biased toward clock speed just smoke on the AMD chips. AMD is doing Solidworks, or rather their users, a huge favor :)

And we all should know by now that Intel is working on their next generation to compete with AMD, just as they did with P4 vs AthlonXP. And they will likely have the ability to scale clock speed ahead of AMD. My analogy between AMD and Intel is that Intel has slightly higher water pressure but AMD has bigger pipes :) It's going to be an interesting year chip fans!

- Eddy

kellnerp wrote in news:Ju7Tb.155703$sv6.849723@attbi_s52:

The AMD64 chips have the memory controller integrated on the CPU, running at full clock speed. The decreased latency from this arrangement allows the AMD64 chip to compete pretty well against the Xeon's huge cache. Also, some applications don't seem to care that much about cache memeory, beyond a certain point. IIRC, SW is one of them. Compare a P4 3GHz against a similar P4 Xeon in SW, and they're similar. This may change when Xeon gets the 800 MHz memory bus.

The short way to say it is this: the AMD64 integrated memory controller is currently offsetting any need for a Xeon-sized cache on the AMD64 chip.

I do agree that it should be interesting to see what effect a 64bit port of SW will have on performance. I don't know what to think myself. I've heard different his from different people who's opinins I normally respect on such matters. All bets may be off because of quality of MS compilers for AMD64. I dunno.

Benchmark rant: Since it's a hardware benchmark, it makes a certain amount of sense to report the results in terms of hardware subsystems. But it can be difficult to interpret how the numbers will affect actual user experience. I'd rather see numbers that more directly tell me how a machine is going to affect my workflow.

Graphics performace expressed in seconds is particularly unhelpful. You need to know the score of a machine you're familiard with to evaluate the results given for another machine. Consider bszotko's graphics score of

25sec. I know that's good, because it's a lot lower than my machine's scores. What would be more useful would be reporting a minimum framerate at several resolutions for several files of varying complexity. Then I could look at the test files, compare them to the work I'll do, and know what video card will be adequate for my work. Do I need a QuadroFX 500 or 3000? The benchmark can't tell me that. I have to try one and find out.

Also, the SPEC benchamrk is unaware of HLR in shaded mode. So if you're going to be selling the hardware, make sure to turn this off to make your hardware look better.

You mentioned the fact that running the test multiple reduces the dependency on the HD. Shouldn't the machine reboot between tests? Anyhow, the broad category of IO doesn't tell when I need a SCSI RAID array or just a fast ATA drive to achieve reasonable load times. Several load to edit scores and save scores would tell me how long it takes to handle small, medium and large parts, assemblies and drawings, and give me some idea of what system I might need.

The CPU score involves not only the CPU, but the memory system, and if there's not enough of that, the HD system. So it also can involve a large part of the IO system. Perhaps I should just relax and take this as a measure of rebuild performace in general. Anyhow, I think naming this score after a specific part of the system mischaracterizes what's measured, and doesn't help me undestand how different machines compare on different types of files and data.

There. I've succeeded in complaining about every part of the benchmark. I feel better.

Dale, maybe I misunderstood your intent but I think you might be missing the bigger picture. If a system takes 100 seconds, it's twice as fast at completing a given set of "average" tasks than a machine that scores 200 seconds. That part's obvious. You have the option of running the more exhaustive APC series if you wish but the base test is an excellent tool for timing how long it takes to open and manipulate real SW files. The fact that Spec runs *your* Solidworks installation using real SW models, assy's and drawings makes it a fair comparison as far as benchmarks go. It's about as non-synthetic as you can get (compared to something like SiSoft Sandra). Obviously you have to see how an installation performs with your real files but, like 0-60 times for a car, less is more.

My only interest in defending the benchmark is to throw perspective at how it works and that the results are revealing. Look at the comparisons on Spec's site and see how the various P4, P4 Extreme, Xeon, and Opteron systems pan out. The time it takes to complete tasks in Solidworks gets better as you move up the food chain. And *that* is what we need to know.

Everyone's mileage varies but it's as close as we're gonna get to a fair comparison tool. Mike's ship is the other fair comparison but doesn't tell us much about spinning a loaded assy around. Our bottlenecks are (were) shading and spinning, file IO, and regen times. The spec test is the only benchmark that glimpses each of those categories. And I have never found their results falling short, when comparing one machine to another. In other words, I've tried to find reason to disagree with it and I can't. When I run the spec bench on two different machines and it says one is "twice" as fast as the other; I try it myself using my files and it's always true.

- Eddy

I think maybe you did miss the nub of my gist. My gripe is that the benchmark can ONLY tell me which is relatively faster on the set of average tasks. I'd like the result presented in a way that's more meaningful to a user, and differentiated by task, with more explanation for how the given scores are generated. In short, I wish it was more specific.

I agree that it's probably the best we can expect. It IS useful for comparing various platforms as far as it goes.

I really like Mike's ship as a benchmark of rebuild speed. It very specifically tells me how fast the machine can do the rebuild without significant graphical or IO systems (apart from memory speed). It runs quickly, and does a good job of isolating the CPU from graphics and storage.

I agree completely. The benchmark can't tell us how much faster lofting, or drafting, etc. will be. So I see your point. It's good as a non-synthetic aggregate. Maybe we can get Mike to volunteer more time :) Come on Mike, get to it. We need more, we need more!? Haha.

- Eddy

I wouldn't want to lean on Mike if I have some skills myself. I'm starting to believe I can do some useful macro programming. I was seriously considering making some benchmark tools that seem more reasonable to me. For example, the minimum frame rate idea for graphics performance. I like the idea, but it would be some time before I get to it. I have other projects I should actually finish first.

Eddy,

Could you clue me in to what I might be doing wrong in trying to run the 2003 benchmark on 2004? You said you did it by batch converting & fixing what files had errors. I did that, but some part files said they were from sw97plus and they wouldn't open. I got all the assemblies to open up ok, so I think the test would run, but when I try to run it I get: Run-time error '713' Class not registered. You need the following file to be installed on your machine. MSSTDFMT.DLL

I found that file in my Windows\system32 directory and even copied it to the sw2003 benchmark directory so maybe it could find it, but that didn't help.

Any ideas?

Thanks, Bob

Yep, been there done that. For some reason, SW2004 (or the '03 benchmark trying to run '04) breaks the registration of that file. Open up a command prompt and do the following...

- open a command prompt window

- change directory to "c:\windows\system32"

- type the following (without quotes) and hit enter: "regsvr32.exe c:\windows\system32\MSSTDFMT.DLL"

That should do it!