Maximum PC

it can be seen indeed as a good thing or a bad thing, because if you've got all the money in the world you don't care who has the cheaper product... you just want what's best. while that's honorable, i just don't think maximumpc's readers are in such a situation. on some articles, such as the one comparing microsoft office to openoffice.org, you seem to have little or no regard to the price. while i understand the value of stating one product is better than the other, i think it would be cool to see a new delta added on the scores you give things as the "bang for your buck" score or something similar to that.

i know microsoft's office is indeed better than OOo, but lets be honest here -- OOo is free, microsoft office is like $100 or more easily. i feel most of us could set aside our minor knit-picks about OOo and save the extra $100 for some good software or a computer part. of course, users should make that assessment on their own, but it's a bit hard to do at times when the review doesn't seem to have any emphasis on this at all.

anyway, not bitching or anything, just would like to see price have a little more pull in what you rate products than it currently does.

thanks, and great magazine

Kamel, you raise a good point, but I think you are missing an obvious point: the price of a product fluctuates constantly, AND is something that the reader can easily find on their own.

If we use your example of a review of MS Office vs. Open.Office, we can see how the price isn't relevant to the review. The reviewer will compare the products for you ... listing their pros and cons and the features that set them apart. Once you have a read the reviews, you can compare prices and decided which of the products to spend money on.

Essentially, what I'm saying is that the quality of the software is something that the reviewer can judge for you ... but the decision to spend your money, and how much of it, remains a personal one.

I hope that answers your question. Heck, I hope I'm correct.

not saying there should be put a lot of emphasis on price, i'm just saying that the emphasis put on price is so little it might as well not exist -- and that in itself was a new establishment. when i first subscribed to maxpc back in '99 or 2K, forgot which, they put absolutely no emphasis on price, and would not even list the price on the magazine.

i understand it's up to the user to make that decision, but it is also not fair to compare a geforce 2 mx400 to an ati x800 pro saying that the x800 pro is far superior, while leaving out the minor detail that you could probably buy a thousand gf2's with the amount it would cost to get an x800 pro. of course, you can argue they aren't in the same series, but that's really not the point. put a $50 speaker against any $300 speaker on the market and the $300 speaker is gonna be a hell of a lot better, that's just all there is to it.

allow me to reiterate, however. i said "more" emphasis should be put on price -- not a large amount or anything like that. also, i suggested it be under possibly a different section not to taint the rest of the review (for example when they mention the good and the bad, maybe put an orange block or something saying bang-for-buck rating, so that users can know when reading an older magazine to investigate the price).

I would venture that most readers picking up a magazine entitled "MaximumPC" are aware of the different pricing tiers that exist within each piece of hardware/software (and if they aren't aware, they should be). Being as that is, nearly every review of a product will list an alternative in the same class and how it compares. From there, the reader can evaluate on their own the pros and cons of each, and determine for themselves which is the best bang for the buck. In most cases (not all), that's a highly subjective decision.

To use your example of the video cards, the x800 Pro should not be knocked in the slightest for it's price compared to a GeForce2 MX400. They represent drastically different classes and there's generally a price range that we've come to expect in each class.

Taking into consideration the subjective nature of bang/buck determinations, and taking into consideration the common pricing found within various tiers of hardware/software, the only fair answer I see is to to continue with the way they have been doing it. If a product ventures out of the expected price range, it will get mentioned and I've seen it be the difference between a "kick ass" rating or not receiving it. For products that excel in pricing advantages (such as a free open source productivity app, or a piece of hardware that is lower than what we're accustomed to paying), by all means award it an extra point for doing so. But it doesn't make sense to deduct a rating from another product that falls within the appropriate pricing tier. Knock it for not delivering the performance, but not on price in the context above.

My opinion anyway...

I am so glad you guys brought this up and so pissed I don't have time to finish a project I am working on.

I have a algorythym I have been working on that is modified from a product review package for software.

The output score is on a 1-100 scale, but you can't simply lok at the score and say "oh - that one got a 52 - it must suck".

You must compare it to other products of it same class.

My example run was done with OS's. Linux, OSX and XP.

I think they all scored in the 70's. Linux got a good boost for being free.

The algorythm doesn't care if the product is worth the price. It simply deducts points for certain price ranges. There is a subjective section - but basically, if you charge for the product you cannot get 100 points. Why is this fair? Because its designed for scores to be compared to competative products. If a product is free, there will likely be things about it that lower its scores as well.

So what does a score of 78 means when your competition is at 76? It means you have the better product as tested. It also means you have room to improve. And, your competitor is right on your ass.

This method (OK - a similar one) is used in product consulting. The raw score is not an indication of if you are a "10", but one of room to move with the product and a scale you may rate competitors by.

Think about it like taking the MPC score and only letting it count for 10 points - then the other 90 are based on price, footprint, flexability, compatability, stability (as tested), etc... (there are like 20 or so things IIRC)

Manta

You just wouldn't be 'Manta' if you didn't come with an algorithm every couple of months

This one is interesting, I'd like to see you post some real world results when you have them and when you have the time. That being said, my only beef w/ it would be if, using your above example, that 78 product was inexpensive and/or free but wasn't as good a piece of software/hardware as the 76pt one, assuming the one that scored 76 was priced in it's appropriated tier. I don't know if you have safeguards against that happening, or agree/disagree with the concept, but I'd hate to see a lesser product outscore a better product simply because the lesser one was priced cheaper, IF the better one was within the appropriate range.

I'm fighting off being pretty sick today, so hope the above makes sense.

I already posted prelim stuff at Commport sometime back. But the program to run the algrythm never got finished (its on my list). I used OS for that - but that isn't ideal as they alwasy score lower then other stuff.

Anyways, a cheaper program can outscore a more expensive one based on price - but not price alone.

I'm guessing here, but I bet MS Office and OOo would score close with OOo coming out ahead. That is only if you are comapring MS Standard - not Pro. That would be a good test though. Price alone is not enough to make or break a program. I think it only influences 10-12 points IIRC.

I really need to get back to my fun work.

Manta

It's difficult to make a universal algorithm though. There are always exceptions that aren't included in the formula, which add or subtract value.

Take your OS example, right? Many Linux distros make it really easy to download and install new software via tools like apt-get or emerge. Microsoft doesn't include functionality like that in Windows, but it's absolutely crucial for a Linux distro because traditionally, installing and updating apps has been one of the things new Linux users have problems with.

Do you dock Microsoft because they don't make it that easy to install new apps, or do you penalize Linux because installing apps requires maintaining dependencies and compiling stuff, while installing Windows apps is as easy as clicking an installer?

Ultimately, you have to do things kind of subjectively. At Maximum PC we score using two things. First, we have general descriptions for each verdict level. These are descriptions ranging from "The perfect PC product. Everyone should own this" to "Does one thing really well, but has several annoying traits" all the way down to "This product failed to function and did harm to either the tester or the testbed"

We use that as a starting score. From that point, we add or subtract points for flaws. Major flaws are docked one point, minor flaws a docked a half point, and showstoppers are docked multiple points. Major bonuses get an extra point, minor bonuses get a half point.

Here are some examples:

A minor flaw would be slightly slow DVD-R burn times, but top class DVD+R burn times. An abnormally high price can sometimes suffer from a half point loss.

A major flaw would be a gamepad without two analog sticks.

A showstopper would be a mouse that actually hurts your hand after long periods of use.

A minor bonus would be a good price compared to the competition or an extra feature that competitors lack. It could also be something that's just cool. Note, features that are minor bonuses now will probably be required in time. Think including a compact flash card reader with a digital camera.

A major bonus is something that makes using the product better than the competition. Good music management software for a MP3 player, a quieter cooling solution for a videocard are good examples.

Of course, some things are not only excellent, but also are so undeniably cool that we can't help but give them high scores. These products evoke lust at a primal level of the geek subconscious. Think products like the Western Digital Raptor and the iPod.

That's the short, short, Will Smith edition of How Maximum PC gives scores.

And the short version is, price is only one of our primary concern if it's out of line with all the other products in the same class.

///Will

That was a great answer ... thanks Will.

Someone should throw that up as a sticky .. I'm sure you get many questions about how MPC reviews products.

I like the system, btw ... a good blend of subjective opinion and objective data. Nice.

Hey Will,

I understand what your are saying. I am not knocking MPC's reviews.

I think what I really need is to finish it and then let you see it work. I think maybe you think I am taking out all subjectivity out. It only looks that way.

I did make a typo above me thinks.....the algorythm (lets say overly complicated flow chart) is based on a product (not software) review method.

What I am talking about is not new. Maybe its just not applied to software often.

It does take into account some of what you said.

Nothing is universal - and thats the point. You would not compare the score of an OS to one of an application. It would not tell you much.

But yes, it addresses many of the points you listed. Installation has its own sub section.

Draw backs? Many. One is that it requires a good amount of time to complete - maybe 30 minutes + mandatory review time. Another is that something like a serious security problem is not realized in the review. Just an example.

It may not be right for a mag. But for a client its better than a fully subjective review.

TTYL

Have fun Willl

Manta