# How would you like to see the reviews graded?



## Dragongirl (Sep 22, 2002)

There has been a lot of discussion about the merits of the current rating system used in the reviews area of the site.  Please don't use this thread to continue those discussions there are other threads for that.  This is simply to try to find out what you, the users of this site, would like to see.

Edit : Oh bother!  Can a mod change the typo in the poll from "wan" to "want".  Thanks.


----------



## Dragongirl (Sep 22, 2002)

I voted for no ratings.  By this I mean no stored and accumulated ratings by the site.  This would not stop people from posting a rating within their review, but if there is no stored rating for a product then that would probably stop people from posting reviews just to boost a rating.


----------



## Nightfall (Sep 22, 2002)

There's my vote.


----------



## Crothian (Sep 22, 2002)

The rating system seems to be the fast way for people to get an idea of how a product is.  It is too bad that people seem to want to abuse it to try to make certain products rate higher or lower for no other treason then they want to.  Personally, I like the ratings, but I read the reviews.  However, becasue of the abuses in the rating system, I voted no rating syastem.  It seems like the easiest way to get rid of that.


----------



## Chun-tzu (Sep 22, 2002)

I prefer a 10 point scale to a 5 point one, because for me, most products (the ones I buy, anyway) fall into the 7-10 range, and with a 5 point scale, it all seems to get a 4.

But we've already got a substantial database, and I don't know what changing the scale would mean (i.e., having 2 separate scales, or averaging, or asking reviewers to re-score products, or what).

You could also go with the standard American grading system, from an A+ to an F.


----------



## Dragongirl (Sep 22, 2002)

Chun-tzu said:
			
		

> *You could also go with the standard American grading system, from an A+ to an F. *




Darn, wish I thought of that when I made the poll.


----------



## rounser (Sep 22, 2002)

Almost completely a suggestion of whimsy rather than being realistic:

There may be a call for "halo effect" and reviewer bias checkboxes like:
Reviewer admits to being a fan of the author/dislikes the author
Reviewer admits to having judgement significantly influenced by nostalgia
Reviewer admits to what they see as being a cool factor of concept of the product (e.g. "big, big dungeon!") significantly affecting their judgement of the actual execution of said product
Reviewer admits to what they see as being a poor concept of product affecting their judgement of the actual execution of said product.
Reviewer admits to judging product more harshly or more favourably because it's published by WotC/not published by WotC
Reviewer admits to being a fan of/disliker of cthulhu elements
Reviewer admits to being a fan of/disliker of psionics
Reviewer admits to being a fan of/disliker of the lower planes
Reviewer admits to being a fan of/disliker of low magic and "grim n gritty"
Reviewer admits to being a fan of/disliker of high magic and high fantasy
Reviewer admits to being a fan/disliker of settings X, Y and Z
Reviewer admits to preferring crunch over fluff
Reviewer admits to preferring fluff over crunch
Reviewer admits to only having done review to bump up the rating of said product
etc. etc.

That way you could match your own preferences and biases to those of the reviewer, and decide that because their priorities don't match yours, to take their words with a grain of salt or more.  Reviewers don't like to think of themselves as being biased by stuff like nostalgia or being a fan of the author, and it's pretty easy to tell when they are, but some of them hide it pretty well and you end up buying something with dodgy execution based on their halo effected or "I'm a fanboy/hater of X so I'll flagwave for it/throw mud at it" reviews.  Caveat emptor.

Altogether, that's way too many checkboxes.  Perhaps reviews could have an "admitted reviewer bias" section instead.  But then again, most of us are blind to our own biases, or see them as "being objective", so the above suggestions are probably pointless. 

Perhaps the more biases a reviewer admits, the more their review is respected.  It would cut down the amount of reading time of multiple reviews as part of "getting to know the reviewer" and trial and error in buying products they recommend to see how your preferences and biases match those of the reviewer.


----------



## Khan the Warlord (Sep 22, 2002)

I voted for a simple "thumbs up or down" rating.

The reviewers should provide a clear, complete review and simply give a nod to if they believe one should purchase the product or not.


----------



## MeepoTheMighty (Sep 22, 2002)

I would like to see a 1 to 10 rating on a number of different aspects of the product.   For instance, 


Fluff:  7/10
Crunch:  9/10
Art:  3/10
Layout:  9/10
Cartography:  3/10
Editing: 8/10
Value: 6/10


The "Fluff" would cover the concept behind the product, the writing style, and the overall feel evoked by the text.  It would be a measure of how inspiring the plot devices are, or how solid the story behind an adventure is.

The "Crunch" section would focus on things like rules balance, proper statting of NPCs, and the mechanical represantations of the fluff.  How well do the rules presented support a concept or idea?  How consistent are they with accepted d20 conventions?

Art would obviously rate the interior and exterior art, both on its artistic merits and in relation to the subject matter.

Layout would cover things like use of margins, whitespace, fonts, and shading.  How readable is the book?  Does it make your head hurt just to look at it?

Cartography is self-explanatory for products which have maps.

Editing would rate the ability of the publishers to go over their texts with a fine-toothed comb, looking for rules discrepancies or obvious misspellings or omissions.

Value would be a gut reaction of "was it worth what I paid for it?"

These could either be averaged to find a composite score, or given as a total # of points.   Maybe a few other categories could be added to make it a clean 100 point scale.


----------



## Chun-tzu (Sep 22, 2002)

I had a similar thought, Meepo, but was thinking of the following categories:

Ideas                 7/10
Mechanics          5/10
Presentation      9/10

Often, I'll find that supplements have some interesting ideas that might inspire me in some way, but there's something about the mechanics that bothers me.

Then again, mechanics can be tough to rate, since your first impression might be flawed.

Presentation collapses art, layout, cartography, and probably editing.


----------



## dren (Sep 22, 2002)

Doesn't matter the scale used. 

I look for the reviewers comments and depending upon who the reviewer is, that will sway me yea or nay.


----------



## Morrus (Sep 22, 2002)

Dragongirl said:
			
		

> *There has been a lot of discussion about the merits of the current rating system used in the reviews area of the site.  Please don't use this thread to continue those discussions there are other threads for that.  This is simply to try to find out what you, the users of this site, would like to see.
> *




As with your previous suggestion, unless you're prepared to rewrite the code, we're stuck with it the way it is.  Sorry!


----------



## Neo (Sep 22, 2002)

I think most people who write a review tend to say whether they recommend it to others or not, and as part of the review list its merits and flaws, as well as describe the waulity of the art, layout, page count, binding etc...

As such its probably okay just to go on the review itself, as different people tend to look for different things in a product so will likely scan numerous reviews of the same thing for different perspectives in order to guage whether or not its what they want (I know I do).


----------



## Grazzt (Sep 22, 2002)

Dragongirl said:
			
		

> *
> 
> Edit : Oh bother!  Can a mod change the typo in the poll from "wan" to "want".  Thanks. *




Fixed.


----------



## Grazzt (Sep 22, 2002)

Dragongirl said:
			
		

> *I voted for no ratings.  By this I mean no stored and accumulated ratings by the site.  This would not stop people from posting a rating within their review, but if there is no stored rating for a product then that would probably stop people from posting reviews just to boost a rating. *




Gotta go with this one as well...


----------



## Dragongirl (Sep 23, 2002)

I would like to know why this was moved.  It was to the public who views ENWorld, not to the mods/owners.  You allow all kinds of OT threads to remain but you move this just because it involves this website?  Morrus himself responded to it, and did not find cause to remove it to meta at that time.

I would like to request that this thread is returned where you found it.


----------



## Morrus (Sep 23, 2002)

Actually, I meant to move it.  Must have been distracted by something.

It's a thread about the site; thus it belongs in Meta.  And the forum description makes no mention of Mods: "questions or comments about the messageboards or the site in general".


----------



## Dragongirl (Sep 24, 2002)

Too bad.  Most people don't bother to come to meta.  IMO


----------



## CRGreathouse (Sep 24, 2002)

I voted 1-10, but really that's 1-5 with half-points.

I'm not too familiar with the code behind it, but I did go over it a while back and it didn't look too complex.  It's quite possible to change it over.



			
				Dragongirl said:
			
		

> *Too bad.  Most people don't bother to come to meta.  IMO *



Well, I did...


----------

