Steve's Golden Oldie #7: My Rating System
Golden Oldies are miscellaneous snatches of writing of mine scattered around the web. I'm republishing them here from time to time as a way of collecting everything in one place. These items are generally unrelated to the themes of this blog, so feel free to move on to other posts that are more on-topic. Or, enjoy ...
This is an article I posted to Epinions in 2000. It might not make total sense to you if you aren't familiar with Epinions and especially its former system by which readers could rate other people's product reviews. Still, you should be able to glean enough to figure out most of it from the following.
My Infallible System (No, no. No need to thank me.)
by Steve Smith
Having been active on Epinions for about a month, I've concluded that rating reviews is a highly subjective process, except when I do it. My ratings are objective, scientifically arrived at, and exact. When I rate an opinion of yours, you can safely ignore any other ratings you may have received for it. My rating will be the truth.
"How does he do it?" you ask. "I will tell you," I answer.
By sharing my system with the Epinions community, I look forward to seeing everyone's opinion ratings gain in precision, to the benefit of all. (By the way, in case you don't want to work through the whole thing right now, the proper application of this system would award this opinion an "HR.")
Epinions has given us these ratings to work with: Highly Recommended, Recommended, Somewhat Recommended, and Not Recommended. Here are what these mean in my ratings system:
Highly Recommended (HR). I award this rating to a review that is well written (no speling errors, and they uses good grammar), complete, detailed, based on the author's obvious familiarity with the product or service reviewed, and preferably marked by originality, wit and intelligence. It is usually subject only to the "green eyes" adjustment (see below). If Epinions is a brave new world, then HR reviews (and by extension, their authors) are the Alphas of that world.
Note: If a review does not meet the above criteria, it is still acceptable for you to award it an HR if the author's Epinions user name begins with "S," has exactly two vowels in it, and ends with the postal abbreviation for an eastern seaboard state.
Recommended (R). I give this rating to a review if it is decently written, workmanlike, and contains enough detail for the reader to make an informed decision about the product or service discussed. R's get the job done, competently but without a lot of flash. They're our Betas and Gammas, and awfully glad just to be who they are -- they don't want to work as hard as those frightfully clever Alphas.
Somewhat Recommended (SR). I give SR's to reviews that would otherwise get a "Not Recommended" but for the inclusion of at least one actual piece of real information. A review that goes on for a hundred words saying "This movie sucks" might yet pull off an SR if somewhere in the stream of unanchored ranting the author drops in something concrete like, "This movie sucks because it only has one car chase." SR's are our Deltas. Oh, no, I don't want to play with Delta children, and Epsilons are still worse ...
Not Recommended (NR). Refer to the above example, but drop all data points whatsoever. At least in Aldous Huxley's world Epsilons performed some useful chores; not so at Epinions. I really try not to give NRs, though, and not because I worry about retaliatory ratings (OK, maybe retaliatory mail bombs); I always look for some way to justify upgrading to SR. Thus the importance of adjustments.
Adjustments. So far I have described a system and an interpretation of the ratings that may not differ very much from your own. The real elegance of my system lies in the adjustments. It is these that allow me to rate people's opinions with scientific precision.
Adjustments are made up or down starting from the baseline HR, R, SR, or NR. Of course, you can't actually go up from HR or down from NR, so in those cases I leave the rating alone and post a comment or send an email stating that the review was "doubleplus good" or "doubleplus ungood" (to borrow from another dystopian vision).
The significance-of-the-product adjustment. A one-paragraph review of corn flakes is OK; a one-paragraph review of a car probably is not. Adjust downward in the latter case. However, adjust upward if a writer goes on for 30 paragraphs about corn flakes, in sheer admiration of the breathtaking lack of proportion.
The green eyes adjustment. If an opinion is so brilliantly written that it makes me jealous, adjust downward.
The kissing up/bringing down adjustment. If the writer is a recognized Epinions celebrity, adjust upward to try to gain favor. Then adjust downward to gain attention by attacking the mighty. Lesson: Celebrity will get you nowhere.
The rebellion-for-the-hell-of-it adjustment. If everyone before me has given the same rating to a review, and it happens to be the rating I would have given, go one step upward or downward (doesn't matter which).
The "What can I say? I'm a guy" adjustment. If the profile photo reveals the writer to be an attractive woman, adjust upward.
The kindly-but-dumb adjustment. This refers to a line from a Simon and Garfunkel song about animals at the zoo: "...and the elephants are kindly but they're dumb." Some people are like this, and when they write a poor review but one that shows their heart is in the right place, I might adjust upward.
The eye-for-an-eye adjustment.* Don't actually use this one, because it would be wrong. If I did use it, which I don't, I would adjust my rating downward for any reviewer who gave me a poor review. But I don't do that. Honest.
The tightrope walking adjustment. Does it suddenly strike me that I've been rating either too permissively or too harshly lately? Then adjust upward or downward accordingly.
The bombs-away adjustment. If the writer uses the phrase "It's the bomb" anywhere in the review, adjust downward. If he or she says "It's da bomb," adjust downward two steps.
The home field adjustment. If the writer's profile indicates that he or she lives near me, adjust upward. There's nothing special about North Carolinians, but I might run into one of these folks one day and want something from them.
The "Stupid or just plain evil?" adjustment. If a writer expresses a political, philosophical or religious opinion at odds with my own -- or even if they just say something good about disco music -- adjust downward while telling myself that's a terrible thing for me to do. Or grit my teeth and let it go.
The "How should I know?" adjustment. If the review is of something I have no interest in, need of, or knowledge about (e.g., breast pumps), and therefore I can't really say if it's a helpful review or not, give the writer the benefit of the doubt and adjust upward.
The personal mood adjustment. Self explanatory. Adjust upward or downward accordingly.
There's one final step to using the system correctly (although the scientific rationale for this is too complicated to explain here):
Apply the Smith Randomizer(TM). For every 10th review that I read, flip a coin. Heads, move the rating up one step; tails, down one step.
That's it, and again, I'm happy to put this system out into the Epinions universe. Feel free to use it to improve your own ratings. And if you don't mind, please notify Epinions that they can close this category now that the final word has been said on the subject.
* Can you believe it? I originally called this the "___-for-tat adjustment" (fill in the blank) but Epinions wouldn't let me submit this opinion until I edited out the "objectionable" word. Geez!
Technorati Tags:
humor, epinions, reviews, review, writing, opinion, web sites, websites, internet, ratings, entertainment, Aldous Huxley, writers, op-eds, communities, community, readers
This is an article I posted to Epinions in 2000. It might not make total sense to you if you aren't familiar with Epinions and especially its former system by which readers could rate other people's product reviews. Still, you should be able to glean enough to figure out most of it from the following.
My Infallible System (No, no. No need to thank me.)
by Steve Smith
Having been active on Epinions for about a month, I've concluded that rating reviews is a highly subjective process, except when I do it. My ratings are objective, scientifically arrived at, and exact. When I rate an opinion of yours, you can safely ignore any other ratings you may have received for it. My rating will be the truth.
"How does he do it?" you ask. "I will tell you," I answer.
By sharing my system with the Epinions community, I look forward to seeing everyone's opinion ratings gain in precision, to the benefit of all. (By the way, in case you don't want to work through the whole thing right now, the proper application of this system would award this opinion an "HR.")
Epinions has given us these ratings to work with: Highly Recommended, Recommended, Somewhat Recommended, and Not Recommended. Here are what these mean in my ratings system:
Highly Recommended (HR). I award this rating to a review that is well written (no speling errors, and they uses good grammar), complete, detailed, based on the author's obvious familiarity with the product or service reviewed, and preferably marked by originality, wit and intelligence. It is usually subject only to the "green eyes" adjustment (see below). If Epinions is a brave new world, then HR reviews (and by extension, their authors) are the Alphas of that world.
Note: If a review does not meet the above criteria, it is still acceptable for you to award it an HR if the author's Epinions user name begins with "S," has exactly two vowels in it, and ends with the postal abbreviation for an eastern seaboard state.
Recommended (R). I give this rating to a review if it is decently written, workmanlike, and contains enough detail for the reader to make an informed decision about the product or service discussed. R's get the job done, competently but without a lot of flash. They're our Betas and Gammas, and awfully glad just to be who they are -- they don't want to work as hard as those frightfully clever Alphas.
Somewhat Recommended (SR). I give SR's to reviews that would otherwise get a "Not Recommended" but for the inclusion of at least one actual piece of real information. A review that goes on for a hundred words saying "This movie sucks" might yet pull off an SR if somewhere in the stream of unanchored ranting the author drops in something concrete like, "This movie sucks because it only has one car chase." SR's are our Deltas. Oh, no, I don't want to play with Delta children, and Epsilons are still worse ...
Not Recommended (NR). Refer to the above example, but drop all data points whatsoever. At least in Aldous Huxley's world Epsilons performed some useful chores; not so at Epinions. I really try not to give NRs, though, and not because I worry about retaliatory ratings (OK, maybe retaliatory mail bombs); I always look for some way to justify upgrading to SR. Thus the importance of adjustments.
Adjustments. So far I have described a system and an interpretation of the ratings that may not differ very much from your own. The real elegance of my system lies in the adjustments. It is these that allow me to rate people's opinions with scientific precision.
Adjustments are made up or down starting from the baseline HR, R, SR, or NR. Of course, you can't actually go up from HR or down from NR, so in those cases I leave the rating alone and post a comment or send an email stating that the review was "doubleplus good" or "doubleplus ungood" (to borrow from another dystopian vision).
The significance-of-the-product adjustment. A one-paragraph review of corn flakes is OK; a one-paragraph review of a car probably is not. Adjust downward in the latter case. However, adjust upward if a writer goes on for 30 paragraphs about corn flakes, in sheer admiration of the breathtaking lack of proportion.
The green eyes adjustment. If an opinion is so brilliantly written that it makes me jealous, adjust downward.
The kissing up/bringing down adjustment. If the writer is a recognized Epinions celebrity, adjust upward to try to gain favor. Then adjust downward to gain attention by attacking the mighty. Lesson: Celebrity will get you nowhere.
The rebellion-for-the-hell-of-it adjustment. If everyone before me has given the same rating to a review, and it happens to be the rating I would have given, go one step upward or downward (doesn't matter which).
The "What can I say? I'm a guy" adjustment. If the profile photo reveals the writer to be an attractive woman, adjust upward.
The kindly-but-dumb adjustment. This refers to a line from a Simon and Garfunkel song about animals at the zoo: "...and the elephants are kindly but they're dumb." Some people are like this, and when they write a poor review but one that shows their heart is in the right place, I might adjust upward.
The eye-for-an-eye adjustment.* Don't actually use this one, because it would be wrong. If I did use it, which I don't, I would adjust my rating downward for any reviewer who gave me a poor review. But I don't do that. Honest.
The tightrope walking adjustment. Does it suddenly strike me that I've been rating either too permissively or too harshly lately? Then adjust upward or downward accordingly.
The bombs-away adjustment. If the writer uses the phrase "It's the bomb" anywhere in the review, adjust downward. If he or she says "It's da bomb," adjust downward two steps.
The home field adjustment. If the writer's profile indicates that he or she lives near me, adjust upward. There's nothing special about North Carolinians, but I might run into one of these folks one day and want something from them.
The "Stupid or just plain evil?" adjustment. If a writer expresses a political, philosophical or religious opinion at odds with my own -- or even if they just say something good about disco music -- adjust downward while telling myself that's a terrible thing for me to do. Or grit my teeth and let it go.
The "How should I know?" adjustment. If the review is of something I have no interest in, need of, or knowledge about (e.g., breast pumps), and therefore I can't really say if it's a helpful review or not, give the writer the benefit of the doubt and adjust upward.
The personal mood adjustment. Self explanatory. Adjust upward or downward accordingly.
There's one final step to using the system correctly (although the scientific rationale for this is too complicated to explain here):
Apply the Smith Randomizer(TM). For every 10th review that I read, flip a coin. Heads, move the rating up one step; tails, down one step.
That's it, and again, I'm happy to put this system out into the Epinions universe. Feel free to use it to improve your own ratings. And if you don't mind, please notify Epinions that they can close this category now that the final word has been said on the subject.
* Can you believe it? I originally called this the "___-for-tat adjustment" (fill in the blank) but Epinions wouldn't let me submit this opinion until I edited out the "objectionable" word. Geez!
Technorati Tags:
humor, epinions, reviews, review, writing, opinion, web sites, websites, internet, ratings, entertainment, Aldous Huxley, writers, op-eds, communities, community, readers
0 Comments:
Post a Comment
Links to this post:
Create a Link
<< Home