You are currently browsing the Online Reputation Management weblog archives for February, 2008.
28 February 2008 by Olivier Zara.
Through face to face interactions, discussions, and other regular social intercourse, we are able to gauge and evaluate the character and actions of a person. If the relationship becomes important (business, partnership, hiring, love, leisure,…) and we need to know more, we can even sound out this person’s circle of acquaintances. But how can we evaluate reputation on the Internet? What is new and disconcerting is that on the Internet we have to trust people we’ve never met and are never going to meet. Besides, in certain cases, we don’t want to meet them at all! When it comes to e-Commerce for example, we’re interested in the seller’s DVD, not the seller himself. They send you the DVD you’ve bought; you pay them and that’s where the relationship ends. But before entering into this or any other transactions and since you don’t know the sellers, how can you know if they’re trustworthy. Based on this realization, eBay designed and built a system to help you evaluate a seller’s or buyer’s reputation, based on their completed transactions with other users. The system makes use of positive, neutral or negative ratings.
In the preceding post, we listed a certain number of services offering a way to evaluate reputation by means of a score (Read that post here).
The lesson from this post is that, while Google may be the leader in evaluating reputation by means of references, eBay is the clear leader in evaluating reputation by means of a score. But eBay presents a major problem; its system is a closed one. You can’t easily promote the reputation you’ve earned on eBay’s system outside of it. That eBay wouldn’t want its evaluation system to be used by competing e-Commerce sites is understandable since this would remove its differentiating factor and eat away at its sales. In any event, if it were advantageous for them to open their system, they would have already done so.
Numerous services have therefore been created to address this problem. These services have set up “open” evaluation systems, meaning that they’re not connected to the e-Commerce site hosting the current transaction. Notable among these services are iKarma, Rapleaf, TrustPlus and Gorb (Opinity has shut its doors). These last have copied eBay’s system and improved upon it in their own way, making theirs more user-friendly and universal (in terms of the reach of the evaluation) systems.
As an e-Commerce site, eBay can only provide an evaluation of you as a seller or a buyer. These new services, however, don’t sell anything except an evaluation of your reputation. To broaden their client base, they have thus applied the methods used to evaluate transactions to the evaluation of people, content or relationships.
Yet, the relevance of an evaluation by scoring is inversely proportional to how much the evaluator and the person being evaluated know about each other.
In other words, the more we know people, the less relevant an evaluation by scoring becomes because the relationship between the evaluator and the person being evaluated carries “more weight.” More weight in terms of sub-text and shared history of ups and downs. This means that friendship, love and the return of favors will push any score higher (oftentimes way above what it should be). Conversely, settling of scores and grudges will push any score lower (oftentimes way below what it should be).
I. How the level of acquaintanceship impacts the evaluation of reputation
Here are the 5 levels of acquaintanceship:
The evaluation by scoring is relevant for levels 1 through 3: weak or non-existent level of acquaintanceship. At these levels, the evaluator will judge the facts: “He delivered what I ordered in the agreed-upon timeframe,” “I’ve been paid,”…
At levels 4 and 5, the history of the relationship will start to weigh on the mind of the evaluator. Subjectivity (opinions, value judgments) will start to supersede objectivity (observable and quantifiable facts). At these levels, the chance of over-valuing (“he’s the best”) or under-valuing (“he’s lousy”) the other person becomes more likely.
On eBay, the level of acquaintanceship between users hovers somewhere between 1 and 2. In addition, it’s not possible to evaluate a person in the absence of a transaction. In fact, it’s the transaction that’s evaluated, not the person. Thus, a new evaluation is required for each new transaction between the same people. Indeed, it’s quite possible for a seller to perform efficiently during one transaction and fall short on another.
Conversely, on the new services such as iKarma, you’ll see that most evaluators share a level 4 acquaintanceship with the person evaluated, a fact which can be ascertained by visiting user profiles. While your best friends and colleagues may visit your profile, so may your worst enemies. Since no particular transaction is required, it’s up to each party to decide which criteria to use to evaluate the other. You may be evaluated on the basis of one act or several, one single day or several years, one transaction or a mix of several kinds of actions (transaction, relationship and content publishing).
On eBay, the relationship’s context is crystal clear. You’re either a buyer or a seller! On other services, however, there often lacks a field to specify the relationship’s context. Some users provide this information of their own accord in their feedback and some don’t. This context is helpful in determining the potential for over- or under- valuation.
II. The operating principles of eBay’s competitors
An analysis of the evaluation methods and communications of eBay’s competitors reveals that they operate on 2 basic principles.
1st Principle: Since no one’s perfect, neutral or negative ratings have to be allowed.
The goal is to create an evaluation of the person and/or their acts that comes as close to reality as possible. When a person behaves badly, it’s to be expected that this should be shared. In fact, this is what underpins eBay’s system. A small measure of discretion, however, is in order. Indeed, studies of eBay’s system have shown the limitations of neutral or negative feedback. If you give a negative rating, you expose yourself to in-kind retaliation from the other party. As a result, the system can sometimes get mired in an implicit understanding of non-aggression (i.e. mutually assured destruction). There have been instances where some buyers have even gone so far as to threaten sellers to try to get more than what was agreed upon: “If you don’t give me more, I’ll give you a negative rating”. A credible threat given that a seller’s reputation is his working capital!
More importantly still is the fact that while some criticisms may be well-founded, others may be fabricated; some may be fair and others unfair; some may rely on readily observable and quantifiable facts and others on prejudices, opinions or value judgments.
Linkedin is the most often cited example of why this principle is needed. If you take a look on that service at the recommendations on users’ profiles, you’ll see that the content of these is always positive and sometimes even panegyric. Given that a user may refuse a recommendation, this leads to a situation where de facto only positive feedback is allowed. It’s not true, however, that feedback is necessarily false just because it is positive. Conversely, it’s just as likely that neutral or negative feedback may be false.
In addition, if we consider the recruiter’s concerns, her decision whether to recruit someone is based on the skills that person possesses, and not on those which they have yet to master. User testimonies work on the same principle. While they do not give any indications about a person’s missing skills, they do validate part of this person’s skill set. As such, Linkedin’s system is imperfect, in the sense that a recruiter still has some legwork to do. The candidate and his references must still be interviewed in order to validate the skills not mentioned in any recommendations. The recruiter no longer needs, however, to validate the candidate’s full skill set. Regardless, this is an improvement on the current system.
It’s also important to realize that the person making the recommendation is putting his or her reputation on the line. You will be destroying your own reputation if you write that such and such is a computer genius, when in fact the person has a marginal grasp of IT. It follows that the value of a recommendation is tied to the reputation of the recommender; the higher the reputation the greater the value of the recommendation, and vice-versa.
Similarly, the more recommendations you garner the more likely it is, statistically speaking, that you already have a good reputation. It’s the same principle as Google’s PageRank. The more people recommend a site, the more they contribute to its credibility and good reputation.
Finally, giving negative feedback may lead to a form of vigilantism. On the Internet as anywhere else, laws defining the balance of rights between the accuser and the accused must be respected. The Internet must not become a lawless place where private disputes are settled in public through acts of vigilantism (libel) or where the lure of a positive rating is used to shake down merchants (see the example above).
From opinion to libel: While the laws concerning libel differ from country to country, in most of them, you leave yourself open to lawsuits and to adverse legal judgments if you leave a negative comment that’s liable to be interpreted as libelous.
2nd Principle: Anonymous ratings and comments have to be allowed in order to let people say what they truly think.
How can we believe an underling’s glowing feedback about his manager when this same manager is responsible for allocating year-end bonuses and promotions? Who will openly complain about their clients’ late payments?
Thus, anonymity would appear to be THE solution for venting, sight unseen, about one’s manager or clients with the full freedom of expression required.
However, who will want to promote their reputation on a service where just about anyone might publicly vilify them? Strangely enough, when you visit sites that allow anonymous posting, you immediately notice that on average 98% of feedback is positive. Weird, isn’t it? It’s not so weird if you take into account the fact that people with negative scores have closed their accounts and numerous others never even opened one for fear of negative feedback. Under such circumstances, the system of anonymous posting doesn’t meet its disclosure goals.
To counter user flight, some services such as Rapleaf have resorted to creating profiles, without asking for anyone’s approval, from email addresses harvested from the Internet. However, nothing is more easily changed than an email address (we already do so to escape from spam). In case this isn’t enough, we can have a “clean-up” service dispatch their lawyers to have the offending content removed. Either the content will be taken down or the offending service will have to lay out a great deal of money to pay for legal costs.
In summary, unless a certain number of conditions are respected, anonymity and neutral or negative ratings produce pernicious effects.
III. The conditions necessary for a successful system of evaluation by scoring
Analyzing eBay’s system allows us to define the conditions necessary for a successful system of evaluation by scoring. Neutral or negative ratings and comments are socially acceptable provided that:
-> the evaluation focuses on objective, factual, observable and quantifiable elements. This is the case for commercial transactions.
-> the evaluation takes place between people who share an acquaintanceship level between 1 and 3.
-> the numerical score is accompanied by a qualitative comment which provides a broader context for the score. “If I’ve made a mistake, I’d like to know what it is so that I may improve”.
-> the stakes of the evaluation be low (evaluation of a person’s one-time action) and not high (general, long-term evaluation of the person as a whole). Therefore, the evaluation must focus on a person’s DOINGS and not on their BEING. “Mr Smith MADE a mistake” is a temporary state of affairs which affords him the opportunity to improve, to do better next time. “Mr Johnson is lousy” is categorical and gives him the STATUS of being lousy.
-> I have more to lose than to gain by leaving the service if I receive a negative rating, either because the service provides other benefits (e.g. eBay is the leader in its market segment) or because there are no alternatives.
-> I can verify the identity of the evaluator (not necessarily his digital id card, but at least his pseudonym’s bona fides). If you’re a seller, you accept the negative rating from a buyer to whom you made a late delivery because you recognize him. But never would you accept a neutral or negative rating from somebody whom you weren’t even sure you’d done business with. This would come across as a clear injustice. We all have a touch of paranoïa in us so we don’t need the Internet to feed it! The reasoning behind anonymous scoring is unimpeachable (freeing speech) and yet it makes no sense in terms of fairness and social acceptance.
In conclusion, eBay is a great success mainly due to its reputation evaluation system which was designed for and implemented within a specific context. A few brave entrepreneurs have attempted to outdo eBay. In the attempt, they’ve taken eBay’s methodology out of its ecosystem, namely: only evaluating actions, evaluating each action and evaluating simple actions (buying or selling).
eBay’s competitors should therefore refocus their evaluations on the actions of people and not on the people themselves. The evaluator should identify him- or her- self and be made to address specific elements of the action under review, as is the case with eBay.
eBay, however, is not THE universal solution for managing reputation. eBay’s clones are even less likely contenders for that title. Here are the main reasons why:
With the advent of Web 2.0, software has started to integrate social components and, as such, it can no longer be content with being a combination of marketing and programming, with a dash of AJAX thrown in. It opens the way for the invention of new social relationships. I hope that this post will have shown that what is technically feasible or financially profitable is not necessarily socially acceptable or useful.
Posted in Definitions - Concepts, Evaluation Systems | 1 Comment »