Need your advice for step 2) of ATARNotes.com expansion: Tutor Listing + Review

[enwiabe]

Today, I am announcing the next phase of our expansion, offering a new service to Australian students - a comprehensive, user-generated tutoring listings and reviews. Unfortunately, I'm a bit perplexed as to how to resolve a conundrum I'm having in implementation! However, if there's one thing I've learnt in my time running this community, there are many highly intelligent people on this forum with great ideas. And, so, I'm deferring to you. HELP!

Firstly, a bit about what this is going to be. We're going to archive the 'Wanted' and 'Advertising' boards for tutoring services. David has coded a dedicated section of the site to be a comprehensive tutor listing and review. There are two sections to the system. The first is a section devoted to individual tutors. You sign up to that service yourself with your account. You'll put your rate, the postcode around where you tutor and which subjects you're tutoring, your e-mail, a little bit about yourself and you'll also put in your study scores (if applicable). You'll then scan in a copy of your results and e-mail it to [email protected] to verify your scores (or you can leave them unverified, but you'll be at a disadvantage to those who verify them). You may also scan in a copy of some ID if you were in the high achievers' lists and we'll cross-reference those. You'll then be in the system. Tutors in the listings will be sorted by Respect by default. That is why Respect was so important. You now have a real incentive to get it and I really wanted an uncorrupted system that could not be easily manipulated. So when you hit, say, "Accounting" as a subject to browse, you'll be taken to a page of the top 10 tutors sorted by AN Respect. You'll then of course have more sorting options like Price, Location, Study Score etc.

The second section will be tuition companies. Every single tuition company will be entered, by hand, by us. If we miss any, those companies will e-mail us and we'll put them in immediately. Reviewing the tuition companies (and tutors as well) is where we're coming unstuck.

We could have a 1000 character comment box, along with a rating system out of 5 like for the notes and let you guys have at it. The pros of this is that you can give quite detailed reviews and have free reign over them. Unfortunately, there are cons, 1) that makes all of the information painstaking and difficult to wade through (reading all those different reviews looking for one that helps) and 2) opens up to defamation claims for negative reviews.

The other idea I was spitballing was guided questions. (I currently have 6 questions worked out. They'd be (at present, subject to change):

1) "Rate the fairness of the price of the service from 1 to 10."
2) "Rate the depth of knowledge displayed by the instructor delivering the service from 1 to 10"
3) "Rate the communication skills and manner of the instructor delivering the service from 1 to 10"
4) "Rate the fairness of the amount of personalised attention you received from this service from 1 to 10."
5) "Rate the quality of the resources provided (if applicable) from 1 to 10. Hit N/A if resources were not applicable."
6) "Rate the impact that this service had on your marks from 1 to 10."

We'd then average these 6 questions to get overall score. What I was thinking was displaying the overall score on the search results page and then hovering over the overall score brings up a bar chart of the individual component scores.

And then we might also have a small additional comments box to add any salient points (I'm even hesitant about this. If someone writes 'THEY FUCKIN SUCK' and it goes unnoticed, someone might try to sue and nobody wants a law suit). The pros of this is it leaves us less liable to defamation and it also generates good, easy to access information that will be valuable when you're sifting through all the tutors. You could then also sort by overall rating, or price, or instructor knowledge etc. The problem is, I don't know how good the questions I've written are. I'd love if you guys would help refine them or throw out new questions that would help guide good reviews.

I also want to know what you think about having more emphasis on user-written reviews or the questions and answers. I am given to believe that you can't sue over vote results. No company would even see a trial trying to sue us over the results of a vote. We already have been c+d'ed by mental blank for actual user written reviews. This is why I'm hesitant.

David pretty much has all the backend done, we're waiting on the layout to be done (but that shouldn't take too long). But one thing is making it all come unstuck. That sticking point is how the reviews should work. Please help!

Edit: I've also been spitballing with Sam about having a link under each individual tutor to utute for ease of scheduling and timetabling. Would you guys like that function? Utute is sam.utute's (AN User) startup to help tutors and students schedule their timetables, and can be found at www.utute.com.au

Edit2: Also very important! I should note that like all other AN services, this is free for all users and even the tuition companies! Except (yes there's always a but, but come on we need to make some money somehow), there will be a promoted spot for people who pay to advertise in that spot. They'll get preferential listing in a coloured box entitled 'Sponsored listing'. More on this later.

[enwiabe]

Other grave issues to consider: Verifying the veracity of the tutoring reviews. As in, did the reviewer actually attend that service?

Should we maybe make a 20 post limit to giving a review? Otherwise other tuition services might make sockpuppet accounts to try and vote down other services etc.

[Russ]

Other grave issues to consider: Verifying the veracity of the tutoring reviews. As in, did the reviewer actually attend that service?

I think this is the biggest one, I see all the "endorsed" posts in advertising and just wonder why people who have never actually had a session are posting it. Perhaps the tutor needs to maintain a list in his/her post of students that he/she has actually tutored? Or something similar to automate it (maybe the tutor can submit the list for the software to use). Postcounts...I guess you could base it on whether the votes come from a regular visitor to the site or a single, unique visit (multiple times).

WRT to the questions vs. comments section, I reckon that having a list of suitable questions with median/individual scores etc. is the best way to go. Have a message saying that if a student feels particularly strongly they can submit a review (either positive/negative) via email and you'll consider it for publication. That would cut down on the volume of reviews you'd need to moderate.

1) "Rate the fairness of the price of the service from 1 to 10."
2) "Rate the depth of knowledge displayed by the instructor delivering the service from 1 to 10"
3) "Rate the communication skills and manner of the instructor delivering the service from 1 to 10"
4) "Rate the fairness of the amount of personalised attention you received from this service from 1 to 10."
5) "Rate the quality of the resources provided (if applicable) from 1 to 10. Hit N/A if resources were not applicable."
6) "Rate the impact that this service had on your marks from 1 to 10."

I don't think 4 is really that relevant, it's sort of folded into 2/3/5. Unless you mean for group sessions, in which case you could create group/solo tags for adverts.
Question 6 is also kinda hard to judge without waiting for at least a month or so. Nobody goes to a tutor and expects to get 100% on a test the next day, so maybe if that's a follow up question that can only be answered [time period] later?

edit, on the sponsored links thing, does AN run a donations box? I'm sure people would chip in if you have a page with a graph of monthly costs.

[taiga]

Would there be any way to incorporate this "rating" or verification system into the current threads which exist. I don't think I can quite put my finger on it, but there is something about the whole thread aspect (being able to bump it, non student endorsements, clarifying any questions openly) which I think makes our current system better than a basic tutor finding website (eg. Tutor Finder/Monash Careers Portal etc.)

It also filters out who is still active and still tutoring (many tutors on VN with threads do not return, and stop tutoring overall). My concern lies in the fact that a lot of other websites which offer a similar service (itute, utute, etc.) end up with hundreds of prospective tutors being left in the dark.

Another thing is that several tutors (including myself), have the bulk of their students not actually being from VN. I've got only ~3 of the 10 or so students I have/had actually on VCE notes, and these students don't even use the website regularly. Whilst I could easily get those 3 students to come back, log in and post a review, the bulk of my actual students aren't on this website. Not having those endorsements right now is annoying, but if I don't have those aaaand I don't have endorsements from others on a non-professional basis (which I think are still somewhat valid) I don't have much
Unfortunately I think this is a case with the bulk of ATARNotes tutors (aside from Shinny), in the respect that they only really get 2-3 students from here, so the rating system being the only basis for judgement can be a real detriment. The unfortunate thing is there are few people on this forum that are tutored by other VN members, and I don't see us being easily able to expand to a tutoring service website (similarly to how we could not easily expand to a national forum).

That said, I think this may be an improvement for commercial services, as there are a greater amount of students wanting to know about company run lectures, there aren't currently many companies actively advocating their product here, and it'll also reduce the amount of threads to do with them as well.

Verifying is a massive problem and I can't think of any easy solutions atm, but I will post.


tl;dr:
excellent idea, may be of detriment to us small tutors, good feature for big companies, verification is problem.

[jane1234]

The only problem with ordering tutors by Respect is often it's based on how long you have been around, or your post count. So obviously the people who have been here since 2007 will have 10x the Respect of users who are newer but still have valuable tuition services.

Verification of individual tutors is a good idea though, maybe you should make this compulsory? It would cut down on dodgey people, and also stop an overload of random tutors who can't be bothered putting the effort in to scan results (which would help with clutter). Compulsory verification also puts people's minds at ease, because this is an internet forum after all and people are going to be skeptical about tutors they read about over the internet.

For reviews, I think a combination of questions and actual reviews would be helpful. Have the 6 (or more) ratings but then maybe a 200-word box (to make it easy for people to read) to add extra comments. Otherwise the reviews aren't going to be very personal.
Or maybe you should just have a box for "pros" and a box for "cons" like I've seen on other sites.

[Respect]

The only problem with ordering tutors by Respect is often it's based on how long you have been around, or your post count. So obviously the people who have been here since 2007 will have 10x the Respect of users who are newer but still have valuable tuition services.

Verification of individual tutors is a good idea though, maybe you should make this compulsory? It would cut down on dodgey people, and also stop an overload of random tutors who can't be bothered putting the effort in to scan results (which would help with clutter). Compulsory verification also puts people's minds at ease, because this is an internet forum after all and people are going to be skeptical about tutors they read about over the internet.

For reviews, I think a combination of questions and actual reviews would be helpful. Have the 6 (or more) ratings but then maybe a 200-word box (to make it easy for people to read) to add extra comments. Otherwise the reviews aren't going to be very personal.
Or maybe you should just have a box for "pros" and a box for "cons" like I've seen on other sites.

it discriminates against me :/

[enwiabe]

Would there be any way to incorporate this "rating" or verification system into the current threads which exist. I don't think I can quite put my finger on it, but there is something about the whole thread aspect (being able to bump it, non student endorsements, clarifying any questions openly) which I think makes our current system better than a basic tutor finding website (eg. Tutor Finder/Monash Careers Portal etc.)

It also filters out who is still active and still tutoring (many tutors on VN with threads do not return, and stop tutoring overall). My concern lies in the fact that a lot of other websites which offer a similar service (itute, utute, etc.) end up with hundreds of prospective tutors being left in the dark.

Another thing is that several tutors (including myself), have the bulk of their students not actually being from VN. I've got only ~3 of the 10 or so students I have/had actually on VCE notes, and these students don't even use the website regularly. Whilst I could easily get those 3 students to come back, log in and post a review, the bulk of my actual students aren't on this website. Not having those endorsements right now is annoying, but if I don't have those aaaand I don't have endorsements from others on a non-professional basis (which I think are still somewhat valid) I don't have much
Unfortunately I think this is a case with the bulk of ATARNotes tutors (aside from Shinny), in the respect that they only really get 2-3 students from here, so the rating system being the only basis for judgement can be a real detriment. The unfortunate thing is there are few people on this forum that are tutored by other VN members, and I don't see us being easily able to expand to a tutoring service website (similarly to how we could not easily expand to a national forum).

That said, I think this may be an improvement for commercial services, as there are a greater amount of students wanting to know about company run lectures, there aren't currently many companies actively advocating their product here, and it'll also reduce the amount of threads to do with them as well.

Verifying is a massive problem and I can't think of any easy solutions atm, but I will post.


tl;dr:
excellent idea, may be of detriment to us small tutors, good feature for big companies, verification is problem.

I'd be lying if I said this was a perfect system. It's not. But I think it's the best we can do in the circumstances. You'll be able to see the usernames of the people reviewing you. If we get suspicious reviews, we will investigate. Beyond that, I think this is the best we can do.

The reviews section will be viewable to everyone. This won't be a locked part of the site. You won't need to register to view the information. You will need to register to view the e-mail and contact info, but other than that, it'll all be searchable. I think there's a blinding market hole for this. There is a complete dearth of information on which services are good or bad. This will fill that hole, and I think it will get significant use. Of course, I could well be wrong I hope not, though!

The only problem with ordering tutors by Respect is often it's based on how long you have been around, or your post count. So obviously the people who have been here since 2007 will have 10x the Respect of users who are newer but still have valuable tuition services.

Verification of individual tutors is a good idea though, maybe you should make this compulsory? It would cut down on dodgey people, and also stop an overload of random tutors who can't be bothered putting the effort in to scan results (which would help with clutter). Compulsory verification also puts people's minds at ease, because this is an internet forum after all and people are going to be skeptical about tutors they read about over the internet.

For reviews, I think a combination of questions and actual reviews would be helpful. Have the 6 (or more) ratings but then maybe a 200-word box (to make it easy for people to read) to add extra comments. Otherwise the reviews aren't going to be very personal.
Or maybe you should just have a box for "pros" and a box for "cons" like I've seen on other sites.

Again, the idea is to reward VN contributors with boosted exposure. But also remember that that's only the default sorting. If you have a valuable service it will be highly rated and will show up prominently when people sort by ratings.

I love love LOVE the idea of pros and cons and I think that's the form 'additional comments' will take. Thanks for all your feedback guys and please keep it coming!

[enwiabe]

Would there be any way to incorporate this "rating" or verification system into the current threads which exist. I don't think I can quite put my finger on it, but there is something about the whole thread aspect (being able to bump it, non student endorsements, clarifying any questions openly) which I think makes our current system better than a basic tutor finding website (eg. Tutor Finder/Monash Careers Portal etc.)

It also filters out who is still active and still tutoring (many tutors on VN with threads do not return, and stop tutoring overall). My concern lies in the fact that a lot of other websites which offer a similar service (itute, utute, etc.) end up with hundreds of prospective tutors being left in the dark.

Another thing is that several tutors (including myself), have the bulk of their students not actually being from VN. I've got only ~3 of the 10 or so students I have/had actually on VCE notes, and these students don't even use the website regularly. Whilst I could easily get those 3 students to come back, log in and post a review, the bulk of my actual students aren't on this website. Not having those endorsements right now is annoying, but if I don't have those aaaand I don't have endorsements from others on a non-professional basis (which I think are still somewhat valid) I don't have much
Unfortunately I think this is a case with the bulk of ATARNotes tutors (aside from Shinny), in the respect that they only really get 2-3 students from here, so the rating system being the only basis for judgement can be a real detriment. The unfortunate thing is there are few people on this forum that are tutored by other VN members, and I don't see us being easily able to expand to a tutoring service website (similarly to how we could not easily expand to a national forum).

That said, I think this may be an improvement for commercial services, as there are a greater amount of students wanting to know about company run lectures, there aren't currently many companies actively advocating their product here, and it'll also reduce the amount of threads to do with them as well.

Verifying is a massive problem and I can't think of any easy solutions atm, but I will post.


tl;dr:
excellent idea, may be of detriment to us small tutors, good feature for big companies, verification is problem.

Again, see my reply to Russ and jane (covers many of your concerns). All I can say is, offer improvements/suggestions and I'll be happy to try to implement them!

[DavidB3ck]

edit, on the sponsored links thing, does AN run a donations box? I'm sure people would chip in if you have a page with a graph of monthly costs.

It's been suggested. Dan said no. Can't recall why.

[kushwoho44]

In principal this is a good idea, but in my honest opinion it will never work. Everybody endorses everybody willy nilly on this forum and it has no credibility. Have read very very many times,"haven't actually met the guy....endorsed."

[enwiabe]

In principal this is a good idea, but in my honest opinion it will never work. Everybody endorses everybody willy nilly on this forum and it has no credibility. Have read very very many times,"haven't actually met the guy....endorsed."

Any obviously faked review will be deleted.

[luken93]

Maybe, rather than people from the forum endorsing others, could the rating on a person's reply to a question be linked somehow to their endorsement.

ie, If someone asks a question and a tutor answers it, and people rate their answer for the help being provided, this somewhat proves their skill?

Just a thought.

[taiga]

Look it's probably worth giving a try, but at the moment there aren't enough VN members who are tutored by other VN members, nor is there a huge demand for private tutors on our website (compared to lectures etc.)

Though respect is a decent way of sorting (I shouldn't be complaining because I would probably be listed around #1), it doesn't really capture our total contribution to the forum. I don't mind endorsements from other VN members (even if they haven't been tutored) because I think it often encapsulates exactly how much a member has contributed to the forum. Respect can often fail in that respect because I can easily get respect by cracking funny posts, whereas when I help someone out with their maths/physics/chem on the forums I will rarely get more than +2 (which doesn't count due to the nature of the system).

In all honesty, I think the current thread system should stay, perhaps with an added "rating" function as opposed to the current poll options.

On the other hand, I think this idea will function well if it is used to rate services which are marketed to moderate/medium size groups of students.

[burbs]

Honestly, is there any reason to change the system at all? Seems to work pretty well.

[DavidB3ck]

Look it's probably worth giving a try, but at the moment there aren't enough VN members who are tutored by other VN members, nor is there a huge demand for private tutors on our website (compared to lectures etc.)

Though respect is a decent way of sorting (I shouldn't be complaining because I would probably be listed around #1), it doesn't really capture our total contribution to the forum. I don't mind endorsements from other VN members (even if they haven't been tutored) because I think it often encapsulates exactly how much a member has contributed to the forum. Respect can often fail in that respect because I can easily get respect by cracking funny posts, whereas when I help someone out with their maths/physics/chem on the forums I will rarely get more than +2 (which doesn't count due to the nature of the system).

In all honesty, I think the current thread system should stay, perhaps with an added "rating" function as opposed to the current poll options.

On the other hand, I think this idea will function well if it is used to rate services which are marketed to moderate/medium size groups of students.

Perhaps disabling respect on certain forums wouldn't be such a bad idea.