I think that instead of these rating systems there should be rules in every graduate program that prevents jerk PIs from being the way they are. It's just unacceptable right now. But if all we can do is rate the PIs and not be able to establish certain laws, I'd say make the best damn rating system available and advertise it everywhere. I bet you'll make the PIs we talk about crap in their pants. And I'd definitely contribute in helping you.

Long story short: by working with other connections I managed to get a reasonable postdoc, which I left after a year to take a job at a top tier school as an administrator/non-tenure track lecturer - taught some courses, helped run a graduate program, etc. That was a lot of fun, but fairly limiting in upward mobility. Did that for 4 years, and now I'm making low 6 figures working as an assistant dean at a top tier school. I'm in my early 30's. How many people in academic science are earning 6 figures in their early 30s? How many are even earning $50K?

Point is: don't give up. Struggle against the system and individuals who you see as destroying your career. Find meaningful work and kick ass at it. Forget the tenure track, there are other ways to have a great career. Including alternative academic careers, which are great IMO.

I'm quite happy with the way things have turned out. I think it would have been a horrible mistake for me to go on the TT. I stop work at 5pm and don't care about it again until 9am the next day.

Oh, and another thing: academia is a small world. Science, in general, is a small world. Try not to piss people off.

But if something like this worked out well and gained momentum, I think the benefits would be quite substantial. Even if less than 10% of students and postdocs participate (I think 10% is quite optimistic, actually). The only way to know what kind of a professor you are signing up to work for now is to talk to the current students and postdocs. It will always be the thing to do. But for the current lab members, the danger of "sticking your neck out" in a face-to-face conversation with a perspective student is even more serious. I've seen that reluctance to say anything specific time and time again. The perspective student has no other option but to attempt to read between the lines and decipher facial expressions.

Any information is helpful in this kind of situation. If you read or hear something, then when you visit the lab you can ask someone "Is it true that your supervisor did this and that?" That way you'll know something that the lab members might not have volunteered to tell you otherwise.

The other, less obvious, reason for doing this is to influence the student-advisor dynamics. Right now there is a lot of room for abuse and very little accountability. Tenured professors vs. students/postdocs largely dependent on their supervisors in their future careers — careers they have already invested so much time and effort into? It's just not a healthy situation. But suppose I'm a professor. Imagine also, that there is a website, well-known and accessible to anyone, where anything I do as a supervisor might happen to come up. What's more, that page will be right up there if you google my name. And I won't know who told — could be someone in my lab right now or a former lab member, a PhD student or a postdoc, or a summer student. All of a sudden, I have to think twice before acting like a total prick, don't I?

Not all professors would have stuff written about them. But every professor who thinks of doing anything morally questionable would face the risk of his/her actions being exposed. That's the beauty of a deterrent.