Good challenges all.

First, on personnel reviews, read carefully: I’m not talking about _managers_ doing the evaluation, I’m talking about people on the (non-managerial) technical track doing the evaluations. As a Director at PwC, I certainly knew and often worked with the Senior Managers, Managers, Senior Associates, and even the Associates.

If (for example) the Senior Engineers don’t know and can’t evaluate the Associate Engineers in their same IT group, then you have serious problems with your organization. Beyond that, the Sr Engineers are certainly free to ask for feedback and insight from the Engineers between them and the Associate Engineers. But I still hold to my opinion that there should be a layer of separation there. Ditto on up the tech track.

Second, on the thermocline: in my experience, the barrier most often happens in the transition from engineering to management, because management is responsible for and has to report on the project’s progress. Both as a consultant (on troubled IT projects) and as an expert witness (on IT systems failure lawsuits), I have found that the best way to get to the truth is to go down in the trenches and interview the engineers. The technical career path means that your most competent engineers — which usually means your most honest ones — will be free to go into projects and find out what’s really going on without the temptation — as a manager over the project — to fudge. I’ve filled that exact role as a consultant in several different companies and have often been the bearer of bad news to senior management.

On measuring: I’m not sure that this is what you were saying, but I certainly don’t advocate using utilization as a measurement (unless, of course, you’re in a law or consulting firm and your goal is to ensure an income stream), and it would make no sense in an in-house IT shop anyway. My sense is that the best means of evaluation probably parallels what I suggested for hiring excellent engineers in the first place: have a relevant set of engineers at level X each interview each engineer at level X-2 and then get together to reach a consensus. ..bruce..

How does your proposed technical track interrupt the thermocline of truth? I’m guess I’m not sure I understand who would be managing projects and what the reporting structure would look like.

Also, doesn’t the evaluation scheme you propose (two levels up) fall prey to thermoclines? I know that my second level managers never had any interaction with me and I was completely unaware of whether they were conscious of my skills and strengths.

The real problem with evaluations is what you mention: you’ll get what you measure. If all you measure is utilization that is what you’ll get: lots of hours billed. It seems that it is very difficult to come up with a good set of metrics that can’t be gamed. What would be ideal is a system in which those being evaluated didn’t know the metrics and/or weights applied to them but trusted the system to be fair and line up well with their impression of who the good engineers were.