In my point of view, a top trainer is a top horseman. - someone who can take a crap horse and make it average, take an average horse and make it ok, and take an ok horse and make it 'good'. However, often the real horsemen never get the opportunity of having a 'great' horse, because their focus is on the animal - not the people.
I feel that what often makes a trainer appear to be 'good' is, as TBL alluded to, the 'brand' of the trainer. In order to be a 'brand' they are generally fabulous talkers - basically salesmen, who are able to capture wealthy clients, who invest heavily in the horses. They therefore will often have a huge number of horses in work. In regards to actual 'training', they may sit at the track in the morning spinning yarns, but not actually dealing with the animal. These trainers also have a huge number of horses However, many of those in work never go to the races because they either:
1. break down
2. aren't good enough.
As the trainer is basically running a 'sausage factory' operation, if one horse doesn't stand up, the wealthy owners have another horse waiting in line. As a result, their horses tend to be well bred and their winning stats will automatically be high because they have, essentially, the pick of the crop. It then turns into a cycle of what appears to be success.
However, the horsemen isn't often a salesman, so doesn't get wealthy owners at their door. This isn't to say they are rude or poor communicators (often they are anything but), but they are not a really charismatic personality. Because they don't get the wealthy owners, they often have shares in the majority of horses they do have, and because they don't have the wealthy owners, often have to sell their better horses just to survive. As a result their appearance of 'success' (having really high strike rates, or winning group races) is less. And, like the salesman trainers cycle of success continues, the horsemen's cycle of average horses continues.
If I was looking for a new trainer, I would approach jockeys and ask their opinion about which trainers they think are the best 'horsemen' and go with them - the jocks know who they are.