Speculation regarding inappropriate and unreliable subgroup analyses being performed
The tendency to perform inappropriate subgroup analysis which leads to incorrect conclusions will be alive and well in the medical literature 10 years from now.
There can be an almost irresistible urge for investigators of a medical trial with a negative outcome to subdivide the data in a way to find a positive outcome for some subgroup. Similarly, if the authors have any other preconceived opinions, the data is often subdivided to see if that opinion can be supported in some fashion by subgroup analysis.
It is not that the trial leaders want to come up with something false. Frequently a trial is performed regarding a treatment for which there is a great deal of optimism. The investigators do not want to miss some positive aspect of the treatment. In addition appropriate subgroup analysis can reveal important information.
Unfortunately, excessively subdividing the trial, particularly in ways that were not planned prior to the trial results were known, will often lead to conclusions that are subsequently shown to be incorrect.
The best hope for decreasing the number of inappropriate and erroneous conclusions resulting from excessive subgroup analysis is a rigorous editorial policy by the major medical journals. The journals through their review process, which occurs prior to publication of an article, provide the best opportunity to decrease the frequency of conclusions derived from inappropriate and excessive subgroup analysis.