- Home
- Judith Rich Harris
No Two Alike Page 15
No Two Alike Read online
Page 15
One of the most troubling concerns a meta-analysis [Sulloway] conducted of a subset of the birth order literature presented in a book by Ernst and Angst (1983). Sulloway does not list which studies he selected for his analysis, an essential element in any presentation of meta-analytic results.52
Ernst and Angst might sound like a good name for a rock group, but in fact they are a team of hardworking Swiss researchers who published a comprehensive review of birth order research in 1983 and concluded that birth order has no important effects on personality. Sulloway disagreed. He reanalyzed the data in Ernst and Angst’s review and claimed to have found results that strongly supported his theory: of 196 findings from properly controlled studies, 72 were in line with his predictions, versus only 14 that came out in the wrong direction. (The remaining 110 did not differ significantly from zero.)53
I was not the only one who questioned this claim. I was not the only one who decided to check it by going through Ernst and Angst’s book to see if I could find 72 positive findings from controlled studies (I couldn’t).54 A sociologist who reviewed the book for the journal Science also tried and failed to figure out where Sulloway got so many positive findings. Another who admitted bafflement was Jules Angst himself. An American science writer who asked him about Sulloway’s meta-analysis reported, “Angst informed me by e-mail that he could ‘neither reconstruct nor understand’ how the data gathered by him and Ernst for their 1983 book had been reanalyzed by Sulloway.”55
But of all those who tried to get to the bottom of the data presented in Born to Rebel, no one worked longer or harder at it than Frederic Townsend. Like me, Townsend is an independent scholar, lacking in academic credentials—he has a law degree but that doesn’t count. Though he gave up being an attorney some years ago (he is currently a bond options trader at the Chicago Board of Trade), he hasn’t lost the motivation that led him to go to law school in the first place: an urge to see truth and justice prevail. I presume that is his motivation in this case. It can’t be a desire for fame or fortune—he will never receive either for the hundreds of hours he has put into his investigation of Sulloway’s methods. Townsend is a married man with young children, but for seven years he dedicated much of his free time to this investigation.
The problem Fred Townsend and Toni Falbo and the Science reviewer and I had with Sulloway’s tables and graphs was that Born to Rebel—though tail-heavy with 245 pages of appendixes, endnotes, and bibliography—does not contain the kind of information Sulloway would have had to provide in order to get his work through the peer-review process of a journal. It’s like the missing N in Suomi’s story of the cross-fostered monkeys. If Sulloway’s reanalysis of Ernst and Angst’s data had been held to the standards of a peer-reviewed journal, he would have had to provide a list of the studies that produced the 196 findings.56
“The principal reason for keeping science secret,” observed the physicist Robert Park in his book Voodoo Science, “is that the science is questionable.” Townsend sent Sulloway a letter requesting a list of the 196 findings. Sulloway turned him down. He was not obligated, he told Townsend, to make the results of his research available to “unqualified individuals.” Sulloway also refused to send Townsend his list of 24 Protestant martyrs who died during the Reformation—of whom, he claimed in Born to Rebel, 23 were laterborns. Townsend was persistent and Sulloway was evidently annoyed. “You have identified yourself,” he told Townsend in a letter, “as a kind of small-minded and ill-informed crank.”57
If Sulloway thought he could discourage Townsend by calling him names, he had misjudged his opponent. Townsend kept plugging away. Since Sulloway wouldn’t send him the list of 196 findings included in his meta-analysis, Townsend attempted to reconstruct the list himself. He used two different methods, based on what Sulloway had said at various times. Neither produced results close to those reported in Born to Rebel. And Townsend doesn’t ask us to take his word for it: he has published lists of the studies he included in his analyses, and their positive, negative, or null findings.58
Townsend discovered many other anomalies and inconsistencies in Sulloway’s work. For example, Born to Rebel contains a table (table 2) showing support for various scientific revolutions as a function of birth order. Later in the book, the data in this table are replotted as a graph (figure 14.1). The graph shows a linear relationship between firstborn/laterborn support and “trend for social attitudes.” Townsend found that the points in the graph don’t always agree with the numbers in the table. Out of seven data points that are different, six had moved in the “correct” direction—closer to the diagonal line drawn through the graph.59
Each of Townsend’s examples sounds trivial in itself—“nitpicking,” as Sulloway later complained to a journalist. But taken together they are devastating. A few errors or discrepancies could be put down to carelessness, but Townsend found too many of them and a very high percentage were in the direction favorable to Sulloway’s theory.60
In 1998, Townsend turned the results of his investigations into a sharply worded critique of Born to Rebel and submitted it to the journal Politics and the Life Sciences. The editor, Gary Johnson, chair of the political science department at Lake Superior State University, sent it out for peer review. Three of the four reviewers recommended publication and Townsend’s article was provisionally accepted, contingent on his making appropriate revisions in response to the reviewers’ comments.61
Johnson didn’t plan on publishing Townsend’s critique by itself. He intended to make it part of a “roundtable” issue of the journal. Sulloway was invited to contribute a reply, and both he and Townsend were asked to nominate other people who would provide commentaries on Townsend’s paper. I was one of Townsend’s nominees and I accepted the invitation from Johnson to contribute a short commentary.62
Townsend revised his manuscript, it was accepted, and copies were sent to Sulloway and to the commentators in December 1998. But Townsend’s critique, Sulloway’s reply, and the ten commentaries weren’t published until February 2004, in an issue of Politics and the Life Sciences labeled September 2000. That issue of the journal also contains a thirty-five-page editorial by Gary Johnson, explaining the long delay in publication. What I will tell you here is based on Johnson’s account.
In January 1999, Johnson reported, he received a letter from Sulloway. In it, Sulloway claimed that Townsend’s manuscript was “defamatory in its present form” and demanded that it be revised. “It would be unfortunate,” he told Johnson, “if your decision to publish Townsend’s manuscript in its present form plunges you and your journal into serious legal difficulties.” Just in case Johnson missed the point, Sulloway added this explanation: “Under the law, you cannot knowingly permit the publication of a pattern of defamatory claims without becoming a party to defamation yourself.”63
Sulloway’s demands went beyond a request that Townsend’s article be expunged of defamatory material. He also said it would be “legally mandatory,” if Johnson published a revised version of Townsend’s article, to preface it with an editorial disclaimer. This disclaimer (for which Sulloway provided some sample wording) would say that the editor had decided to publish Townsend’s paper in spite of its flaws, and would warn readers that Townsend’s claims are not to be trusted because his evidence is “erroneous” and his paper contains “blatant errors of fact and interpretation.”64
Again, Sulloway had underestimated his man. Johnson was not about to let Sulloway make his editorial decisions for him. On the other hand, nobody wants to be sued. Before he could publish Townsend’s critique, Johnson decided, Townsend would have to revise it carefully, to make sure that it contained no defamatory remarks and that every factual statement was well documented. For his own good as well as Townsend’s, Johnson explained, he wanted Townsend’s critique to be as lawsuit-proof as possible.65
Townsend’s re-revised manuscript was again sent out for peer review and again accepted. In July 2000, the final version was sent out to the commentato
rs and to Sulloway.
“It was Sulloway, of course, who had demanded revisions in Townsend’s manuscript,” Johnson noted in his editorial. “However, the revisions Townsend made must not have been the revisions Sulloway had in mind.” In fact, the final manuscript, though less sharply worded than the earlier version, was stronger and more convincing. Sulloway reacted as angrily to this manuscript as he had to the previous one, and again his attack was aimed, not at its author, but at the editor of the journal. “After receiving his copy of the revised Townsend manuscript,” Johnson reported, “Sulloway made a variety of accusations against me.” Sulloway’s primary accusation was that Johnson had behaved improperly by allowing Townsend to make extensive changes to his manuscript after an earlier version had been sent out to the commentators. Johnson acknowledged that this was a departure from normal procedure, but pointed out that these were not normal circumstances. “Under normal circumstances,” he explained, “participants in the peer commentary process do not threaten to bring suit if revisions are not made in the roundtable article.”66
When Sulloway learned that Johnson was planning to go ahead with publication, he escalated his attack. He sent a letter to the president of Johnson’s university, with copies to other prominent officials, saying that he was planning to file “formal charges of scientific misconduct against Gary Johnson with the American Political Science Association, the Human Behavior and Evolution Society, members of Congress who have shown a concern about science fraud,” and so on.67 (Sulloway did not follow through on these threats. In May 2004, he explained to a journalist that it “wasn’t worth the trouble.”68)
Johnson remained steadfast in his determination to publish, but the fight was long, wearisome, and costly. When the publisher of the journal got wind of the threat of litigation, he refused to print the issue in question unless both Sulloway and Townsend would agree in writing not to sue him, the printer, or the journal. (They would still be free to sue each other.) Townsend signed the release but Sulloway refused, so Johnson had to find another way to get the issue printed. He and his assistants ended up doing most of the work themselves.69 That was one reason why the September 2000 issue of Politics and the Life Sciences didn’t appear in print until February 2004.
The other reason was Johnson’s decision, made late in the day, not to remain on the sidelines but to conduct his own investigation into Sulloway’s research methods. As he explained in his editorial, he had started out with an “favorable predisposition toward Born to Rebel” he approved of Sulloway’s Darwinian approach. But when Sulloway finally submitted his reply to Townsend’s revised critique, Johnson found it disappointing. Sulloway “oversimplifies some of the issues Townsend raises,” Johnson said, “and he sidesteps or ignores others. In general, he seems to have used a ‘dismissive onslaught’ strategy…. The response to Townsend is dismissive and patronizing.”70
The specific thing that caused Johnson to enter the fray was Sulloway’s response to one small piece of Townsend’s evidence. Townsend had obtained a copy of Sulloway’s 1991 National Science Foundation grant proposal through the Freedom of Information Act. Both the proposal and Born to Rebel contain tables showing laterborn support for various scientific revolutions, but Townsend found that the numbers didn’t agree: the ratios of laterborn support were considerably higher in Born to Rebel. In his reply to Townsend, Sulloway accounted for the discrepancies as follows: “It turns out that Townsend mistook relative risk ratios, which I provided in my 1991 grant proposal, for odds ratios, which I provided in my 1995 book.” In a footnote he described Townsend’s error as one of those “problems that can arise when unpublished documents and data are acquired by a nonprofessional who should have done the minimal amount of checking.”71
Johnson, who is a professional, decided to do the minimal amount of checking. He examined Sulloway’s NSF grant proposal. The table in the grant proposal, Johnson discovered, explicitly said that the numbers were odds ratios; the term “risk ratios” was used nowhere in the document. Having made this discovery, Johnson felt that he had to say something. If he hadn’t—if he had allowed Sulloway to get the last word—readers would have come away believing that Townsend had made a stupid mistake, confusing odds ratios with risk ratios.72
In for a penny, in for a pound. Once aroused, Johnson was as tenacious as Townsend. He decided to look more closely at Born to Rebel. He soon discovered that attempting to find out how Sulloway had obtained the data in his graphs and tables was like being “involved in a shell game.”73
One of the things Johnson looked at was Sulloway’s claim about the religious martyrs killed during the Reformation. Sulloway classified the Protestant martyrs as rebels—which means, according to his theory, that they should be laterborns—and the Catholic martyrs as conservatives. He reported in Born to Rebel that “23 out of 24” Protestant martyrs had been laterborns (a list of their names was one of the things he had refused to send Townsend), whereas “most” of the Catholic martyrs (no numbers given) had been firstborns.74
Sulloway was talking about events that happened in the fifteenth century. In endnotes he listed some of the sources he used for obtaining information on the religious beliefs of these martyrs. But nowhere, Johnson discovered, did Sulloway say where he had found information on their birth order. And that was the tricky part, because good records weren’t kept in those days. Dates of birth were often approximate; family arrangements were complicated by deaths, remarriages, step-siblings, adoptions, and illegitimate offspring. After a good deal of historical research, Johnson succeeded in obtaining birth order information on 28 Protestant martyrs and 22 Catholic martyrs.75
Both groups contained a preponderance of laterborns, because these people came from families with an average of seven children, of which only one was a firstborn. But Johnson found no difference between the (supposedly rebellious) Protestants and the (supposedly conservative) Catholics: 3 of the 28 Protestants and 2 of the 22 Catholics were firstborns. “It appears that there was no empirical basis,” Johnson concluded, for Sulloway’s statement that most Catholic martyrs were firstborns.76
According to a recent news article in the British journal Nature, Gary Johnson has submitted a formal request to the National Science Foundation, asking that an independent inquiry be made into the research methods used by Frank Sulloway in his book Born to Rebel. Johnson’s request is based on the fact that Sulloway received a grant from the NSF to do the research for the book. “The evasiveness, the varying methodological accounts, the data discrepancies and the seemingly desperate attempts to interfere with publication together suggest that an independent review of Sulloway’s research should be undertaken,” Johnson explained to the journalist from Nature. Sulloway, for his part, accused Johnson of “unprofessional editorial tactics” and denied his allegations. As for the inquiry, “The only concern to me is the amount of time it takes to make all the data available,” Sulloway told the journalist. “I’m just swamped with work.”77 An example of the kind of data that Sulloway would presumably be asked to provide is a list of the Protestant and Catholic martyrs he referred to in his book.
Other journalists have questioned Sulloway about the disputed issue of Politics and the Life Sciences. Here’s an account written by a newspaper reporter in Berkeley, California, who interviewed Sulloway in person not long after the issue was published and recorded his reaction to it:
Sulloway’s intense focus becomes clear within minutes of meeting him. He is angry about what has happened and harbors a deep disdain for his critics, which he doesn’t attempt to hide. They are unsophisticated detail freaks, he says, who are unable to grasp, replicate, or even understand the way he put his book together. He speaks of his detractors as if they were third-tier nuisances with insubstantial and irritating complaints, and says he is not alone in this thinking. He produces an e-mail from a colleague who calls Sulloway’s critics a “confederacy of dunces.”…This is a theme to which Sulloway and his supporters are sticking—they make mu
ch ado of credentials or lack thereof, noting that this is a case of non-pedigreed folks going after one of academia’s brightest lights. “It’s like being Tom Cruise or somebody else and people start going after you the way they go after stars in the tabloids,” Sulloway says.78
I won’t comment on Sulloway’s similarity to Tom Cruise, only on his logic. To say that A is right and B is wrong because A has better credentials is an error in reasoning. The truth of a statement doesn’t rest on who said it. As Johnson declared in a letter to Sulloway, “Unless science is to be a credentialed aristocracy, we must surely be open to participation by the uncredentialed and the unorthodox. The history of science seems to suggest that the work of amateurs deserves, if anything, special protection.”79
A curious thing about this story is the fact that Sulloway is a laterborn. According to his theory, it’s supposed to be firstborns who are dominant and aggressive and who “employ their superior size and strength to defend their special status.”80
Birth order effects are real; they affect behavior within the family. Siblings compete with each other for parental attention. Older ones are nurturant toward younger ones and boss them around. Parents treat firstborns differently from laterborns, and older children differently from younger ones, and the children are keenly aware of it. All these things go on within families and, for children who have siblings, are a deeply felt part of their early experiences at home. People remember them all their lives.
If these things don’t have measurable effects on personality—if their effects can’t be detected in the way kids behave on the playground or in the responses adults check off on personality tests—is it plausible to put the blame on within-the-family differences of the upper-versus-lower-bunk-bed sort? If the substantial, persistent differences between the family experiences of a firstborn and a laterborn don’t leave visible marks on their personalities, is it likely that minor differences—due, say, to family members’ reactions to the random differences between identical twins or to the fact that one of them made a mess of the mousse—are going to do the job? If personality isn’t shaped by big things that happen to children at home, is it likely to be shaped by little things that happen to them at home?