Saturday, October 22, 2022

Negativity bias in the media is increasing over time (2000-2019), mostly in right-leaning news outlets

Longitudinal analysis of sentiment and emotion in news media headlines using automated labelling with Transformer language models. David Rozado ,Ruth Hughes,Jamin Halberstadt. PLOS One, October 18, 2022. https://doi.org/10.1371/journal.pone.0276367

Abstract: This work describes a chronological (2000–2019) analysis of sentiment and emotion in 23 million headlines from 47 news media outlets popular in the United States. We use Transformer language models fine-tuned for detection of sentiment (positive, negative) and Ekman’s six basic emotions (anger, disgust, fear, joy, sadness, surprise) plus neutral to automatically label the headlines. Results show an increase of sentiment negativity in headlines across written news media since the year 2000. Headlines from right-leaning news media have been, on average, consistently more negative than headlines from left-leaning outlets over the entire studied time period. The chronological analysis of headlines emotionality shows a growing proportion of headlines denoting anger, fear, disgust and sadness and a decrease in the prevalence of emotionally neutral headlines across the studied outlets over the 2000–2019 interval. The prevalence of headlines denoting anger appears to be higher, on average, in right-leaning news outlets than in left-leaning news media.

The chronological analysis of headlines emotionality shows a growing proportion of headlines denoting anger, fear, disgust and sadness and a decrease in the prevalence of emotionally neutral headlines across the studied outlets over the 2000–2019 interval. The prevalence of headlines denoting anger appears to be higher, on average, in right-leaning news outlets than in left-leaning news media.

Discussion

The results of this work show an increase of sentiment negativity in headlines across news media outlets popular in the United States since at least the year 2000. The sentiment of headlines in right-leaning news outlets has been, on average, more negative than the sentiment of headlines in left-leaning news outlets for the entirety of the 2000–2019 studied time interval. Also, since at least the year 2008, there has been a substantial increase in the prevalence of headlines denoting anger across popular news media outlets. Here as well, right-leaning news media appear, on average, to have used a higher proportion of anger denoting headlines than left-leaning news outlets. The prevalence of headlines denoting fear and sadness has also increased overall during the 2000–2019 interval. Within the same temporal period, the proportion of headlines with neutral emotional valence has markedly decreased across the entire news media ideological spectrum.

The higher prevalence of negativity and anger in right-leaning news media is noteworthy. Perhaps this is due to right-leaning news media simply using more negative language than left-leaning news media to describe the same phenomena. Alternatively, the higher negativity and anger undertones in headlines from right-leaning news media could be driven by differences in topic coverage between both types of outlets. Clarifying the underlying reasons for the different sentiment and emotional undertones of headlines between left-leaning and right-leaning news media could be an avenue for relevant future research.

The structural break in the sentiment polarity and the emotional payload of headlines around 2010 is intriguing, although the short nature of the time series under investigation (just 20 years of observations) makes the reliability uncertain. Due to the methodological limitations of our observational study, we can only speculate about its potential causes.

In the year 2009, social media giants Facebook and Twitter added the like and retweet buttons respectively to their platforms [33]. These features allowed those social media companies to collect information about how to capture users’ attention and maximize engagement through algorithmically determined personalized feeds. Information about which news articles diffused more profusely through social media percolated to news outlets by user-tracking systems such as browser cookies and social media virality metrics. In the early 2010s, media companies also began testing news media headlines across dozens of variations to determine the version that generated the highest click-through ratio [34]. Thus, a perverse incentive might have emerged in which news outlets, judging by the larger reach/popularity of their articles with negative/emotional headlines, started to drift towards increasing usage of negative sentiment/emotions in their headlines.

A limitation of this work is the frequent semantic overloading of the sentiment/emotion annotation task. The negative sentiment category for instance often conflates into the same umbrella notion of negativity text that describes suffering and/or being at the receiving end of mistreatment, as in “the Prime Minister has been a victim of defamation”, with text that denotes negative behavior or character traits, as in “the Prime Minister is selfish”. Thus, it is uncertain whether the increasing prevalence of headlines with negative connotations emphasize victimization, negative behavior/judgment or a mixture of the two.

An additional limitation of this work is the frequent ambiguity of the sentiment/emotion annotation task. The sentiment polarity and particularly the emotional payload of a text instance can be highly subjective and intercoder agreement is generally low, especially for the latter, albeit above chance guessing. For this reason, automated annotations for single instances of text can be noisy and thus unreliable. Yet, as shown in the simulation experiments (see S1 File for details), when aggregating the emotional payload over a large number of headlines, the average signal raises above the noise to provide a robust proxy of overall emotion in large text corpora. Reliable annotations at the individual headline level however would require more overdetermined emotional categories.

The imbalanced nature of the emotion labels also represents a challenge for the classification analysis. For that reason, we used performance metrics that are recommended when handling imbalanced data such as confusion matrices, precision, recall and F-1 scores. Usage of different algorithms such as decision trees are often recommended when working with imbalanced data. However, since Transformer models represent the state-of-the-art for NLP text classification, we circumscribed our analysis to their usage. Other techniques for dealing with imbalanced data such as oversampling the minority class or under sampling the majority class could have also been used. However, our relatively small number of human annotated headlines (1124 for sentiment and 5353 for emotion), constrained our ability to trim the human-annotated data set.

Another limitation of this work is the potential biases of the human raters that annotated the sentiment and emotion of news media headlines. It is conceivable that our sample of human raters, recruited through Mechanical Turk, is not representative of the general US population. For instance, the distribution of socioeconomic status among raters active in Mechanical Turk might not match the distribution of the entire US population. The impact of such potential sample bias on headlines sentiment/emotion estimation is uncertain.

A final limitation of our work is the small number of outlets falling into the centrist political orientation category according to the AllSides Media Bias Chart v1.1. Such small sample size limits the sample representativeness and constraints the external validity of the centrist outlets results reported here.

An important question raised by this work is whether the sentiment and emotionality embedded in news media headlines reflect a wider societal mood or if instead they just reflect the sentiment and emotionality prevalent or pushed by those creating news content. Financial incentives to maximize click-through ratios could be at play in increasing the sentiment polarity and emotional charge of headlines over time. Conceivably, the temptation of shaping the sentiment and emotional undertones of news headlines to advance political agendas could also be playing a role. Deciphering these unknowns is beyond the scope of this article and could be a worthy goal for future research.

To conclude, we hope this work paves the way for further exploration about the potential impact on public consciousness of growing emotionality and sentiment negativity of news media content and whether such trends are conductive to sustain public well-being. Thus, we hope that future research throws light on the potential psychological and social impact of public consumption of news media diets with increasingly negative sentiment and anger/fear/sadness undertones embedded within them.

Children’s self-regulation abilities are key predictors of educational success and other life outcomes such as income and health

Teaching self-regulation. Daniel Schunk, Eva M. Berger, Henning Hermes, Kirsten Winkel & Ernst Fehr. Nature Human Behaviour, October 13 2022. https://www.nature.com/articles/s41562-022-01449-w

Abstract: Children’s self-regulation abilities are key predictors of educational success and other life outcomes such as income and health. However, self-regulation is not a school subject, and knowledge about how to generate lasting improvements in self-regulation and academic achievements with easily scalable, low-cost interventions is still limited. Here we report the results of a randomized controlled field study that integrates a short self-regulation teaching unit based on the concept of mental contrasting with implementation intentions into the school curriculum of first graders. We demonstrate that the treatment increases children’s skills in terms of impulse control and self-regulation while also generating lasting improvements in academic skills such as reading and monitoring careless mistakes. Moreover, it has a substantial effect on children’s long-term school career by increasing the likelihood of enroling in an advanced secondary school track three years later. Thus, self-regulation teaching can be integrated into the regular school curriculum at low cost, is easily scalable, and can substantially improve important abilities and children’s educational career path.


Across experiments, & contrary to our expectations, threat increased liberal—but not conservative—men’s preference for a wide range of aggressive political policies and behaviors (e.g., the death penalty, bombing an enemy country)

DiMuccio, Sarah, and Eric Knowles. 2022. “Something to Prove? Manhood Threats Increase Political Aggression Among Liberal Men.” PsyArXiv. October 21. doi:10.31234/osf.io/qnpw4

Abstract: Manhood is a precarious state that men seek to prove through the performance of masculine behaviors—including, at times, acts of aggression. Although correlational work has demonstrated a link between chronic masculine insecurity and political aggression (i.e., support for policies and candidates that communicate toughness and strength), experimental work on the topic is sparse. Existing studies also provide little insight into which men—liberal or conservative—are most likely to engage in political aggression after threats to their masculinity. The present work thus examines the effects of masculinity threat on liberal and conservative males’ tendency toward political aggression. We exposed liberal and conservative men to various masculinity threats, providing them with feminine feedback about their personality traits (Experiment 1), having them paint their nails (Experiment 2), and leading them to believe that they were physically weak (Experiment 3). Across experiments, and contrary to our initial expectations, threat increased liberal—but not conservative—men’s preference for a wide range of aggressive political policies and behaviors (e.g., the death penalty, bombing an enemy country). Integrative data analysis (IDA) reveals significant heterogeneity in the influence of different threats on liberal men’s political aggression—with the most effective being intimations of physical weakness. A multiverse analysis suggests that these findings are robust across a range of reasonable data-treatment and modeling choices. Possible sources of liberal men’s heightened sensitivity to manhood threat are discussed.

The Role of Ideology

The link between masculinity and conservative political ideology is well-established. In our own work, we have found that chronic masculine insecurity predicts voting for Republican presidential and congressional candidates (DiMuccio & Knowles, 2020). In other research, threats to masculinity increased men’s support for Donald Trump—an effect mediated by the desire for a highly masculine president (Carian & Sobotka, 2018). Other studies have revealed a strong link between masculinity and conservatism including robust cultural associations between “Republican” and “masculine” (Katz, 2016; M. L. McDermott, 2016; Winter, 2010) and a tendency for political conservatives to endorse traditional gender and sex-role beliefs (Feather et al., 1979; Sharrow et al., 2016). Given this link, we were surprised to find that it was liberal—not conservative—men who reacted with increased political aggression to manhood threat. We propose three potential explanations for this unexpected finding.

First, it may be that our dependent measures of political aggression (e.g., support for military intervention and the death penalty) failed to allow sufficient room for movement among conservative participants, who already strongly endorsed such positions. Indeed, we observed a ceiling effect in which 17% of our conservative male participants scored at or near the scale maximum across studies and measures (Terwee et al., 2007). This was not the case for liberal participants, who either opposed aggressive policies less (Experiments 1 and 2) or became supportive them (Experiment 3) after a threat to their manhood. Manhood threat may nonetheless cause conservative men to venture outside the range of socially-sanctioned political aggression (e.g., military intervention) into the realm of violent extremism (as exemplified the 2021 Capitol insurrection). If this is correct, then more extreme measures of political aggression would allow such an effect to emerge. By increasing the extremity of aggressive political options, researchers can allow for effects of masculinity threat to emerge among conservative men, while also shedding light on the recent rise of right-wing extremism in the U.S. (Kapur, 2021).

Second, it may be that liberal men are genuinely more vulnerable to masculinity threat in political contexts. In light of the fact that people stereotype liberals as feminine and conservatives as masculine (Katz, 2016; Rudman et al., 2013; Winter, 2010), it stands to reason that liberal men are especially eager not to exhibit feminine traits in the political realm. In other words, perhaps liberals experience stereotype threat (Spencer et al., 1999) with respect to their masculinity. In our studies, then, liberal men may have reacted to threat with heightened political aggression in order to avoid confirming a (presumably) negative stereotype of their ideological group. Suggesting that this stereotype is, in fact, negative, accusations of femininity constitute a recurring attack line against liberal politicians, presidents, and laypeople—from both the left (Dowd, 2006; Prabhu, 2016) and the right (Fahey, 2007; French, 2015). Conversely, aggression and masculinity are widely regarded as positive political qualities in American politics (Ducat, 2004; Fahey, 2007; Katz, 2016; Messner, 2007), rendering “feminized” liberal men stereotype-incongruent in political contexts (Bauer & Carpinella, 2018). Future research should further examine the possibility that liberal men experience a form of gendered stereotype threat in the realm of politics.

Research has found that liberals become more conservative in their attitudes when exposed to system threats (a phenomenon termed conservative shift; (Bonanno & Jost, 2006; Nail et al., 2009; Nail & McGregor, 2009). This raises the question of whether our findings might reflect a conservative shift among liberals rather than an increase in their political aggression per se. We believe the answer may be found in our findings regarding nonaggressive policies, such as attitudes toward Obamacare, affirmative action, and other social-welfare policies. Such stances have clear (liberal) ideological content. Thus, if masculinity threat were simply causing liberals to become more conservative, we should have observed liberals endorse such policies less under threat. We did not, however, observe any reliable effect of masculinity threat on such ideologically-laden, yet nonaggressive, attitude dimensions. We therefore believe the present findings reflect a “aggressive shift” that is not reducible to a conservative shift.

Limitations and Future Directions

This research has several limitations. First, our experiments did not include manipulation checks. We chose not to include such checks out of concern that doing so would have hinted at the true intentions of the research. This, unfortunately, means that we cannot know whether and to what extent the participants perceived each threat manipulation to challenge their masculinity. While our manipulations had face validity, future research should systematically measure the extent to which each type of manipulation employed in the present research is experienced as a threat to masculinity.

Second, our samples were disproportionately politically left-leaning. It is possible that we would have seen differences by threat across the political spectrum (and not only for left-leaning men) if we had had access to a greater number of highly conservative participants. To investigate this possibility, using the combined dataset, we plotted the estimated effects of masculinity threat on aggressive policy endorsement at every point on the conservatism scale. As can be seen in Figure S1, despite widening of the 95% confidence intervals due to the relatively small number of conservatives in the sample, there is no suggestion of a threat effect emerging at the highest levels of conservatism. Despite the lack of evidence for masculinity threat among conservatives, a definitive test awaits research that includes a large number of extremely conservative men.

Finally, our indicators of political aggression tend to be ones that conservatives endorse. This necessarily makes it difficult (though, as discussed above, not impossible) to tease apart aggressive responses from merely conservative ones. As such, future research should examine the extent to which threats might cause men to employ more aggressive methods to reach whatever their political ends may be. Perhaps, for instance, manhood threats would cause conservative men to embrace more aggressive means of passing gun rights legislation while also causing liberal men to embrace more aggressive means of passing social-welfare legislation. Such a study of political strategy (as opposed to outcomes) is a promising avenue for future investigation.

---

Final version

Something to Prove? Manhood Threats Increase Political Aggression Among Liberal Men. Sarah H. DiMuccio & Eric D. Knowles. Sex Roles, Mar 10 2023. https://link.springer.com/article/10.1007/s11199-023-01349-x

Abstract: Manhood is a precarious state that men seek to prove through the performance of masculine behaviors—including, at times, acts of aggression. Although correlational work has demonstrated a link between chronic masculine insecurity and political aggression (i.e., support for policies and candidates that communicate toughness and strength), experimental work on the topic is sparse. Existing studies also provide little insight into which men—liberal or conservative—are most likely to display increased political aggression after threats to their masculinity. The present work thus examines the effects of masculinity threat on liberal and conservative men’s tendency toward political aggression. We exposed liberal and conservative men to various masculinity threats, providing them with feminine feedback about their personality traits (Experiment 1), having them paint their nails (Experiment 2), and leading them to believe that they were physically weak (Experiment 3). Across experiments, and contrary to our initial expectations, threat increased liberal—but not conservative—men’s preference for a wide range of aggressive political policies and behaviors (e.g., the death penalty, bombing an enemy country). Integrative data analysis (IDA) reveals significant heterogeneity in the influence of different threats on liberal men’s political aggression, the most effective of which was intimations of physical weakness. A multiverse analysis suggests that these findings are robust across a range of reasonable data-treatment and modeling choices. Possible sources of liberal men’s heightened responsiveness to manhood threats are discussed.


General Discussion

The present findings provide support for our hypothesis that threats to men’s (but not women’s) gender status leads to an increase in political aggression, defined as attitudes or behavior that communicate toughness, strength, or force. At the same time, our findings run directly counter to our initial prediction as to which men—liberals or conservatives—would be most affected by masculinity threat. Although we hypothesized that conservative men would increase in political aggression after masculinity threat, they did not; instead, across our three experiments, it was liberal men who exhibited increased political aggression after a manhood threat.

In Experiment 1, we found that a personality-based false feedback manipulation, in which men learned they possessed traits resembling that of the average woman, significantly increased liberal—but not conservative—men’s endorsement of aggressive political policies (e.g., the death penalty). Consistent with our hypotheses, gender threat had no effect on endorsement of nonaggressive policies, or on women’s endorsement of either policy type. In Experiment 2, we found that a behavioral manipulation, in which men engaged in a stereotypically feminine behavior (applying pink nail polish) increased liberal—but not conservative—men’s support for an aggressive approach to a foreign-policy dilemma. We again observed no effect of threat on men’s endorsement of nonaggressive approaches. Finally, in Experiment 3, we found that a strength-based false feedback manipulation, in which men learned their handgrips were only as strong as the average woman’s, caused liberal—but not conservative—men to become more supportive of both aggressive policies and foreign-policy strategies. Once again, manhood threat affected support for aggressive, but not nonaggressive, political policies and strategies.

While liberal men increased significantly in political aggression in all three experiments, and conservative men in none of the experiments, the difference between the threat effect for liberals and conservatives was not always significant. Specifically, ideology did not significantly moderate men’s threat-induced political aggression in Experiment 2, and did so in Experiment 3 only for vignette responses. Nonetheless, our Integrative Data Analysis (IDA; Curran & Hussong, 2009) pooling the evidence across experiments suggests a significant difference in liberals’ and conservatives’ threat responses overall. A multiverse analysis (Steegen et al., 2016) of the combined data indicated that our results are highly robust to different data-treatment and modeling choices. Taken together, then, our results tell a consistent story in which liberal men show a greater tendency to reaffirm their masculinity after manhood threats by embracing more aggressive political views.

Manhood Threats

The present research allows us to compare the relative impact of three different kinds of manhood threats on liberal men. Predictions derived from the IDA revealed that the strength-based threat manipulation in Experiment 3 produced the strongest effect on aggressive policy endorsement. This suggests that intimations of physical weakness represent an especially powerful threat to liberal men’s masculinity—and, more broadly, that the onus on men to display physical strength is highly salient in American society (Frederick et al., 2017). Our personality-based threat manipulation in Experiment 1, in which men were led to believe they possessed stereotypically feminine personalities, yielded the second-strongest effect on political aggression among liberal men. Interestingly, Experiment 2’s behavioral threat manipulation, in which men were compelled to paint their nails pink, produced the weakest threat effect. This may imply that engaging in a feminine behavior at the clear behest of an experimenter does not impugn men’s masculinity as drastically as does information that one’s personal qualities, whether physical (Experiment 3) or psychological (Experiment 1), are truly feminine. This pattern of results suggests that, in the political domain, advertising that impugns men’s physical strength, or suggests that men possess feminine personality traits, may be highly effective in shifting liberal men’s electoral and policy preferences in an aggressive direction. These results are also consistent with previous research showing that public (vs. private) threats tend to be most problematic for men (Weaver et al., 2013). In our research, the experiment that induced threat in a public space (Experiment 3) produced stronger results than threats experienced in private (Experiments 1 and 2). It is worth noting, however, that both public and private threats “worked”— suggesting that, to some extent, participants wish to demonstrate to themselves that they are adequately masculine.

The Role of Ideology

The link between masculinity and conservative political ideology is well-established. Past work has found that chronic masculine insecurity predicts voting for Republican presidential and congressional candidates (DiMuccio & Knowles, 2021). In other research, threats to masculinity increased men’s support for Donald Trump—an effect mediated by the desire for a highly masculine president (Carian & Sobotka, 2018). Other studies have revealed a strong link between masculinity and conservatism including robust cultural associations between “Republican” and “masculine” (Katz, 2016; McDermott, 2016; Winter, 2010) and a tendency for political conservatives to endorse traditional gender and sex-role beliefs (Feather et al., 1979; Sharrow et al., 2016). Given this link, we were surprised to find that it was liberal—not conservative—men who reacted with increased political aggression to manhood threat. We propose four potential explanations for this unexpected finding.

First, it may be that our dependent measures of political aggression (e.g., support for military intervention and the death penalty) failed to allow sufficient room for movement among conservative participants, who already strongly endorsed such positions. Indeed, we observed a ceiling effect in which 17% of our conservative male participants scored at or near the scale maximum across studies and measures (Terwee et al., 2007). This was not the case for liberal participants, who either opposed aggressive policies less (Experiments 1 and 2) or became supportive them (Experiment 3) after a threat to their manhood. Manhood threat may nonetheless cause conservative men to venture outside the range of socially-sanctioned political aggression (e.g., military intervention) into the realm of violent extremism (as exemplified the 2021 Capitol insurrection). If this is correct, then more extreme measures of political aggression would allow such an effect to emerge. By increasing the extremity of aggressive political options, researchers can allow for effects of masculinity threat to emerge among conservative men, while also shedding light on the recent rise of right-wing extremism in the U.S. (Kapur, 2021).

Second, it is possible that, compared to liberals, conservatives are higher in chronic concern for masculinity—and that this blunts the impact of transient threat inductions on their political attitudes. If this is the case, then the effects of masculinity threat may already be “baked in” to conservative men’s political attitudes. Future research should carefully parse out the effects of, and interactions between, trait vs. state levels of masculine insecurity.

Third, it may be that liberal men are genuinely more vulnerable to masculinity threats in political contexts. In light of the fact that people stereotype liberals as feminine and conservatives as masculine (Katz, 2016; Rudman et al., 2013; Winter, 2010), it stands to reason that many liberal men are especially eager not to exhibit feminine traits in the political realm. In other words, perhaps liberals experience stereotype threat (Spencer et al., 1999) with respect to their masculinity. In our studies, then, liberal men may have reacted to threat with heightened political aggression in order to avoid confirming a (presumably) negative stereotype of their ideological group. Suggesting that this stereotype is, in fact, negative, accusations of femininity constitute a recurring attack line against liberal politicians, presidents, and laypeople—from both the left (Dowd, 2006; Prabhu, 2016) and the right (Fahey, 2007; French, 2015). Conversely, aggression and masculinity are widely regarded as positive political qualities in American politics (Ducat, 2004; Fahey, 2007; Katz, 2016; Messner, 2007), rendering “feminized” liberal men stereotype-incongruent in political contexts (Bauer & Carpinella, 2018). Future research should further examine the possibility that liberal men experience a form of gendered stereotype threat in the realm of politics.

Fourth, research has found that liberals become more conservative in their attitudes when exposed to system threats (a phenomenon termed conservative shift; Bonanno & Jost, 2006; Nail et al., 2009; Nail & McGregor, 2009). This raises the question of whether our findings might reflect a conservative shift among liberals rather than an increase in their political aggression per se. We believe the answer may be found in our findings regarding nonaggressive policies, such as attitudes toward Obamacare, affirmative action, and other social-welfare policies. Such stances have clear (liberal) ideological content. Thus, if masculinity threat were simply causing liberals to become more conservative, we should have observed liberals endorse such policies less under threat. We did not, however, observe any reliable effect of masculinity threat on such ideologically laden, yet nonaggressive, attitude dimensions. We therefore believe the present findings reflect a “aggressive shift” that is not reducible to a conservative shift.

We see another possible—though more speculative—explanation for the fact that masculinity threat increased political aggression more among liberals than among conservatives. Specifically, it may be that liberals and conservatives are distinguished by a differential tendency to repair masculinity in public vs. private contexts. If conservatives only see the utility of performing reparative behaviors in public, while liberals engage in such behaviors in public or private, it might explain liberals’ relatively large threat-induced increase in political aggression in the present experiments. Indeed, two of three studies (Experiments 1 and 2) measured political aggression in private, laboratory contexts. Only Experiment 3 was conducted in a public space (a park), and perhaps not coincidentally produced the largest threat effects among conservative men. Although we know of no data or theory that specifically suggests a public–private distinction along ideological lines, little is known about factors that make public vs. private performance of manhood preferable. (We thank one of our reviewers for raising these issues.)

We wish to caution that, while our masculinity threats increased political aggression only among relatively liberal men, threat failed to close the gap between liberals’ and conservatives’ overall levels of political aggression. Indeed, conservatives displayed consistently and drastically stronger support for aggressive policies and vignette responses. This could be due to people’s longstanding ideological affinities, as aggressive political policies tend to be more conservative than nonaggressive policies. Despite these caveats, political liberals’ heightened susceptibility to messages that impugn their masculinity suggests that left-leaning men should be vigilant against attempts to manipulate their politics through such means.

Limitations and Future Directions

This research has several limitations. First, our experiments did not include manipulation checks. We chose not to include such checks out of concern that doing so would have hinted at the true intentions of the research. This, unfortunately, means that we cannot know whether and to what extent the participants perceived each threat manipulation to challenge their masculinity. While our manipulations had face validity, future research should systematically measure the extent to which each type of manipulation employed in the present research is experienced as a threat to masculinity.

Second, our samples were disproportionately politically left-leaning. It is possible that we would have seen differences by threat across the political spectrum (and not only for left-leaning men) if we had had access to a greater number of highly conservative participants. To investigate this possibility, using the combined dataset, we plotted the estimated effects of masculinity threat on aggressive policy endorsement at every point on the conservatism scale. As can be seen in Figure S1, despite widening of the 95% confidence intervals due to the relatively small number of conservatives in the sample, there is no suggestion of a threat effect emerging at the highest levels of conservatism. Despite the lack of evidence for masculinity threat among conservatives, a definitive test awaits research that includes a large number of extremely conservative men.

Finally, our indicators of political aggression tend to be ones that conservatives endorse. This necessarily makes it difficult (though, as discussed above, not impossible) to tease apart aggressive responses from merely conservative ones. As such, future research should examine the extent to which threats might cause men to employ more aggressive methods to reach whatever their political ends may be. Perhaps, for instance, manhood threats would cause conservative men to embrace more aggressive means of passing gun rights legislation while also causing liberal men to embrace more aggressive means of passing social-welfare legislation. Such a study of political strategy (as opposed to outcomes) is a promising avenue for future investigation.