Gender variation in writing: Analyzing online dating ads

Transcription

1 Patrick Schultz Coyote Papers 21 (2013) UA Linguistics Tucson, AZ, U.S.A. Gender variation in writing: Analyzing online dating ads Patrick Schultz University of Texas at Austin Abstract In the present study, a corpus of more than 18,000 online dating ads (downloaded from Craigslist.com, ~ 1.4 million words) is used to investigate differences in language use between men and women in the online dating context. Few studies have investigated gender differences in written texts, Newman, Groom et al. (2008), Mulac and Lundell (1994) and Koppel, Argamon et al. (2002) being the notable exceptions. These papers, however, differ remarkably in methodology and results. In the dataset studied here, regression analysis reveals marked differences the use of linguistic features such as emoticons or abbreviations. Writer gender and addressee gender emerge as predictors of variation.

2 Schultz, p.2 1 INTRODUCTION 1 Introduction Since Lakoff s (1975) pioneering work, the interaction between language and gender has been studied in quite some detail (for an overview, see Cheshire (2007) or Holmes (2007)). However, most of the research deals with spoken language. Few studies have investigated gender differences in written texts, Newman et al. (2008), Mulac & Lundell (1994) and Koppel et al (2002) being the notable exceptions. These papers, however, differ remarkably in methodology and results. Newman et al (2008) studied gender differences in a corpus of more than 45 million words. The results relevant to this study are the findings that women tend to use more pronouns and verbs, while men commonly use longer words and more articles and numbers. Mulac & Lundell (1994) compared essays written by female and male students; among their findings is the tendency for men to use more numbers while female writers are more likely to use progressive verbs and writer longer sentences. Koppel et al (2002) designed a text classifier that was able to quite reliably group texts from the British National Corpus according to author gender. The most important features their algorithm made use of included noun specifiers (determiners, numbers etc.) as an indicator of male writing and pronouns as an indicator of female writing. A variety of sociolinguistic studies find women to use more standard variants than men (Labov 1990). The research has reached a kind of consensus on certain features: Articles and numbers are generally used more frequently by male writers. Female writing is positively correlated with verbs although there is disagreement about what type of verbs and pronoun frequencies. Results on other variables such as word count or word length remain inconclusive. All the authors quoted above point out that differences between male and female language seem to be more pronounced in the spoken than the written register. Differences in writing, then, can only be studied in a sizeable dataset. In this paper, a corpus of more than 18,000 online dating ads will be used to investigate differences in language use between men and women in the online dating context. These dating ads are not only readily available for download, they are also categorized for gender, represent a rather informal type of writing and offer few incentives for authors to play down gendered language features. In addition to that, the data also allow us to take into account the sexual orientation of the writer and the gender of the addressee.

3 Schultz, p.3 2 METHODOLOGY 2 Methodology Several Python scripts were employed for data download and extraction of features. All statistical analysis was done in R (R Development Core Team 2011). Logistic regression models were built for the binary variables gender and addressee. After running the model with all possible predictors, only those with p < 0.01 were retained. The resulting models were afterwards validated by bootstrapping. Significant predictors of the author s sexual orientation were determined by Principal Component Analysis. The data set had to be simplified to make any kind of graphical representation possible: feature numbers were calculated for the 80 corpus files (one for each category and city) rather than the individual ads. Category Ads Words Female ,084 Male 11, ,123 men4men ,909 men4women ,270 women4women ,859 women4men ,169 Total 18,884 1,430,207 Table 1: Number of ads and words for each category. The data was coded for gender (male, female), addressee of the ad (to women, to men) as well as the sexual orientation of the writer (heterosexual male, gay male, heterosexual woman, gay woman). The following linguistic variables were extracted for each ad:

4 Schultz, p.4 2 METHODOLOGY Feature Ad length Avg. word length Number of long words Number of sentences Avg. sentence length Abbreviations Emoticons Misspellings Part of speech tags Defined as Number of words Number of characters/number of words Words longer than six characters/number of words Number of sentences Number of words/number sentences Number of abbreviations and acronyms/number words. Only abbreviations that occurred more than 10 times were used. Number of emoticons/number of words (list of emoticons compiled from: Wikipedia 2011.) Number of misspelled words/number of words, determined by the Open Office (2011) spellchecker for American English. Part of speech tag/number of words. The data was tagged with the Natural Language Toolkit POS-tagger. Table 2: Linguistic variables 3 Results Initial data exploration suggested that gender might not be the only or even the most important predictor for linguistic differences between writing samples. The plot of average word count per ad below illustrates this point:

5 Schultz, p.5 Figure 1: Mean number of words per ad, by gender, sexual orientation, addressee The first barplot suggests that ad length is about the same for men and women. Plotting the numbers according to sexual orientation shows that this is only due to the fact that heterosexual men write ads longer than any other group while gay men write very short ads. Several other variables show similar distributions. Concentrating on gender as the dependent variable only might fail to reveal some of the linguistic variability in the data set. The questions to be addressed in the following are therefore: What are the defining linguistic characteristics of gender and sexual orientation? Are there significant differences between writing samples addressed to men and samples addressed to women? And, ultimately, is one of these predictors more important than the others? 3.1 Gender The logistic regression model yields the following results for gender. (See appendix for detailed graphic representations):

6 Schultz, p.6 Variable Coefficient S. E. Wald Z P Intercept Ad length Long words Emoticons Abbreviations Verbs Cardinal Numbers Determiners Common Nouns (sg.) Pronouns Frequency of responses: female=7073, male=11811, Model L.R.=1636.2, d.f=.9, p=0, C= Table 3: Logistic regression for author gender (success=male 1 ) Two surprisingly strong predictors emerge: The number of emoticons, which seem to be typically used by women, and the use of numbers, which in this corpus is a feature of masculine writing. 1 i.e. positive values indicate a masculine feature. 2 Unfortunately, it is impossible to change the labels of the variants in this plot. These are the POS tags from the UPenn tagset POSind=possessives, NNPind=proper nouns sg, NNind=common nouns sg, CCind=coordinating conjunctions, CDind=numerals, cardinals, JJind=adjectives, PRPDOLLAR=possessive pronouns, VBZind=verb, 3 rd person present tense, VBINGind=verb, present progressive, INind=preposition, DTind=determiner, VBPind=verb, present tense, not 3 rd person

7 Schultz, p.7 Figure 2: Emoticons and numbers by gender 3.2 Addressee The same method was applied to addressee differences: Are ads directed at women different from ads written to men? To make sure that both categories have the same number of female and male writers, some ads were deleted. Variable Coef S.E. Wald Z P Intercept Ad length Sentence length Misspellings Emoticons Abbreviations Numerals Determiners Common nouns (sg.) Pronouns Frequency of responses: to female=7516, to male=6678. Model L.R.= , d.f.=9, p=0, C=0.681, Table 4: Logistic regression for addressee of ad (success=to male)

8 Schultz, p.8 This model introduces misspellings and abbreviations as strong predictors for female- and male-directed communication respectively. Figure 3: Misspellings and abbreviations per word 3.3 Sexual orientation The third dimension of variation concerns differences between four groups: heterosexual men, heterosexual women and gay men and women. A Principal Component Analysis (PCA) was conducted on the part of speech counts. The PCA combines those factors in various ways to account for as much of the variation as possible without taking any non-linguistic categories into account. The grid created in this way is shown in Figure Unfortunately, it is impossible to change the labels of the variants in this plot. These are the POS tags from the UPenn tagset POSind=possessives, NNPind=proper nouns sg, NNind=common nouns sg, CCind=coordinating conjunctions, CDind=numerals, cardinals, JJind=adjectives, PRPDOLLAR=possessive pronouns, VBZind=verb, 3 rd person present tense, VBINGind=verb, present progressive, INind=preposition, DTind=determiner, VBPind=verb, present tense, not 3 rd person sg, RBind=adverb, comparative. POS tags with less than 100 occurrences were excluded from analysis.

9 Schultz, p.9 Figure 4: Principal Component Analysis If we map the different orientation groups into this chart, they cluster together quite nicely. Figure 5: Principal Component Analysis, sexual orientation

10 Schultz, p.10 We see that the gay men (gm) cluster together; their use of noun phrases, numerals, and conjunctions is above average. The gay women (gw) differ from the rest of the population mainly in their use of possessives. Heterosexual women (hw) cluster high in the verb categories; however, they are quite similar to heterosexual males (hm) in several respects. If we now do the same thing for the gender difference, we get a much less conclusive graph where the men seem to be randomly split into two groups. (The shape of the plot for addressee can easily be inferred from the orientation plot above). Figure 6: Principal Component Analysis, gender This suggests that at least for the PCA analysis, a categorization according to sexual orientation makes the most sense. We must be careful, however, not confuse gender with genre here: especially the big difference between gay males and the other groups might be due to them writing a different kind of ad for example an ad looking for a casual encounter versus an ad looking for a

11 Schultz, p.11 long-term relationship. The counts (percentage of words in brackets) for three terms indicating the kind of relationship sought below suggest something like this: 3 NSA, no strings FWB, friends LTR, long sex attached with benefits term relationship Gay males 209 (0.07%) 37 (0.01%) 31 (0.01%) 119 (0.04%) Gay females (0.02%) 113 (0.03%) 116 (0.03%) 119 (0.03%) Heterosexual males (0.01%) 91 (<0.01%) 360 (0.06%) 283 (0.04%) Hetereosexual females (0.01%) 39 (0.01%) 251 (0.1%) 96 (0.03%) Table 5: Frequencies of relationship indicators 4 Conclusion The findings above suggest that there is no singular key to explaining variation in this dataset. Some features however, emerge as almost singular predictors for certain categories: Gender: Addressee: Sexual orientation: Emoticons, Numbers Misspellings, Abbreviations Nouns, Possessives Some of these findings are consistent with previous research presented in the introduction, such as significantly higher frequencies of pronouns and verbs in female writing. Numbers show up as significant predictors of male writing in this study, too. Just as it did in Mulac s study, sentence length is a predictor for gender, with long sentences indicating a female writer. The sociolinguistic truthism that women use more standard forms than men seems to be reflected in the data as well (cf. 3 I think a better (but more time-consuming) way of doing this would be comparing ads to data from the category that craigslist has for casual encounters (and maybe to the strictly platonic section on the other hand). The counts above are interesting and are probably telling us something, but they are probably a little distorted by the fact that men use a lot more abbreviations overall. Also, it is a simple word count that ignores negation etc.

12 Schultz, p.12 the misspellings variable). Newman s finding about long words being typical of male writing is reversed for this study (but then, both effect sizes are quite small). Contrary to Koppel s findings, determiners are positively correlated with female, not male writers. The results show that the two additional dimensions, sexual orientation and addressee, influence language use to a considerable extent and add to the explanatory power of the model. As shown above, some variables seem to be gender indicators while others are indicative of sexual orientation or gender of addressee. It is also interesting to note that the features that seem to be used very differently by respective groups (emoticons, abbreviations) are specific to the medium of computer-mediated-communication and therefore rather new linguistic phenomena. The data suggest that the groups adapt these new features in different ways. Besides the stronger findings for each category presented above, the smaller effects show a certain pattern, too: several of the smaller effect features for addressee seem to parallel the results for gender. Determiners, for example, are positively correlated with female writers; they are also characteristic of female-directed writing. (this is also true for pronouns). The same pattern is found for male writers and addressees (nouns and numbers). It looks like a kind of linguistic assimilation or style matching to the imagined addressee. References Cheshire, J. (2007). Sex and gender in variationist research. In: Chambers, J. (ed). The Handbook of Language Variation and Change. Malden: Blackwell. Holmes, J (ed.). (2007). The Handbook of Language and Gender. Malden: Blackwell. Koppel, M, Shlomo Argamon & Anat Shimoni. (2002). Automatically categorizing texts by author gender. Literary and Linguistic Computing (17.4). Labov, W. (1990). The intersection of sex and social class in the course of linguistic change. Language Variation and Change (2): Lakoff, R. (1975).Language and the woman s place. New York: Harper.

13 Schultz, p.13 REFERENCES Mulac, A & Torborg Lundell. (1994). Effects of gender-linked language differences in adults written discourse: Multivariate test of language effects. Language and Communication (14.3). Newman, M, et al. (2008). Gender differences in language use: An analysis of 14,000 text samples. Discourse Processes (45). OpenOffice.org. (2011). Spell Checker American English. Retrieved from R Development Core Team. (2011). R: A Language and Environment for Statistical Computing. Wikipedia. List of Emoticons. Retrieved from

14 Schultz, p.14 APPENDIX Appendix Appendix 1: Probability plot logistic regression model for gender

15 Schultz, p.15 APPENDIX Appendix 2: Probability plot logistic regression model for addressee