Document Type : Original Article
Authors
Imam Khomeini International University, Qazvin, Iran
Abstract
Keywords
1. Introduction
Because many researchers (e.g., Oller, 1979, 1983) regard cloze test as one of the best integrative methods, many others (e.g., Eckes & Grotjahan, 2006) claim that C-tests can provide an integrative assessment of general language proficiency. Some researchers have criticized cloze tests (Khodadady & Hashemi, 2011; Klein-Braley, 1983) and have attempted to modify its format in some ways (Read, 2000). Consequently, C-test was constructed as an alternative integrated approach which, according to Alderson (2000), is the deletion of "every second half of every second word and has to be restored by the reader" (p. 225).
In addition to language ability, Bachman (1990) emphasized cognitive characteristics as one of the important sources of bias in language tests. Therefore, the effect of test takers' characteristics needs to be identified, controlled, and minimized because it may affect the test takers' scores (Bachman, 1990). Although several studies have been conducted on the relationship between C-test and some cognitive characteristics like field-dependency (Heidari, 2012), tolerance of ambiguity (Babaii & Shahri, 2010; Zarei, 2012), willingness to communicate (Baghaei & Dourakhshan, 2012), and problem-solving tasks (Ghodrati, Rjaei, & Ebrahimpour, 2014) among Iranian EFL learners, studies on C-test have given us few accounts of the relationship between test takers' personality traits such as Extroversion/Introversion, Sensing/iNtuition, Thinking/Feeling, and Judging/Perceiving and their performance on C-test. This study attempts to bridge this gap by exploring the possible relationship between personality traits and performance on C-test. Additionally, the study aims at investigating which type of personality plays the most influential role contributing to language learners' better performance on C-test. To achieve these two goals, we formulated the following research questions to focus this study:
2. Literature Review
2.1 Personality Traits
Trait theory in psychology plays a pivotal role in personality (Quenk, 2009), which has a great deal to do with environments (Ryckman, 2007). According to Ashton (2013) and Harris (2014), in the theory of personality traits, people can be categorized into one of the following classes: Extraversion/Introversion, Sensing/Intuition, Judging/Perceiving, and Thinking/Feeling. However, DiTiberio (1996) noted that each personality trait has its own strengths and weaknesses in education and learning.
Over the last two decades, the importance of personality traits in learning process has been demonstrated in several studies. Indeed, personality variables have mediating and moderating roles in learning process (Sadeghi, 2012). More recent evidence (e.g., Ehrman, 2008; Leaver, Ehrman, & Shekhtman, 2005; Sharp, 2008) indicates that personality variables can effectively influence language learning process. Meeusen, Brown-Mahoney, Dam, Zundert, and Knape (2010) claimed that depending on their personality traits, "people create their own environment, and influence their job satisfaction through cognitive, affective, and behavioral processes" (p. 573). Ehrman and Oxford (1990) posited that personality types are one of the dimensions which are most definitely associated with second language (L2) learning. Bachman (1990) convincingly argued test takers' characteristics could be regarded as one of the critical factors that influence test performance. Therefore, the effect of various test takers' characteristics on test performance need to be considered in the interpretation of test scores and the design of language tests (Phakiti, 2006, 2008).
Although some researchers have failed to find any direct relationship between language proficiency and personality traits of learners (e.g., Carrell & Anderson, 1994; Ehrman & Oxford, 1995; Sharp, 2008), many teachers and educators believe that test takers' performance is not only affected by the ability being measured but also by factors that affect language performance in any assessment situations. Among those factors, test takers' personal characteristics abound (Dewaele, 2007; Howard, 2010; Sharp, 2004; Wakamoto, 2009). In fact, test takers' characteristics determine how language ability is actualized in their language performance (Zhang, Goh, & Kunnan, 2014).
In this regard, Noftle and Robins (2007) showed that there could be a positive effect of personality traits on academic performance. In line with previous findings, Robinson, Gabriel, and Katchan (1994) concluded that language learners' performance in oral and written language assessments might differ, depending on their personality traits. Furthermore, in their study on 247 students of two British universities, Chamorro-Premuzic and Furnham (2003) concluded that personality traits seem to correlate with the students’ scores in examinations and their academic achievement.
2.2 C-Test
Cloze procedures (CPs) have been proposed as integrative alternatives to test reading comprehension (Eckes & Grotjahan, 2006; Fotos, 1991). Because the ability to fill deleted words indicates power of comprehension, it has been recommended for assessment in L2 reading research (Alderson, 2000; Hughes, 2003; Koda, 2005). Global integrative testing (Spolsky, 1976), such as C-test, according to Oller (1979), provides a measure of the candidates' ability to apply language skills in the way they are combined and used for actual language use. McNamara (2000) also points out that these methods of language testing can assess the practical language skills of the foreign language learners in the way his/her "knowledge is used to achieve communication" (p. 14).
Klein-Braley (1981) modified cloze tests and coined a new cloze procedure, C-test, to overcome the problems of standard cloze test. Indeed, the 'C' in 'C-test' was abbreviated from the word 'cloze' to confirm to the relationship between cloze test and C-test (Klein-Braley, 1997). This test method, according to Read (2000), "tends to make a very embedded assessment of vocabulary, to the extent that it is difficult to unearth the distinctive contribution that vocabulary makes to test performance" (p. 115). Therefore, Lee (2008) regards C-test as a proper tool for measuring "previously targeted vocabulary" (p. 644). In C-test, "instead of the complete words, the second half of every second word is deleted" and "examinees get scores for only exact words" (Hughes, 2003, p. 71). Alderson (2000) asserts that deletions start in the second sentence of the text to provide a general perception of the topic and style of the materials. Hastings (1996) argues that applying the rule of two maximizes the representativeness of the text. In C-test, one-letter words are ignored, and "if the word has an odd number of letters, the larger half is deleted" (Babaii & Ansary, 2001, p. 211). This mutation process continues till its number reaches 100 (Khodadady, 2007, p. 3) which takes little space and time to complete (Hughes, 2003, p. 71). Examinees get credit for exact word restorations (Alderson, 2000; Hughes, 2003; Weir, 1990). The lower scores of native speakers on C-tests illustrate the necessity of a careful examination of the text for removing nonfunctioning items if needed (Klein-Braley, 1997). There are studies supporting the efficacy of C-test in measuring the same latent variable that most other test types measure (Eckes & Grotjahn, 2006; Klein-Braley, 1985; Lee & Ellis, 2009). Evidence on high practicality, reliability, and validity of C-test has turned it to a proper tool for reading comprehension (Lee-Ellis, 2009).
In recent years, C-test has been widely used as measurement of general proficiency (Eckes & Grotjahn, 2006) to assess the relationship between language proficiency of Second Language Learners and Foreign Language Learners and their individual differences (Alderson, Clapham, & Steel, 1997), motivation (Dornyei & Kormos, 2000), cognitive styles (Grotjahn, 2010), and personality traits (Ghodrati, Bavarian, & Attaran, 2014 ). Very few studies have also been conducted to examine the relationship between language learners' personality traits and their performance on C-test. The major problem with these studies is they are methodologically flawed: they used very few participants, so the findings are by no means generalizable. The study fills this gap, using a relatively large number of participants.
3. Methodology
3.1 Participants
Initially, 362 Iranian EFL language learners participated in this study, 68 of whom were left out of the study, because they were beginners. Further, another 11 language learners were left out because they did not complete the tests and the MBTI questionnaire properly. Finally, 283 Iranian EFL language learners, ranging in age from 13 to 37, participated in this study. One hundred and sixty-seven language learners were female (59%) and 116 of them were male (41%). They were all native-Persian and Turkish speakers who were currently living in Karaj and studying English for variety of purposes in six most well-known institutes of this city. These language learners were conveniently selected (Dornyei, 2007).
3.2 Instruments
Materials for instrumentation comprised three measuring tools, a newly designed researcher-made C-test, the Michigan Test of English Language Proficiency (MTELP), and the Myers-Briggs Type Indicator, form M (MBTI-M).
A new C-test was developed for the purposes of this study. Thus, certain steps seemed necessary to be followed to construct C-test. The first step was selecting the most suitable topic. Thus, a list of the first 20 popular topics selected from the most recent commercial English conversation textbooks currently taught in English language institutes was submitted to 38 intermediate and advanced English language learners who were selected randomly to survey the topic list and rank them. The results revealed that the first interesting topic that students preferred to read was 'sport'. Two texts enjoying similar characteristics were taken from an online website (wordville.com). The estimated Fog's readability for the texts turned out to be 7.3 and 7.1, respectively which indicates an average difficulty level (Farhady & Keramati, 1994). To construct the C-test, the guidelines given by Klein-Braley's (1997) were considered. The C-test was constructed, with every second half of every other word omitted. One-letter words were ignored. For the words with odd number of letters, the larger half was deleted (Babaii & Ansary, 2001). The newly developed C-test (See Appendix) comprising two short texts, with 23 and 17 mutilations, respectively, was piloted on 31 randomly selected students. The Cronbach's alpha showed that the C-test was reliable (alpha = .868). To establish the validity of the test, the researchers asked six testing experts to comment on the clarity, wording, content, and appropriateness of the test. The finalized C-test was ready to be administered to language learners.
Distinguishing between the intermediate and advanced learners urged the necessity to make use of a standard language proficiency test. Participants' English proficiency was measured by the standardized Michigan Test of English Language Proficiency (MTELP) including 20 grammar, 20 vocabulary, and 10 reading comprehension items. The students who obtained 70% of total scores were classified as advanced language learners, those scoring between 46% and 69% as intermediate, and those whose scores were below 45% as beginners (Phakiti, 2003). Thus, 68 students were excluded from this study since they could answer less than 45% of this proficiency assessment.
The fourth instrument in this investigation was the Myers-Briggs Type Indicator-form M (MBTI-M) questionnaire. This personality type indicator has been one of the most extensively used personality questionnaires in the world (Harris, 2014), including four pairs of dichotomies. These dichotomies are signified by their initial letters. The major difference between this instrument and others is the ability to measure personality types whereas other instruments tend to assess personality traits (Harris, 2014). The form that was utilized in this research was the MBTI-M which involved 93 items. The Persian version of the MBTI-M was administered to language learners (Hosseini, 2003). According to previous studies (see Abedin, Fathabadi, & Ahangi, 2010), reliability indices for the four strands of this inventory in the translated version include Extroversion/Introversion = .82, Sensing/Intuition = .65, Thinking/Feeling = .86 and Judging/Perceiving =.84. Furthermore, the construct validity of the Persian version of the MBTI-M is confirmed in many relevant studies (Marefat, 2006).
3.3 Procedure
The required data for this investigation were collected over a span of two months. The allocated time for the testing session was about 90 minutes. The language learners were presented with the three assessment tools of this study in the following order (with maximum time allowed in parentheses): the MTELP (50 minutes), C-test (20 minutes), and the MBTI-M (20 minutes). Language learners were informed about the goal of the study and ensured that the results would not influence their final scores. They were asked to write their age and gender, but not their names.
3.4 Scoring
The collected data needed to be scored. To this end, it was attempted to follow the original instructions proposed by the developers of the instruments to score the collected data.
The first administered test to be scored was the MTELP which was given to language learners to divide them into two levels of proficiency. Based on the instructions, language learners got one point for answering any of the correctly answered items. Two hundred and eighteen language learners were grouped as intermediate, and the other 76 language learners as advanced. The purpose was to exclude beginners for two reasons. First, C-test is a measure of reading comprehension that beginners are not proficient enough to do well on. Second, the designed C-test for this study was developed based on the texts suitable for intermediate and advanced language learners. Therefore, those who could answer less than 45% of the proficiency test (Phakiti, 2003), were excluded from this study.
The second step was evaluating the language learners' performance on C-test. To score their responses on C-test, we followed the exact guidelines proposed by Klein-Braley (1997). In other words, only the complete and correct spelling of the mutilated words was considered correct. The words with incorrect spellings did not get any point. Besides spelling, only the words which closely corresponded to the original mutilated words were scored. Therefore, language learners got one credit for completing each blank (Alderson, 2000; Hughes, 2003; Weir, 1990). The blanks which were left unanswered did not get any credit either. In conclusion, language learners got one point for every correct and exact word restoration.
For MBTI-M, there were two options (A/B) for each question; thus language learners needed to choose and circle only one of them. Each option of the items indicates that language learners are more likely to be at one of the two extremes of each dichotomy (Harris, 2014). All responses were computed to determine the respondents' personality types.
3.5 Data Analysis
The data was analyzed utilizing the IBMSPSS (version 22). The Cronbach's alpha was employed to determine the reliability of C-test. Then, descriptive statistics such as frequencies and percentages were calculated for analyzing language learners' demographic information such as age and gender. Next, the Pearson correlations were also utilized to determine any possible relationship between the variables of this study. Finally, the standard multiple regression was conducted on the variables to illustrate if any of the personality traits measured by MBTI-M could predict language learners' success in performing on C-test.
4. Results
4.1 Investigation of the First Research Question
Regarding MBTI-M, firstly, a summary of descriptive statistics is given. The eight personality traits of this questionnaire are shown in Table 1. As can be seen, tendencies are towards I, S, T, and P.
Table 1 Frequency of Personality Traits |
|
|
|
|
|
|||
Trait |
E |
I |
N |
S |
T |
F |
J |
P |
Frequency |
75 |
208 |
155 |
128 |
191 |
92 |
139 |
144 |
Note. E = Extravert; I = Introvert; N = iNtuitve; S = Sensing; T = Thinking; F = Feeling; J = Judging; P=Perceiving |
In the study, all sixteen personality types were observed. However, language learners were not equally distributed into these sixteen personality types. The majority of them (17.7%) turned out to be ISTJ type; whereas very few language learners (0.7%) tended to be ESFJ type.
The first research question was aimed at establishing the relationship between the Iranian English language learners' performance on C-test and their personality traits. Pearson correlation was run to answer this research question. The results are presented at Table 2.
Table 2 Correlations between Traits and C-Test
Correlation between Traits and C-test |
||
Trait |
C-Test |
Sig. |
Extravert |
-0.316** |
0.000 |
Introvert |
0.316** |
0.000 |
iNtutive |
-0.026 |
0.660 |
Sensing |
0.026 |
0.660 |
Thinking |
0.168** |
0.005 |
Feeling |
-0.168** |
0.005 |
Judging |
0.109 |
0.066 |
Perceiving |
-0.109 |
0.066 |
Note. **p< .01. |
Only four traits are correlated with language learners' performance on C-test (p < .05): Extravert, Introvert, Feeling, and Thinking. As Table 2 shows, there is a moderate, negative correlation between Extravert language learners and their performance on C-test (r = -.316, n = 283, p = .000, R2 = .09) with a very small effect size. By contrast, there is a moderate, positive correlation between Introvert language learners and their performance on C-test (r = -.316, n = 283, p = .000, R2 = .09) with a very small effect size. There is a weak, negative correlation between Feeling language learners and their performance on C-test (r = -.168, n = 283, p = .005, R2 = .02) with a very small effect size. By contrast, there is a weak, positive correlation between Thinking language learners and their performance on C-test (r = .168, n = 283, p = .005, R2 = .02) with a very small effect size.
4.2 Investigation of the Second Research Question
After ensuring that the data of this study did not violate the assumptions of multiple regressions, a standard multiple regression was run to find the most effective personality traits in predicting the language learners' performance on C-test.
Based on the results shown in Table 3, R square value was .120. This means that our model explains 12% of language performance on C-test.
Table 3 |
||||
Model Summary of Predictors |
||||
Model |
R |
R Square |
Adjusted R Square |
Standard Error of Estimate |
1 |
.347 |
.120 |
.108 |
7.882 |
The results of the ANOVA test (Table 4) show that the predictive power of the model is statistically significant (F (4,278), = 9.505, p<.000).
Table 4 |
|||||
ANOVA Test for Predictors |
|||||
Model |
Sum of Squares |
Df |
Mean Square |
F |
Sig. |
Regression Residual Total |
2361.985 17270.015 19632.000 |
4 278 282 |
590.496 62.122 |
9.505 |
.000 |
In Table 5, the largest beta value belongs to Introversion, indicating that this personality trait makes the strongest contribution to explaining language learners’ performance on C-test. After Introversion, Feeling makes significant contribution to explaining language learners' performance on C-test. The researchers checked the Sig. value for each independent variable. If it was less than 0.05, the variable was considered to make a significant unique contribution to the prediction of the dependent variable. Accordingly, Introversion and Feeling make statistically significant contributions to the prediction of language learners' performance on C-test. It can be concluded that Introversion and Feeling are significant predictors of language learners' performance on both C-test.
Table 5
Coefficient for Predictors
Model |
Unstandardized Coefficient |
Standardized Coefficient |
|||
1 |
B |
Std. Error |
Beta |
t |
Sig. |
(Constant) |
20.599 |
1.628 |
|
12.651 |
.000 |
Total Introvert |
.395 |
.075 |
.298 |
5.257 |
.000 |
Total Sensing |
-.014 |
.047 |
-.018 |
-.307 |
.759 |
Total Feeling |
-.148 |
.068 |
-.132 |
-2.170 |
.031 |
Total Perceiving |
-.040 |
.060 |
-.040 |
-.672 |
.502 |
5. Discussion and Conclusions
The study set out to examine the relationship between language learners' performance on C-test and their personality traits. The study also aimed to identify which of personality traits could best predict language learners' performance on C-test.
The first finding of this study is that performance on C-test is correlated with Introversion, Extraversion, Feeling, and Thinking. Introversion and Thinking were positively correlated with performance on C-test; by contrast, Extroversion and Feeling were negatively correlated with performance on C-test. The findings of the study support that of Veisi (2007). She maintained that Introversion significantly correlates with the C-test performance; Introvert language learners were better performers on C-test in her study. Moreover, the findings of this study echo those of Sanchez-Marin et al. (2001), who found that Introverts are better language learners at performing on C-test.
However, the results of this study contradict those of some others. For example, the findings do not support Pfister's (2000) results which showed that Extravert college students can perform better at overcoming reading comprehension problems and obtain better scores on reading comprehension questions than Introvert ones. He concluded that there is a positive relationship between Extraversion and reading comprehension test scores. The result of the study is also in contrast with Ehrman and Oxford's (1995) findings. They found that there is no significant relationship between language learners' Extraversion and their performance on reading comprehension tasks. Moreover, in Ehrman and Oxford's study, no significant relationship was found between Introversion and language learners' performance on reading comprehension tests.
Thinking was the other personality trait which showed a statistically significant correlation with language learners’ performance on C-test. This finding supports the findings of Ghodrati et al.'s (2014) study. They found that there was a high positive correlation (.838) between Thinking and their scores obtained on C-test. The results from the present study can also confirm their report which revealed that Feeling personality trait weakly and negatively correlates with language learners' performance on C-test. However, the finding of the present study diverges from Pfister's (2000) results. He showed that Feeling significantly correlates with college students' performance on reading comprehension multiple-choice questions. Also, the finding is in contrast with another study done by Ehrman and Oxford (1990). These researchers concluded that Feelers are better at performing on tests measuring reading comprehension
The other finding is that Introversion and Feeling can predict language learners' performance on C-test. However, Feeling, according to Pallant (2011), cannot make a unique contribution due to its low beta value (B = 13%). The finding of this study echoes the study conducted by Veisi (2007). Her investigation showed that Introversion is the most effective contributor to language learners’ performance on C-test; however, she also reported Feeling and iNtuition as other personality traits influencing language learners’ performance on C-test. In another study, Boroujeni, Roohani, and Hasanimanesh (2015) investigated the effect of personality traits on 50 undergraduate Iranian university students' writing performance. In their study, the MTELP and the MBTI-M questionnaire were used to establish language learners' proficiency and personality traits, respectively. The findings of regression analysis revealed that Introversion is the most effective personality trait which can best predict language learners' success in essay writing. The findings of the study also support the previous findings by Leaver et al. (2005) who demonstrated Introversion is the strongest predictor of language learning achievements. However, there are findings of other studies which are in contrast with the results of the present investigation. For example, MacIntyre and Charos (1996) maintained that Extraversion is the best predictor of language learners' learning achievements. These researchers reported Extraversion as the strongest affecting factor on the learners' performance. The result of this study is in contrast with Razmjoo and Shaban's (2008) findings. These researchers reported that Introversion cannot predict language learners' performance on English grammar tests, although language proficiency turned out to be the best predictor of language learners' performance on grammar tests in their study.
The first conclusion of the study is that the correlation between Introversion and Thinking and language learners' performance on C-test is positive. Introvert language learners are not active oral performers especially in EFL contexts. Moreover, in settings like Iran where making mistakes may be regarded as losing face, language learners with this personality trait are more reticent and reserved. Instead, when introverts perform on reading comprehension tests like C-test, they perform better. In Iranian settings, language learners' achievements are currently assessed by a series of typical test methods that are either subjective, like writing an essay, or objective, like multiple-choice questions. However, the nature of C-test necessitates language learners to analyze sentences grammatically as well as considering the main idea and the topic of the text in order to detect the mutilated words and complete them. Thus, thinking language learners who are able to link ideas together by making logical connections (Myers-Briggs & MacCaully, 1985) can perform better. In such situations that completing mutilated words needs high levels of concentration and longer stretches of time, as well as analytical and critical thinking abilities, Introverts and Thinkers are at an advantage.
The second conclusion of this study is that Introversion is the best predictor for language learners' performance on C-test. Despite many EFL settings like Germany, in Iranian settings, stakes-holders and authorities still follow the traditional test methods for measuring language learners' reading comprehension; thus, performing on new assessment tools such as C-test seems overwhelming and strange. When this occurs, the most influencing contributions are personality traits. In such situations, Introverts, who would not be worn out and confused by unfamiliarity, will make efforts and concentrate on the task patiently until it is successfully completed. In other words, Introverts make conscientious decisions and are more devoted to the task. They never sacrifice accuracy for speed which consequently results in better performance. Additionally, Introverts do not usually seek for social attention or others’ compliments; instead, they have intrinsic motivation. As a result, when they participate in a study and perform a task, they do their best; even if they are not going to benefit from the result individually. In fact, Introverts can have an inner strength of trait that enables them to complete the task more accurately than their Extravert counterparts.
The findings may have some pedagogical implications for language teaching and language testing. Language teachers may consider the influence of personality traits on language learners' test performance to make more accurate decisions. Test developers may also benefit from the findings of this study to design more appropriate tests for language learners’ with different personality traits. Therefore, language learners' level of proficiency can be measured more precisely by a test which correlates better with their personality traits. In other words, the effect of test bias on the test scores can be reduced. Hence, C-test seems to be a suitable measure to assess the reading comprehension performance of Introvert and Thinking language learners while utilizing this test for Extravert and Feeling individuals should be done with caution since it may reduce the interpretability of test scores.
This study suffers from some limitations. One of the limitations is that it was focused on only one variant of cloze procedures, C-test. Another limitation of this research was that although the MBTI personality traits of the language learners were briefly reported in this study, an in-depth investigation of other individual differences of the language learners was not the focus of this study. Therefore, further studies should be done to make a relative comparison between language learners’ personality traits in terms of some other variants of cloze procedures like multiple choice cloze tests, summary cloze tests, cloze-elide tests, and so on. Besides, further studies can make use of other forms of MBTI-M or even more recent personality inventories such as dominance, influence, steadiness, and consciences (DISC) (Turnasella, 2002).