What is, then, the right assessment instrument(s)? Is there one? Suming up....

Joana Almeida's picture

According to the posts of our discussion group, it seems that choosing the right instrument(s) is a concern of all of us. Which conclusions emerge from our discussions? What is, then, the right instrument? Is there one? Is there a size that fits all?

Some of the answers/ideas that emerged from our discussion:
a) It’s important to precise what we intend to assess, how can the construct be operationalized

b) When adapting an instrument it’s important to match it to our research context and target population needs.
Suggested reading: Harkness, J.A., Van de Vijer, F. R., Moher, P. P. (2004). Cross-Cultural Survey Methods. New Jersey: Willey.

c) A multi-method assessment perspective is crucial (Deardorff, 2009), as well as to triangulate data. Can a single instrument alone work or convey the complexity of the assessment task or even the complex and subjective nature of IC?
Suggested readings:
- Deardorff, D. K. (2009). Implementing intercultural competence assesssment. In Darla K. Deardorff (Ed.), The sage handbook of intercultural copetence (pp. 477-491). Thousand Oaks: Sage.
- Almeida, J., Simões, A. R., & Costa, N. (2012). Bridging the gap between conceptualisation & assessment of Intercultural Competence. Procedia - Social and Behavioral Sciences, 69(0), 695-704. Doi: http://dx.doi.org/10.1016/j.sbspro.2012.11.463

Also, stimulating students’ reflection trough critical incidents emerged as a way to grasp the students experiences and their (inter)cultural learning.
Suggested reading: Williams, T. R. (2009). The reflective model of intercultural competency: a multidimensional, qualitative approach to study abroad assessment. Frontiers, 18(Fall), 289-306.

d) It’s important to bear in mind criteria, such Reliability and Validity, Theoretical basis of the instrument, Biases of instrument (including cultural), Limitations of the instrument and most important understanding exactly what the instrument measures… among other criteria

Other suggested readings in our discussion:
1) Paige’s chapter on instrumentation in Bolen's Guide to Outcomes Assessment in Study Abroad - published by The Forum on Education Abroad
2) Stuart’s chapter on tools in Moodian's book on intercultural competence and leadership - published by Sage
3) Fantini, A. (2006). Exploring and assessing intercultural competence. Brattleboro, VT: Federation of the Experiment in International Living. Retrieved from www.worldlearning.org or www.experiment.org.
4) Fantini, A. E. (2009). Assessing intercultural competence. In Darla K. Deardorff (Ed.), The sage handbook of intercultural competence (pp. 456-476). Thousand Oaks: Sage
5) Paige, M. (2004) Instrumentation in ntercultural training, in, Landis, Dan, Bennett, J.M., and Bennett, M. J. (Eds.) Handbook of intercultural training. Thousand Oaks, CA : Sage Publications, Inc
6) Abbe, A., Gulick, L. M. V, & Herman, J. L. (2007). Cross-cultural competence in Army leaders: A conceptual and empirical foundation. U.S. Army Research Institute for the Behavioral and Social Sciences, Study Report 2008-1. Arlington, VA: ARI.

In my opinion, maybe the quest is not the lack of variety of instruments, but rather making sense of this variety (Almeida, Simões & Costa, 2012) and when choosing an instrument “understand what the [chosen] instrument measures and to be sure that its purpose is compatible with the goals and objectives being assessed” (Fantini, 2009, p. 465).

These are the main conclusions I draw from our discussions. I am looking forward to hearing your opinions and thoughts on this attempt to sum up our discussions...