Debating the future of international scholarships at NAFSA 2015

How can we understand the outcomes of international scholarship programmes? And what shapes the long-term impact of programmes on their alumni and society? These were the central questions posed to a panel of experts at the NAFSA Association of International Educators annual conference in Boston, last month. 

Participating were Dr John Kirkland from the ACU, Professor Joan Dassin of Brandeis University, Martha Loerke of the Open Society Foundations, and Shona Bezanson of The MasterCard Foundation. Between them they represented views from higher education institutions, donors, and programming organisations, with experience spanning several decades of both national government- and charitably-funded scholarship programmes.   

Analysis from the panel focused on three areas:

1. Major trends in international scholarship provision
2. Evaluation strategies for international scholarships
3. How can universities build productive relationships with sponsors and programming organisations? 

As the audience ranged from faculty academics and staff to directors of international education associations, some additional questions were posed to tap into their expertise:

1. What types of programmes and opportunities are provided for international scholarship students that enable them to engage with the broader community and gain direct knowledge and experience of the host country?
2. The issue of return to a student's home country is often paramount for donors, but does 'return' matter from the perspectives of institutions? What types of policies and programmes create an incentive (or disincentive) for international students to return to their home countries? 
3. What types of data do institutions collect about international students?  What is the information used for (e.g. fundraising, storytelling, alumni relations)? 

Highlights 

Summarising the rich discussion in a few lines would be impossible, but below are a few highlights that seem especially relevant to the ongoing work of the ACU and colleagues elsewhere involved in scholarship programmes: 

Aligning aims and analysis 

John Kirkland reminded us that international scholarships broadly fall into four categories: 

1. Institutional scholarships to attract the most able students
2. Scholarships to transform individual lives
3. Scholarship with a focus on externalities (i.e. catalytic impact)
4. Scholarships for public diplomacy 

To understand the outcomes of programmes, evaluation approaches must closely relate to the programme’s objectives, since the variables to be measured and most appropriate strategies of measurement will vary. For example, scholarships primarily aimed at providing access to higher education for under-represented groups may be judged on their capacity to provide that access. Whereas scholarships such as the Fulbright Awards are often awarded to those who may still have entered higher education, and therefore access is not a useful measure of success. 

Supporting long-term studies 

It is vital to invest in long-term studies and not view evaluation as a phase that is ‘tacked on’ to the end of a funding cycle or administrative period. Several colleagues previously and currently involved in tracking the outcomes of the Ford Foundation International Fellowship Program (IFP) highlighted the long-term relationships involved, firstly between IFP administrators and their evaluation partners at the Centre for Higher Education Policy Studies, and now with the Institute of International Education. Simply put, detailed, long-term data takes an extensive time to collect. 

Analysing the whole process 

Martha Loerke reminded us that a scholarship is more than the academic programme a recipient undertakes; it is part of a much longer engagement from pre-scholarship through to alumni, all of which can impact outcomes. The evaluation process is also part of this long-term engagement. Although the phases of selection, study, and post-scholarship may involve very different activities on the part of programming institutions, they can all be considered part of a unified monitoring, analysis, and evaluation process. Designing analytical approaches to understand these components – such as exploring the features of academic programmes that affect propensity to return home – is a vital part of understanding the scholarship experience. 

Understanding transition post-scholarship 

Finally, the panel emphasised the importance of ‘bridging back’: facilitating reintegration into home countries, communities, and employment sectors. Reintegration – particularly in employment and/or after several years away – is often cited as one of the most difficult elements of translating gains from a scholarship at home. 

In situations where scholarships have been offered to displaced communities (e.g. DAFI scholarships) reintegration may be an even more challenging issue. Both the panel and audience highlighted that funding organisations frequently have strong networks in recipients’ home countries which could be used to help ease the social and professional transition of scholarship alumni. Most organisations involved with an international scholarship programme are working on this challenge at some level, but there is clearly progress yet to be made. 

Ways forward 

Although the panel discussion lasted only an hour, it represented a microcosm of the current debate  happening around international scholarship programme analysis. Earlier in the day, participants from 11 organisations involved in running and evaluating international scholarship programmes held a workshop to discuss how they could better collaborate. For some time, momentum has been gathering around the idea that funders, administrators, and beneficiaries can work together more closely, both to improve policymaking and our understanding of international scholarships. 

The next step is to put these discussions into practice. For instance, is it possible for organisations to collaborate on data collection and analysis without the risk of data being used to draw politically damaging (and potentially simplistic) comparisons? Conversely, if direct comparisons are not made, then what can realistically be achieved by further collaboration? In other words, is there a safe political environment in which international organisations can collectively examine their outcomes with the aim of better understanding and better practice? Some effective groups certainly exist, such as the Donor Harmonisation Group, but it seems that pioneering work remains to be done in this field. 

Here at the ACU, we have been examining the methodology and results of the Postgraduate Taught/Research Experience Survey (PTES and PRES) administered by the Higher Education Academy in the UK. It reminded us that it is possible to develop unified instruments to measure complex topics in our sector. Benchmarking results in this way seems an unlikely direction for analysis of international scholarships to turn, although the ACU is working on initiatives at an institutional level. Yet working together to build a shared understanding of crucial outcome metrics and evaluation tools is perhaps not all that farfetched. In 2014, the Commonwealth Scholarship Commission in the UK’s analysis of current trends in evaluation methodology suggested that many outcome variables crosscut scholarship programmes –  a view echoed in the opinions expressed by NAFSA participants. 

Increasingly, the operation and analysis of scholarship programmes is becoming – or perhaps has become – a field in its own right. Although scholarships are deployed to achieve a variety of aims, as a tool and an experience they merit an analytical process that offers a deep understanding of how they impact on both individual lives and policy aims. It is, to borrow from a well-known media theorist, a case of the medium being the message. Like international education as a whole, we can reasonably be interested in understanding the capacity of our medium to shape, embody, modify and synergise with the policy message of the initiatives that create them. 

Exactly how the field emerges in the coming years remains somewhat uncertain but, as international actors talk seriously about consolidation and analytical coherence, there is good reason to hope it may look increasingly rigorous, collaborative, and synergistic. 

Dr Matt Mawer is a Programme Officer working in the Commonwealth Scholarship Commission's Evaluation team, based at the ACU in London.You can find out more about Matt on his website

If you would like to contribute to the ACU Voice, please email [email protected]