Measuring success?

Drawing from the ACU’s experience in scholarship design, administration, and analysis, our new blog series ‘Measuring success?’ explores the outcomes of international scholarship schemes for higher education.

Launched in October 2015, the aim of the series is to generate dialogue between evaluators, programme designers, and policymakers about the cutting edge of the field.

New posts are published regularly, authored by experts from all around the world, as well as by ACU staff. 

If you are interested in contributing to the series, or if you have any feedback, please email [email protected]

A growing need for Sanctuary Scholarships: Lessons learned from the University of Bristol
Dr Katie Bales, Lecturer in Law, University of Bristol - published 07/02/2018

On 8 June 2016, the University of Bristol announced the launch of the 'Sanctuary Scholarship scheme', which provides access to higher education for forced migrants facing major barriers to accessing education. In doing so, Bristol joined a cohort of likeminded universities seeking to provide space and sanctuary for those forced to flee their countries of origin. At present, for example, there are approximately 40 universities in the UK offering scholarships to forced migrants. Read more

Consolidating research, policy and practice in the emerging field of scholarships and social change
Joan Dassin, Brandeis University - published 26/01/2017

Scholarships for individuals to pursue international education can lead to more equitable, sustainable, inclusive, and prosperous communities. This is the central claim explored in the new book International Scholarships in Higher Education: Pathways to Social Change that I recently edited with Robin Marsh and Matt Mawer. The book draws together research and critical analysis by nearly two dozen authors from Africa, Asia, Australia, Europe, North America, and Latin America.

Through a social justice lens, Pathways to Social Change examines and analyses the role of scholarships – in our view much more than simple funding mechanisms – in broadening access to international education and extending its benefits beyond individual recipients. Read more

How can a robust measurement of the impact of scholarships be achieved when a panel survey is not feasible? Answers from an evaluation of the Belgian university development cooperation
Lennart Raetzell and Olga Almqvist, Syspons GmbH - published 27/10/2017

Panel surveys collect information from the same individuals at multiple points in time to capture long-term changes... However, it is often not feasible to implement panel surveys in impact evaluations, as evaluation assignments have to be completed in a given timeframe that does not allow for data collection over the course of several years. This is precisely the situation we are facing with the impact evaluation of Belgian higher education cooperation. Read more

The Scholarship Program Research Network – a new international community for studying and evaluating scholarship programs
Aryn Baxter, Anne Campbell and Mirka Martel - published on 07/08/2017

Those who administer, research, and evaluate scholarship programmes often work in small circles, focusing on a specific programme, country, or area of study, and within agencies or institutes whose work goes beyond scholarships. Historically, it has been difficult to share information and resources across administrators, evaluators, and academics because these communities only had limited opportunities for exchange...The communication gap appears even larger between national and international scholarship programmes. These two communities rarely cross over at an international level...

To begin addressing these concerns, a small group of researchers have decided to combine their bibliographic resources and pull together those who are working in this field to form the nucleus of a new research community: The Scholarship Program Research Network (SPRN). Read more

Tracking international scholarship outcomes: the CSC Longitudinal Research Framework

Dr Matt Mawer, Shireen Quraishi and Rachel Day, Commonwealth Scholarship Commission in the UK, published on 16/06/2017
In stark contrast to common practice only five years ago, much of the prominent new research on scholarship programme outcomes adopts a longitudinal model. Like others in the field, the Commonwealth Scholarship Commission in the UK (CSC) has recently invested considerable time and energy into building an effective longitudinal study of mid-range scholarship outcomes (professional development, organisational change, etc.). Unlike most others in the field, however, the CSC Longitudinal Research Framework has been designed to continue as a component of basic scholarship administration in the long term. We introduce the logic and methodology of the framework in our article. Read more

A longitudinal study on the Queen Elizabeth II Diamond Jubilee Scholarships-Advanced Scholars programme
Anna Seifried and Cate Lawrence, Universities Canada, published on 19/05/2017
The QES and QES-AS programmes aim to activate a dynamic community of global leaders across Canada and the world, and create lasting impacts through cross-cultural exchanges encompassing international education, discovery and inquiry, and professional experiences. QES-AS will build on the success of QES by identifying scholars who show leadership potential and supporting them in acquiring world-ready research experience through a university, in collaboration with a non-profit organisation or industry. Through these networks, QES-AS aims to strengthen university linkages with industry, community and international partners, explore new opportunities, and better position Canadian universities to engage in solutions to complex global development challenges. Read more

Looking beyond the alumni: reflections on the institutional legacy of IFP in Mexico
Dr David Navarrete, Centre for Research and Advanced Studies in Social Anthropology (CIESAS), Mexico, published on 21/04/2017
Social inclusion and equity in higher education are high on Mexico's educational agenda. Within this framework, guaranteeing indigenous students access to, and successful conclusion of, a university education is an urgent undertaking that is being addressed – although there is still much to do. The purpose of this post is to highlight some contributions of the Ford Foundation International Fellowships Program in Mexico (IFP Mexico) in this field, particularly at the institutional level. Read more

Can developing countries influence the transnational arena using science and education as foreign policy tools? The Brazilian case and the need for policy evaluation
Gabriela Ferreira, University of São Paulo and King's College London, published on 17/03/2017
The use of education as a foreign policy tool is usually related to developed countries. However, major developing countries are designing strategies adapted to the new transnational arena... One of the most recognised transnational strategies has been developed by Brazil, which offers educational opportunities to students from the global south through two specific programmes... The main question is: can a country like Brazil thrive on such policies? Read more

Evaluating the legacy of 'Ciência sem Fronteiras'
Frederico Menino, The New School for Social Research, published on 10/02/2017
When the 'Ciência sem Fronteiras' ('Science without Borders') programme was launched by President Dilma Rousseff in the end of 2011, few could foresee how deeply the economic, social and political outlook of Brazil would be transformed five years later. An emerging power in the early 2010s and soon to host the FIFA World Cup and Summer Olympics, Brazil had resisted the worst impacts of the global financial crisis fairly well and remained an increasingly influential geopolitical actor. The favourable winds pushed the country to also strengthen its position in the 'global knowledge economy' via a series of initiatives in the field of international education. Read more

Ciência sem fronteiras: What next for the programme?
Vivienne Stern, Universities UK International, published on 20/01/2017
...Several things are clear. The new programme will start from a different perspective. It will be less concerned with individual opportunities to study abroad, and more concerned with seeing student mobility as part of a broader approach to internationalisation of Brazilian institutions. The programme will be dramatically smaller in scale... Read more

Australia Awards: Measuring and evaluating development impact
Rachel Parker, Australian Council for Educational Research, published on 29/11/2016 
The Australian Government's prestigious Australia Awards programme offers scholarships to emerging leaders from the Indo-Pacific region to undertake full-time undergraduate or postgraduate study, or short-term fellowships and short courses. In order to monitor and evaluate the impact of these scholarships and fellowships, the Department of Foreign Affairs and Trade (DFAT) has designed the Australian Awards Global Tracer Facility. Read more

'Measuring Success?' year one – The scene, the motifs, the actors
Dr Matt Mawer and Sian Julian, the Association of Commonwealth Universities, published on 04/11/2016
The end of October 2016 marks the one-year anniversary of 'Measuring Success?'. The blog series has been developed – or perhaps curated – without a strong editorial line. We have helped develop some themes in the blog, but have not guided the topics much beyond defining the basic parameter that posts had to be relevant to understanding the outcomes of scholarship programmes. Read more

The Atlantic Philanthropies-University of Queensland Vietnamese scholarship programme experience: stepping stone and personal transformation
Authored by Trevor Grigg, published on 30/09/2016
Over the period 2000 to 2006, The Atlantic Philanthropies (AP) funded The University of Queensland's Vietnam Coursework Masters and Doctoral Development Scholarship Programs, with grants totalling AUD 17 million. The primary objective of the programmes was: 'to contribute to the sustainable development of Vietnam through education and training initiatives that enhanced the knowledge, competitiveness and skills of talented young professionals'. Read more

The soft power of government-funded scholarship schemes – how to measure impact?
Authored by Robin Hart, published on 29/07/2016
International scholarships offer huge benefits to the sponsoring country; scholars are an important source of talent, skills, and diverse perspectives. Scholarships are also effective tools in promoting and enhancing a country's soft power, by investing in future leaders, providing access and equity to higher education, and increasing research excellence. Read more

Schwarzman Scholars: using scholarships to build cultural diplomacy
Authored by Professor Sir Nigel Thrift, published on 08/07/2016
Leadership is a nebulous term... In the world we now live in, we need an enlightened cadre of leaders who can negotiate with China from the ground up. Educating leaders means fashioning situations in which students can encounter both leadership and China simultaneously. This is the goal of the Schwarzman Scholars. Read more

Meaning-making through storytelling – the ways in which students use storytelling and reflection to reauthor the academic journey (part 2)
Authored by Aiden Choles, published on 17/06/2016
This blog post follows on from a previous entry that discussed how a Participatory Narrative Inquiry (PNI) methodology was used in the evaluation of the Sasol Inzalo Foundation's (SaIF) bursary programme in South Africa... While it was generally acknowledged that the storytelling methodology would allow students to reflect on their experiences, the extent of the benefits and the resultant influence in how students' reauthored their journey was not fully anticipated. Read more

Do international scholarships lead to 'brain drain': tracing the impact of international alumni on their countries of origin
Authored by Robin Marsh, published on 10/06/2016
...Current research on African graduates from the University of California at Berkeley (UC Berkeley) and partner universities, indicates widespread and enduring social contributions by alumni, and counterfactual evidence of 'brain drain'. As universities, such as UC Berkeley, aspire to become ever more global in their reach and impact, there is an increased rationale for investing in retrospective tracer studies of international alumni, particularly from developing regions of the world. Read more

Can individual outcomes lead to communal impacts? Measuring social change using longitudinal studies... A challenge and an opportunity
Authored by Mirka Martel, published on 20/05/2016
'A transformative experience for me and my community'. These are words I often hear when speaking to alumni of the Ford Foundation International Fellowships Program (IFP), a programme for 4,305 emerging social justice leaders from marginalised communities in 22 countries, which ran from 2001 - 2013. But how can we transfer these individual stories to measure the success of a global social change programme like IFP? Read more

Tracking science and engineering students' experiences using participatory narrative approaches (part 1)
Authored by Aiden Choles and Dr Murray Hofmeyr, published on 22/04/2016
This blog post will discuss how a Participatory Narrative Inquiry (PNI) approach was used in the evaluation of the Sasol Inzalo Foundation's bursary programme in South Africa. As part of the evaluation process, students were encouraged to record their experiences using self-quantified narratives, with the aim of using the insights to inform the design of student support during the bursary programme. Read more

Not always at ease: the interaction between policymaking and evaluations of the Netherlands Fellowship Programmes
Authored by Ad Boeren, published on 01/04/2016
One would expect and wish for a direct link between evaluation studies and policymaking. In an ideal world, policy making is influenced by the outcomes of evaluation studies, and evaluation studies are designed to improve and inform policy decisions. In practice, this is not always the case. Read more 

Evaluating the re-integration process of the Australian Development Scholarship programme's Indonesian alumni 
Authored by Dr Halil Chalid, published on 11/03/2016
Historically, the benefits of Australian scholarships for Indonesian students have been regarded as self-evident, yet my research indicates they are harder to quantify. Read more

Examining outcomes from a distance learning partnership
Authored by Lynne Elliott, published on 19/02/2016
The need to enhance support for countries to improve progress towards development was highlighted by the 2005 Paris declaration on Aid Effectiveness and the 2008 Accra Agenda for Action. Distance learning partnerships, supported in part by student scholarships, are one way of measurably developing capacity. Read more

The impact of home-country context on scholarship outcomes
Authored by Anne Campbell, published on 29/01/2016
The theory of change that underpins many international scholarship programmes suggests that with exposure to quality education found in leading universities, students will bring newfound knowledge and experiences back home to positively shape the future of their countries. Read more

Defining success in light of student experiences
Authored by Dr Aryn Baxter, published on 18/12/2015
A reflection on how the experiences of international scholarship recipients from low-income contexts might inform the ways we choose to define and measure success. Read more

Methodologies for evaluating short fellowship programmes
Authored by Dr Caroline Cage and Shireen Quraishi, published on 30/11/2015
Short fellowship programmes offer academics a tailored opportunity to develop professional skills. Given the individual nature of the fellowships, evaluation presents some challenges. Here, the authors outline the methodologies in place to assess impact of two specific schemes administered by the ACU. Read more

International scholarships and the SDGs
Authored by Dr John Kirkland, published on 06/11/2015
Half a cheer for the new Sustainable Development Goals (SDGs). On the one hand it's good to see increased prominence given to tertiary education, including a welcome reference to scholarships. But problems will emerge from the targets in themselves. Read more

Geographical mobility and the Rhodes Scholarships across the 20th century
Authored by Dr Tamson Pietsch, published on 16/10/2015
The Rhodes Scholarships scheme is one of the longest running programmes of scholarly exchange, and as such it offers a century's worth of invaluable data for examining patterns in the geographical mobility of awardees post-scholarship. The author's analysis has revealed some striking patterns. Read more