Cross-posting research I’ve led, which examines how to help students complete their qualifications. Our research shows that more apprentices and trainees will complete their training if students are given six behaviourally informed SMS prompts. Messages provided timely and practical advice on workplace rights, and where to seek support if they were struggling. Our results equate to 16% fewer learners dropping out. Our intervention led to a 7:1 return on investment.
The challenge
Completion rates for apprenticeships and traineeships have been a persistent challenge in recent decades. In 2015, the New South Wales (NSW) Premier set a State Priority to increase apprenticeship and traineeship completion from 50% to 65%.
The annual cost of non-completion to the state and federal governments had been estimated in 2011 to be $91 million. The total cost of non-completion including productivity forgone plus budgetary impacts was $348 million.
Apprentices and trainees (‘learners’) spend much of their time in the workplace. Learners also deal with various state and federal agencies but often do not have awareness of where to get help for specific workplace and training issues. As a result, learners who are struggling with their qualification were rarely seeking help from NSW Government services.
What did we do?
My team, the NSW Behavioural Insights Unit, worked with Training Services NSW (TSNSW) and the Centre for Education Statistics & Evaluation (CESE) in the Department of Education to help learners get timely support to complete their apprenticeships and traineeships.
We sent behaviourally informed text messages to the learners. The messages were designed to encourage them to seek help when they needed it, and persevere during their apprenticeship or traineeship.
From May to November 2019, 13,065 first year learners were randomly assigned into three groups:
- One third of learners were sent six SMS about how to build self-efficacy and get a ‘fair go’ at work (e.g. seeking mentorship, finalising their training plan, and other workplace rights).
- One-third of learners received six SMS about incentives (e.g. travel concessions, pay progression, and early completion).
- Both groups were sent a link to visit the Training Services NSW website for resources to help them persevere with their training, or they could get further help by calling or texting their local Regional Office.
- The other third of learners did not receive any SMS (the control group), however, they still had access to the same online resources and support from their local TSNSW Regional Office. More apprentices & trainees will complete their training if they are given 6 low-cost behaviourally informed SMS prompts.
Does it work?
The results of the trial were very positive. The Fair Go message was most effective in reducing 12-month dropout rates.
The dropout rates for learners who received the Fair Go message was 2.8 percentage points lower than those who received no SMS (15.1% for treatment vs 17.9% for the control group). The Fair Go messages were also more effective than those who received messages about Incentives (who had 16.7% dropout rate).
If we implement this behavioural intervention and send the Fair Go messages to all 13,100 first year learners around NSW, an additional 370 learners will continue their training.
These six simple and effective SMS will retain an additional 1,850 qualified workers over the next five years alone.
The text messages also led to increased customer engagement, with 552 direct phone calls and 512 text messages from learners seeking support, plus almost 6,000 clicks to TSNSW online resources.
Why do the results matter?
Our low-cost intervention led to $2.0M avoided costs for NSW Government and a further $1.1M avoided costs for business. For 2020-2025, the net present value is $2.4M.
Economic analysis shows the messages offer a sevenfold return on investment. This means that for every $1 the NSW Government spent on supporting learners in this way, $7 is returned in benefits, primarily of students progressing with their qualifications, as well as other benefits to businesses.
Our trial shows that low cost, behaviourally informed text messages improve outcomes for young, vulnerable customers, many of whom governments struggle to engage with. This insight has broad application across the NSW Government.
Several recommendations have been made in the report to support the implementation of this proven intervention to support learner completions.
- Implement the behavioural messages to boost proactive self-help behaviour
- Include personalised messages, and send timely messages throughout the training calendar, from April to November, on Wednesdays, from 11am to 12pm
- Measure longer term impact of the interventions
What were the behavioural insights?
Our intervention was co-designed with TSNSW stakeholders and drew on several behavioural insights concepts.
- Personalisation and messenger effect. Our messages included the learners’ first name, it specifically referenced their apprenticeship or traineeship (instead of a generic message about their ‘qualification’), and it was signed off from their local Regional Manager (personalisation). Regional Managers are senior figures within TSNSW, which reinforced that the organisation was taking seriously the responsibility of supporting learners (messenger effect).
- Salience. Our messages had a clear call to action to visit a relevant TSNSW webpage, or call their local Regional Office if learners needed more specific help.
- Timeliness. Because learners from different trades work varying hours (from 5am to 5pm), we tested sending our texts during common break times, in the morning, lunchtime, and early afternoon. Each campaign was timed at pivotal points in the apprenticeship/ traineeship journey. For example, when learners needed to complete key training documents.
- Address information bias. Learners are overwhelmed by too many documents at the time of enrolment, and cannot recall which government agency sent them helpful information about their services. Our SMS campaigns called attention to useful resources from TSNSW that learners may have overlooked, and which could help them at specific points in their first-year journey.
- Change habits. The current default for learners who have problems is to do nothing, suffer in silence, and eventually quit because they see no other option (default bias). Our messages sought to change the default by helping learners develop new habits. We established an early routine at the time of enrolment, sending behavioural texts every two months, to encourage learners to stay motivated to progress through their first year, and proactively seek help before problems got out of control.
Additionally, for each intervention group, we tested a different set of behavioural principles.
- Self-efficacy: Our Fair Go messages encouraged learners to build up their capacity to better manage stressful situations, exercise their workplace rights, and persevere with their training (this principle had the best result in our test).
- Incentives: Our Incentives messages appealed to both intrinsic incentives (self-improvement) and extrinsic incentives (financial and material rewards), to make staying in their training more attractive.
The Other Sociologist 