At the height of the COVID-19 pandemic in 2020, the key focus for policy makers and industries was to assist Australians back into work in jobs that were in-demand and resilient to the immediate economic shocks of the public health crisis.

As Australia emerges from the economic impact of the pandemic, there needs to be a shift towards ensuring the education and skills system delivers the skills and knowledge for the economy now and in the future. There is an inherent uncertainty in estimating the future skills needs of the economy, therefore it is crucial to ensure that our education and skills systems are resilient enough to deliver the skills we need despite this uncertainty.

To do this, it is important to better understand the pathways from education to employment. Before the pandemic, the Australian labour market had been progressively shifting towards higher skilled employment and this trend is likely to continue as we recover from the economic impacts of COVID-19 (NSC, 2020). This emphasises the importance of post-school qualifications in the labour market going forward. There is a need for a more detailed understanding of how to better match skills developed in education and training to the skills required by employers. A granular view allows policy makers and education providers to design better qualification pathways that are timely and cost-efficient for users and employers.

The nexus between education and employment is an important aspect of the work of the National Skills Commission (NSC). The NSC was established by the Australian Government as a recommendation of the Strengthening Skills: Expert Review of Australia’s Vocational Education and Training System by the Honourable Steven Joyce (Joyce, 2019).

The review identified a need for better careers information and data on skills needs and future demand. The NSC aims to provide trusted and independent intelligence on Australia’s current and future skills, education, and jobs. The NSC also contributes to a labour market that effectively aligns skills needs with education and training. This work is an important part of the linking piece between the education and employment spheres to drive better advice in careers information, VET sector outcomes, and future skills demand. Qualification reform and rationalisation is an important part of building a resilient VET system to deliver the skills required by employers and industries.

Machine learning techniques have been used previously to explore the labour market. The World Economic Forum (2018) and the former Australian Government Department of Employment, Skills and Small and Family Business (2019) used machine learning and natural language processing techniques in their work to compare occupations. And Kern et al., (2019) analysed twitter data to assess the common personality traits and values associated with a range of occupations. This research can assist students, job seekers and other members of the public to determine occupations that they would be suited to.

Machine learning can handle large quantities of data in a way which could not be done with other techniques. Machine learning techniques can also be applied to better understand the education and skills system to compare the similarity of qualifications. This paper analyses the similarities between VET courses across, and within, training packages in Australia. This has been done using natural language processing, which is a branch of machine learning, to calculate similarity scores. The analysis relied on language models that represent text as vectors which allows the calculation of similarity scores based on the text contained in the course names, the units within the course, the course description and the frequency of keywords within these texts.

The language embedding model, data sets and the model outputs and validation are discussed in detail in the next section.