Measuring the impact of ELIXIR’s training

As a distributed infrastructure, ELIXIR’s training provision is spread throughout ELIXIR’s 22 Nodes. The training aims to enhance data and software skills in the life sciences, for example building expertise in data management, tools, workflows and standards.

There are many benefits of distributed training: each Node runs training events in their particular specialisms, and all training sessions are open to participation from other Nodes. However, with over 60 different training providers, the coordination, tracking and evaluation of all training events is a substantial task. 

The Training Metrics Database

In 2015, the ELIXIR Training Platform initiated work on a quality and impact strategy. As a result, in 2019, the Training Metrics Database was born. The database enables reporting on core training metrics, including audience demographics, training quality and training impact.

Metrics are predominately collected through a standard feedback survey, although in some cases the demographic information comes from the training registration form. The experience of ELIXIR training providers, along with discussions with stakeholders and external training providers, was used to draw up a shortlist of potential indicators which was whittled down to a core set of training metrics for quality and impact.

Learning from the numbers

Querying the Training Metrics Database gives an indication of the scale of the operation. Since 2015, there have been over 1,600 events comprising over 5,000 days of training. More than 40,000 people have been trained by nearly 6,000 trainers. 

For events where short-term feedback (quality metrics) was requested, 88 percent of participants filled in the post-training survey. Of these, 76 percent scored the training as very good or excellent, and a further 17 percent as good. Having a consistent and standardised set of metrics across the different Nodes is important as it allows training quality and impact to be monitored across all training providers and over time.

Measuring impact

The impact of a training course refers to its long-term effect, which makes measurement particularly challenging. To understand impact, the Training Coordinators of each Node came up with a list of questions to be answered from six months after the training event onwards. 

In response to the question How many people have you shared the skills and/or knowledge that you learned during the training with? 58 percent of respondents had shared skills and knowledge with at least one other and a further 22 percent planned to. In addition, sixty percent of participants pointed to a direct outcome from the training, for example new collaborations following training (14 percent), facilitating a publication (18 percent), and changing career (5 percent). The full list of impact metrics is freely available on the resource help page.

The data suggest that the training events are having ripple effects on colleagues and building capacity in ELIXIR Nodes organisations and throughout Europe.

A valuable resource

The metrics data are valuable for quality control and also provide queryable answers to requests for information from funding bodies and other professional organisations. Corinne Martin, Senior Impact Officer at ELIXIR, has used the database in numerous occasions:

“I have found the Training Metrics Database extremely useful in the context of external relations work. Some of our funders are keen to have nationally relevant information on training activities and related outcomes and impact, which the database easily provides. When aggregated across countries, the information becomes very powerful and concrete to illustrate the coordination role of ELIXIR as a distributed, pan-European research infrastructure. ELIXIR’s Training Metrics Database was a forerunner in the organisation and we are now building systems to track a wider range of indicators on the ELIXIR Impact Dashboard.”

In summary, the ELIXIR Training Metrics Database enables the provision of a wide range of training from a large pool of training providers whilst maintaining a connection to the experience of the participants. The Database, designed and run by trainers in the ELIXIR Nodes, has enabled training provision in ELIXIR to be scaled up, without sacrificing quality or impact.

Find out more

Posted