Theoretical Background

The FIRE Toolbox is based on Donald Kirkpatrick's four-stage evaluation model. This assumes that a course can only be considered successful if the content is put into practice and makes a significant difference there. In terms of emergency services, this means that courses should lead to more successful operations.

This relationship is usually difficult to measure. It often takes some time before the content of a course becomes relevant in practice. In an operation, content from different courses is then relevant at the same time. The contribution of a single course is difficult to estimate.

The evaluation model we use in FIRE therefore divides the evaluation into four stages, which build on each other. First, the participants' reaction to the course should be positive. This is a prerequisite for learning success. Only when learning is successful can the content of the course be transferred into behavior. Finally, the change in behavior can lead to the course being reflected in concrete results (e.g., lower number of errors). These four stages are illustrated in the following image.

 

While the behavior and results stages require larger-scale studies over time and with comparison groups, the first two stages of evaluation, reaction and learning success, can be captured well using questionnaires. The FIRE Toolbox was developed for this purpose. The questionnaires it contains make it comparatively easy to check whether the conditions are in place for successful behavioral change, which is ultimately reflected in the results.

In order to be able to examine stages three and four as well, we have developed the online panel FIRE Feedback. IdF NRW course participants will be able to register here so that we can survey them over time.