Newcastle University

This report discusses the development of a robust and objective framework for the assessment of the effectiveness of delivering core chemical engineering knowledge and employment competencies. The outcomes of the extensive review of intended learning outcomes set by various national accreditation and governmental bodies are presented here, indicating a very consistent approach across Europe and worldwide. A detailed methodology for collecting responses from all three stakeholder groups (academics, employers and graduates) on the importance of various areas of CE knowledge and employment competencies, current methods of delivering these as well as the current methods of assessing the effectiveness of their delivery is also described. 

iTeach must provide robust performance indicators of Chemical Engineering training centers and of teaching effectiveness. This exercise is part of the modes of governance because it aims at transforming boring and complex data and into "eloquent" and clearly legible figures.

Two frameworks are then developed :

• The first one deals with the assessment of a whole training center gathered in 7 global indicators taking into account the specificity of the activities in Chemical Engineering.

• The second one deals with the evaluation of a single teaching unit and is based on a literature analysis of different methods developed for assessment of teaching effectiveness. The impact of the formation, its relevance, the notions of transfer, acquisitions, competencies are in particular considered. The proposed framework is then based on 6 metrics, that can be assessed through questionnaires completed by different stakeholders involved in the teaching process : students, graduates, academics and employers.

The assessment of performance is both measurement (since associated with quantification), meaning (because it must be interested in the meanings by the interpretation of the measurements), decision (indicators represent what efforts should be accomplished to achieve “excellence”).

The document is preceded by a preamble defining a number of characteristics of the complex relationships between teaching activities, and their environments and translation in terms of indicators, followed by a proposal which has been discussed and approved by the iTeach EU partners. 

The two frameworks developed in the former deliverable D.3.1. are here clearly defined and quantified.

The first one, related to the effectiveness of a whole formation is based on 7 global indicators gathering more than 150 parameters. All the parameters are quantified, according to Gaussian laws or threshold values, and their mean values, standard deviations and weights are also defined. This work was based on the literature analysis, on the recommendations of the European Federation of Chemical Engineering, and on several internal discussions within the consortium.

The second framework, related to the evaluation of the teaching effectiveness of a single module is assessed through questionnaires, by different stakeholders involved in the teaching process : students, graduates, academics and employers. The questionnaires are organised into 6 metrics.

The questions inside each metrics have been discussed within the consortium, and are quantified by Likert scale. The global values of the metrics are finally quantified according to the responses of the stakeholders, and to different weights that were also discussed between the project partners.

After a first survey in academic year 2015-16, a revised version of that second framework has been developed, for utilisation during the 6 last months of the project. 

This report provides guidance for the use of the proposed assessment frameworks of the effectiveness of core knowledge and competency, for a whole training center and for a single teaching unit, for their pilot implementations.

This deliverable reports on the development of an assessment framework template. The framework proposed in WP3 allows to assess an individual unit/course and is divided in six metrics, as follows:

1. Strategic nature of the course/discipline;

2. Relevance of the proposed formation;

3. Pedagogical relevance of the teaching approach;

4. Perception of relevance of the pedagogical approach;

5. Evaluation of acquisitions and;

6. Evaluation of transfer .

Each stakeholder (Academics, Employers, Graduates and Students) is asked to assess on specific metrics. To this end, four surveys - one for each stakeholder – were made available online on the project’s website and disseminated by email to the consortium partner’s contacts (see questionnaires in Appendixes A-D). The fifth metric – Evaluation of acquisitions – is assessed through grades/marks obtained by students during exams/tests.