Intended Learning Outcomes on programme level
Programme ILOs are not only programme specific, but can also be more general applicable to programmes, for example ‘being able to write a scientific paper’. Also, these ILOs do not necessarily be specifically assessed by one assessment method. For example, the thesis (period) assesses different ILOs that are integrated in one assignment at the end of the programme.
Before you start formulating or redesigning your programme ILOs, you need an overview of the requirements your programme has to meet. These requirements consist of internal aspects – the institutional level as described above – and external aspects. For our UM programmes, external aspects consists of the Dublin descriptors, European and national qualification frameworks (Lokhoff et al., 2010). The requirements for programme ILOs according to the NVAO assessment framework for higher education in the Netherlands are defined on the right.
- Explanation: The intended learning outcomes demonstrably describe the level of the programme (Associate Degree, Bachelor’s, or Master’s) as defined in the Dutch Qualifications Framework, as well as its orientation (professional or academic). In addition, they tie in with the regional, national or international perspective of the requirements currently set by the professional field and the discipline with regard to the contents of the programme. Insofar as is applicable, the intended learning outcomes are in accordance with relevant legislation and regulations.
Concequently, a first step in defining ILOs is to determine the relevant frameworks for your particular study programme. While some frameworks are relevant to all programmes at UM (Dublin descriptors), other frameworks are specific for the particular professional field and/or discipline of the programme. Moreover, each institute or faculty can put its specific emphasis. As argued by Biggs & Tang (2007), programme ILOs need to reconcile the graduate attributes (institutional level) with the aims of the programme in that particular study field.
For example, the frameworks and attributes that the UM Bachelor Psychology needs to take into account when designing its programme ILOs are described in the table below.
ILOs for Employability
Employability
- Keep in mind the graduate profile of the programme
- Make sure course ILOs are aligned with programme ILOs
ILOs for International/Intercultural learning
International/Intercultural learning
Under construction
On the programme level, Lokhoff et al. (2010) distinguish ‘programme competences’ and ‘programme learning outcomes’. They consider the competence as a quality, ability, capacity or skill that is developed by and belongs to a student. Each programme is based on a set of key competences; these are the cornerstones of the programme. These key competences consist of both generic and subject-specific competences: Generic competences are general academic skills, relevant and transferable to any degree programme. Subject-specific skills are typical for a particular subject area (Lokhoff et al, 2010).
The learning outcomes are formulated as a reference to verify the achievement of the competences: they express the level of competence to be obtained by the student. A learning outcome is defined as ”a measurable result of a learning experience which allows us to ascertain to which extent/level/standard a competence has been formed or enhanced. The learning outcomes are not properties unique to each student, but statements which allow higher education institutions to measure whether students have developed their competences to the required level.” (Lokhoff et al., 2010). Lokhoff and colleagues (2010) distinguish the following five components in an ILO:
- An active verb form indicating the level of achievement;
- An indication of the type of learning to be achieved: knowledge, cognitive processes, skills, or other competences;
- The topic area of the ILO: this can be specific or general and refers to the subject matter, field of knowledge or a particular skill;
- An indication of the standard or the level that is intended / achieved by the ILO;
- The scope and/or context of the ILO.
According to Lokhoff et al. (2010), a programme should consist of up to 15 (generic and subject-specific) competences and 15 to 20 programme learning outcomes. The programme learning outcomes can be formulated for every cycle/formalized level.
If there are several tracks/specializations within a programme, additional learning outcomes or competences are needed to specify the tracks/specializations (Lokhoff et al., 2010). E.g. the Master Psychology at Maastricht University consists of 6 specializations. Each specialization has set ILOs regarding: 1) Knowledge and understanding, 2) Applying knowledge and understanding, 3) Making judgements, 4) Communication, and 5) Learning skills. However, since each specialization focuses on another subfield of psychology, these ILOs and competences are specified for each track. For instance:
- The specialization Psychology and Law requires that graduates are able to develop a critical attitude towards legal psychology (as part of ‘making judgements’), and are able to write an expert witness report and perform as an expert witness in court (as part of ‘applying knowledge and understanding’).
- The specialization Work and Organisational Psychology requires that graduates are able to evaluate conflict management and negotiation techniques (as part of ‘making judgements’) and to perform a job analysis (as part of ‘applying knowledge and understanding’).
- Who are the relevant stakeholders and how are they involved?
The best person to coordinate the programme ILOs and the linkage between course and programme ILOs is the director of studies and/or programme coordinator. Policy advisors and educational consultants can assist through the process of developing ILOs. Other relevant stakeholders can be members of the Faculty Board, Programme Committees, course coordinators, students and alumni etc. - What is the aim of the programme?
Programme coordinators should decide on the programme aims in consultation with the Faculty Board, programme committees and course coordinators. They should be informed about these aims to be able to design coherent courses. - What is the level of your programme?
Bachelor, Master or Research Master. - What are the key competences in this programme?
Think about general academic skills and subject-specific skills.
- Which frameworks are relevant to your programme?
The NVAO assessment framework for higher education in the Netherlands and the Dublin descriptors are relevant to all programmes aiming an NVAO accreditation. Besides, there might be other frameworks you are obliged to incorporate specifically for your professional field and/or discipline (e.g. a study leading towards a position as a physician, needs to fulfil specific criteria in order to grant this title). - Which requirements do you want and/or need your programme to meet?
Besides the frameworks you are obliged to follow, there are also requirements you can decide upon for yourself. For instance, you’ll probably consider whether you want your students to meet criteria for relevant post-academic studies, for specific jobs or criteria set by non-Dutch countries were graduates often want to continue studying or start working. (e.g. if many psychology students want to continue post-academic education to become a psychotherapist, you’ll probably consider to make sure that your graduates can meet the entrance criteria set).
Aligning course ILOs and programme ILOs
- Who guards the alignment between course and programme ILOs?
The best person to coordinate the programme ILOs and the linkage between course and programme ILOs is the director of studies and/or programme director.
- Evaluate per course how the course ILOs address the programme ILOs?
E.g. Curriculum map.
- Are all programme ILOs addressed throughout the courses?
Consider whether (1) all programme ILOs are addressed; (2) the alignment is balanced (no overemphasize); (3) there are no gaps of programme ILOs not (sufficiently) being addressed. Consider using curriculum maps to gain a good overview of the alignment.
- Is the alignment balanced? (e.g. no overemphasis on specific topics)
If a topic is overemphasized, you should consider how to regain balance. Probably there is room to address other (innovative) topics at some points in your curriculum.
- Are there no gaps of programme ILOs not (sufficiently) being addressed?
If a topic doesn’t get enough attention within your curriculum, you can search for possibilities to adjust this.
Stakeholders on programme level
It is important that someone keeps track on the programme. The best person to coordinate the programme ILOs and the linkage with course ILOs is the programme coordinator (also called programme director/director of studies in faculties). In addition, input from other stakeholders is required. Policy advisors and/or educational consultants can assist the programme director/programme coordinator/director of studies through this process. To define or adjust the programme ILOs, opinions of different stakeholders can be gathered such as:
- Board members: since they are responsible for the education offered at your faculty;
- Programme committees: since they determine the level of knowledge, skills and insight that students have obtained at the end of their study;
- Boards of Examiners: since they have to provide a guaranty for the adopted knowledge, skills and insights of students at the end of their study;
- Course coordinators: since they are involved in the course content and often specialist within their field;
- Students and alumni: since they might be the first to notice strengths and pitfalls of the programme;
- And other persons with a clear view on the requirements of the labour market.