Evaluation of sustainability using the AISHE Instrument: case study in a Brazilian University
Brazilian Journal of Science and Technology volume 1, Article number: 4 (2014)
The role of Higher Education Institutions (HEI), which are in the forefront of the challenges of sustainable development, is becoming increasingly predominant, given the importance of these institutions in the dissemination of knowledge and the implementation of sustainable conscientiousness in society. This article presents the application of the Auditing Instrument for Sustainability in Higher Education (AISHE) at the University of Passo Fundo, which is situated in Southern Brazil. The research evaluated the level of sustainability in 30 undergraduate courses at this university, focusing on Education for Sustainability. To do this, interviews were conducted with course coordinators and quickscans were utilized with professors and students. The way in which the University of Passo Fundo applied AISHE is innovative and interesting: first of all, it is important to emphasize that during the research, a wide variety of university departments were assessed; secondly, this approach allowed a comparison between the courses, producing interesting outcomes that can easily be used for a successive phase of sustainability integration. The most innovative element in the UPF approach is the way in which the outcome of the twenty AISHE criteria are related to the results of the quickscans, which are part of the ‘Check’ criteria of AISHE. Considering the priorities within sustainability in the UPF, the need exists of an initial drive in the managerial stage, such as definition of vision and policy for the institution.
There are many studies that discuss the concept of “sustainability in the universities” and what can be observed is the existence of many different ways of integrating the practice of sustainability, the most common being the inclusion of curriculum, research and outreach in the dimensions of the campus operations (Weenen, 2000; Halac et al., 2005; Tauchen and Brandli 2006; Lidgren et al., 2006; Ferrer-Balas et al., 2009). In this article, however, the focus is on the inclusion of the elements of Sustainable Development - SD, on “greening” the curriculum.
As Lozano (2010) pointed out, some research has been carried out explaining the incorporation of SD in university curricula (Velazquez et al., 2005; Lidgren et al., 2006), but there is potential for still further research. According to Velazquez et al. (2006), what is lacking is clear and objective orientation about what exactly sustainability should be in HEI.
According to Adomssent et al. (2007), in the inclusion of sustainability in higher education, the institutions should act as agents of change. Following this line of thought, it is important that the institutions divulge their implanted changes in benefit of sustainability and evaluate their effectiveness, serving as a model and inspiration for others and playing the role of an accelerator. To gauge this, the use of instruments of evaluation becomes necessary. Lozano (2006) mentions that one of the steps for the inclusion of SD in the curriculum is the use of tools to evaluate the effectiveness of the efforts that have been made.
In this context, mediation through evaluation indexes of involvement of the university with sustainability becomes important, beginning with the development of a general picture, which can facilitate evaluation of the current situation or progress. These indexes, however, should be instruments orientated towards the particularities of the higher education institution, for example, the Graphical Assessment of Sustainability in Universities – GASU (Lozano, 2006), the Sustainability Tool for Auditing Universities Curricula in Higher Education – STAUNCH (Lozano, 2010), the Framework-Level-Actors - FLA Analysis (Ferrer-Balas, Buckland, Mingo, 2007) and the Auditing Instrument for Sustainability in Higher Education - AISHE (Roorda, 2001; 2008).
Shriberg (2002) presented in his article an outlook of the strengths and weaknesses of the assessment tool and concluded that it is necessary in the future to think about a universal assessment tool which can facilitate the ranking of universities in terms of levels of sustainability. Shiberg (2002), also comments that the tool, AISHE, the focus of this study, is an excellent example of an approach which makes it possible to evaluate sustainability from evaluation indexes with orientation. The aim attached to the creation of AISHE was to construct a flexible platform where it is possible to stimulate and make sustainability operable at higher education level. In fact, this tool has the potential of achieving global appeal. However, the author also highlights that its weak point is that its criteria are quite abstract and that there may be difficulties to understand it.
Many universities find it appealing to use AISHE in a more informal way, i.e. as a self-assessment tool (Roorda, 2004; 2008). Every university is free to do so, as all necessary equipment, such as the AISHE book (Roorda, 2001) and the computer application ‘AISHE Reporter’, can simply be downloaded. Consequently, universities in more than a dozen countries have applied AISHE in a variety of ways.
In 2008, the University of Passo Fundo (UPF) together with other universities from various other countries, committed itself to participating in the group of universities which applied the AISHE tool in its internal and external environments, with the aim of evaluating the methodology and contributing to future adjustments. The aim of this article was to evaluate environmental sustainability on the graduation course curricula of UPF, making it possible to identify priorities in relation to the endeavour for creating sustainable environments and to contribute to the knowledge of their evaluation.
This case study is innovative if compared to other AISHE applications (Roorda, 2004; 2008). Firstly, it is important to emphasize that during the research, a wide variety of university departments were assessed, ranging from physics, chemistry and life sciences to, for example, Geography, History, Business Administration, Journalism and Music. This approach allows for a comparison to be made between all these subjects, delivering outcomes that can easily be used for a successive phase of sustainability integration.
Another distinguishing feature in the Passo Fundo approach is the way in which the outcomes of the twenty AISHE criteria are related to the results of the quick scans, which make part of the ‘Check’ criteria of AISHE. In these quick scans, the level of appreciation of the sustainability process and its results of several stakeholders could be assessed: course coordinators, professors, students. This idea of making a comparison between AISHE criteria and quick scans was never thought of before and it did not make part of the procedure when AISHE was originally developed in 2000–2001.
Aim of the study
The University of Passo Fundo, in southern Brazil, is a community university with around 20 thousand students, 1100 teachers and 1000 employees. It is located in 7 campuses with 54 graduation courses, 7 technical courses, 45 specialization courses, 7 master degree courses and 1 doctorate.
The focus of the study was the 40 graduation courses of the main Campus I, 30 of which agreed to participate in the research, which represents 75% of the Campus I population. The sample size was random, depending on acceptance of participation in the study. It should be noted that, at this stage of the research, the 14 higher education technical courses were not included. The application of the questionnaires occurred in 2010.
Instrument for evaluation of sustainability
The Auditing Instrument for Sustainability in Higher Education (AISHE) is an instrument designed specifically for evaluating sustainability in educational institutions (Roorda, 2001; 2008). When such assessments are used as repeated elements of an iterative process, a ‘Deming cycle’, this process has proved to be able to lead towards a true transition of the university towards ‘System Integration of Sustainable Development’ (SISD), a concept which was introduced and thoroughly discussed in Roorda (2010) PhD dissertation. The author also shows that this SISD state can very well act as the final goal for a university’s sustainability strategy. This state can be considered as equal to the systemic integration of sustainable development described by Lozano-Garcia et al. (2006), at least in terms of the education and the underlying mission and philosophy of the institution.
As illustrated in Figure 1, AISHE is based on the PDCA cycle and within each of the phases of the cycle, fields of attention are evaluated, each one with four criteria.
The criteria can be classified in accordance with five stages (level of sustainability). Each level has a specific description which reflects its level of sustainability for which the people that are responding classify their course.
The stages have an ordinal scale for the five phases: stage 1 – activity; stage 2 – process; stage 3 – system; stage 4 – part of a chain of process; stage 5 – society; In this way, it is only possible to conclude that a particular stage has been reached if the previous phases have been completely achieved because all the stages of a criterion are designed to be cumulative; for this reason intermediate values can be defined for each criterion.
The research was performed according to the scheme presented in Figure 2.
The AISHE methodology was applied to campus I in Passo Fundo for the graduation course coordinators, with the presupposition that these people would be the most likely to respond in relation to their courses as they are directly involved in the day-to-day activities and are very familiar with them.
Criteria established in AISHE evaluation were answered individually by coordinators, for each (selected) stage the participant formed his/her own opinion about the situation on the course. For the appraisal of results, AISHE methodology allows the use of punctuation median.
After this, a quickscan, in the form of a questionnaire, was used for professors and students, with the aim of confirming the data obtained with the coordinators. In total, 60 professors (two from each course) and 90 students (three from each course) took part. The amount was defined in such a way as to make the research feasible and the sample was selected randomly, taking into consideration all courses in the institution.
Discussion and evaluation
Sustainability evaluation according to the coordinators
The sustainability level attributed by the coordinators to each one of the criteria when the situation of their course was evaluated is presented in the Additional file 1: Table S1.
In Figure 3 the different levels of sustainability reached for each course can be observed; this level was obtained by the median of the punctuation attributed for each criteria. Some attained considerably elevated levels (stage 4 e 5). Though it can be observed that these are exceptions, it can be seen that around 70% of the evaluated courses fit into stage 1, demonstrating that their actions are in line with ESD, from the point of view of isolated practices.
Figure 4 shows the results from the median of each criterion in relation to the points attributed to the total sum of the evaluated courses. It can be observed that the majority of the criteria reached Stage 1.
The criterion educational methodology (3.2) showed the best results, reaching Stage 3, which signifies that the teaching and learning methodology gives the student the opportunity to encounter real situations, which gives rise both to reflection and development of his/her future professional development in a sustainable fashion.
The interdisciplinary criterion (4.2) attained Stage 2, which shows that the curriculum is structured in such a way that the subjects covered by the courses are interconnected. The other criteria of this field of attention (4.1 curriculum; 4.3 stage, graduation; 4.4 speciality), were met in Stage 1. This demonstrates that analysing the interdisciplinary criterion by itself does not make it possible to say that the subject of sustainability is embraced by the courses satisfactorily.
The criterion vision (1.1) presented a median of intermediate value between stages 1 and 2, signifying that the administration has taken a position on sustainability, even if only implicit, because it offers work opportunities with objectives which, aside from this awareness, can bring concrete consequences for the University, albeit through individual actions. It can also be observed that both this awareness and sustainability cease being merely implicit when they are formulated in documentation.
The existence of vision is important but it does not necessarily produce changes; it merely represents an opinion. The policies are a structure which solidify this vision because it translates itself into plans and from these, goals are formulated The criteria policy (1.2) achived stage 1. The criteria Communication (1.3) attained 1.5, the communication of the university with society and it’s stakeholders is fundamental although it is necessary that the subject of sustainability be properly approached.
The criteria of internal environmental management presented sustainability level 1 and should be given special attention because it plays a fundamental role. It demonstrates the respect which the institution has towards the environment, making the whole community learn and change its practices and develop a new life style concept that takes into account SD (Disterheft et al., 2012). There are various programmes and projects at the UPF for improving the quality of the environment, although these are in fact isolated practices because the university does not have an environmental management system (EMS).
The students’ criterion (5.2) considers the existence of university data concerning students’ perception in relation to the sustainability of the institution. As the university did not make available singificant data for valuation, the values stated in the evaluation of this criterion were low, resulting in a median value equal to zero.
Analysis of the items responded to by the students and professors
Figure 5 presents the different responses of professors for each analysed item, where it can be seen that the professors opted more for ‘partly agree’.
Some key points are possible to be concluded by the answers of the professors. The majority believes that a significant number of employees, students and even professors do not value and/or respect the principals of sustainable development, showing the weakness of the EMS of the institution. Also, that sustainability is still seen as a speciality and not as something interdisciplinary in teaching institutions. Lastly, they believed that what is being taught regarding sustainability is not perhaps ideal or sufficient to meet the requirements and form adults who have responsibility regarding sustainability. This is in compliance with the partial agreement demonstrated by the professors in relation to the presence of the subject of sustainability on the curriculum courses which they teach.
Figure 6 presents the percentage of the replies for each of the items by the students. It can be observed that the students’ replies are divided between ‘neither agree nor disagree’ and ‘partly agree’.
The quickscan applied for the students showed a lack of knowledge about what the university, still in an isolated or partial way, is doing concerning its environmental sustainability. They state that environmental sustainability should be treated in a more effective manner in their curriculum, leading to the development of a sharper sense of ethics in relation to sustainability in their future professional lives. As the professors, they also seen sustainability merely as a speciality subject and not something interdisciplinary, such a position demonstrates uncertainty or even a lack of knowledge on the part of the students concerning the relationship between sustainability and the structure of the course curriculum. Considering that, environmental sustainability should be revised, principally within the curricula context, so that it can offer greater security for future graduates in relation to what represents sustainability attitudes around them.
Comparison between the replies of the coordinators, professors and students
The aspects evaluated by professors and students from the use of the quickscans are related to one or more criteria of the AISHE methodology which was answered by the coordinators. In this way it was possible to compare the opinions of students, professors and coordinators about the same subject. This is presented in Table 1.
When the opinions of the coordinators, professors and students are compared, it can be observed that the professors were the most positive group of the three because they partly agreed with most of the aspects related to the presence of sustainability in the institution. The students showed insecurity at the moment they were asked to affirm something about sustainability at UPF, showing a lack of knowledge about the subject or even about the activities that are being developed at the university. This became clear as most of the students’ answers were “neither agree nor disagree”. The more negative evaluation of the coordinators in some aspects could be a reflection of the instrument of evaluation. The coordinators evaluate the courses through the vision of AISHE, which is far more detailed than quickscan. Because of this, even if they are covering the same subjects, the instruments use different evaluation scales, which make it difficult to make comparisons.
Another aspect that can be identified by the comparison of the three responses is the unanimous opinion that internal environmental management is not being observed and respected by everybody and that even when the institution strives to implement action towards sustainability, the involvement of the employees, professors and students is ineffective both in their commitment and responsibility.
In addition, the students who graduate from the institution only have knowledge of certain aspects of sustainable development; the professors feel more secure about their level of knowledge but it depends on individual initiatives. The necessity for specialists in this area was also perceived and it is recognized that the demand for such professionals is on the increase. Currently, the graduate student does not have sufficient knowledge and only some curriculum material is concerned with sustainable development.
Definition of the priority actions to incorporate SD in the university
For sustainable development to be included in the courses of an institution, four distinct stages can be used which consist of motivation, planning, implementation and expected results. Figure 7 presents these 4 stages with their components. The motivators were applied to the case in study (UPF).
The motivators include requirements from the Ministry of Education (MEC), a Brazilian governmental body which requires that Law no 9795 for Environmental Education at the Universities is applied; the necessity of an institution to prepare a professional adequately for the work market and to prepare him/her to deal with questions related to sustainability; the decade of UNESCO for SD (UNESCO, 2004); and the EMSU event, 2008 where the UPF committed itself to applying the AISHE 1.0 tool (results obtained with this application, which demonstrated the need for better integration of the subject in the teaching practices of the institution).
The planning for inclusion of SD on the course curriculums of the institution should be envisaged as long term. They should also be implemented with interdisciplinarity and transdisciplinarity as their base.
In relation to implementation, action can be developed, not just in the university management, education, research and extension but also in university life in general. The results evoke expectation in a change in the profile of the students as a result of the inclusion of sustainability in their competencies, responsibility and personal involvement. Likewise, as the image of the Institution is improved, the better its involvement with social, economic and environmental problems through its research and extension activities.
Lozano (2006) states that “university policies and strategies must be designed to holistically integrate SD as the golden thread throughout the university system”. Thus, the first stage would be the construction of an institutional policy for sustainability, through the Vice director of teaching, which would involve the commitment of the superior levels of the university and would extend to all the institution courses, regardless of their specialty.
For a university to be able to progress in this subject, it is suggested that a system of environmental management be implemented and structured in accordance with the specific norms and in accordance with its own context, having as a general directive aspect, a clear definition of its policies; in this way education about sustainability at the UPF should make part of a programme of systematic aims and a plan of action for concrete application of them. The environmental management system should include activities directed towards teaching, research, outreach and university life.
Sustainability should not be understood to be a specialist subject but should be incorporated into the practices of each course. This does not mean to insert into the curriculum a subject about environmental issues or sustainability but it means introducing an attitude and behavior pattern into each student as a future member of society and a future professional.
This paper showed a case study in which institutional approaches to sustainability are described in an Brazilian University. Even as, the article performed an evaluation of sustainability in the curriculums of the UPF and through this study, showed that the university faces many difficulties to include this theme of sustainable development in a satisfactory manner in its curriculums; a reflection perhaps of a lack of policy directed towards sustainability in its academic and operational practices. This case study, the UPF could contribute to knowledge for other universities, especially in Brazil, or even in other contexts where there is on going discussion about the incorporation of SD in their curriculums.
It has been observed during the research that the introduction of SD curriculums should be interdisciplinary and transdisciplinary. However, this is not sufficient; SD should also permeate into the research and extension activities and general university campus life. Only in this way, through daily practice in the academic activities of students, professors and employees, will SD be incorporated in the professional preparation of these individuals.
In relation to the application of AISHE for the course coordinators, it has been noticed that on numerous occasions, difficulties arise in the understanding of certain terminology used in the questionnaire, a fact that corroborates the results of other research, which may well reflect a general lack of familiarity in some areas.
Considering this, it can be confirmed that the 30 courses which were evaluated at the University of Passo Fundo using the AISHE tool have been shown to be at the beginning of a process of insertion of sustainability into their curriculums.
At the moment of publication of this paper, a new version of AISHE, called AISHE 2.0, is about to be published after a careful international development process. Whereas the first version of AISHE paid ample attention to the education within a university and the underlying mission and philosophy, AISHE 2.0 is also dedicated to the other roles of a university: the research, the operations and the societal role (the community outreach). These four main roles together form four modules, to which a fifth (or rather zero-th) module, ‘Identity’ is added, forming the new assessment tool.
The repeated application of AISHE 2.0 in an iterative process will allow a university to reach and verify the state of system integration of sustainable development (SISD), not only considering its education and its mission, but in all elements of the university activities and in its very identity. For this reason, it is recommended for the University of Passo Fundo to start using AISHE 2.0, as this will allow the institution to reach the full state of SISD.
Adomssent M, Godemann J, Michelsen G: Transferability of approaches to sustainable development at universities as a challenge. Int J Sustain High Educ 2007,8(4):385–402. 10.1108/14676370710823564
Disterheft A, Da Silva Caeiro SSF, Ramos MR, De Miranda Azeiteiro UM: “Environmental Management Systems (EMS) implementation processes and practices in European higher education institutions – Top-down versus participatory approaches”. J Clean Prod 2012, 31: 80–90.
Ferrer-Balas D, Bruno J, de Mingo M, Sans R: Advances in education transformation towards sustainable development at the Technical University of Catalonia, Barcelona. Int J Sustain High Educ 2004,5(3):251–266. 10.1108/14676370410546402
Ferrer-Balas D, Buckland H, Mingo M: “El rol de la universitat a la societat en el desenvolupament sostenible des d'una perspectiva de canvi sistèmic: el cas de la UPC”. In 1r Congrés UPC Sostenible 2015, Barcelona, Spain. Barcelona: Centre per a la Sostenibilitat; 2007:247–250.
Ferrer-Balas D, Bucland H, Mingo M: “Explorations on the University’s role in society for sustainable development through a systems transition approach. Case-study of the Technical University of Catalonia (UPC)”. J Clean Prod 2009, 17: 1075–1085. 10.1016/j.jclepro.2008.11.006
Halac R, Schiller S, Venturini E: “Sustainable Universities: New Knowledge and Innovative Actions”. In The 2005 World Sustainable Building Conference, Proceedings. Tokyo, Japan: SB05Tokyo; 2005:2316–2322.
Lambrechts W, Mulà I, Ceulemans K, Molde I, Gaeremynck V: The integration of competences for sustainable development in higher education: an analysis of bachelor programs in management. J Clean Prod 2012. doi:10.1016/j.jclepro.2011.12.034
Lidgren A, Rodhe H, Huisingh D: A systemic approach to incorporate sustainability into university courses and curricula. J Clean Prod 2006, 14: 797–809. 10.1016/j.jclepro.2005.12.011
Lozano R: Incorporation and institutionalization of SD into universities: breaking through barriers to change. J Clean Prod 2006, 14: 787–796. 10.1016/j.jclepro.2005.12.010
Lozano R: Diffusion of sustainable development in university curricula: an empirical example from Cardiff University. J Clean Prod 2010, 18: 637–644. 10.1016/j.jclepro.2009.07.005
Lozano-Garcia FJ, Kevany K, Huisingh D: Sustainability in higher education: what is happening? J Clean Prod 2006, 14: 757–760. 10.1016/j.jclepro.2005.12.006
Roorda N: AISHE-Assessment Instrument for Sustainability in Higher Education. Amsterdam: Stichting Duurzaam Hoger Onderwijs (DOH); 2001.
Roorda N: Policy development for sustainability in higher education – results of AISHE audits. In Higher Education and the Challenge of Sustainability: Problematics, Promise, and Pactice. Edited by: Corcoran PB, Wals AEJ. Dordrecht: Kluwer Academic Publishers; 2004:305–318.
Roorda N: “Assessment, policy development & certification of education for sustainable development: AISHE 2.0”. In Environmental Management of Sustainable Universities - EMSU, Barcelona, Spain, 2008, Conference contributions. Barcelona: UPC/UAB/RCE; 2008:493–502.
Roorda N: “Sailing on the winds of change: the odyssey to sustainability of the Universities of Applied Science in the Netherlands”. In PhD Dissertation. Netherlands: Maastricht University Press; 2010.
Shriberg M: Institutional assessment tools for sustainability in higher education. Int J Sustain High Educ 2002,3(3):254–270. 10.1108/14676370210434714
Tauchen J, Brandli LL: A gestão ambiental em instituições de ensino superior: modelo para implantação em campus universitário. Gestão Produção 2006,13(3):503–515.
United Nations Educational, Scientific And Cultural Organization (UNESCO): “Education for sustainable development. Draft international implementation scheme for the UN decade of education for sustainable development (2005–2014)”. 2004.
Velazquez L, Munguia N, Shanchez M: Deterring sustainability in higher education institutions. Int J Sustain High Educ 2005,6(4):383–391. 10.1108/14676370510623865
Velazquez L, Munguia N, Platt A, Taddei J: Sustainable university: what can be the matter? J Clean Prod 2006, 14: 810–819. 10.1016/j.jclepro.2005.12.008
Weenen HV: Towards a vision of a sustainable university. Int J Sustain High Educ 2000,1(1):20–34.
The authors would like to thank CNPQ and FAPERGS for their financial support for the project, for their productivity research grant and the grant for scientific initiation.
The authors declare that they have no financial competing interests and non-financial competing interests (political, personal, religious, ideological, academic, intellectual, commercial or any other) in relation to this manuscript.
LLB participated in the methodology choice and conception of the manuscript, analysis of the data and reviewed the paper before publication and the research coordination, MALF participated in the methodology choice and conception and reviewed the paper before publication and made some contributions to the text. NR helped on the implementation of the tool, analysis of the results. KTG helped on the acquisition of data and analysis of the results, LCV helped on the acquisition data, analysis of the results and graphical content of the paper. All authors read and approved the final manuscript.
Electronic supplementary material
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Brandli, L.L., Leite Frandoloso, M.A., Roorda, N. et al. Evaluation of sustainability using the AISHE Instrument: case study in a Brazilian University. Braz J Sci Technol 1, 4 (2014). https://doi.org/10.1186/2196-288X-1-4