Abstract

We have investigated the performance and achievement of precocious young persons in school and at work (Stamm & Stutz, 2007), and of gifted trainees during their apprenticeship (Stamm/ Müller/ Niederhauser, 2006), and we have shown that the choice of indicators to be used in measuring those constructs has a determining influence on the resulting appraisals.

For instance, whereas self-assessment data indicate increasing achievement throughout the apprenticeship of gifted trainees, reliance on third-party assessments (e.g., the trainees' masters) would indicate stagnation once a certain, high level of achievement has been reached. What does this discrepancy tell us about the validity of the assessment methodology?

Methods

The method we used to measure the development of the trainees' achievement is based on a design by Ripper and Weißschuh (1999), originally aimed at technical apprenticeships and now commonly used in the field and in research. (cf. Stein, 2003).

Although the requirements of any two occupations can differ greatly, Ripper and Weißschuh specify a set of competencies that they claim are of central importance for every occupation. These include methodological competencies, such as the successful application of expert knowledge; social competencies, such as conflict-resolution skills; and so-called self-competencies, such as autonomy.

The measurement method used in both of the aforementioned studies includes twelve items that may be used as the basis for both self assessment and third party assessment. However, aside from the issue of whether this short list is generally valid, the instrument suffers from validity and reliability problems that become especially poignant when the subject of the assessment is gifted:

How accurately can a master appraise the performance of a gifted trainee whose domain-specific skills rival or surpass her own?

How can we be certain that the self assessment is not, whether consciously or unwittingly, a degenerate case of third-party assessment by proxy?

How can we be assured of the integrity of the assessment when the parties are clearly aware of the bearing of the reports on their career opportunities and/or professional standing?

Achtenhagen and Baethge (2007) approach this complex set of problems within the context of VET-LSA, an initiative for the development of a framework for the comparative study of vocational education and training (VET) in Europe. They group competencies into basic competencies, such as reading, writing, mathematics, and problem solving (as in the PISA study); cross-occupational competencies, including the ability to meet the demands of today's labor markets, organizations, and workplaces; and domain-specific competencies, including the ability to apply knowledge and experience in an actual occupational situation. To measure achievement, they (like Ripper and Weißschuh) rely on the concept of key competencies, but they broaden the concept of competencies and (unlike Ripper and Weißschuh) do not assume that the importance of each key competence is the same for every occupation.

Along similar lines, but from a somewhat different vantage point, a department head has declared that '[C]ompetencies are observable forces in practical situations that are managed successfully and which may be accredited in an observable and measurable qualification process. Behavioral dispositions, expert knowledge, attitudes, etc., are (only) preconditions for these competencies.' (Stalder, 2008)

And, underlying these theoretical proclamations, there is the tacit concern that, in practice, achievement is still sometimes assessed not in terms of competency, but rather in terms of perceived efficiency and efficacy.

Results

Against this backdrop of differing understandings of achievement, diverse occupational contexts, and heterogeneous vocational education infrastructure, the following questions will be addressed in the workshop:

• How might one choose a working definition of, and an approach to, measuring achievement for a given context?
• How might one determine which competencies correlate with achievement in a given context?
• What means of assessing competence - besides standardized aptitude tests - are available or suitable for a given context?
• What methods - beyond the standardized questionnaire - are suitable for my project?

Special consideration will be given to the following ancillary questions:

• Do the answers to the above questions facilitate or hinder a comparison of results across occupational and national contexts? (Brockmann et al.; 2007)
• What are the implications of the resulting methodology for the case of gifted young persons in a vocational education program?

Bibliography

• Achtenhagen, F. & Baethge, M. (2007). Large Scale Assessment for Vocational Education and Training (VET-LSA). http://www.uni-goettingen.de/en/71738.html [Retrieved 15.2.2008].
• Brockmann; M.; Mehaut, Ph.; Hanf, G.; Westerhuis, A.; Clarke, L.; Brockmann, M.; Winch, C. (2007). Cross-national equivalence of vocational qualifications and skills. Papers for a Symposium at ECER 2007. http://www.vet-research.net/ECER_2007/sep20/session_5a/ [Retrieved 15.2.2008].
• Ripper, J. & Weissschuh, B. (1999). Ausbildung im Dialog. Das ganzheitliche Beurteilungsverfahren für die betriebliche Berufsausbildung. Stuttgart: DaimlerChrysler AG.
• Stalder, M. (2008). Höhere Berufsbildung 'eine Erfolgsstory, die es noch zu entdecken gilt'. Referat im Rahmen der Informationstagung der Ausbildungsverantwortlichen der Sportverbände. http://www.baspo.admin.ch/internet/baspo/de/home/themen/ausbildung/trainerbildung/infos_verbaende.ContentPar.0004.DownloadFile.tmp/H%C3%B6here%20Berufsbildung%20deutsch.pdf [Retrieved 15.2.2008].
• Stamm, M., Müller, R. & Niederhauser, M. (2006). Hochbegabt und nur' Lehrling? Eine empirische Studie zu den Ausbildungsverläufen besonders befähigter Jugendlicher im Schweizer Berufsbildungssystem. Schlussbericht zuhanden der KTI-Berufsbildungsforschung.
• Stamm, M. & Stutz, M. (2007). Die Wirkungen von vorschulischen Kompetenzen auf die Berufsausbildung. Eine Längsschnittstudie zu den Schul- und Bildungslaufbahnen von Jugendlichen, die bei Schuleintritt bereits lesen und/oder rechnen konnten. Bericht vom Mai 2007. Fribourg: Departement für Erziehungswissenschaften.
• Stein, M. et al. (2003). Berufliche Begabung erkennen und fördern. Der Modellversuch 'Leistungsstarke Auszubildende nachhaltig fördern (LAnf)'. Bonn: Bundesinstitut für Berufsbildung. Heft 67.