[Advisory] Re: [Secondary] RE: The ICT Continuum Hoax

[Ian Lynch]

On Thu, Mar 11, 2010 at 1:26 PM, Crispin Weston
<crispin.weston@alphalearning.co.uk> wrote:
> Ah-ha! The very interesting question of competencies.
>
> I think this is an important discussion because we have yet to create an
> acceptable way of codifying competencies (a key pre-requisite for many new
> learning technologies, including portfolios and systems for managing
> differentiation and remediation automatically). I hope someone from QDCA is
> listening in.
>
> I agree with Ian that competency is connected to outcome.
>
> But I would go further and define it along the following lines:
>
> A characteristic which was predictive of a person's ability to achieve a
> range of desirable outcomes.
>
> If you say that someone is "good at tennis" you are predicting that he will
> beat someone who you say is "bad at tennis". If you say the two people are
> good and bad respectively at racket sports, then you would predict the
> outcome of matches in tennis, squash and ping pong and even some new racket
> game which hasn't been invented yet. This last point is important, because:
>
> a) competencies are transferrable (which is why employers recruit graduates,
> even though they know nothing about the business in which they are being
> employed);

Some competencies are transferable in some contexts. A physics
graduate is not necessarily going to be competent in human
communication, table tennis or classroom teaching.

> b) the nature of the competency cannot be defined exclusively by the range
> of outcomes of which it is taken at any particular point in time to be
> predictive.

No measure ever can. An ICT graduate that does no practice for 10
years won't necessarily be a competent IT user.

> In other words, competency is abstract.

Uncertain I would say is a more significant property than being
abstract and the degree of uncertainty is variable. But then again an
academic exam result has similar properties, it too is uncertain - was
that a particularly good or bad day? Did the right questions come up?
How much of the knowledge needed is remembered a few months on etc.
it's fairly easy to understand that there are degrees of competence
with factors such as context making a significant difference.

> The transferability of a competency is crucial. If you define a competency
> very narrowly, as I understand a lot of vocational courses do, to say "saw a
> piece of wood in half"

Actually if you look at the QCF you will find this is not the case at all.

ITQ Bespoke software unit at Level 2

A competent person will use appropriate structures to organise and
retrieve information efficiently

Assessment criteria

B2.1 Describe what functions to apply to structure and layout
information effectively

B2.2 Select and use appropriate structures and layouts to organise information

B2.3 Apply local and/or legal guidelines and conventions for the
storage and use of data where available

That seems pretty broad to me. In some respects rather better than the
vagueness of NC attainment target statements. In fact combining both
is probably  better than either in isolation but that is not too
surprising.
	
> and you demonstrate the competency by performing the
> action, then all you get to show is that we know that x can saw a piece of
> wood in half because he has been observed sawing a piece of wood in half.

Could be important if that person is building your shed :-). OTOH, it
is inaccurate to give the impression that is typical of how good
vocational qualifications are written.

> (In fact, it might have been a fluke - he should really be observed
> performing the action several times in several different contexts). But the
> main point I am making is that these very concrete, yes/no competencies are
> in fact tautological: they say nothing

Just as passing an academic exam on a one off at the end of two years
could be a fluke? Sawing in a straight line for any distance is
unlikely to be achieved by a fluke.

> In order to *infer* an abstract competency, you must observe a range of
> concrete performances which achieve defined outcomes, preferably performed
> repeatedly in a range of different contexts. These observed performances,
> the leaves on the competency tree if you like, I would call "capabilities".
>
> You could then say that if someone demonstrated a range of capabilities,
> from sawing to screwing to drilling to sanding etc. you could *infer* a
> competency at "basic carpentry" which would *predict* that he could make a
> simple bookcase - even though no-one had ever observed him making a simple
> bookcase before. Magic!

Which is how qualifications like the ITQ are designed. There is
nothing about using a WP to write a particular document, there is
stuff about the typical things one does in a WP that if demonstrated
in a range of context would make it unlikely that the candidate could
not design that unique form or brochure. But getting practice with
real and meaningful tasks is better than doing imaginary and
theoretical ones because for most people it is likely to be more
motivating.

> Competencies form hierarchies, from concrete "leaf" capabilities through to
> very abstract "trunk" competencies, like saying someone is intelligent,
> "sporty" or "good with their hands".
>
> A further important point is that, being abstract, any measure of competency
> has a degree of uncertainty about it, illustrated by the degree of
> uncertainty in any predication which is based on the competency. I would
> call any supposed measure of competency a "competency claim", also adding
> that the confidence with which the claim is made will vary depending on the
> amount of evidence you have of its predictive reliability.
>
> How you define a particular competency (rather than the term) is the subject
> of another post.
>
> Anyone interested in this subject might like to look at the discussion I had
> a while back with some LETSI people at
> https://letsi.org/resources/Performance_Correspondence.html. The most
> interesting piece of the discussion, to my view, is with Allyn Radford,
> starting some way down the page.
>
> Crispin.
>