THOUGHT PROVOKER 13

Let's Tell The Evaluators

Agood question here. A graduate student writes in with the following
request. Read it and respond back on what you feel we need to tell her.
Let's try and nail this one down, once and for all.

"Dear Robert

I would appreciate it if you could put the following forward as a
thought provoker. If you have comments of your own, or know of anyone
else who may be knowledgeable in this area, I would welcome any
advice/information that could be offered to me.

Thank You"

Lynsey Dobbie

Thought Provoker:

I am a student at The Queen's University of Belfast in Northern
Ireland studying for a Masters in Occupational Psychology. I am
currently working on a project exploring psychometric testing of
candidates with visual impairments, with specific reference to timing
issues.

I am interested in finding out about your experiences of taking
psychometric tests, whether for selection or other purposes. I have
prepared some questions, however, any other insights would also be
greatly appreciated.

What have been your specific needs when taking psychometric tests ?

What are the main problems you have encountered when taking these tests?

What sort of time limits were you given?

Was the time limit sufficient?

Do you think candidates with visual impairments should be given
unlimited time or is this counterproductive?

I would be very grateful if you could also tell me your age and briefly
tell me about the nature of your visual impairment, your education
background, work experience, the types of tests you have taken and the
format you have taken them in.

Any thoughts will be gratefully appreciated and very valuable to this
piece of research.

I look forward to hearing your views.

Lynsey Dobbie"

In explaination: The psychometric tests, the tests are
either verbal or numerical in nature. They are standardized tests used
for the purposes of selection. The tests are more often than not in the
form of multiple choice, where the question is read from an answer book
and answers are written on a separate booklet. The Civil Service
Commission in Ireland, who I am carrying this piece of research out
with, currently offer the tests in Braille, large print and computer
synthesized speech (HAL).

Standardized tests compare an individual's performance with the
performance of a representative sample group. The tests are timed,
although candidates (including sighted candidates) are not necessarily
expected to finish the tests within the allocated time.

When such test are used appropriately, they can provide significant
information about the individual being tested. However, if standardized
tests are used inappropriately (for some reason other than for what
they were developed) test results may actually screen out individuals
seeking job possibilities. Therefore, it is essential that standardized
tests be administered by someone who has formal training and is
licensed to administer such tests. It is equally important that the
evaluator understand the specialized issues relative to visual
impairment.

As I mentioned, I am particularly interested in the time limits given
to candidates with visual impairments."

RESPONSES
e-mail responses to newmanrl@cox.net

**1. " HI, I'd be very interested in getting copies of the answers that you get, I am on a committee in my college, in charge of determining which if any special considerations students should receive as a function of their specific disability in order to insure that they can learn and compete on equal footing with students who have no such disabilities. Thanks

Tirzah Horminer (Alon Shvut, Israel)

FROM ME: "Picture the globe; from Ireland to Israel we fine evaluators who are interested. I'm sure others will log in as they log on."

**2. " Psychometric measuring of blind persons is hardly new. Around 1955, when I was still in Vancouver, British Columbia, and an under-graduate at the University of British Columbia, some staff person at the Canadian National Institute for the Blind thought it would be interesting to find out if the aptitudes of blind persons corresponded to what they might rationally expect to be able to do. The test consisted chiefly of some three hundred questions in the form: "Would you sooner be a cow grazing in the meadow or an alien in a spacecraft approaching the strange planet Earth or a Chinese philosopher meditating as you floated down the Yangtse River." (That wasn't actually one of them, but you get the idea.) Each of us was then scheduled to meet with the industrial psychologist who worked with the agency person. My interview resulted in several interesting conclusions on the part of the fellow: 1. You have a very analytic mind and should consider being a lawyer; 2. You have a tendency toward sarcasm; and 3. You have a somewhat above average "psychiatric deviate" and that meant I was at a higher than usual risk to have a sudden dramatic departure from ordinary behavior and do something outrageous. Well, I didn't become a lawyer and have never had one of those sudden outbreaks and those who know me will affirm that there's not an ounce of sarcasm in my system. Incidentally, near the same time as these tests, Queens University, Belfast, sent a rugger team of theology students to play against U. B. C. and they beat the hell out of us. The same night I met them at the Royal Vancouver Yacht Club and they were all drunk as--pardon the expression--lords. The moral of both stories is that generally things don't work out the way your measurements or stereotypes might predict."

James Nyman (Lincoln, Nebraska, USA)

3. "Besides being a lawyer I also have an MA in counseling, so it is an area in
which I have some training and experience and is also near and dear to my
heart.

In thinking about psychometric tests one needs to know the difference
between a test and an inventory. A test is a measure of one's knowledge,
skill or ability to perform. It is designed to elicit both right and wrong
answers. An inventory is not designed to have right and wrong answers, but
to give the taker an array of equally valid choices. The choices one makes
will presumably tell us something definite and knowable about the taker.

Most psychometric tests are in reality inventories. They are designed to
evaluate such things as likelihood of mental illness, personality traits,
career preferences, etc. The purpose is to give the trained evaluator the
same quality of information that that evaluator would get if he or she had
conducted extensive one on one interviews with the taker.

In order to make sure that the quality of the test results is high, test
developers will routinely do validity studies. This usually involves giving
the test (inventory) to a large number of subjects under controlled
conditions and checking the results against information about the subjects
that has already been established as accurate.

In order to make sure that the test is measuring what we think it is and not
some other variable, other factors are rigorously excluded from the
validation studies. One of the variables is testing conditions.
Individuals who are not able to take a written test under timed conditions
are systematically excluded from study groups, which is why you will often
get a disclaimer about the validity of the results when the testing company
knows that the taker has taken the test orally, by Braille or large print or
untimed.

With that said, it is ultimately impossible to control for all the variables
in social science research as one can in a physics or chemistry experiment.
The way social scientists deal with this is to require that a test only be
accurate enough of the time that we can say with some statistical certainty
that it is not due to random chance. This means that the test may be
considered valid even though it misses the mark completely with a certain
number of people.

Some African-American and ethnic activists have criticized psychometric test
validation studies as failing to take into account cultural differences in
interpreting test results. In the studies that have been conducted in this
area, there appears to be something to this. We have no data on this with
respect to blind people. We assume that blind people mirror the attitudes,
beliefs and expectations of the ethnic and cultural groups from which they
come, where it may be that the experience of living as a blind person in our
society makes enough of a difference to warrant drawing different inferences
from test results. It is worth further study."

Tom Elkins
(THOMAS C. ELKINS (BlindLaw, >
(773)477-1440; Fax (773)477-1207)

**4. "No matter which way a blind person takes the test, they will need more time. Meaning if it was normed on people using the visual medium, doing it in any alternative method means the loss of standard results. Think of it this way, if the test had been normed on blind persons reading Braille and if you gave it to sighted people who didn't use Braille and forced them to take it in print or verbally, then you'd have the same non-standard results.

What I think is best is give the blind person all the time she or he wants; let them choose to finish the test or not. But let them go far enough to get a reading on what they know or feel or what ever else you are looking for. The timed factor is not as important as the rest.

I've taken all manner of tests and have done them on the computer, read out loud, on tape and in Braille. Aloud was a pain, open to the reader's interpretation. I'm not the fastest reader of Braille and that was tiring (should have gotten better with it when I was younger). Computer was where I best had control.

I'm totally blind, female, am 48 years old, a college graduate (with a masters in social work).

**5. " What have been your specific needs when taking psychometric tests ?
Experienced readers to read the test materials and fill out the response
forms. Also an increase in the allotted time to allow for the time it takes
to have the test read, give the answer back to the reader, and the reader to
fill out a response form.

What are the main problems you have encountered when taking these tests?
The main problem is having a reader that can smoothly verbalize the
material.

What sort of time limits were you given?
Typically I was given double the allotted time which was given to a sighted
test participant.

Was the time limit sufficient?
I felt the time limit was sufficient.

Do you think candidates with visual impairments should be given unlimited time or is this counterproductive?
I do not believe that an unlimited time frame is necessary for a fair test
administration.

I would be very grateful if you could also tell me your age and briefly
tell me about the nature of your visual impairment, your education background, work experience, the types of tests you have taken and the format you have taken them in. I am a 28-year-old college graduate holding a BA in speech communication. I
currently work as a PBX operator for the world headquarters of Alcon
Laboratories. The tests that I have experienced were those given to
potential job candidates. They were all multiple choice scantron form
tests."

Wendy McCurley (USA)

**6. "How about classes over the Internet?"

The Profit

**7. "I hesitate in some ways to respond to this issue, but on the other hand I
feel there are some things to be said. Of course, here in the United States
we allow time and a half for any type of testing related to entering either
educational situations or employment, which must be looked at from two
points of view. On one hand there are simple physical difference in the
manner in which blind individuals obtain information, and in the case of
many blind persons, for whom the use of Braille or other alternatives is
less familiar or less effective than the use of print is to their sighted
counterparts, more time during testing will allow the individual's true
potential to be evaluated. On the other hand, the blind individual should
be expected to function at a competitive level if he or she is to make the
type of contribution society is expecting of either a student or a worker.
Both of these arguments are valid, and this fact points out an underlying
issue that is not being considered in this whole process, and that this form
of testing is not a truly appropriate method of determining the fitness of
any person for employment in a specific field, but rather for determining
who is less fit for such a position. Therefore, such testing is by its very
design a method of discriminating against those the educator, employer, or
evaluator perceives as being unfit for such education or employment. I am
certain that the majority of persons involved in creating or administrating
these tests do not perceive them in such a manner; however, I would point
out that if these test do not function in this manner, then why do they not
function to direct the majority of the individuals that complete such a
testing process to the specific types of careers that they will occupy
during the course of their lives. I believe that the use of such testing
methods in determining fitness for either educational advancement or
employment is as likely to reinforce preconceptions as it is to reveal human
potential. In other words I believe that this is more likely to prevent the
right person from being placed in the appropriate position than it could
possibly improve the odds that a poor educational or employment match will
be avoided. I have been involved with the hiring process enough to know
that the most effective approach involves exploring the individual's
experiences, interests, and that person's reaction to having the reality of
the situation, whether educational or in regard to employment, presented as
straight forward as possible. A decision is made on this basis, and the
testing of fitness comes after. Not testing in the normal sense, but on the
job training and evaluation, and the appropriate persons are retained, and
those that do not fit the desired role are dismissed. I believe that before
Psychometric testing is viewed as a desirable tool, a careful study of the
long term job performance of those selected upon the basis of this testing
for employment should be conducted, and a follow-up on those turned away by
it should also be pursued. I would not say that there is not value in
psychometric testing, but I believe that there is also great danger. When it
is used as a determiner of fitness. When it is offered as a tool for the
individual to utilize in determining personal interests and abilities, then
it provides a great benefit, and could be a far better means of weeding out
those that do not fit the desired role and assuring that the right person is
found."

Jeff Altman (Lincoln, Nebraska, USA)

**8. "I am thirty-seven years old and I have a BA in music. I am a registered
music therapist, working part-time at a nursing home and volunteering at
various facilities in the community that serve senior citizens.

I have not yet taken any psychometric tests but I feel that some of the
issues discussed here apply to all tests. It is important to have good
readers and visually impaired candidates should not be given time limits
because it may be necessary to have a question (and in the case of multiple
choice questions the possible answers) read more than once, which may take
longer. If the topics covered in these psychometric tests are related to
the job being sought, these tests probably are good tools for determining
who is qualified for the job."

Abbie Johnson (Sheridan, Wyoming, USA)

**9. "The best determiners of what is needed at test time for any specific visually impaired student is the student themselves! Most VI students know what they need and will advocate for themselves. However, in general; 1. For the partially sighted, we mostly use various sizes of large print. This can be created cheaply and quickly by use of a good copy machine; needing good contrast, black on white (a general rule). Also good lighting, either natural sun light or high intensity lighting. They will also, in the main, be slower in reading and need more time.
If the exam is presented upon a computer screen, then measures will be needed to accommodate the student. Either a larger monitor or software that enlarges the font or just larger fonts.

2. For the person who can not read print, they may wish either Braille, tape or a live reader. If the test is normally given or is requested to be given upon a computer, a screen reader will be needed (speech for the computer). Also, additional time will be needed.
Double the allotted time is enough. Though it time factor can be also individualized for each student.

I may also bring up a minor point that may or may not be of significance. If the psychometric testing is normed upon the "normally sighted" person who has lived a "normal sighted" type of life, then if the blind person has not had as similar of life to the time of testing, then they may not score the same, yet may really be suited via a different set of life's experiences. This argument is the same for some one who is black or American Indian or Asian and is taking a test normed upon the average white person in the USA. So cultural differences may give a false reading.
I am female, 32, unemployed, have a college degree in psychology, " have taken psychometric testing for employment and have taken most other types of testing tools. My favorite method of testing is essay."

Diana Webber (Geneva, Wisconsin, USA)

9. "I agree with number 8 and several others where the message is when using a test to determine “suitability” the test better be normed with your characteristics in mind. Or its going to misread qualifications. Do something practical, functional relating to the duties of the job.”

**10. “Give any student or testie the right tools, time and they will all test well.”

**11. "Personal note to your readers; I have known Robert, now for
better than five years, and has helped me with many problems,
personal and visual. I am a 44 year-old male, who has held jobs
in the restaurant field, electronics, and most of my jobs have
been in the customer service area of business. I do use adaptive
equipment daily.

The letter from Lynsey, interested me so, I thought I'd put my two
cents worth in. I broke it down into an eight question's format.

1. The word Psychology threw me a bit, so I looked in my American
Heritage Dictionary and this, is what I found. Another word I
wanted the definition for, was psychometric which I found to be:
     Psychology: The branch of metaphysics that studies the soul, the
mind, and the relationship of life and mind to the functions of the body.
     The branch of psychology that deals with the design,
administration, and interpretation of quantitative tests for the
measurement of psychological variables such as intelligence,
aptitude, and personality traits. Deals with the design,
administration, and interpretation of quantitative tests for the
measurement of psychological variables such as intelligence,
aptitude, and personality traits
The key word is: variables and the relationship of life and mind to the functions of the body.

2. Each person is an individual so, it would be a difficult task
to fit everyone into the same mold. It would be like, fitting a
square peg in a round hole.

3. I think there is more of a need, not to be lead by the hand.
Than to do for one's self.

4. I personally don't remember taking them even though I did.

5. Time was a bit of a learning curve when I went to UNO
(University of Nebraska at Omaha), where test became only as easy
or as difficult as the proctor.

6. Most of the time, I had an hour of the class period, if
I needed more an extra half-hour.

7. It's more of an individual comfort zone, and depending on the
task at hand. If employment is the issue then the person want's
to do there task as quickly as possible. Because of the human
element.

8. Most of this question, has been answered. As to the amount
of vision I personally have, I'm considered blind."

Robin Rush (West Point, Nebraska, USA)