Statistically, getting less than 25% in a multichoice test should not happen

[simon43]

I inherited my Mathayom classes from the previous teachers who were asked by immigration to make a quick exit

This week was mid-term test time, and I checked the test papers that had been previously prepared by my agency for the students. I corrected some typos and grammar errors, and also commented to my agency that in the short time that I had been teaching the students, I realised that their English language ability was basic, and that most of them would fail the exam - miserably.

But since it was a 4-choice, multiple-choice paper, I consoled myself that if the students just shut their eyes and chose blindly, then statistically, they should get 25% of the answers correct.

The students did the test - there was no copying or cheating.

I marked the papers today. <deleted>!! How can almost the entire class achieve no more than 15% correct answers?

Do Bell curves and probability theory etc not work like in the West?

[Rayk]

"How can almost the entire class achieve no more than 15% correct answers?"

Easy. They answered 85% of the questions incorrectly.

[gandalf12]

It is Probability which is good for a very large sample. You had a tiny sample which is why it did not work.

[impulse]

If you do grade on the same curve as Miss Heinlein at Normady Villa Jr High, with 10% A's, 20% B's 40% C's, 20% D's and 10% F's, does that mean someone scoring 10% correct may pass with a C?

That may explain why I got 2 hamburgers and no drink to go when I ordered a hamburger and 2 waters to eat in.

[stubuzz]

I marked the papers today. <deleted>!! How can almost the entire class achieve no more than 15% correct answers?

If the whole class is getting low grades, the teacher should really start by evaluating themselves.

[Rhys]

I marked the papers today. <deleted>!! How can almost the entire class achieve no more than 15% correct answers?

If the whole class is getting low grades, the teacher should really start by evaluating themselves.

Student Iodine deficiency... a possible cause..

[aidenai]

There's a mismatch between the content of the test and what the students have actually learned in class.

[SlyAnimal]

Do Bell curves and probability theory etc not work like in the West?

Bell curves are distributed around the mean, and then the brackets of students are generally defined by how many standard deviations they are from the mean, not around a set score. However some schools will define a set score/grade that they want to give and then map student's positions within the class onto those grades (e.g. top 5 get As, 6-10 get Bs, 11-15 get Cs etc), this isn't really how bell curves work though (And is more akin to dividing the class into quartiles or quintiles). Bell curves are also sometimes used by schools in order to scale grades up/down if the tests are too difficult/easy by moving the mean and scaling grades in accordance with a bell curve.

The first time that I wrote a maths exam, I had a similar experience. The median was around 35% (It wasn't multi choice), with only 3x students passing the exam (One of which got 98%!!). All of the questions were taken from the homework that the students had been assigned (And modified slightly), however it turned out that almost all of the students had just been copying the 98% student's homework and so while I was under the impression that they were all capable students when they most definitely weren't. I've since modified how I do things (Since if the entire class fails, it's upto the teacher to change, rather than the teacher to simply expect the students to magically improve), and so now most classes would likely have a mean of around 65% and a distribution of scores that should fit onto a bell curve.

But yeah, if you want the students to pass the tests, then you need to either:

1/ Teach better (There is always more to learn and ways to improve).

2/ Write the tests yourself and set them to an appropriate level.

3/ Teach to the tests, so in addition to ensuring that you've taught the content that will be covered in the tests, try to give examples that, while not the same as the ones in the test, are similar enough that the students will at least have a fighting chance. As there is a huge amount of vocabulary in English, and often the vocab "surrounding" the vocab that's being tested are unfamiliar to students and throws them off (e.g. The question might center around the correct use of win, won, winning, but if the rest of the vocabulary in the sentence is unfamiliar to the students it will throw them off, even if they know how to use the correct tense).

Alternatively, get used to scaling up your student's scores, since you can fail a few students, but not the entire class (If you do, it's a reflection on the teacher/test, rather than the students, and the agency will likely pass the buck onto you, rather than taking that on the chin themselves).

Edited July 14, 2016 by SlyAnimal

[Reigntax]

The OP is correct. Selecting any group of people who have not even studied for the test, nor even understood the content, the average score of the group achieved by random answers should have been 25% plus a standard deviation.

An average of 15% even achieved by a small group would be highly unusual without other external factors especially considering that any student would generally know the answers to at least some questions.

Maybe the just failed the subject of " how to answer multiple guess exams"?

The results suggest they know less about the subject than any random person who has never studied nor been taught the subject. Now that is hard to achieve.

[muzmurray]

Interesting read.

http://alohonyai.blogspot.com/2013/08/guessing-on-multiple-choice-tests.html

[gandalf12]

There's a mismatch between the content of the test and what the students have actually learned in class.

Didn't thnik you could put questions about Candy Crush on an exam paper

[maewang99]

there's quite a few more surprises in store for you if even this puzzles you. enjoy them while they are still at least a little bit humorous, in some way.

Edited July 15, 2016 by maewang99

[wotsdermatter]

There is a difference between statistics and reality in Thailand. No name but it is located near one of the world's top hotels. A friend and I co-taught at one of Thailand's "top" high schools for a year. Everything for their first test, only 25 questions, multiple choice, was from our teaching, although we did add in one question and we provided an answer choice of MICKEY MOUSE, which was utter rubbish. (We were astonished how many answered that question with MM as their answer.) No student received a mark higher than 5 for the test. The head of the department, a Thai, had been shown the test and approved it. He was astonished, disappointed, and upset at the students because of the results. C'est la vive.

[impulse]

Just out of curiosity, the test scoring didn't deduct points for wrong answers, did it?

Back in the days of slide rules when I grew up, we had tests that would deduct points for wrong answers, calculated so that anyone randomly guessing would get a zero, not 1/x with x being the number of choices on each question.

They used to coach us to guess if we could eliminate some of the choices as being nonsensical. Statistically, that would have improved our scores, and getting into a good college was all about the scores...

Edited July 15, 2016 by impulse

[simon43]

If the whole class is getting low grades, the teacher should really start by evaluating themselves

A valid statement, if I were their teacher and if I wrote the test paper. But I am the replacement teacher and I didn't write the test paper. I know that the paper was written to meet the learning objectives of that grade. But realistically, that doesn't mean that the students have achieved those learning objectives.

Some lower grades did OK in the test. But the M4, M5, M6 grades had very poor results. So this gives me a challenge in my teaching plans

[MissAndry]

Design future exams so that 40% of the correct answers are 'a'.

Students don't guess at random, they choose 'a' when they don't know.

[partington]

This conclusion is flawed though. Students choosing answers at random should on average get 25% right, given a large enough group. However two things have to be borne in mind:

1. These students are not making random choices, they are making directed choices, and if these are based on errors of understanding, or superstition ( the answer is right if it has the number 4 in it) they will result in lower than randomly selected scores, because they aren't "random". In the same way humans asked to write a list of random numbers will almost never repeat the same numbers next to each other, for psychological reasons, but this will happen fairly frequently in machine generated number lists, which are truly random.

2. Even in the case of random choices, probability is chance: it is not guaranteed. This single result could be a statistically improbable, but completely possible, outlier result. In this case doing the test over again would result in the well known "reversion to the mean" effect, where repeats become closer to the statistically expected result simply because the first result was an unlikely one (like getting heads 5 times in a row when you toss a coin-it can happen but usually doesn't).

[Briggsy]

Following on from and with reference to part 1 of the excellent post above, many multiple choice question compilers use the following technique to check understanding and weed out clueless guesses.

They use a key 2 or 3-word phrase in the question and then repeat it in one or two of the four possible answers. These answers will be wrong using this technique.

Your students, who appear out of their depth may have fallen for this very basic trick. Thus, the examiner has successfully demonstrated which students have little or no understanding. Unfortunately, it is almost all of them.

The result of only 15% correct answers may well be due to this. The students have been successfully misdirected due to their lack of understanding.

p.s. Are you quite sure there was no copying?

[MissAndry]

^^ People who make up exams in Thailand, don't think that much.

(Thai or foreigner)

[Briggsy]

^^ People who make up exams in Thailand, don't think that much.

(Thai or foreigner)

^^ People who make up exams in Thailand like to copy them off the internet

[Briggsy]

Another possible theory to explain the OP's surprising results is that his answer key / stencil is incorrect and thus despite the average score being over 25% as expected, the stencil gives an average score of under 25% because it refers to a different test.

[Scott]

Off-topic posts removed.

[gk10002000]

Unusual for sure, but certainly possible. My experience in looking over many of the English tests is that I would likely fail them also. It has been decades since I could speak to the technical details of English as far as past participles, passive voice definition, etc. I no doubt forgot that as a youngster in English classes here in the USA we were taught those things, but frankly all I remember that was useful and that stuck a little bit were my Junior and Senior High school composition and writing classes. Fortunately, my parents and all the kids in the neighborhoods I hung out with wrote and spoke much better than the average drop out.

Edited July 19, 2016 by gk10002000

[puukao]

A kitten is like a cat like a puppy is like a __________

a. purple dinosaur on crack

b. invisible cloak from harry potter

c. banana under a bus falling off a cliff

d. dog

[BruceMangosteen]

The correct answers also have to be random and most farangs us "A" as the correct answer. Filipino's like "C" as the correct answer. In addition, "D" will often be "all of the above" or "none of the above" which is common for the English subject exams. The best students will choose All of the Above if he/she knows two of the choices are correct or good enough answers, even if they know the third is wrong. Two out of three means All of the above, not "best" of the two correct answers.

As for "five" in a row with a coin flip, that theory has long ago been debunked. There are many occasions on a roulette table where five red/black/odd/even/high/low come in sequences of five or more in a row. The infinity theoretical application doesn't apply to roulette or English exams in Thai Government schools. O O

[Rhys]

Consider that the student is so deep into thought...they mark the test and forget to circle the answer key.

[Emster23]

True-false test only test I can think of that is worse than "multiple guess".

Back when I was student teaching, I got an old IQ test that was multiple guess. I gave out the quiz answer sheets. "Where's the questions?" students asked. "Ah, don't worry about it, just mark the paper".

I got the sheets back, and returned results for IQ per test. Ranged from around 50 to 140. The aspect that surprised me was some students bragging about their high scores, others dejected at their low scores.... and so it goes

[how241]

gandalf12, on 14 Jul 2016 - 18:14, said:

It is Probability which is good for a very large sample. You had a tiny sample which is why it did not work.

Yes, correct. The same way that you can flip a coin and get heads 8 times in a row, when it is a 50-50.

[Suradit69]

I consoled myself that if the students just shut their eyes and chose blindly, then statistically, they should get 25% of the answers correct.

That's a bit like saying if you flip an "honest" coin 10 times you should get 5 heads. Everything from 0 to 10 heads could easily come up. If you manage to flip the coin an infinite number of times ... which you can't ... getting 50% heads is a certainty.

Presumably your students aren't actually blindly ticking answers so their choices are not truly random. And the number of students is slightly less than infinite. Coming up with a result several standard deviations off in either direction from μ is undoubtedly frustrating but not surprising. 68% of event results will be within +/- one standard deviation from μ ... which means 32% event results will be even more than one standard deviation from μ .

Edited July 19, 2016 by Suradit69

[Ulic]

When I was in school there was a correction factor in multiple guess tests. So if you were

completely guessing you would get a zero. The teachers did not want you guessing. Of

course you could easily eliminate the "stupid/joke" answer, leaving three options. And

another that was unlikely, so after that it was 50-50.