Statistically, getting less than 25% in a multichoice test should not happen

[Johpa]

Those who designed the test probably did not randomly, and thus equally, assign the correct answer to the four slots and, as others noted, the students tended to select one slot when they did not know the answer, such as choice 'A'.

[GOLDBUGGY]

Well it is because you are not using the "probability theory" correctly.

If your Multi-choice Test only consisted with one question, with the correct answer being A,B,C, or D, and a person were to guess, then you would be correct in saying he had a 25% chance of guessing correctly.

If you Multi-choice Test consisted of 4 questions, and you could not use the same letter twice as the correct answer, say letter "A" or "B" two times in 4 questions, then if a person were to guess "A" or "B" 4 times in a row he would be bound to get one right or 25%.

But as you know Multi-choice Tests never consist of only one question nor does the answer have to be different each time with only 4 questions. For example in only having 4 questions, if the correct answers were C,C, D, and B, and the test guesser chose "A" thinking he would get at least one correct, he wouldn't get any correct.

Look at the local Lotto Ticket for example. To get the last number correctly your chances are 1 in 10 as there are only 10 numbers to choose from (0 to 9). But what about getting the last 2 numbers correctly? Now you have 100 numbers to choose from to get 1 correct answers (00 to 99) some now the odds go down and your chances are only 1 in 100. For 3 correct numbers it is 1 in 1,000 and so on. But it is important to know that if you already have 2 correct numbers then the chances of getting the 3 number correct is still 1 in 10.

Maybe Dice is easier to work with when looking at Multi-choice Probability, What are the chances of rolling a 12 in Dice? To do that you need two 6's. So for one Dice your chances are 1 in 6 and also the same for the other Dice. So your chances of rolling a 12 is 1/6 x 1/6 or 1 in 36. Keep in mind this does not apply to the number 7 or other numbers as with two Dice there are several different ways to achieve 7. For example 6 and 1, 5 and 2, 4 and 3, 1 and 6, 2 and 5, 3 and 4. So your 1 in 36 only applies to the number 12 in Dice, and also the number 2.

So now that we know the formula on "Probability" with Dice, what are the chances of getting 12 two times in a roll? Is it 1 in 72? No, actually it is 1/6 x 1/6 x 1/6 x 1/6 or 1 in 1,296. But what if you did roll a 12 the first time, what are your chances of rolling 12 two times in a roll then? It goes back to only 1 in 36. So like the Lotto and if you have the first one correct it is easier to get the second one.

So now you can apply this "Dice Probability Formula" to a Multi-choice Test. Let's say you have 20 Multi-choice Question with 4 answers to choose from, so what are the chances of getting 25% by guessing? Well to get 25% from 20 questions he would need to have 5 correct answers. So his odds are: 1/4 x 1/4 x 1/4 x 1/4 x 1/4 or 1 in 1,024. But he also gets 20 guesses as he has 20 questions, so his actual odds of guessing 25% is 20/1,024 or 1 in 51 or about a 2% chance.

So this is how someone can get 15% in a test when you expect them to get at least 25%. To get even 15% they still need 3 correct answers which the odds say are 1 in 64 by pure guessing, but he also gets 20 guesses. So now his chances are 20 in 64 or about 1 in 3 or 32%, which still seems difficult to get an overall 15% score. But there are other things to consider first.

Firstly, even when the student doesn't know much on this Multi-choice Test chances are he is going to read all the questions at least. Usually not all the answers are logical ones. In my mind usually one answer is way off base, one (or two) a maybe, and two are close. So in reality the student really isn't guessing from 4 answers but rather 2 or 3 answers, which improve his odds.

The other thing to consider is that not all questions are difficult ones. Even if this student only knows 10%, or 2 questions right, this improves his chances a lot. Just like when you already have 2 correct last numbers in the Lotto. To now get 15% he only needs 1 more correct guess in 18 questions, and to get 25% he needs 3 more. So for a student who knows 2 questions, his chances of getting 25% in this Multi-choice Test is about 25% if all the answers are difficult, and about 50/50 is one answer is not very logical.

So in short if a students knows 10% his chances of getting 25% are about 50/50. His chances of passing and getting 10 correct answers by guessing is next to impossible. But still getting 25% when you know only 10% is a bit false. Perhaps why Multi-choice isn't the best way to test student knowledge. With long answer you either know the answer or you don't. But in an age where you get marks for showing your work and still come up with the wrong answer, who knows? .

[GOLDBUGGY]

When I was in school there was a correction factor in multiple guess tests. So if you were

completely guessing you would get a zero. The teachers did not want you guessing. Of

course you could easily eliminate the "stupid/joke" answer, leaving three options. And

another that was unlikely, so after that it was 50-50.

Yes! Good Point! Now that you brought me back to my school years what they used to do was take away 2 points if your answer was wrong and you got 1 point if it was right. So if you got 10 right and guessed 10 wrong you did not end up with 50% but rather you got "0" and a talking to.

In Theory this should have worked, but like in any Brainy Idea that was not well thought out, this was full of flaws and was dropped from the school system. For one thing, if you did not know the answer you were expected to leave that question blank. But what if you just missed this question? Or put it aside to go back to and forgot? I guess you could say too bad, but yet it still didn't prove if the student knew this answer or not.

But the worst part was the uncertainty that goes with taking huge marks away from the wrong answer. In Multi-choice Test at least 2 answers are close. So even though you may be 90% sure of the correct answer you were afraid to make the wrong answer and thus left it blank. So when the brightest students saw there marks drop like a rock because of this new testing theory, they threw that out the window and went back to normal testing and as they still do today.

[Gulfsailor]

Well it is because you are not using the "probability theory" correctly.

If your Multi-choice Test only consisted with one question, with the correct answer being A,B,C, or D, and a person were to guess, then you would be correct in saying he had a 25% chance of guessing correctly.

If you Multi-choice Test consisted of 4 questions, and you could not use the same letter twice as the correct answer, say letter "A" or "B" two times in 4 questions, then if a person were to guess "A" or "B" 4 times in a row he would be bound to get one right or 25%.

But as you know Multi-choice Tests never consist of only one question nor does the answer have to be different each time with only 4 questions. For example in only having 4 questions, if the correct answers were C,C, D, and B, and the test guesser chose "A" thinking he would get at least one correct, he wouldn't get any correct.

Look at the local Lotto Ticket for example. To get the last number correctly your chances are 1 in 10 as there are only 10 numbers to choose from (0 to 9). But what about getting the last 2 numbers correctly? Now you have 100 numbers to choose from to get 1 correct answers (00 to 99) some now the odds go down and your chances are only 1 in 100. For 3 correct numbers it is 1 in 1,000 and so on. But it is important to know that if you already have 2 correct numbers then the chances of getting the 3 number correct is still 1 in 10.

Maybe Dice is easier to work with when looking at Multi-choice Probability, What are the chances of rolling a 12 in Dice? To do that you need two 6's. So for one Dice your chances are 1 in 6 and also the same for the other Dice. So your chances of rolling a 12 is 1/6 x 1/6 or 1 in 36. Keep in mind this does not apply to the number 7 or other numbers as with two Dice there are several different ways to achieve 7. For example 6 and 1, 5 and 2, 4 and 3, 1 and 6, 2 and 5, 3 and 4. So your 1 in 36 only applies to the number 12 in Dice, and also the number 2.

So now that we know the formula on "Probability" with Dice, what are the chances of getting 12 two times in a roll? Is it 1 in 72? No, actually it is 1/6 x 1/6 x 1/6 x 1/6 or 1 in 1,296. But what if you did roll a 12 the first time, what are your chances of rolling 12 two times in a roll then? It goes back to only 1 in 36. So like the Lotto and if you have the first one correct it is easier to get the second one.

So now you can apply this "Dice Probability Formula" to a Multi-choice Test. Let's say you have 20 Multi-choice Question with 4 answers to choose from, so what are the chances of getting 25% by guessing? Well to get 25% from 20 questions he would need to have 5 correct answers. So his odds are: 1/4 x 1/4 x 1/4 x 1/4 x 1/4 or 1 in 1,024. But he also gets 20 guesses as he has 20 questions, so his actual odds of guessing 25% is 20/1,024 or 1 in 51 or about a 2% chance.

So this is how someone can get 15% in a test when you expect them to get at least 25%. To get even 15% they still need 3 correct answers which the odds say are 1 in 64 by pure guessing, but he also gets 20 guesses. So now his chances are 20 in 64 or about 1 in 3 or 32%, which still seems difficult to get an overall 15% score. But there are other things to consider first.

Firstly, even when the student doesn't know much on this Multi-choice Test chances are he is going to read all the questions at least. Usually not all the answers are logical ones. In my mind usually one answer is way off base, one (or two) a maybe, and two are close. So in reality the student really isn't guessing from 4 answers but rather 2 or 3 answers, which improve his odds.

The other thing to consider is that not all questions are difficult ones. Even if this student only knows 10%, or 2 questions right, this improves his chances a lot. Just like when you already have 2 correct last numbers in the Lotto. To now get 15% he only needs 1 more correct guess in 18 questions, and to get 25% he needs 3 more. So for a student who knows 2 questions, his chances of getting 25% in this Multi-choice Test is about 25% if all the answers are difficult, and about 50/50 is one answer is not very logical.

So in short if a students knows 10% his chances of getting 25% are about 50/50. His chances of passing and getting 10 correct answers by guessing is next to impossible. But still getting 25% when you know only 10% is a bit false. Perhaps why Multi-choice isn't the best way to test student knowledge. With long answer you either know the answer or you don't. But in an age where you get marks for showing your work and still come up with the wrong answer, who knows? .

True, except you are calculating the odds to get exactly 25% of the answers correct. To the question at hand it is perfectly fine to get a higher percentage correct. In your example of 20 4-way questions the odds of getting 25% or more questions correct by pure chance is 58.5%. If 10% of the answers are known the student will have a chance of 86.5% of getting to a minimum score of 25%.

In the OP's example of his students only getting 15% on average, the chance of this low score or a lower score (based on 20 4-way questions) is 22.5% if only 1 student was making the test. The probability rapidly decreases to near 0% when more students make the test. I'll work out the exact odds when the OP tells me how many students took the test.

[ozyjon]

There's your answer,,,

that's the best you'll get

[chilli42]

Probability theory says all/-any outcomes are possible on any given day. If you are going to grade to curve then just shift the mean. Alternatively, kick some butt and get them to do what they should be doing.

[wprime]

It is Probability which is good for a very large sample. You had a tiny sample which is why it did not work.

Actually we can work out a lot from these results. Considering the margin of error in this case of 40%, you just need 6 students within the sample before the confidence level falls below 95%.

If there were more than 6 students in the class, it's likely that they were copying (wrong) answers.

[zyphodb]

The OP is correct. Selecting any group of people who have not even studied for the test, nor even understood the content, the average score of the group achieved by random answers should have been 25% plus a standard deviation.

An average of 15% even achieved by a small group would be highly unusual without other external factors especially considering that any student would generally know the answers to at least some questions.

Maybe the just failed the subject of " how to answer multiple guess exams"?

The results suggest they know less about the subject than any random person who has never studied nor been taught the subject. Now that is hard to achieve.

My educated guess is that most of the class managed to copy the answers from 1 or 2 people, probably using mobiles, (they're very creative at cheating in exams), & those one or 2 were very bad...

[SlyAnimal]

But anywho I think that the point the OP was really getting at was that his students did extremely poorly on their test.

Do you have some sample questions at all from the test we can check out? I'm interested in seeing the difficulty level of this test & how "tricky" the questions were.

Sent from my iPhone using Tapatalk

Edited July 20, 2016 by SlyAnimal

[simon43]

Do you have some sample questions at all from the test we can check out? I'm interested in seeing the difficulty level of this test & how "tricky" the questions were.

OK, here is a typical question and choice of answers. This question was for M4 students.

Remember - I did not set this question

John wins the lottery. What does he say?

a) Oh dear

b ) Great!

c) Never mind

d) Whoosh (I wasn't sure about that answer, but some students chose it...)

Edited July 20, 2016 by simon43

[BruceMangosteen]

But as you know Multi-choice Tests never consist of only one question nor does the answer have to be different each time with only 4 questions.

Not true when picking a wife who is also a prostitute in SEA.

The mistake many teachers make is numbering the questions. There is no reason for a test question to have a number. Without numbers, cheating becomes more difficult. There are also software programs which shuffle the choices but that's going to an extreme. Regarding the non-answer aspect,(not counting as much as a wrong answer) I remember that from SAT exams but as mentioned, it was a short lived trial.

[BruceMangosteen]

Do you have some sample questions at all from the test we can check out? I'm interested in seeing the difficulty level of this test & how "tricky" the questions were.

OK, here is a typical question and choice of answers. This question was for M4 students.

Remember - I did not set this question

John wins the lottery. What does he say?

a) Oh dear

b ) Great!

c) Never mind

d) Whoosh (I wasn't sure about that answer, but some students chose it...)

d, Whoosh, would be the best answer. The others could be correct depending on the student's attitude. Questions like that are the norm unfortunately. This is IMHO why grammar is taught and "error detection" is so common on Thai exams.

[simon43]

Here is another set of questions about positionals, for an M3 class.

Chose the correct word to describe the position:

The dog is xxxxx the house

The car is xxxxx the house

The cat is xxxxx the house

The tree is xxxxx the house

next to

in front of

besides

behind

Unfortunately, no drawing or photo of the house/tree/car/cat/dog /was provided for the students! So I gave full marks to all students, regardless of their answers.

[GOLDBUGGY]

Well it is because you are not using the "probability theory" correctly.

If your Multi-choice Test only consisted with one question, with the correct answer being A,B,C, or D, and a person were to guess, then you would be correct in saying he had a 25% chance of guessing correctly.

If you Multi-choice Test consisted of 4 questions, and you could not use the same letter twice as the correct answer, say letter "A" or "B" two times in 4 questions, then if a person were to guess "A" or "B" 4 times in a row he would be bound to get one right or 25%.

But as you know Multi-choice Tests never consist of only one question nor does the answer have to be different each time with only 4 questions. For example in only having 4 questions, if the correct answers were C,C, D, and B, and the test guesser chose "A" thinking he would get at least one correct, he wouldn't get any correct.

Look at the local Lotto Ticket for example. To get the last number correctly your chances are 1 in 10 as there are only 10 numbers to choose from (0 to 9). But what about getting the last 2 numbers correctly? Now you have 100 numbers to choose from to get 1 correct answers (00 to 99) some now the odds go down and your chances are only 1 in 100. For 3 correct numbers it is 1 in 1,000 and so on. But it is important to know that if you already have 2 correct numbers then the chances of getting the 3 number correct is still 1 in 10.

Maybe Dice is easier to work with when looking at Multi-choice Probability, What are the chances of rolling a 12 in Dice? To do that you need two 6's. So for one Dice your chances are 1 in 6 and also the same for the other Dice. So your chances of rolling a 12 is 1/6 x 1/6 or 1 in 36. Keep in mind this does not apply to the number 7 or other numbers as with two Dice there are several different ways to achieve 7. For example 6 and 1, 5 and 2, 4 and 3, 1 and 6, 2 and 5, 3 and 4. So your 1 in 36 only applies to the number 12 in Dice, and also the number 2.

So now that we know the formula on "Probability" with Dice, what are the chances of getting 12 two times in a roll? Is it 1 in 72? No, actually it is 1/6 x 1/6 x 1/6 x 1/6 or 1 in 1,296. But what if you did roll a 12 the first time, what are your chances of rolling 12 two times in a roll then? It goes back to only 1 in 36. So like the Lotto and if you have the first one correct it is easier to get the second one.

So now you can apply this "Dice Probability Formula" to a Multi-choice Test. Let's say you have 20 Multi-choice Question with 4 answers to choose from, so what are the chances of getting 25% by guessing? Well to get 25% from 20 questions he would need to have 5 correct answers. So his odds are: 1/4 x 1/4 x 1/4 x 1/4 x 1/4 or 1 in 1,024. But he also gets 20 guesses as he has 20 questions, so his actual odds of guessing 25% is 20/1,024 or 1 in 51 or about a 2% chance.

So this is how someone can get 15% in a test when you expect them to get at least 25%. To get even 15% they still need 3 correct answers which the odds say are 1 in 64 by pure guessing, but he also gets 20 guesses. So now his chances are 20 in 64 or about 1 in 3 or 32%, which still seems difficult to get an overall 15% score. But there are other things to consider first.

Firstly, even when the student doesn't know much on this Multi-choice Test chances are he is going to read all the questions at least. Usually not all the answers are logical ones. In my mind usually one answer is way off base, one (or two) a maybe, and two are close. So in reality the student really isn't guessing from 4 answers but rather 2 or 3 answers, which improve his odds.

The other thing to consider is that not all questions are difficult ones. Even if this student only knows 10%, or 2 questions right, this improves his chances a lot. Just like when you already have 2 correct last numbers in the Lotto. To now get 15% he only needs 1 more correct guess in 18 questions, and to get 25% he needs 3 more. So for a student who knows 2 questions, his chances of getting 25% in this Multi-choice Test is about 25% if all the answers are difficult, and about 50/50 is one answer is not very logical.

So in short if a students knows 10% his chances of getting 25% are about 50/50. His chances of passing and getting 10 correct answers by guessing is next to impossible. But still getting 25% when you know only 10% is a bit false. Perhaps why Multi-choice isn't the best way to test student knowledge. With long answer you either know the answer or you don't. But in an age where you get marks for showing your work and still come up with the wrong answer, who knows? .

True, except you are calculating the odds to get exactly 25% of the answers correct. To the question at hand it is perfectly fine to get a higher percentage correct. In your example of 20 4-way questions the odds of getting 25% or more questions correct by pure chance is 58.5%. If 10% of the answers are known the student will have a chance of 86.5% of getting to a minimum score of 25%.

In the OP's example of his students only getting 15% on average, the chance of this low score or a lower score (based on 20 4-way questions) is 22.5% if only 1 student was making the test. The probability rapidly decreases to near 0% when more students make the test. I'll work out the exact odds when the OP tells me how many students took the test.

Do you care to share and show your work in how a student who only knows only 10% on what is on this test of 20 questions has a 86.5% chance of obtaining a score of 25%?

Of course "Probability" is not an exact science. "Probability" means the chances are, or likelihood off, or the odds are. So even though the "Probability" of throwing two 12's twice in a row from Dice are slim, it doesn't mean it is impossible. Of course when you add more people (students) your chances of hitting 25% go up. For example in the 6/49 Lottery, your chances of winning the top money is 14 Million to 1, which seems impossible. Yet with millions of people buying tickets weekly, someone always wins eventually.

Of course there are many other factors that need to be considered in "Probability" which we don't have the information for, but still needs to be considered, For example if the Class Average is very high, then the odds of more people getting 25% or higher is greater. The difficulty of this test also needs to be considered. If the range is 1 to 10 on the difficulty scale and you give a really hard test rated at 9, more students are going to fail. If the test you give is rated at 2, then more will pass.

My Reference Point and where I started was a student who is strictly guessing and who doesn't know any of the answers to this 20 Question Test, and all 4 answers are possible answers For this student his chances of getting at least 25% still remains at 1 in 51 or about 2%, I did not factor in the chances of having a student who doesn't know at least 1 correct answer, as this may be 1 in 100, or more. Or the difficulty of this test. Or the amount of students taking this test and their class average.

With the information we have it is impossible to come up with an exact "Probability" and like we can for Dice or a Lotto as there is a human factors involved here. My point was, and is, that it is not so easy to get 25% when you know nothing on this test. That getting the first question right is a 1 in 4 chance but each question after that the odds go against you.

[GOLDBUGGY]

Here is another set of questions about positionals, for an M3 class.

Chose the correct word to describe the position:

The dog is xxxxx the house

The car is xxxxx the house

The cat is xxxxx the house

The tree is xxxxx the house

next to

in front of

besides

behind

Unfortunately, no drawing or photo of the house/tree/car/cat/dog /was provided for the students! So I gave full marks to all students, regardless of their answers.

Well, using Logic, there is really only one correct answer here.

Since there is only one correct answer per question then the answer can't be "next to" or "besides" as that means the same thing. If everything was behind the house you wouldn't be able to see it. So the only correct answers is "in front of". But here is another question for your M.3 Class which I think is better than this. .

What is the correct answer to a "cup that is half full" and/or a "cup that is half empty"?

The correct answer is that this cup is twice as big as it needs to be. If you cut your cup in half it would be full all the time, thus never be half full or half empty ever again.

[JAS21]

Another possible theory to explain the OP's surprising results is that his answer key / stencil is incorrect and thus despite the average score being over 25% as expected, the stencil gives an average score of under 25% because it refers to a different test.

That is thinking outside the box ...

[DP25]

How can almost the entire class achieve no more than 15% correct answers?

Because they are not answering randomly. They are trying to answer correctly but there is a fundamental problem in their knowledge of the subject (and likely a poorly written test as well if it is a typical Thai English test), and they are choosing the wrong answers as a result. If they were to answer randomly they would get a better score, instead they are attempting to answer correctly and because of their poor knowledge of the subject matter are choosing the wrong answers.

Edited July 20, 2016 by DP25

[Gulfsailor]

Well it is because you are not using the "probability theory" correctly.

If your Multi-choice Test only consisted with one question, with the correct answer being A,B,C, or D, and a person were to guess, then you would be correct in saying he had a 25% chance of guessing correctly.

If you Multi-choice Test consisted of 4 questions, and you could not use the same letter twice as the correct answer, say letter "A" or "B" two times in 4 questions, then if a person were to guess "A" or "B" 4 times in a row he would be bound to get one right or 25%.

But as you know Multi-choice Tests never consist of only one question nor does the answer have to be different each time with only 4 questions. For example in only having 4 questions, if the correct answers were C,C, D, and B, and the test guesser chose "A" thinking he would get at least one correct, he wouldn't get any correct.

Look at the local Lotto Ticket for example. To get the last number correctly your chances are 1 in 10 as there are only 10 numbers to choose from (0 to 9). But what about getting the last 2 numbers correctly? Now you have 100 numbers to choose from to get 1 correct answers (00 to 99) some now the odds go down and your chances are only 1 in 100. For 3 correct numbers it is 1 in 1,000 and so on. But it is important to know that if you already have 2 correct numbers then the chances of getting the 3 number correct is still 1 in 10.

Maybe Dice is easier to work with when looking at Multi-choice Probability, What are the chances of rolling a 12 in Dice? To do that you need two 6's. So for one Dice your chances are 1 in 6 and also the same for the other Dice. So your chances of rolling a 12 is 1/6 x 1/6 or 1 in 36. Keep in mind this does not apply to the number 7 or other numbers as with two Dice there are several different ways to achieve 7. For example 6 and 1, 5 and 2, 4 and 3, 1 and 6, 2 and 5, 3 and 4. So your 1 in 36 only applies to the number 12 in Dice, and also the number 2.

So now that we know the formula on "Probability" with Dice, what are the chances of getting 12 two times in a roll? Is it 1 in 72? No, actually it is 1/6 x 1/6 x 1/6 x 1/6 or 1 in 1,296. But what if you did roll a 12 the first time, what are your chances of rolling 12 two times in a roll then? It goes back to only 1 in 36. So like the Lotto and if you have the first one correct it is easier to get the second one.

So now you can apply this "Dice Probability Formula" to a Multi-choice Test. Let's say you have 20 Multi-choice Question with 4 answers to choose from, so what are the chances of getting 25% by guessing? Well to get 25% from 20 questions he would need to have 5 correct answers. So his odds are: 1/4 x 1/4 x 1/4 x 1/4 x 1/4 or 1 in 1,024. But he also gets 20 guesses as he has 20 questions, so his actual odds of guessing 25% is 20/1,024 or 1 in 51 or about a 2% chance.

So this is how someone can get 15% in a test when you expect them to get at least 25%. To get even 15% they still need 3 correct answers which the odds say are 1 in 64 by pure guessing, but he also gets 20 guesses. So now his chances are 20 in 64 or about 1 in 3 or 32%, which still seems difficult to get an overall 15% score. But there are other things to consider first.

Firstly, even when the student doesn't know much on this Multi-choice Test chances are he is going to read all the questions at least. Usually not all the answers are logical ones. In my mind usually one answer is way off base, one (or two) a maybe, and two are close. So in reality the student really isn't guessing from 4 answers but rather 2 or 3 answers, which improve his odds.

The other thing to consider is that not all questions are difficult ones. Even if this student only knows 10%, or 2 questions right, this improves his chances a lot. Just like when you already have 2 correct last numbers in the Lotto. To now get 15% he only needs 1 more correct guess in 18 questions, and to get 25% he needs 3 more. So for a student who knows 2 questions, his chances of getting 25% in this Multi-choice Test is about 25% if all the answers are difficult, and about 50/50 is one answer is not very logical.

So in short if a students knows 10% his chances of getting 25% are about 50/50. His chances of passing and getting 10 correct answers by guessing is next to impossible. But still getting 25% when you know only 10% is a bit false. Perhaps why Multi-choice isn't the best way to test student knowledge. With long answer you either know the answer or you don't. But in an age where you get marks for showing your work and still come up with the wrong answer, who knows? .

True, except you are calculating the odds to get exactly 25% of the answers correct. To the question at hand it is perfectly fine to get a higher percentage correct. In your example of 20 4-way questions the odds of getting 25% or more questions correct by pure chance is 58.5%. If 10% of the answers are known the student will have a chance of 86.5% of getting to a minimum score of 25%.

In the OP's example of his students only getting 15% on average, the chance of this low score or a lower score (based on 20 4-way questions) is 22.5% if only 1 student was making the test. The probability rapidly decreases to near 0% when more students make the test. I'll work out the exact odds when the OP tells me how many students took the test.

Do you care to share and show your work in how a student who only knows only 10% on what is on this test of 20 questions has a 86.5% chance of obtaining a score of 25%?

Of course "Probability" is not an exact science. "Probability" means the chances are, or likelihood off, or the odds are. So even though the "Probability" of throwing two 12's twice in a row from Dice are slim, it doesn't mean it is impossible. Of course when you add more people (students) your chances of hitting 25% go up. For example in the 6/49 Lottery, your chances of winning the top money is 14 Million to 1, which seems impossible. Yet with millions of people buying tickets weekly, someone always wins eventually.

Of course there are many other factors that need to be considered in "Probability" which we don't have the information for, but still needs to be considered, For example if the Class Average is very high, then the odds of more people getting 25% or higher is greater. The difficulty of this test also needs to be considered. If the range is 1 to 10 on the difficulty scale and you give a really hard test rated at 9, more students are going to fail. If the test you give is rated at 2, then more will pass.

My Reference Point and where I started was a student who is strictly guessing and who doesn't know any of the answers to this 20 Question Test, and all 4 answers are possible answers For this student his chances of getting at least 25% still remains at 1 in 51 or about 2%, I did not factor in the chances of having a student who doesn't know at least 1 correct answer, as this may be 1 in 100, or more. Or the difficulty of this test. Or the amount of students taking this test and their class average.

With the information we have it is impossible to come up with an exact "Probability" and like we can for Dice or a Lotto as there is a human factors involved here. My point was, and is, that it is not so easy to get 25% when you know nothing on this test. That getting the first question right is a 1 in 4 chance but each question after that the odds go against you.

Google Bernoulli.

In your example of 20 questions with 2 already known, there are 3 more correct answers required from the remains 18 questions to get 25% in total.

However I don't want to know the probability of getting exactly 25%, but 25% or higher, or in the example case getting 3 or more correct answers from 18 questions. It is quickest to calculate the probability of getting 0, 1 and 2 answers right, and deducting that number from 1 to get the probability from 3 or more right.

0 right; 3/4^18

1 right; 1/4*3/4^17*(18over1)

2 right; 1/4^2*3/4^16*(18over2)

Add these together and you get 0.135

1- 0.135 = 0.865 = 86.5%

[simon43]

This discussion of stats and probability re answering multiple-choice exam questions is quite fascinating, but from in-class experience, I don't think some of the assumptions made actually apply.

...even when the student doesn't know much on this Multi-choice Test chances are he is going to read all the questions at least.

Many of my students cannot read English. This makes the whole idea of a test where the student must read and understand the question simply ridiculous. Many of the students simply look for an English word that they recognise in the answers.

(There was an English listening and speaking test last week for all grades, but as the only NES in the whole school, I wasn't asked to implement this - a Tinglish-speaking Thai teacher prepared and executed the test - Duh!)

Despite still being laid low with Dengue/Typhoid/Unknown fever, I will get back to school tomorrow and prepare a plan of action to assist my students to learn something useful for them.

BTW, not all of my students fall into the 'low achiever' box. Out of a typical class of 35 students, perhaps 10 students never turn up for classes. Of the remaining 25 students, about 6 boys will do nothing at all, even with one-to-one encouragement in Thai from me. About 6 girls in each lesson have permanent period pains and need to sleep. There are about 6 students who want to learn, but have almost zero ability to read or understand basic English. Of the remaining students, perhaps 2 can meet the learning objectives of a grade 2 levels below, whilst I'd grade the other students at 4 levels below.

Since I previously taught in private schools, where most of the students were good to high achievers, I find this all a very sad reflection of the Thai educational system and especially the manner in which English is taught...

Anyway, back to the stats discussion

[GanDoonToonPet]

Well it is because you are not using the "probability theory" correctly...

...So now you can apply this "Dice Probability Formula" to a Multi-choice Test. Let's say you have 20 Multi-choice Question with 4 answers to choose from, so what are the chances of getting 25% by guessing? Well to get 25% from 20 questions he would need to have 5 correct answers. So his odds are: 1/4 x 1/4 x 1/4 x 1/4 x 1/4 or 1 in 1,024. But he also gets 20 guesses as he has 20 questions, so his actual odds of guessing 25% is 20/1,024 or 1 in 51 or about a 2% chance.

So this is how someone can get 15% in a test when you expect them to get at least 25%. To get even 15% they still need 3 correct answers which the odds say are 1 in 64 by pure guessing, but he also gets 20 guesses. So now his chances are 20 in 64 or about 1 in 3 or 32%, which still seems difficult to get an overall 15% score. But there are other things to consider first.

You can't use this kind of probability for this type of question. Besides that there are more than 20 ways to get 5 correct answers from 20 questions; combination theory gives 20C5 = 15504 different ways.

You need to use a binomial distribution as the test is a series of identical trials with 2 possible outcomes (success or failure). Using this method, this probability is:

P(x=5) = 15504*(1/4)^5 * (3/4)^15 = 0.2 or 20%

Also the probability of getting 5 or less correct answers is quite high at 62% !!!

Simon, we need to know the number of questions as the probabilities are different for different number of trials, even for the same % of correct answers. For example:

the probability of scoring 25 out of 100 questions is 9%.

the probability of scoring 25 or less out of 100 questions is 55%.

There's a calculator here if you want to work it out for yourself:

http://stattrek.com/m/online-calculator/binomial.aspx

[talisman01]

I experimented with my collection of recent Thai English mid-term, end of term and O-Net exam papers and found that, more often than not, by choosing only answer "a" or answer "b", or "c" or "d" the resulting score was often far higher than the average of the results gained when students picked the answers they believed to be correct. If I was a student in their shoes with minimal grasp of English this is the route I would go down, choosing all "a"s or "b"s or "c"s or "d"s.

As has been mentioned, randomly picking answers (where the student has no idea of the correct answer) is akin to picking lottery numbers with less chance of hitting upon a right answer. I knew a very popular student, an ace school footballer, who was hopeless at English, who relied on this technique during his M5 and M6 school years and his Thai English teachers were duly highly impressed which helped lift his overall grades significantly.

[Gulfsailor]

Okay, just to come back to the statistical bit. I was thinking on how to calculate the probability of the class as a whole scoring 15% or less on average on the test.

I will make the following assumptions;

20 multiple choice questions

Each question has 4 possible answers

Answers are given as a pure guess

30 students took the test

Now if 30 students on average scored 15% on 20 questions, that means 3 answers correctly on average, which means together the students had 90 questions guessed correctly out of a total of 600.

Then using the binomial formula and an excel sheet to add up all the individual probabilities of getting 1, 2, 3, etc up to 90 correct it results in a probability of 0.000000000169 (I may mistyped a 0 or two there). Anyway the odds that this test really was made this bad or worse is 1 in 600 million!!!

So either statistical rules don't apply in Thailand as proffered by the OP, or he exaggerated, or the students deliberately screwed up.

[Mudcrab]

And the crux of the problem is multiple choice examinations.

Draw a picture and ask for a written description...i.e the dog is next to the house, or the dog is sitting next to the house, the dog lives at the house. That would be , and used to be, a far better method of gauging comprehension and language skills and creative thinking than silly multiple choice exams which were invented by so called educational experts (supported by lazy teachers and their unions).

How confident would you be knowing that the guy operating on the cornea of your eye gained his degree by answering multiple choice exam questions? Hope he guesses right with the scalpel better than 1 in 4.

Education is about preparing people for the big bad world as an adult, unfortunately many teachers never have to live in that world since they never leave the school environment. University lecturers can be particularly bad for this.

[simon43]

Education is about preparing people for the big bad world as an adult, unfortunately many teachers never have to live in that world since they never leave the school environment.

The other side of the coin is that many teachers - like myself - have spent a lifetime working, designing and managing in the big bad world, and have entered teaching as a second or third career.

As for multiple-choice exams, I have no option re this. The exam style set by the school authority is 4-answer multiple-choice. I am merely the 'teacher'

[Scott]

Multiple choice allows for a rather quick assessment of a large number of students' ability in a subject. It is good for testing concrete knowledge about the subject matter. It is a valuable tool in the testing arena, but it's like a hammer -- it may be useful, but other tools are also needed.

I am not a big fan of using multiple choice in language exams because language is much less concrete than say Social Studies, where you can ask 'What is the Capital city of Thailand?' There are certain questions that can be done, but other types of testing methods should also be employed.