It is, though, when the selection is functionally binary.
Better / Worse / No Opinion isn’t going to get you a ton of extra information with more responses.
You might be inclined to interrogate individual responses and ask how things have improved / worsened / remained unchanged. And, at that point, a surveying a guy who became a Bitcoin millionaire against a guy who simply enjoys watching his browner neighbors get The Purge treatment matters more. But from the perspective of the “Are things better?” question, the answer is the same.
I’m not concerned about getting more information, I’m interested in getting more accurate information.
I recognize there isn’t room for diverse answers when the question is ‘choose 1, 2, or 3’. My thought is that turning up in Boulder, Colorado and asking the first person you see if they like chocolate, vanilla, or strawberry ice cream then claiming everyone in the city likes vanilla is misrepresentative.
My thought is that turning up in Boulder, Colorado and asking the first person you see if they like chocolate, vanilla, or strawberry ice cream then claiming everyone in the city likes vanilla is misrepresentative.
You don’t ask the first person you see. You ask fifty or sixty people, get their demographic data, and then feed that into a big pot. Then you pull some of them back out again based on the statistical norms across the whole country.
The principle being that you’re not trying to get the “average” person in Colorado. You’re trying to get the “average” person nationally, with a random sample of Colorado residents feeding that model.
I understand that. However I didn’t choose Boulder, Colorado to ask a national question - I specifically posed a question irrelevant to location because the question being asked is not important to what I was attempting to illustrate.
I chose Boulder for its population size, which is proportionally the same as what the NYT has done. If the survey were completed by 50/100,000 of voters, the sample size would be 0.0005%, which in my opinion is much better than 0.00001%.
It is, though, when the selection is functionally binary.
Better / Worse / No Opinion isn’t going to get you a ton of extra information with more responses.
You might be inclined to interrogate individual responses and ask how things have improved / worsened / remained unchanged. And, at that point, a surveying a guy who became a Bitcoin millionaire against a guy who simply enjoys watching his browner neighbors get The Purge treatment matters more. But from the perspective of the “Are things better?” question, the answer is the same.
I’m going to go on a limb and say old conservatives are more likely to respond to a phone survey than Gen Z leftists.
Sure, but you can control for that in your sampling.
I’m not concerned about getting more information, I’m interested in getting more accurate information.
I recognize there isn’t room for diverse answers when the question is ‘choose 1, 2, or 3’. My thought is that turning up in Boulder, Colorado and asking the first person you see if they like chocolate, vanilla, or strawberry ice cream then claiming everyone in the city likes vanilla is misrepresentative.
You don’t ask the first person you see. You ask fifty or sixty people, get their demographic data, and then feed that into a big pot. Then you pull some of them back out again based on the statistical norms across the whole country.
The principle being that you’re not trying to get the “average” person in Colorado. You’re trying to get the “average” person nationally, with a random sample of Colorado residents feeding that model.
I understand that. However I didn’t choose Boulder, Colorado to ask a national question - I specifically posed a question irrelevant to location because the question being asked is not important to what I was attempting to illustrate.
I chose Boulder for its population size, which is proportionally the same as what the NYT has done. If the survey were completed by 50/100,000 of voters, the sample size would be 0.0005%, which in my opinion is much better than 0.00001%.
It’s in the nation. I don’t see why you wouldn’t.
Pulling a sample is going to get you results consistent with the national average when the people you select are representative of the average