Is the Quiz Biased?

A large number of people taking Quiz2D score in the Libertarian quadrant, yet the Libertarian Party usually does very poorly in the polls. What's the deal? Is the Quiz ridiculously biased? Are or there other factors at play?

Actually, at least four factors that lead to the discrepancy:

  1. Just What is the definition of "Libertarian?"
  2. Bias in the demographics of who takes Quiz2D.
  3. Bias in the quiz itself.
  4. Bias in our political system.

Just What is a "Libertarian" Anyway?

The word "Libertarian" has two quite different meanings:

  1. One who desires less government in both the personal and economic realms.
  2. One who believes that it is never right for the government to initiate force, no matter how much societal benefit results.

The first definition properly describes the libertarian quadrant. The second describes only the apex of said quadrant, what is now called "Radical Libertarian" by this quiz.

The original purpose of this quiz was to get the Libertarian Party to fully embrace the first definition, as well as find people who fit this definition. This proved insufficient, so eventually, I spearheaded the Libertarian Reform Caucus to force the issue at the national convention level. In 2006, efforts by the Caucus and others resulted in removing most of the radical planks in the LP platform. However, the membership definition in the bylaws still limits membership to those who fit the second definition.

So, to avoid falsely leading people to think they fit within the Libertarian Party, I have changed the labels to "Left-Leaning Freedom Lover", "Right-Leaning Freedom Lover" and "Radical Libertarian." Non-radical freedom lovers have no political party to represent them in the U.S. (In fact, I am no longer a member of the Libertarian Party, and am distancing myself from the "L" word.)

Should the Libertarian Party change its definition to the broader meaning of the word "Libertarian," I will alter the designations once again.

Demographic Factors

The statistics here are not part of a random scientific sample of voters. If you are running for office, you might want to do your own polling. The sample is biased several ways.

First, many people come to this quiz via libertarian web sites. I have tried to reduce the impact of this factor by encouraging those who link to this site to use a special URL so that data from those sites is segregated from the data from those who come to the site directly. However, not everyone does this, so there is a high predominance Radical Libertarians among the quiz takers. You can get around this bias by looking at the "All Except Radicals" subset. (This also reduces the impact of those who play around with the quiz.)

Second, people who are "politically homeless" are more likely to take a political quiz; that is, those who are less satisfied with the major political parties are more likely to take this quiz. Also, the libertarian Advocates for Self Government has promoted the idea of taking a political quiz, which leads to more libertarian takers.

Third, some people play with the quiz, taking it multiple times thus contaminating the data. In a future update I will use cookies or IP address checks to prevent recording such bogus results. Enough signal exists in the bigger trends to swamp such noise, but for smaller trends the stats cannot be relied upon.

Finally, Web demographics are not general demographics. While this is less true than it was a few years ago, it is still the case that younger people are more likely to search around on the Web than older people. This results in a more socially liberal demographic. Teens and twenty-somethings are more socially liberal then their parents. People become less enthralled with sex and drugs when they have teens of their own. Note how left leaning freedom lovers outnumber right leaning freedom lovers.

I have given (earlier versions) of this quiz as a Windows program at various events and gotten a significantly different distribution of results. Gun shows had very few liberals. Student events had very few conservatives. I have gotten more authoritarians at fair events than elsewhere.

All this said, there are still useful trends in the data. The big trends (such as a surprisingly high sentiment in favor of legal prostituuttion) have held true over different demographics.

Is the Quiz itself Biased?

Perhaps, but less than many people think. The other factors are more important.

When I made the first version of this quiz back in 1999, the background material for each question was blatant libertarian propaganda. What I was trying to measure at the time was how people would respond to the Libertarian message if we had the ad budget to get out that message.

But right from the beginning, I always offered people a chance to disagree with that message—even strongly. To try to offset bias from the explanation, I put the pro-government answers first. I endeavored to match the vocabulary of the pro-government position in those answers. Whether I succeeded is for you to decide.

All this said, since then I have toned down the background paragraphs, especially in Version 6 (which is where all the posted data comes from). Over time I decided that the best way to popularize this quiz is to make it a useful tool for all political alignments. (The result screens point to the appropriate political parties as well.)

In some ways I have worked hard to avoid "false positives." Note how I ask how much people want government spending cut vs. taxes cut.

A remaining source of potential bias is that there are more reduced government options than increased government options for some of the questions. My purpose in doing this is not to try to badger people into making a libertarian choice, but to determine just how much liberty people do want when they want it. Wanting to legalize just marijuana is a bid difference from wanting to legalize all drugs. Wanting a 30% reduction in the size of government is hugely different from wanting zero taxes. Part of the mission of this quiz is reveal these differences preference wise.

Is our Political System Biased?

The center point of the political map I use is not meant to be some objective political center on an absolute scale. It is not meant to reflect a demographic center either. The center point is meant to represent the amount of government we have in the United States today. For each question there is a status quo answer which scores 50 or very close thereto. If you answer for the (U.S.) status quo for each question, you end up in the center.

However, the status quo amount of government is not necessarily the amount of government the average person wants! It is quite possible that the average person does land in the "Centrist, with Libertarian Tendencies" area. It is quite possible that the U.S. government is bigger than the average person wants it to be.

Incumbents become more statist the longer they stay in office. Special interest groups lobby for more largesse and more complicated laws. Public Choice theory says that concentrated interests have more incentive to lobby than dispersed interested. Pork wins over tax cuts.

It is quite possible that the people in general do want less government than we have today, as the quiz data indicates.

But this data does not prove such definitively.

Back to main stats page