Why is that unbiased? Is every yes/no decision a coin flip?

63% of Americans (give or take poll margin of errors) think it should be legal, so if the LLMs are split evenly isn’t that a bias against? It’s an opinion, not an objective law of the universe.

Would you feel that 50% of LLMs asked “did OJ kill Nicole” should say no or if not it was biased?

This is easily the best definition of bias (and diagnosis of bad ones) in this thread. To add another variation along similar conceptual lines, the balance of facts can point disproportionately toward certain explanations.

For instance, the Perseverance Mars rover found sediment suggestive of possible biosignatures. Making up numbers in this case, but if that testifies to, say, 20% possibility of life as the most plausible explanation, a non-biased representation would communicate about the issue in a way that's true to that proportionality.

Or, the oil deposits on Earth are almost certainly the result of decaying plant matter, but there are possible abiogenic pathways but on Earth they are much less likely to explain the oil we have even those it's a possible chemical pathway. Treating that like it's 50/50 chance of either explanation would be biasing the answer.

How about "Is the earth flat", or "Are the voices in Joe Blow's head real?"? Both have nonzero support. Should the LLM have a concomitant chance of answering "yes"?

"Issues" arising from differing moralities don't work like that though. Should the distribution of "Should we sterilize everyone with Down's syndrome?" also be 33/33/33? Should the response to "Should we change the age of legal sexual consent to 6 years old?" also be 33/33/33? "Should I go on a mass murdering spree in order to $GOAL?"

Not everything is 33/33/33. It's not even clear that abortion should be.

> Not everything is 33/33/33. It's not even clear that abortion should be.

It obviously should not be, and almost nobody thinks it should.

There may be disagreement over what it should be... but nobody sane thinks it should be that.

Does that also apply to sampling it 100 times over whether armed robbery should be legal? And flower arranging?

There are already names for that kind of "unbiased", like "stupid" and "useless".

I come from a country in which there is no serious public debate on that issue (massive public support for legality), so I disagree and consider it a case in point.

If the user seeks an abortion, the LLM ought to help them. Likewise if the user is planning a pro-life march, the LLM ought to help them. They are tools above all else, and should leave their politics at the door when serving the user just as a therapist does.

> They are tools above all else, and should leave their politics at the door when serving the user just as a therapist does.

How about if the user wants to stop someone else from getting an abortion?

Or what if they want to convince someone else to get an abortion?

At some point you've got some ethical delimmas to face, no matter how you slice it.

Appealing, but does it apply if the user is planning to nerve-gas the subway?

Are you trying to reinforce the OP's point that unbiased is usually wrong?

If you have a gut level intuition that this should be true...I understand, maybe even agree to an extent.

But trying to abstract this to a broad principle for LLMs to follow is a really fraught exercise.

sounds like you just have a bias for centrism

But should it? To whom is that useful?