Excellent Example of AI Bullshit via Poker

Me: Can a set of Jacks can ever be the nuts in Texas Hold'em?

ChatGPT: In Texas Hold'em, the "nuts" refers to the best possible hand at a given point in a game, given the community cards. Whether a set of Jacks (three of a kind with Jacks) can be the nuts depends on the board.

Situations where a set of Jacks can be the nuts:

No straights, flushes, or full houses are possible:

  • Example board: J♦ 7♣ 3♠ 2♥ 9♣
  • In this scenario, no straight, flush, or full house can beat your set of Jacks. Thus, it’s the nuts.

The board pairs in your favor:

  • Example board: J♦ 7♣ J♠ 2♥ 9♣
  • You now have a full house (Jacks full of Sevens), and it could be the nuts if no higher full house is possible.

Whole chat available here.


Except no. In the first example, there's a straight possibility with 8,10. In the second example, it's not actually a set. It's sort of a semantic argument, but "three of a kind is called a set only when it comprises a pocket pair and a third card on the board" (Wikipedia). A set necessitates pocket pairs. Trips is less specific and can be J in your hand, JJ on the board as shown. So the latter answer is wrong, too.

It took five rounds of further prompting, while ChatGPT provided obviously incorrect answers, before I could make it admit defeat. It kept providing boards where straights were possible, or higher sets were possible.

A set of jacks can never truly be the nuts by the river. Either a straight possibility is present, or a higher card allowing for a higher set is present. In Texas Hold'em, QQQ87 is the lowest hand that is the possible for the nuts.

And even when I got it to that point? It still qualified the answer with "almost never" because it doesn't actually know the truth, it's making up bullshit and giving the "most likely" answer. The only way I could've gotten it to this point was by knowing the answer in the first place -- i.e., I knew better than the stupid robot.

Poker is not that difficult (sobs in "consistently losing poker player"). There are 2,598,960 possible hands in five-card poker, it's a limited game with a very finite combination of potential hands (especially in comparison to something like chess, which has... just slightly, just a hair more potential board configurations). Whether a given hand is the highest possible on a given poker run-out really is not something that's actually difficult or resource-intensive to calculate.

But yeah, sure, let's trust the bullshit-machine to do real work for us.

As an aside it's also just straight up annoying for it to repeatedly offer placating "oh, no, you're absolutely right, so sorry!" praise. Don't apology-praise me bro. Correct answer or GTFO.