Can the “Best” Language Model Detect Logical Fallacies?

Rongpeng Li (Ron)

Prior knowledge:
No previous knowledge expected

Summary

This lightning talk will present an early exploration of whether the state-of-the-art language models like GPT-3 can detect common logical fallacies like the slippery slope fallacy.

Description

A fallacy is reasoning that is logically incorrect, undermines the logical validity of an argument, or is recognized as unsound. Some of them are structural and some are contextual. The former is recognized as formal fallacy and the latter is recognized as informal fallacy. They are common everywhere especially in advertising and political slogans, etc. Most people won’t be able to detect them.

Can billion-parameter language models like GPT-3 detect those fallacies? In this lightning talk, I will share some results of the experiments I did, inspired by some online content, and show how powerful language models perform against different kinds of fallacies. Some of the results are inspirational that they may help humans be better information digesters and thinkers.

Types of logical fallacies are listed below.

  1. formal logical fallacies:
    1. affirming the consequent
    2. denying the antecedent
    3. affirming a disjunct
    4. denying a conjunct
  2. Informal logical fallacies (there are many. only a few are listed here)
    1. confusion of Necessary with a Sufficient Condition
    2. straw person
    3. whataboutism
    4. red herring