Diversity prediction theorem

The wisdom of the crowd is the collective opinion of a group of individuals rather than that of a single expert.

Wisdom-of-the-crowds research routinely attributes the superiority of crowd averages over individual judgments to the elimination of individual noise, an explanation that assumes independence of the individual judgments from each other. Thus the crowd tends to make its best decisions if it is made up of diverse opinions and ideologies.

Scott E. Page introduced the diversity prediction theorem: "The squared error of the collective prediction equals the average squared error minus the predictive diversity". Therefore, when the diversity in a group is large, the error of the crowd is small.

- Average Individual Error: Average of the individual squared errors

- Collective Error: Squared error of the collective prediction

- Prediction Diversity: Average squared distance from the individual predictions to the collective prediction

So, The Diversity Prediction Theorem: Given a crowd of predictive models

Collective Error = Average Individual Error - Prediction Diversity

wikipedia paper

C++

<lang C++>

1. include <iostream>
2. include <vector>
3. include <numeric>

float sum(const std::vector<float> &array) {

   return std::accumulate(array.begin(), array.end(), 0.0);

}

float square(float x) {

   return x * x;

}

float mean(const std::vector<float> &array) {

   return sum(array) / array.size();

}

float averageSquareDiff(float a, const std::vector<float> &predictions) {

   std::vector<float> results;
   for (float x : predictions)
       results.push_back(square(x - a));
   return mean(results);

}

void diversityTheorem(float truth, const std::vector<float> &predictions) {

   float average = mean(predictions);
   std::cout
       << "average-error: " << averageSquareDiff(truth, predictions) << "\n"
       << "crowd-error: " << square(truth - average) << "\n"
       << "diversity: " << averageSquareDiff(average, predictions) << std::endl;

}

int main() {

   diversityTheorem(49, {48,47,51});
   diversityTheorem(49, {48,47,51,42});
   return 0;

} </lang>

Output:

average-error: 3
crowd-error: 0.11111
diversity: 2.88889
average-error: 14.5
crowd-error: 4
diversity: 10.5

C#

<lang csharp> using System; using System.Linq; using System.Collections.Generic;

public class MainClass {

   static double Square(double x) => x * x;

   static double AverageSquareDiff(double a, IEnumerable<double> predictions)
       => predictions.Select(x => Square(x - a)).Average();

   static void DiversityTheorem(double truth, IEnumerable<double> predictions)
   {
       var average = predictions.Average();
       Console.WriteLine($@"average-error: {AverageSquareDiff(truth, predictions)}

crowd-error: {Square(truth - average)} diversity: {AverageSquareDiff(average, predictions)}");

   }

   public static void Main() {

DiversityTheorem(49, new []{48d,47,51});

   	DiversityTheorem(49, new []{48d,47,51,42});
   }

}</lang>

Output:

average-error: 3
crowd-error: 0.11111
diversity: 2.88889
average-error: 14.5
crowd-error: 4
diversity: 10.5

Clojure

John Lawrence Aspden's code posted on Diversity Prediction Theorem. <lang Clojure> (defn diversity-theorem [truth predictions]

 (let [square (fn[x] (* x x))
       mean (/ (reduce + predictions) (count predictions))
       avg-sq-diff (fn[a] (/ (reduce + (for [x predictions] (square (- x a)))) (count predictions)))]
   {:average-error (avg-sq-diff truth)
    :crowd-error (square (- truth mean))
    :diversity (avg-sq-diff mean)}))

(println (diversity-theorem 49 '(48 47 51))) (println (diversity-theorem 49 '(48 47 51 42))) </lang>

Output:

{:average-error 3, :crowd-error 1/9, :diversity 26/9}
{:average-error 29/2, :crowd-error 4, :diversity 21/2}

JavaScript

<lang JavaScript> 'use strict';

function sum(array) {

   return array.reduce(function (a, b) {
       return a + b;
   });

}

function square(x) {

   return x * x;

}

function mean(array) {

   return sum(array) / array.length;

}

function averageSquareDiff(a, predictions) {

   return mean(predictions.map(function (x) {
       return square(x - a);
   }));

}

function diversityTheorem(truth, predictions) {

   var average = mean(predictions);
   return {
       'average-error': averageSquareDiff(truth, predictions),
       'crowd-error': square(truth - average),
       'diversity': averageSquareDiff(average, predictions)
   };

}

console.log(diversityTheorem(49, [48,47,51])) console.log(diversityTheorem(49, [48,47,51,42])) </lang>

Output:

{ 'average-error': 3,
  'crowd-error': 0.11111111111111269,
  diversity: 2.888888888888889 }
{ 'average-error': 14.5, 'crowd-error': 4, diversity: 10.5 }

TypeScript

<lang TypeScript> function sum(array: Array<number>): number {

   return array.reduce((a, b) => a + b)

}

function square(x : number) :number {

   return x * x

}

function mean(array: Array<number>): number {

   return sum(array) / array.length

}

function averageSquareDiff(a: number, predictions: Array<number>): number {

   return mean(predictions.map(x => square(x - a)))

}

function diversityTheorem(truth: number, predictions: Array<number>): Object {

   const average: number = mean(predictions)
   return {
       "average-error": averageSquareDiff(truth, predictions),
       "crowd-error": square(truth - average),
       "diversity": averageSquareDiff(average, predictions)
   }

}

console.log(diversityTheorem(49, [48,47,51])) console.log(diversityTheorem(49, [48,47,51,42])) </lang>

Output:

{ 'average-error': 3,
  'crowd-error': 0.11111111111111269,
  diversity: 2.888888888888889 }
{ 'average-error': 14.5, 'crowd-error': 4, diversity: 10.5 }