A hybrid system composed of human predictions and artificial intelligence algorithms.

The interdisciplinary team at the University of California, Irvine, in the areas of Artificial Intelligence, Law and Society, presented a new mathematical model that could optimize the performance of AI systems.

This improvement is possible by combining human and algorithmic predictions and confidence scores. "Human and machine algorithms have complementary strengths and weaknesses. Each uses different sources of information and strategies to make predictions and decisions," commented co-author of this research, Mark Steyvers, professor of cognitive science at UCI. "We show through empirical demonstrations and theoretical analysis that humans can improve on AI predictions even when human accuracy is somewhat inferior to AI accuracy, and vice versa. And this accuracy is higher than the combined predictions of two individuals or two AI Algorithms."

During trials of this new hybrid system, the team experimented by testing with image classification, where humans and algorithm worked separately. The images were distorted and had to be correctly identified. 

The system presented a continuous score while the human participants rated on the accuracy of identifying each image as low, medium or high, these results accommodated the large differences in confidence between humans and AI algorithms on the images.

"In some cases, the human participants were fairly confident that a particular image contained a chair, for example, while the AI algorithm was confused about the image," said co-author Padhraic Smyth, professor of computer science at UCI Chancellor. "Similarly, for other images, the AI algorithm was able to confidently provide a label for the object shown, while the human participants were unsure whether the distorted image contained any recognizable objects."

The hybrid model led to better performance than either human or machine predictions achieved separately. "While previous research has demonstrated the advantages of combining machine predictions or combining human predictions-the so-called 'wisdom of crowds'-this work forges a new direction by demonstrating the potential of combining human and machine predictions, pointing to new and improved approaches to human-artificial intelligence collaboration," Smyth said.

This convergence could develop artificially intelligent systems that are more accurate and applicable to new domains.

You can read about the study at https://news.uci.edu/2022/03/07/uci-researchers-develop-hybrid-human-machine-framework-for-building-smarter-ai/