When humans and AI work together: A review
Author Information
Author(s): Michelle Vaccaro, Abdullah Almaatouq, Thomas Malone
Primary Institution: Massachusetts Institute of Technology
Hypothesis
When do combinations of humans and AI perform better than either alone?
Conclusion
Human–AI combinations generally perform worse than the best of humans or AI alone, but they can enhance human performance in certain tasks.
Supporting Evidence
- Human–AI combinations performed significantly worse than the best of humans or AI alone.
- Performance losses were noted in decision-making tasks.
- Significantly greater gains were found in tasks involving content creation.
- Human augmentation was observed, meaning human–AI systems performed better than humans alone.
Takeaway
Sometimes, when people and AI work together, they don't do as well as the best person or the best AI alone, but they can help each other in some tasks.
Methodology
A systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes.
Potential Biases
Potential publication bias favoring studies showing human augmentation.
Limitations
The analysis may not include all relevant studies and could be affected by publication bias.
Participant Demographics
Studies included various participant types, such as doctors, crowdworkers, and students.
Statistical Information
P-Value
0.005
Confidence Interval
−0.39 to −0.07
Statistical Significance
p<0.05
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website