Multi-Level Feature Fusion in CNN-Based Human Action Recognition
Author Information
Author(s): Lueangwitchajaroen Pitiwat, Watcharapinchai Sitapa, Tepsan Worawit, Sooksatra Sorn
Primary Institution: National Electronic and Computer Technology Center, National Science and Technology Development Agency, Thailand
Hypothesis
Can a multi-level fusion approach improve human action recognition using only RGB frames?
Conclusion
The study demonstrates that a multi-level fusion technique can significantly enhance human action recognition accuracy using only RGB data.
Supporting Evidence
- The proposed method achieved 91.5% accuracy on the NTU RGB+D 60 dataset.
- Using only RGB data, the model performed comparably to those using additional data sources.
- Multi-level fusion allows for better integration of spatial and temporal information.
Takeaway
This study shows that we can recognize human actions better by combining different types of information, even if we only use regular video frames.
Methodology
The study used RGB frames from the NTU RGB+D dataset, extracting 2D skeleton coordinates and optical flow frames, and evaluated a multi-level fusion approach with EfficientNet-B7.
Potential Biases
The pre-trained pose estimation model was not specifically trained on the dataset, which may affect accuracy.
Limitations
The model may struggle with more complex datasets and lacks an attention mechanism, which could limit performance on intricate class sets.
Participant Demographics
The NTU RGB+D dataset includes 40 unique subjects aged 10 to 35, representing diversity in age, gender, and height.
Statistical Information
P-Value
0.0001
Statistical Significance
p<0.05
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website