Amit Konar, Aruna Chakraborty

Emotion Recognition (eBook, PDF)

A Pattern Analysis Approach

• Format: PDF

Produktbeschreibung

A timely book containing foundations and current research directions on emotion recognition by facial expression, voice, gesture and biopotential signals

This book provides a comprehensive examination of the research methodology of different modalities of emotion recognition. Key topics of discussion include facial expression, voice and biopotential signal-based emotion recognition. Special emphasis is given to feature selection, feature reduction, classifier design and multi-modal fusion to improve performance of emotion-classifiers.

Written by several experts, the book includes several tools and techniques, including dynamic Bayesian networks, neural nets, hidden Markov model, rough sets, type-2 fuzzy sets, support vector machines and their applications in emotion recognition by different modalities. The book ends with a discussion on emotion recognition in automotive fields to determine stress and anger of the drivers, responsible for degradation of their performance and driving-ability.

There is an increasing demand of emotion recognition in diverse fields, including psycho-therapy, bio-medicine and security in government, public and private agencies. The importance of emotion recognition has been given priority by industries including Hewlett Packard in the design and development of the next generation human-computer interface (HCI) systems.

Emotion Recognition: A Pattern Analysis Approach would be of great interest to researchers, graduate students and practitioners, as the book

• Offers both foundations and advances on emotion recognition in a single volume
• Provides a thorough and insightful introduction to the subject by utilizing computational tools of diverse domains
• Inspires young researchers to prepare themselves for their own research
• Demonstrates direction of future research through new technologies, such as Microsoft Kinect, EEG systems etc.

Produktdetails

• Verlag: John Wiley & Sons
• Erscheinungstermin: 10. Dezember 2014
• Englisch
• ISBN-13: 9781118910610
• Artikelnr.: 42009766

Autorenporträt

Amit Konar is a Professor of Electronics and Tele-Communication Engineering, Jadavpur University, India, where he offers graduate-level courses on Artificial Intelligence and directs research in Cognitive Science, Robotics and Human-Computer Interfaces. Dr. Konar is the recipient of many prestigious grants and awards and is an author of 10 books and over 350 research publications. He offered consultancy services to Government and private industries. He served editorial services to many journals, including IEEE Transactions on Systems, Man and Cybernetics (Part-A) and IEEE Transactions on Fuzzy Systems.

Aruna Chakraborty is an Associate Professor with the Department of Computer Science and Engineering, St. Thomas' College of Engineering and Technology, India. She is also a Visiting Faculty with Jadavpur University, where she offers graduate-level courses on Intelligent Automation and Robotics, and Cognitive Science. Her research interest includes human-computer interfaces, emotional intelligence and reasoning with fuzzy logic.

Inhaltsangabe

Preface xix

Acknowledgments xxvii

Contributors xxix

1 Introduction to Emotion Recognition 1
Amit Konar, Anisha Halder, and Aruna Chakraborty

1.1 Basics of Pattern Recognition, 1

1.2 Emotion Detection as a Pattern Recognition Problem, 2

1.3 Feature Extraction, 3

1.4 Feature Reduction Techniques, 15

1.5 Emotion Classification, 17

1.6 Multimodal Emotion Recognition, 24

1.7 Stimulus Generation for Emotion Arousal, 24

1.8 Validation Techniques, 26

1.9 Summary, 27

References, 28

Author Biographies, 44

2 Exploiting Dynamic Dependencies Among Action Units for Spontaneous Facial Action Recognition 47
Yan Tong and Qiang Ji

2.1 Introduction, 48

2.2 Related Work, 49

2.3 Modeling the Semantic and Dynamic Relationships Among AUs With a DBN, 50

2.4 Experimental Results, 60

2.5 Conclusion, 64

References, 64

Author Biographies, 66

3 Facial Expressions: A Cross-Cultural Study 69
Chandrani Saha, Washef Ahmed, Soma Mitra, Debasis Mazumdar, and Sushmita Mitra

3.1 Introduction, 69

3.2 Extraction of Facial Regions and Ekman’s Action Units, 71

3.3 Cultural Variation in Occurrence of Different AUs, 76

3.4 Classification Performance Considering Cultural Variability, 79

3.5 Conclusion, 84

References, 84

Author Biographies, 86

4 A Subject-Dependent Facial Expression Recognition System 89
Chuan-Yu Chang and Yan-Chiang Huang

4.1 Introduction, 89

4.2 Proposed Method, 91

4.3 Experiment Result, 103

4.4 Conclusion, 109

Acknowledgment, 110

References, 110

Author Biographies, 112

5 Facial Expression Recognition Using Independent Component Features and Hidden Markov Model 113
Md. Zia Uddin and Tae-Seong Kim

5.1 Introduction, 114

5.2 Methodology, 115

5.3 Experimental Results, 123

5.4 Conclusion, 125

Acknowledgments, 125

References, 126

Author Biographies, 127

6 Feature Selection for Facial Expression Based on Rough Set Theory 129
Yong Yang and Guoyin Wang

6.1 Introduction, 129

6.2 Feature Selection for Emotion Recognition Based on Rough Set Theory, 131

6.3 Experiment Results and Discussion, 137

6.4 Conclusion, 143

Acknowledgments, 143

References, 143

Author Biographies, 145

7 Emotion Recognition from Facial Expressions Using Type-2 Fuzzy Sets 147
Anisha Halder, Amit Konar, Aruna Chakraborty, and Atulya K. Nagar

7.1 Introduction, 148

7.2 Preliminaries on Type-2 Fuzzy Sets, 150

7.3 Uncertainty Management in Fuzzy-Space for Emotion Recognition, 152

7.4 Fuzzy Type-2 Membership Evaluation, 157

7.5 Experimental Details, 161

7.6 Performance Analysis, 167

7.7 Conclusion, 175

References, 176

Author Biographies, 180

8 Emotion Recognition from Non-frontal Facial Images 183
Wenming Zheng, Hao Tang, and Thomas S. Huang

8.1 Introduction, 184

8.2 A Brief Review of Automatic Emotional Expression Recognition, 187

8.3 Databases for Non-frontal Facial Emotion Recognition, 191

8.4 Recent Advances of Emotion Recognition from Non-Frontal Facial Images, 196

8.5 Discussions and Conclusions, 205

Acknowledgments, 206

References, 206

Author Biographies, 211

9 Maximum a Posteriori Based Fusion Method for Speech Emotion Recognition 215
Ling Cen, Zhu Liang Yu, and Wee Ser

9.1 Introduction, 216

9.2 Acoustic Feature Extraction for Emotion Recognition, 219

9.3 Proposed Map-Based Fusion Method, 223

9.4 Experiment, 229

9.5 Conclusion, 232

References, 232

Author Biographies, 234

10 Emotion Recognition in Naturalistic Speech and Language—A Survey 237
Felix Weninger, Martin W¨ollmer, and Björn Schuller

10.1 Introduction, 238

10.2 Tasks and Applications, 239

10.3 Implementation and Evaluation, 244

10.4 Challenges, 253

10.5 Conclusion and Outlook, 257

Acknowledgment, 259

References, 259

Author Biographies, 267

11 EEG-Based Emotion Recognition Using Advanced Signal Processing Techniques 269
Panagiotis C. Petrantonakis and Leontios J. Hadjileontiadis

11.1 Introduction, 270

11.2 Brain Activity and Emotions, 271

11.3 EEG-ER Systems: An Overview, 272

11.4 Emotion Elicitation, 273

11.5 Advanced Signal Processing in EEG-ER, 275

11.6 Concluding Remarks and Future Directions, 287

References, 289

Author Biographies, 292

12 Frequency Band Localization on Multiple Physiological Signals for Human Emotion Classification Using DWT 295
M. Murugappan

12.1 Introduction, 296

12.2 Related Work, 297

12.3 Research Methodology, 299

12.4 Experimental Results and Discussions, 306

12.5 Conclusion, 310

12.6 Future Work, 310

Acknowledgments, 310

References, 310

Author Biography, 312

13 Toward Affective Brain–Computer Interface: Fundamentals and Analysis of EEG-Based Emotion Classification 315
Yuan-Pin Lin, Tzyy-Ping Jung, Yijun Wang, and Julie Onton

13.1 Introduction, 316

13.2 Materials and Methods, 323

13.3 Results and Discussion, 327

13.4 Conclusion, 332

13.5 Issues and Challenges Toward ABCIs, 332

Acknowledgments, 336

References, 336

Author Biographies, 340

14 Bodily Expression for Automatic Affect Recognition 343
Hatice Gunes, Caifeng Shan, Shizhi Chen, and YingLi Tian

14.1 Introduction, 344

14.2 Background and Related Work, 345

14.3 Creating a Database of Facial and Bodily Expressions: The FABO Database, 353

14.4 Automatic Recognition of Affect from Bodily Expressions, 356

14.5 Automatic Recognition of Bodily Expression Temporal Dynamics, 361

14.6 Discussion and Outlook, 367

14.7 Conclusions, 369

Acknowledgments, 370

References, 370

Author Biographies, 375

15 Building a Robust System for Multimodal Emotion Recognition 379
Johannes Wagner, Florian Lingenfelser, and Elisabeth André

15.1 Introduction, 380

15.2 Related Work, 381

15.3 The Callas Expressivity Corpus, 382

15.4 Methodology, 386

15.5 Multisensor Data Fusion, 390

15.6 Experiments, 395

15.7 Online Recognition System, 399

15.8 Conclusion, 403

Acknowledgment, 404

References, 404

Author Biographies, 410

16 Semantic Audiovisual Data Fusion for Automatic Emotion Recognition 411
Dragos Datcu and Leon J. M. Rothkrantz

16.1 Introduction, 412

16.2 Related Work, 413

16.3 Data Set Preparation, 416

16.4 Architecture, 418

16.5 Results, 431

16.6 Conclusion, 432

References, 432

Author Biographies, 434

17 A Multilevel Fusion Approach for Audiovisual Emotion Recognition 437
Girija Chetty, Michael Wagner, and Roland Goecke

17.1 Introduction, 437

17.2 Motivation and Background, 438

17.3 Facial Expression Quantification, 440

17.4 Experiment Design, 444

17.5 Experimental Results and Discussion, 450

17.6 Conclusion, 456

References, 456

Author Biographies, 459

18 From a Discrete Perspective of Emotions to Continuous, Dynamic, and Multimodal Affect Sensing 461
Isabelle Hupont, Sergio Ballano, Eva Cerezo, and Sandra Baldassarri

18.1 Introduction, 462

18.2 A Novel Method for Discrete Emotional Classification of Facial Images, 465

18.3 A 2D Emotional Space for Continuous and Dynamic Facial Affect Sensing, 469

18.4 Expansion to Multimodal Affect Sensing, 474

18.5 Building Tools That Care, 479

18.6 Concluding Remarks and Future Work, 486

Acknowledgments, 488

References, 488

Author Biographies, 491

19 Audiovisual Emotion Recognition Using Semi-Coupled Hidden Markov Model with State-Based Alignment Strategy 493
Chung-Hsien Wu, Jen-Chun Lin, and Wen-Li Wei

19.1 Introduction, 494

19.2 Feature Extraction, 495

19.3 Semi-Coupled Hidden Markov Model, 500

19.4 Experiments, 504

19.5 Conclusion, 508

References, 509

Author Biographies, 512

20 Emotion Recognition in Car Industry 515
Christos D. Katsis, George Rigas, Yorgos Goletsis, and Dimitrios I. Fotiadis

20.1 Introduction, 516

20.2 An Overview of Application for the Car Industry, 517

20.3 Modality-Based Categorization, 517

20.4 Emotion-Based Categorization, 520

20.5 Two Exemplar Cases, 523

20.6 Open Issues and Future Steps, 536

20.7 Conclusion, 537

References, 537

Author Biographies, 543

Index 545

---

Preface xix
Acknowledgments xxvii
Contributors xxix
1 Introduction to Emotion Recognition 1
Amit Konar, Anisha Halder, and Aruna Chakraborty
1.1 Basics of Pattern Recognition, 1
1.2 Emotion Detection as a Pattern Recognition Problem, 2
1.3 Feature Extraction, 3
1.4 Feature Reduction Techniques, 15
1.5 Emotion Classification, 17
1.6 Multimodal Emotion Recognition, 24
1.7 Stimulus Generation for Emotion Arousal, 24
1.8 Validation Techniques, 26
1.9 Summary, 27
References, 28
Author Biographies, 44
2 Exploiting Dynamic Dependencies Among Action Units for Spontaneous Facial
Action Recognition 47
Yan Tong and Qiang Ji
2.1 Introduction, 48
2.2 Related Work, 49
2.3 Modeling the Semantic and Dynamic Relationships Among AUs With a DBN,
50
2.4 Experimental Results, 60
2.5 Conclusion, 64
References, 64
Author Biographies, 66
3 Facial Expressions: A Cross-Cultural Study 69
Chandrani Saha, Washef Ahmed, Soma Mitra, Debasis Mazumdar, and Sushmita
Mitra
3.1 Introduction, 69
3.2 Extraction of Facial Regions and Ekman's Action Units, 71
3.3 Cultural Variation in Occurrence of Different AUs, 76
3.4 Classification Performance Considering Cultural Variability, 79
3.5 Conclusion, 84
References, 84
Author Biographies, 86
4 A Subject-Dependent Facial Expression Recognition System 89
Chuan-Yu Chang and Yan-Chiang Huang
4.1 Introduction, 89
4.2 Proposed Method, 91
4.3 Experiment Result, 103
4.4 Conclusion, 109
Acknowledgment, 110
References, 110
Author Biographies, 112
5 Facial Expression Recognition Using Independent Component Features and
Hidden Markov Model 113
Md. Zia Uddin and Tae-Seong Kim
5.1 Introduction, 114
5.2 Methodology, 115
5.3 Experimental Results, 123
5.4 Conclusion, 125
Acknowledgments, 125
References, 126
Author Biographies, 127
6 Feature Selection for Facial Expression Based on Rough Set Theory 129
Yong Yang and Guoyin Wang
6.1 Introduction, 129
6.2 Feature Selection for Emotion Recognition Based on Rough Set Theory,
131
6.3 Experiment Results and Discussion, 137
6.4 Conclusion, 143
Acknowledgments, 143
References, 143
Author Biographies, 145
7 Emotion Recognition from Facial Expressions Using Type-2 Fuzzy Sets 147
Anisha Halder, Amit Konar, Aruna Chakraborty, and Atulya K. Nagar
7.1 Introduction, 148
7.2 Preliminaries on Type-2 Fuzzy Sets, 150
7.3 Uncertainty Management in Fuzzy-Space for Emotion Recognition, 152
7.4 Fuzzy Type-2 Membership Evaluation, 157
7.5 Experimental Details, 161
7.6 Performance Analysis, 167
7.7 Conclusion, 175
References, 176
Author Biographies, 180
8 Emotion Recognition from Non-frontal Facial Images 183
Wenming Zheng, Hao Tang, and Thomas S. Huang
8.1 Introduction, 184
8.2 A Brief Review of Automatic Emotional Expression Recognition, 187
8.3 Databases for Non-frontal Facial Emotion Recognition, 191
8.4 Recent Advances of Emotion Recognition from Non-Frontal Facial Images,
196
8.5 Discussions and Conclusions, 205
Acknowledgments, 206
References, 206
Author Biographies, 211
9 Maximum a Posteriori Based Fusion Method for Speech Emotion Recognition
215
Ling Cen, Zhu Liang Yu, and Wee Ser
9.1 Introduction, 216
9.2 Acoustic Feature Extraction for Emotion Recognition, 219
9.3 Proposed Map-Based Fusion Method, 223
9.4 Experiment, 229
9.5 Conclusion, 232
References, 232
Author Biographies, 234
10 Emotion Recognition in Naturalistic Speech and Language-A Survey 237
Felix Weninger, Martin W¿ollmer, and Björn Schuller
10.1 Introduction, 238
10.2 Tasks and Applications, 239
10.3 Implementation and Evaluation, 244
10.4 Challenges, 253
10.5 Conclusion and Outlook, 257
Acknowledgment, 259
References, 259
Author Biographies, 267
11 EEG-Based Emotion Recognition Using Advanced Signal Processing
Techniques 269
Panagiotis C. Petrantonakis and Leontios J. Hadjileontiadis
11.1 Introduction, 270
11.2 Brain Activity and Emotions, 271
11.3 EEG-ER Systems: An Overview, 272
11.4 Emotion Elicitation, 273
11.5 Advanced Signal Processing in EEG-ER, 275
11.6 Concluding Remarks and Future Directions, 287
References, 289
Author Biographies, 292
12 Frequency Band Localization on Multiple Physiological Signals for Human
Emotion Classification Using DWT 295
M. Murugappan
12.1 Introduction, 296
12.2 Related Work, 297
12.3 Research Methodology, 299
12.4 Experimental Results and Discussions, 306
12.5 Conclusion, 310
12.6 Future Work, 310
Acknowledgments, 310
References, 310
Author Biography, 312
13 Toward Affective Brain-Computer Interface: Fundamentals and Analysis of
EEG-Based Emotion Classification 315
Yuan-Pin Lin, Tzyy-Ping Jung, Yijun Wang, and Julie Onton
13.1 Introduction, 316
13.2 Materials and Methods, 323
13.3 Results and Discussion, 327
13.4 Conclusion, 332
13.5 Issues and Challenges Toward ABCIs, 332
Acknowledgments, 336
References, 336
Author Biographies, 340
14 Bodily Expression for Automatic Affect Recognition 343
Hatice Gunes, Caifeng Shan, Shizhi Chen, and YingLi Tian
14.1 Introduction, 344
14.2 Background and Related Work, 345
14.3 Creating a Database of Facial and Bodily Expressions: The FABO
Database, 353
14.4 Automatic Recognition of Affect from Bodily Expressions, 356
14.5 Automatic Recognition of Bodily Expression Temporal Dynamics, 361
14.6 Discussion and Outlook, 367
14.7 Conclusions, 369
Acknowledgments, 370
References, 370
Author Biographies, 375
15 Building a Robust System for Multimodal Emotion Recognition 379
Johannes Wagner, Florian Lingenfelser, and Elisabeth André
15.1 Introduction, 380
15.2 Related Work, 381
15.3 The Callas Expressivity Corpus, 382
15.4 Methodology, 386
15.5 Multisensor Data Fusion, 390
15.6 Experiments, 395
15.7 Online Recognition System, 399
15.8 Conclusion, 403
Acknowledgment, 404
References, 404
Author Biographies, 410
16 Semantic Audiovisual Data Fusion for Automatic Emotion Recognition 411
Dragos Datcu and Leon J. M. Rothkrantz
16.1 Introduction, 412
16.2 Related Work, 413
16.3 Data Set Preparation, 416
16.4 Architecture, 418
16.5 Results, 431
16.6 Conclusion, 432
References, 432
Author Biographies, 434
17 A Multilevel Fusion Approach for Audiovisual Emotion Recognition 437
Girija Chetty, Michael Wagner, and Roland Goecke
17.1 Introduction, 437
17.2 Motivation and Background, 438
17.3 Facial Expression Quantification, 440
17.4 Experiment Design, 444
17.5 Experimental Results and Discussion, 450
17.6 Conclusion, 456
References, 456
Author Biographies, 459
18 From a Discrete Perspective of Emotions to Continuous, Dynamic, and
Multimodal Affect Sensing 461
Isabelle Hupont, Sergio Ballano, Eva Cerezo, and Sandra Baldassarri
18.1 Introduction, 462
18.2 A Novel Method for Discrete Emotional Classification of Facial Images,
465
18.3 A 2D Emotional Space for Continuous and Dynamic Facial Affect Sensing,
469
18.4 Expansion to Multimodal Affect Sensing, 474
18.5 Building Tools That Care, 479
18.6 Concluding Remarks and Future Work, 486
Acknowledgments, 488
References, 488
Author Biographies, 491
19 Audiovisual Emotion Recognition Using Semi-Coupled Hidden Markov Model
with State-Based Alignment Strategy 493
Chung-Hsien Wu, Jen-Chun Lin, and Wen-Li Wei
19.1 Introduction, 494
19.2 Feature Extraction, 495
19.3 Semi-Coupled Hidden Markov Model, 500
19.4 Experiments, 504
19.5 Conclusion, 508
References, 509
Author Biographies, 512
20 Emotion Recognition in Car Industry 515
Christos D. Katsis, George Rigas, Yorgos Goletsis, and Dimitrios I.
Fotiadis
20.1 Introduction, 516
20.2 An Overview of Application for the Car Industry, 517
20.3 Modality-Based Categorization, 517
20.4 Emotion-Based Categorization, 520
20.5 Two Exemplar Cases, 523
20.6 Open Issues and Future Steps, 536
20.7 Conclusion, 537
References, 537
Author Biographies, 543
Index 545