Humanoid robot as an educational assistant – insights of speech recognition for online and offline mode of teaching

Figures & data

Figure 1. Overview of the working of the proposed system with humanoid robot in offline and online settings.

Figure 2. Analysis of features in the online mode of teaching with male participant.

Figure 3. Analysis of features in offline mode of teaching with male participant.

Figure 4. Analysis of features in the online mode of teaching with a female participant.

Figure 5. Analysis of features in the offline mode of teaching with a female participant.

Table 1. Features selected for male participants in the online mode of teaching.

Rouge1_fmeasure	Rouge2_fmeasure	RougeL_fmeasure	bleu_score	Jaccard_Similarity_Score	WER
0.9412	0.9333	0.9412	0.6004	0.5455	0.3333
0.8929	0.8519	0.8929	0.8266	0.8077	0.1071
0.7429	0.6061	0.7429	0.4225	0.5	0.3889
0.8182	0.7	0.8182	0.6989	0.6923	0.1818
0.8276	0.6786	0.8276	0.5197	0.6452	0.3333
0.9091	0.9032	0.9091	0.8702	0.8125	0.125
0.9091	0.8387	0.9091	0.7625	0.8235	0.125
0.8571	0.7368	0.8571	0.7071	0.8182	0.1
0.8	0.6429	0.8	0.4593	0.6667	0.2857
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
0.8889	0.8	0.8889	0.7612	0.8462	0.0769
0.8889	0.75	0.8889	0.7506	0.8	0.1111
0.6857	0.303	0.5714	0.0	0.4783	0.5882
0.7273	0.7097	0.7273	0.3595	0.5	0.9167
0.8718	0.7568	0.8718	0.6859	0.7368	0.1667
1.0	1.0	1.0	0.0	0.4	0.4286
1.0	1.0	1.0	0.726	0.6364	0.2222
0.8214	0.7407	0.8214	0.6824	0.6207	0.25
0.7458	0.5965	0.7458	0.4779	0.6061	0.3
0.8125	0.6667	0.8125	0.5681	0.6471	0.1875
0.8125	0.7333	0.8125	0.7368	0.7222	0.1875
0.7619	0.7368	0.7619	0.7598	0.6667	0.2
0.9286	0.8462	0.9286	0.8555	0.9286	0.0714
0.3462	0.24	0.3077	0.141	0.1707	3.0
0.7692	0.6667	0.7692	0.5784	0.6	0.2308
0.4898	0.2979	0.449	0.1412	0.303	1.2941
0.75	0.6364	0.75	0.5311	0.6	0.25
0.7887	0.6377	0.7606	0.4947	0.6774	0.3548

Table 2. Features selected for male participants in the offline mode of teaching.

Rouge1_fmeasure	Rouge2_fmeasure	RougeL_fmeasure	bleu_score	Jaccard_Similarity_Score	WER
1.0	1.0	1.0	0.0	0.4	0.4286
0.8889	0.75	0.8889	0.3226	0.4167	0.4444
0.9455	0.9434	0.9455	0.8879	0.8462	0.1071
0.9189	0.8	0.9189	0.629	0.7778	0.1667
0.9375	0.871	0.9375	0.7975	0.871	0.1667
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
0.88	0.8696	0.88	0.68	0.6667	0.2857
0.7826	0.7619	0.7826	0.7103	0.6429	0.2727
0.8571	0.8462	0.8571	0.7577	0.6471	0.2143
1.0	1.0	1.0	1.0	1.0	0.0
0.9091	0.9	0.9091	0.8932	0.8182	0.0909
0.9286	0.9231	0.9286	0.8507	0.8571	0.1538
1.0	1.0	1.0	1.0	1.0	0.0
0.9412	0.875	0.9412	0.8282	0.8889	0.0588
1.0	1.0	1.0	0.0	0.4	0.4286
0.9474	0.8235	0.9474	0.6389	0.5833	0.3333
0.9643	0.963	0.9643	0.9622	0.92	0.0357
1.0	1.0	1.0	0.6899	0.8235	0.1111
1.0	1.0	1.0	1.0	1.0	0.0
0.9655	0.8929	0.9655	0.7823	0.8966	0.1
0.9825	0.9455	0.9825	0.7728	0.9231	0.1111
0.9375	0.9333	0.9375	0.9306	0.8667	0.0625
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
0.9286	0.8462	0.9286	0.7825	0.8667	0.0714
0.9524	0.8421	0.9524	0.7109	0.9091	0.0909
0.7857	0.6923	0.7857	0.645	0.6471	0.2143
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
0.96	0.8696	0.96	0.7677	0.9167	0.0769
1.0	1.0	1.0	1.0	1.0	0.0
0.8824	0.75	0.8824	0.6423	0.7895	0.1176
1.0	1.0	1.0	0.7825	0.8182	0.1
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0

Table 3. Features selected for female participants in the online mode of teaching.

Rouge1_fmeasure	Rouge2_fmeasure	RougeL_fmeasure	bleu_score	Jaccard_Similarity_Score	WER
0.6667	0.64	0.6667	0.2646	0.3	jan.33
0.8929	0.8148	0.8929	0.7372	0.7407	0.1429
0.9444	0.8824	0.9444	0.6269	0.7222	0.1667
0.9091	0.9	0.9091	0.8932	0.8333	0.0909
0.918	0.8475	0.918	0.7203	0.7273	0.1667
0.9286	0.8889	0.9286	0.7637	0.8214	0.1111
0.8387	0.6897	0.8387	0.6423	0.75	0.1875
0.9375	0.8667	0.9375	0.8762	0.9333	0.0625
0.8571	0.8421	0.8571	0.7861	0.75	0.2
0.8276	0.6667	0.7586	0.5703	0.6471	0.2857
0.9524	0.9474	0.9524	0.7964	0.75	0.1818
0.8571	0.6923	0.8571	0.6298	0.75	0.1429
1.0	1.0	1.0	1.0	1.0	0.0
0.8182	0.7	0.8182	0.6875	0.5833	0.2727
1.0	1.0	1.0	1.0	1.0	0.0
0.7778	0.5	0.7778	0.3689	0.6364	0.2222
0.8889	0.7059	0.8889	0.4531	0.6364	0.2941
0.7619	0.6316	0.7619	0.4692	0.6154	0.3
0.88	0.7826	0.88	0.6652	0.7143	0.25
0.9474	0.8889	0.9474	0.7966	0.7778	0.1667
0.9143	0.8824	0.9143	0.6126	0.7	0.2581
1.0	1.0	1.0	0.0	0.4	0.4286
0.8889	0.75	0.8889	0.3051	0.3077	0.5556
0.9286	0.8889	0.9286	0.8646	0.8462	0.0714
1.0	1.0	1.0	0.8394	0.9375	0.0556
1.0	1.0	1.0	1.0	1.0	0.0
0.8182	0.7813	0.8182	0.4759	0.5526	0.5556
0.8387	0.7586	0.8387	0.6296	0.6471	0.25
1.0	1.0	1.0	0.8154	0.875	0.0625
0.8421	0.7059	0.8421	0.6717	0.7273	0.2
0.963	0.88	0.963	0.5799	0.6875	0.2143
0.9091	0.8	0.9091	0.7017	0.8333	0.0909
0.8571	0.7692	0.8571	0.6998	0.8	0.1429
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
0.9231	0.8333	0.9231	0.7612	0.8462	0.0769
1.0	1.0	1.0	1.0	1.0	0.0
0.9412	0.875	0.9412	0.8844	0.8889	0.0588
0.75	0.6364	0.75	0.3556	0.5714	0.6
1.0	1.0	1.0	1.0	1.0	0.0
0.8718	0.8108	0.8718	0.7332	0.7778	0.1667
0.9091	0.8438	0.9091	0.5615	0.6667	0.1935

Table 4. Features selected for female participants in the offline mode of teaching.

Rouge1_fmeasure	Rouge2_fmeasure	RougeL_fmeasure	bleu_score	Jaccard_Similarity_Score	WER
1.0	1.0	1.0	0.0	0.4	0.4286
1.0	1.0	1.0	0.5874	0.5	0.3333
0.8364	0.7547	0.8364	0.5722	0.6	0.2857
0.7119	0.5614	0.7119	0.3081	0.5152	0.4828
0.8333	0.6471	0.8333	0.5731	0.7222	0.1667
0.8182	0.8	0.8182	0.7861	0.6923	0.1818
0.8852	0.7797	0.8852	0.7226	0.75	0.1667
1.0	1.0	1.0	1.0	1.0	0.0
0.9375	0.8667	0.9375	0.6865	0.8125	0.125
0.9524	0.8421	0.9524	0.6989	0.75	0.2
1.0	1.0	1.0	0.7825	0.8667	0.0714
0.9091	0.8	0.9091	0.7017	0.8333	0.0909
0.8571	0.6154	0.7857	0.5565	0.75	0.2143
1.0	1.0	1.0	1.0	1.0	0.0
0.7857	0.7692	0.7857	0.6102	0.6667	0.5455
0.6154	0.5	0.6154	0.2928	0.4375	1.1111
0.9714	0.9091	0.9714	0.716	0.75	0.1765
0.9	0.8889	0.9	0.5254	0.6667	0.2
0.9167	0.8182	0.9167	0.7349	0.9167	0.0833
0.7727	0.6667	0.7727	0.5331	0.6667	0.4444
0.9412	0.9333	0.9412	0.6004	0.5455	0.3333
0.9444	0.8824	0.9444	0.6269	0.7647	0.1667
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
0.9375	0.8667	0.9375	0.8762	0.9333	0.0625
0.8571	0.8421	0.8571	0.7861	0.75	0.2
0.8387	0.7586	0.8387	0.6739	0.8125	0.2143
0.6667	0.5714	0.6	0.4561	0.4286	0.9286
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
0.8571	0.7273	0.8571	0.6654	0.75	0.1765
0.9	0.8889	0.9	0.8801	0.8182	0.1
1.0	1.0	1.0	1.0	1.0	0.0
1.0	1.0	1.0	1.0	1.0	0.0
0.9394	0.875	0.9394	0.4682	0.6129	0.2903

Figure 6. Visualization of average features in online and offline modes of teaching.