Improved dilation CapsuleNet for motor imagery and mental arithmetic classification based on fNIRS

Figures & data

Table 1. Details of the four datasets.

Dataset	Subject	Paradigm	Task specification
Dataset A	8 (3males,5females, 26.0 ± 2.8 years old)	Mental arithmetic; Task period (12s); Rest period (28s)	In the instruction phase, participants memorized a subtraction problem on the screen. During the activity phase, they solved problems by subtracting a single-digit number sequentially. In the resting phase, participants took a break, following instructions to stare at a black screen.
Dataset B	29(14males,15females 28.5 ± 3.7 years old)	Mental arithmetic; Task period (10s) Rest period (15–17s)	The MA task in question shares similarities with Dataset D, while the BL task involved subjects being allowed to relax and fixate their gaze on a black cross that appeared before them. To mark the beginning and end of the task period, a brief 250 ms beep was played.
Dataset C	30 (17males,13females, 23.4 ± 2.5 years old)	Motor imagery; Task period (10s); Rest period (17–19s)	The task started with a brief sound, followed by on-screen instructions for movements with the right hand (RHT), left hand (LHT), and feet (FT). After completion, a rest period started with a sound and a “STOP” sign, signaling participants to relax. Volunteers performed assigned movements continuously throughout the task.
Dataset D	14	Motor imagery; Task period (21s); Rest period (15–21s)	Participants gripped with either hand in 10 cycles: 1.55s contraction, 0.55s relaxation, inducing rapid force changes. They followed instructions to exert force (25%–50% MVC), synchronized with the rhythm of cycles

Figure 1. The Conv_dilation extract hemodynamic response information from fNIRS.

Figure 2. The ID-CapsuleNet includes two Conv_dilations for feature extraction. After dynamic routing, the size of the output vectors is computed.

Table 2. Parameter settings of the model.

Datasets	Conv_dilation1	Conv_dilation2	p	q
Dataset A	kernel_size=(1,26)	kernel_size=(1,8)	1664	2
Dataset B	kernel_size=(1,26)	kernel_size=(1,16)	1152	2
Dataset C	kernel_size=(1,26)	kernel_size=(1,22)	640	3
Dataset D	kernel_size=(1,64)	kernel_size=(1,24)	768	2

Table 3. Average accuracy of the test sets.

Method	Dataset A	Dataset B	Dataset C	Dataset D
ANN	75.79 ± 4.32	58.66 ± 2.38	47.54 ± 2.07	56.28 ± 8.35
LSTM	88.45 ± 3.41	63.75 ± 1.65	61.94 ± 1.95	64.23 ± 3.46
CNN	87.00 ± 3.32	74.14 ± 1.78	72.89 ± 1.70	69.62 ± 5.04
CapsuleNet	60.81 ± 5.54	51.74 ± 1.19	36.92 ± 1.94	59.34 ± 3.13
fNIRSNet	95.59 ± 2.44	75.48 ± 1.64	73.16 ± 2.15	78.98 ± 4.41
ID-CapsuleNet	95.01 ± 2.41	76.88 ± 1.95	74.03 ± 2.26	80.29 ± 3.75

Figure 3. Classification accuracy for individual subjects.

Table 4. Average classification results of all individual subjects.

Dataset	Method	Accuracy	Precision	Recall	F1-score	Kappa
Dataset A	ANN	68.84 ± 9.92	70.18 ± 12.05	67.88 ± 8.74	68.73	0.38
	LSTM	83.68 ± 12.42	84.88 ± 12.62	82.99 ± 13.41	83.55	0.67
	CNN	82.47 ± 11.64	93.96 ± 13.59	82.29 ± 10.66	82.70	0.65
	CapsuleNet	85.42 ± 10.96	84.34 ± 12.93	88.89 ± 8.65	86.25	0.71
	fNIRSNet	87.85 ± 11.30	86.87 ± 13.00	90.97 ± 8.65	88.60	0.76
	ID-CapsuleNet	88.72 ± 10.47	88.20 ± 12.41	90.97 ± 8.87	89.25	0.77
Dataset B	ANN	58.68 ± 7.61	59.25 ± 8.71	62.30 ± 12.13	59.80	0.17
	LSTM	66.78 ± 6.81	69.53 ± 9.51	63.45 ± 11.09	65.35	0.34
	CNN	70.43 ± 7.63	72.65 ± 10.59	71.30 ± 13.95	71.41	0.42
	CapsuleNet	73.97 ± 6.80	75.38 ± 10.45	74.71 ± 12.94	73.85	0.48
	fNIRSNet	75.72 ± 7.45	76.67 ± 9.49	75.40 ± 16.53	75.11	0.52
	ID-CapsuleNet	75.80 ± 8.16	76.96 ± 10.04	76.90 ± 14.54	75.59	0.52
Dataset C	ANN	55.68 ± 9.86	56.88 ± 10.56	54.68 ± 10.37	55.47	0.29
	LSTM	69.73 ± 15.75	71.04 ± 15.38	69.73 ± 15.75	69.13	0.55
	CNN	73.27 ± 17.19	75.25 ± 16.51	73.27 ± 17.19	72.85	0.59
	CapsuleNet	33.38 ± 4.18	23.80 ± 9.89	33.38 ± 4.18	23.68	0.00
	fNIRSNet	75.47 ± 17.61	76.89 ± 17.40	75.47 ± 17.61	74.98	0.63
	ID-CapsuleNet	75.78 ± 16.73	77.52 ± 15.83	75.78 ± 16.73	75.16	0.64
Dataset D	ANN	52.97 ± 4.70	38.66 ± 19.12	52.97 ± 4.70	41.07	0.06
	LSTM	56.90 ± 6.37	63.62 ± 17.68	56.90 ± 6.37	48.20	0.14
	CNN	58.61 ± 12.24	58.12 ± 20.26	58.61 ± 12.24	53.56	0.17
	CapsuleNet	66.32 ± 7.61	61.50 ± 19.46	63.96 ± 13.70	62.39	0.25
	fNIRSNet	80.51 ± 11.20	82.05 ± 14.14	82.45 ± 14.08	81.00	0.61
	ID-CapsuleNet	80.34 ± 7.04	81.81 ± 7.28	80.34 ± 7.04	80.11	0.61

Table 5. Average classification results of different convolution.

Method	Dataset A	Dataset B	Dataset C	Dataset D
Convolution	94.96 ± 2.08	76.61 ± 1.37	73.42 ± 1.69	79.78 ± 2.85
Dilation Convolution	95.01 ± 2.41	76.88 ± 1.95	74.03 ± 2.26	80.29 ± 3.75

Table 6. The parameters of different convolution.

Method	Dataset A	Dataset B	Dataset C	Dataset D
Convolution	894784	927552	960320	991296
Dilation Convolution	560704	544320	527936	598336

Table 7. Average classification results of different loss.

Method	Dataset A	Dataset B	Dataset C	Dataset D
CapsuleLoss	91.39 ± 3.04	68.87 ± 2.03	67.24 ± 1.38	63.93 ± 4.13
LabelSmoothing	95.01 ± 2.41	76.88 ± 1.95	74.03 ± 2.26	80.29 ± 3.75

Table 8. Average accuracy of the test sets.

Method	Dataset A	Dataset B	Dataset C	Dataset D
CNN	87.00 ± 3.32	74.14 ± 1.78	72.89 ± 1.70	69.62 ± 5.04
CapsuleNet	60.81 ± 5.54	51.74 ± 1.19	36.92 ± 1.94	59.34 ± 3.13
ID-CNN	93.19 ± 2.56	74.56 ± 1.87	73.59 ± 2.23	79.25 ± 4.87
ID-CapsuleNet	95.01 ± 2.41	76.88 ± 1.95	74.03 ± 2.26	80.29 ± 3.75