Multi-task learning based feature extraction method in signal processing of high resolution remote sensing video images

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Test results of different models on the WHDLD data set

Models	Building	Roads	Walkway	Vegetation	Naked soil	Water	Average (%)
MD TANet	62.61	68.74	49.54	82.01	48.96	93.44	67.55
U-Net	60.41	59.42	42.01	78.71	36.97	90.79	61.39
EANet	53.11	63.63	47.76	79.21	35.15	90.76	61.6
PSPNet	60.12	62.75	46.89	73.29	32.99	92.86	61.54
BiseNetv2	58.35	64.14	48.18	71.91	32.57	91.09	61.04
CCNet	61.51	64.25	45.86	77.95	39.03	93.02	63.6
RefineNet	61.14	63.02	42.01	79.39	37.72	93.29	62.76
EMANet	59.69	62.73	47.87	73.11	37.36	92.18	62.16
Deeplabv3+	61.57	62.93	46.98	79.62	35.05	92.19	63.06

Test results of different models on the DLRSD data set

Models	MD TANet	EANet	CCNet	RefineNet	Deeplabv3+
Airplane	68.30	64.65	62.58	62.89	64.76
Naked soil	59.05	53.16	54.51	53.89	53.49
Architecture	71.44	74.51	73.38	72.40	72.39
Car	67.53	62.20	65.23	64.38	65.20
Church	59.33	53.020	51.380	53.110	53.55
Courtyard	84.99	81.28	80.53	81.72	80.22
Quay	54.60	42.82	46.30	46.88	45.73
Field	91.83	93.6	90.93	94.66	93.25
Grass	64.28	66.18	65.11	64.78	64.88
Mobile house	61.07	63.96	62.1	61.06	60.84
Walkway	77.61	76.94	73.11	75.71	76.83
Sand beach	65.71	71.78	62.75	68.89	68.18
Ocean	96.73	93.25	90.79	92.56	91.86
Ship	77.07	65.04	70.23	72.28	70.15
Tanks	71.69	76.44	75.07	73.61	68.31
Tree	74.76	72.60	73.35	69.93	71.01
Other waters	83.44	82.18	81.75	80.98	80.04
Average (%)	72.20	70.21	69.36	69.98	69.45

Test results of different models on the Vaihingen data set

Models	Roads	Building	Dwarf vegetation	Tree	Car	Other	Average (%)
MD TANet	85.92	87.81	75.71	78.34	68.89	59.22	75.98
U-Net	80.25	80.34	67.55	72.26	65.75	57.55	70.62
EANet	81.49	83.84	71.53	74.96	66.86	55.16	72.31
PSPNet	80.56	80.15	67.71	69.71	67.62	51.85	69.60
BiseNetv2	82.91	82.51	73.74	73.83	68.51	55.41	72.82
CCNet	82.42	83.57	73.15	73.84	70.47	57.82	73.55
RefineNet	83.14	82.21	71.87	70.28	69.02	58.37	72.48
EMANet	81.51	81.84	69.88	69.37	66.18	55.33	70.69
Deeplabv3+	83.93	83.54	73.57	74.21	69.52	56.48	73.54

Experimental platform configuration

	Name	Configuring
Hardware	CPU	Intel(R) Xeon(R) Gold 5115 CPU@2.40GHz
	Memory	128GB
	GPU	Tesla P100
	Display storage	16GB
Software	Programming language	Python 3.7.15
	Compiling environment	Pycharm
	Anaconda version	Conda 4.10.3
	Depth learning framework	Pytorch 1.9.0
	Operating system	CentOS 7.7

The evaluation results of different models in the validation set

Model	Input size	mAP@50	mIoU@50	FPS	Parameters	FLOPS
U-Net	256*256	73.86	70.62	82.42	10.57	25.23
EANet	256*256	75.88	72.31	94.22	7.93	28.13
PSPNet	256*256	72.14	69.60	73.35	11.49	22.12
BiseNetv2	256*256	74.81	72.82	68.32	13.31	33.49
CCNet	256*256	76.94	73.55	76.22	10.77	53.35
RefineNet	256*256	76.42	72.48	88.65	9.42	40.44
EMANet	256*256	75.78	70.69	73.76	12.07	29.65
Deeplabv3+	256*256	76.49	73.54	76.21	11.25	35.37
MD TANet	256*256	78.10	75.98	90.72	10.38	26.54