Abstract

The detection of surface damage is an important part of the process before remanufacturing a retired steel shaft (RSS). Traditional damage detection is mainly done manually, which is time-consuming and error-prone. In recent years, computer vision methods have been introduced into the community of surface damage detection. However, some advanced typical object detection methods perform poorly in the detection of surface damage on RSS due to the complex surface background and rich diversity of damage patterns and scales. To address these issues, we propose a Faster R-CNN–based surface damage detection method for RSS. To improve the adaptability of the network, we endow it with a feature pyramid network (FPN) as well as adaptable multiscale information modifications to the region proposal network (RPN). In this paper, a detailed study of an FPN-based feature extraction network and the multiscale object detection network is conducted. Experimental results show that our method improves the mean average precision (mAP) score by 8.9% compared with the original Faster R-CNN for surface damage detection of RSS, and the average detection accuracy for small objects is improved by 18.2%. Compared with the current advanced object detection methods, our method is more advantageous for the detection of multiscale objects.

References

1.
Sabnavis
,
G.
,
Kirk
,
R. G.
,
Kasarda
,
M.
, and
Quinn
,
D.
,
2004
, “
Cracked Shaft Detection and Diagnostics: A Literature Review
,”
Shock Vibr. Dig.
,
36
(
4
), pp.
287
296
.
2.
Chen
,
Z. S.
,
Yang
,
Y. M.
,
Xie
,
Y.
,
Guo
,
B.
, and
Hu
,
Z.
,
2013
, “
Non-Contact Crack Detection of High-Speed Blades Based on Principal Component Analysis and Euclidian Angles Using Optical-Fiber Sensors
,”
Sens. Actuators, A
,
201
, pp.
66
72
.
3.
Campbell
,
F. C.
,
2013
,
Inspection of Metals: Understanding the Basics
,
ASM International
.
4.
Shanmugamani
,
R.
,
Sadique
,
M.
, and
Ramamoorthy
,
B.
,
2015
, “
Detection and Classification of Surface Defects of Gun Barrels Using Computer Vision and Machine Learning
,”
Measurement
,
60
, pp.
222
230
.
5.
D'Orazio
,
T.
,
Leo
,
M.
,
Distante
,
A.
,
Guaragnella
,
C.
,
Pianese
,
V.
, and
Cavaccini
,
G.
,
2008
, “
Automatic Ultrasonic Inspection for Internal Defect Detection in Composite Materials
,”
NDT E Int.
,
41
(
2
), pp.
145
154
.
6.
Garcia-Martin
,
J.
,
Gomez-Gil
,
J.
, and
Vazquez-Sanchez
,
E.
,
2011
, “
Non-Destructive Techniques Based on Eddy Current Testing
,”
Sensors
,
11
(
3
), pp.
2525
2565
.
7.
Jiang
,
L. X.
,
Sun
,
K. Y.
,
Zhao
,
F. L.
, and
Hao
,
X. Y.
,
2015
, “
Automatic Detection System of Shaft Part Surface Defect Based on Machine Vision
,”
Proceedings of the Conference on Automated Visual Inspection and Machine Vision
,
Munich, Germany
,
June 22
,
p. 95300G
.
8.
Jeon
,
I.
,
Lim
,
H. J.
,
Liu
,
P.
,
Park
,
B.
,
Heinze
,
A.
, and
Sohn
,
H.
,
2019
, “
Fatigue Crack Detection in Rotating Steel Shafts Using Noncontact Ultrasonic Modulation Measurements
,”
Eng. Struct.
,
196
, p.
109293
.
9.
Neslušan
,
M.
,
Bahleda
,
F.
,
Minarik
,
P.
,
Zgutova
,
K.
, and
Jambor
,
M.
,
2019
, “
Non-Destructive Monitoring of Corrosion Extent in Steel Rope Wires via Barkhausen Noise Emission
,”
J. Magn. Magn. Mater.
,
484
, pp.
179
187
.
10.
Sha
,
J. W.
,
Fan
,
M. B.
,
Cao
,
B. H.
, and
Liu
,
B. L.
,
2021
, “
Noncontact and Nondestructive Evaluation of Heat-Treated Bearing Rings Using Pulsed Eddy Current Testing
,”
J. Magn. Magn. Mater.
,
521
, p.
167516
.
11.
Xie
,
S. J.
,
Zhang
,
L.
,
Zhao
,
Y.
,
Wang
,
X. G.
,
Kong
,
Y. Y.
,
Ma
,
Q.
,
Chen
,
Z. M.
,
Uchimoto
,
T.
, and
Takagi
,
T.
,
2020
, “
Features Extraction and Discussion in a Novel Frequency-Band-Selecting Pulsed Eddy Current Testing Method for the Detection of a Certain Depth Range of Defects
,”
NDT E Int.
,
111
, p.
102211
.
12.
Zheng
,
H.
,
Kong
,
L. X.
, and
Nahavandi
,
S.
,
2002
, “
Automatic Inspection of Metallic Surface Defects Using Genetic Algorithms
,”
J. Mater. Process. Technol.
,
125–126
, pp.
427
433
.
13.
Huang
,
X. Q.
, and
Luo
,
X. B.
,
2014
, “
A Real-Time Algorithm for Aluminum Surface Defect Extraction on Non-Uniform Image From CCD Camera
,”
Proceedings of the International Conference on Machine Learning and Cybernetics
,
Lanzhou, China
,
July 13–16
, IEEE, pp.
556
561
.
14.
Tosyali
,
A.
,
Song
,
R.
,
Guo
,
W.
,
Abolhassani
,
A.
, and
Kalamdani
,
R.
,
2020
, “
Data-Driven Gantry Health Monitoring and Process Status Identification Based on Texture Extraction
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
1
), p.
011003
.
15.
Dong
,
X.
, and
Li
,
Y.
,
2022
, “
Online Detection of Turning Tool Wear Based on Machine Vision
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
5
), p.
050903
.
16.
Cha
,
Y.-J.
,
Choi
,
W.
, and
Büyüköztürk
,
O.
,
2017
, “
Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks
,”
Comput.-Aided Civ. Infrastruct. Eng.
,
32
(
5
), pp.
361
378
.
17.
LeCun
,
Y.
,
Bengio
,
Y.
, and
Hinton
,
G.
,
2015
, “
Deep Learning
,”
Nature
,
521
(
7553
), pp.
436
444
.
18.
Krizhevsky
,
A.
,
Sutskever
,
I.
, and
Hinton
,
G. E.
,
2017
, “
Imagenet Classification With Deep Convolutional Neural Networks
,”
Commun. ACM
,
60
(
6
), pp.
84
90
.
19.
Bhatt
,
P. M.
,
Malhan
,
R. K.
,
Rajendran
,
P.
,
Shah
,
B. C.
,
Thakar
,
S.
,
Yoon
,
Y. J.
, and
Gupta
,
S. K.
,
2021
, “
Image-Based Surface Defect Detection Using Deep Learning: A Review
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
4
), p.
040801
.
20.
Wong
,
V. W. H.
,
Ferguson
,
M.
,
Law
,
K. H.
,
Lee
,
Y.-T. T.
, and
Witherell
,
P.
,
2021
, “
Segmentation of Additive Manufacturing Defects Using U-Net
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
3
), pp.
1
44
.
21.
Konovalenko
,
I.
,
Maruschak
,
P.
, and
Brevus
,
V.
,
2021
, “
Steel Surface Defect Detection Using an Ensemble of Deep Residual Neural Networks
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
1
), p.
014501
.
22.
Ren
,
M.
,
Shen
,
R.
, and
Gong
,
Y.
,
2022
, “
A Surface Defect Detection Method Via Fusing Multi-Level Features
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
5
), p.
051005
.
23.
Chen
,
H.
,
Nie
,
Z.
,
Xu
,
Q.
,
Fei
,
J.
,
Yang
,
K.
,
Li
,
Y.
,
Lin
,
H.
,
Fan
,
W.
, and
Liu
,
X.-J.
,
2022
, “
Intelligent Detection and Classification of Surface Defects on Cold-Rolled Galvanized Steel Strips Using a Data-Driven Faulty Model With Attention Mechanism
,”
ASME J. Comput. Inf. Sci. Eng.
,
23
(
4
), p.
041001
.
24.
Huang
,
X. Y.
,
Liu
,
Z. L.
,
Zhang
,
X. Y.
,
Kang
,
J. L.
,
Zhang
,
M.
, and
Guo
,
Y. L.
,
2020
, “
Surface Damage Detection for Steel Wire Ropes Using Deep Learning and Computer Vision Techniques
,”
Measurement
,
161
, p.
107843
.
25.
Ko Leong
,
R. T. J. C.
,
Rodriguez
,
F. E. A.
,
Vidad
,
A. J. P.
,
Dadios
,
E. P.
, and
Billones
,
R. K. C.
,
2022
, “
Deep Learning Applications in Identifying Deep Metallic Surface Defects
,”
2022 IEEE 14th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM)
,
Boracay Island, Philippines
,
Dec. 1–4
, pp.
1
6
.
26.
Ning
,
F.
,
Cheng
,
Z.
,
Meng
,
D.
,
Duan
,
S.
, and
Wei
,
J.
,
2021
, “
Enhanced Spectrum Convolutional Neural Architecture: An Intelligent Leak Detection Method for gas Pipeline
,”
Process Saf. Environ. Prot.
,
146
, pp.
726
735
.
27.
Li
,
D.
,
Li
,
Y.
,
Xie
,
Q.
,
Wu
,
Y.
,
Yu
,
Z.
, and
Wang
,
J.
,
2021
, “
Tiny Defect Detection in High-Resolution Aero-Engine Blade Images via a Coarse-to-Fine Framework
,”
IEEE Trans. Instrum. Meas.
,
70
, pp.
1
12
.
28.
Hu
,
S.
,
Feng
,
A.
,
Shi
,
J.
,
Li
,
J.
,
Khan
,
F.
,
Zhu
,
H.
,
Chen
,
J.
, and
Chen
,
G.
,
2022
, “
Underwater Gas Leak Detection Using an Autonomous Underwater Vehicle (Robotic Fish)
,”
Process Saf. Environ. Prot.
,
167
, pp.
89
96
.
29.
Jiang
,
Q. S.
,
Tan
,
D. P.
,
Li
,
Y. B.
,
Ji
,
S. M.
,
Cai
,
C. P.
, and
Zheng
,
Q. M.
,
2020
, “
Object Detection and Classification of Metal Polishing Shaft Surface Defects Based on Convolutional Neural Network Deep Learning
,”
Appl. Sci.
,
10
(
1
), p.
87
.
30.
Zhao
,
M.
,
Shi
,
P.
,
Xu
,
X.
,
Xu
,
X.
,
Liu
,
W.
, and
Yang
,
H.
,
2022
, “
Improving the Accuracy of an R-CNN-Based Crack Identification System Using Different Preprocessing Algorithms
,”
Sensors
,
22
(
18
), p.
7089
.
31.
Zheng
,
J.
, and
Zhang
,
T.
,
2023
, “
Wafer Surface Defect Detection Based on Background Subtraction and Faster R-CNN
,”
Micromachines
,
14
(
5
), p.
905
.
32.
Su
,
J.
,
Yi
,
H.
,
Ling
,
L.
,
Shu
,
A.
,
Lu
,
E.
,
Jiao
,
Y.
, and
Wang
,
S.
,
2022
, “
Multi-Object Surface Roughness Grade Detection Based on Faster R-CNN
,”
Meas. Sci. Technol.
,
34
(
1
), p.
015012
.
33.
Manyar
,
O. M.
,
Cheng
,
J.
,
Levine
,
R.
,
Krishnan
,
V.
,
Barbič
,
J.
, and
Gupta
,
S. K.
,
2023
, “
Physics Informed Synthetic Image Generation for Deep Learning-Based Detection of Wrinkles and Folds
,”
ASME J. Comput. Inf. Sci. Eng.
,
23
(
3
), p.
030903
.
34.
Xu
,
Y. Y.
,
Li
,
D. W.
,
Xie
,
Q.
,
Wu
,
Q. Y.
, and
Wang
,
J.
,
2021
, “
Automatic Defect Detection and Segmentation of Tunnel Surface Using Modified Mask R-CNN
,”
Measurement
,
178
, p.
109316
.
35.
Akyon
,
F. C.
,
Onur Altinuc
,
S.
, and
Temizel
,
A.
,
2022
, “
Slicing Aided Hyper Inference and Fine-Tuning for Small Object Detection
,”
2022 IEEE International Conference on Image Processing (ICIP)
,
Bordeaux, France
,
Oct. 16-19
, pp.
966
970
.
36.
Teng
,
S.
,
Liu
,
Z.
, and
Li
,
X.
,
2022
, “
Improved YOLOv3-Based Bridge Surface Defect Detection by Combining High- and Low-Resolution Feature Images
,”
Buildings
,
12
(
8
), pp.
10
. 3390/buildings12081225
37.
Li
,
D.
,
Xie
,
Q.
,
Gong
,
X.
,
Yu
,
Z.
,
Xu
,
J.
,
Sun
,
Y.
, and
Wang
,
J.
,
2021
, “
Automatic Defect Detection of Metro Tunnel Surfaces Using a Vision-Based Inspection System
,”
Adv. Eng. Inform.
,
47
, p.
101206
.
38.
Ren
,
S. Q.
,
He
,
K. M.
,
Girshick
,
R.
, and
Sun
,
J.
,
2017
, “
Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
39
(
6
), pp.
1137
1149
.
39.
Girshick
,
R.
,
2015
, “
Fast R-CNN
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 7–13
,
IEEE, pp. 1440–1448
.
40.
Deng
,
Z.
,
Sun
,
H.
,
Zhou
,
S.
,
Zhao
,
J.
,
Lei
,
L.
, and
Zou
,
H.
,
2018
, “
Multi-Scale Object Detection in Remote Sensing Imagery With Convolutional Neural Networks
,”
ISPRS J. Photogramm. Remote Sens.
,
145
, pp.
3
22
.
41.
Qin
,
J. H.
,
Huang
,
Y. J.
, and
Wen
,
W. S.
,
2020
, “
Multi-Scale Feature Fusion Residual Network for Single Image Super-Resolution
,”
Neurocomputing
,
379
, pp.
334
342
.
42.
Zhang
,
L. L.
,
Lin
,
L.
,
Liang
,
X. D.
, and
He
,
K. M.
,
2016
, “
Is Faster R-CNN Doing Well for Pedestrian Detection?
Proceedings of the 14th European Conference on Computer Vision (ECCV)
,
Amsterdam, The Netherlands
,
Oct. 11–14, 2016
, pp.
443
457
.
43.
Lin
,
T.-Y.
,
Dollár
,
P.
,
Girshick
,
R.
,
He
,
K.
,
Hariharan
,
B.
, and
Belongie
,
S.
,
2017
, “
Feature Pyramid Networks for Object Detection
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Honolulu, HI
,
July 21–26
, pp.
936
944
. .
44.
Junxu
,
C.
,
Qi
,
C.
,
Jun
,
G.
, and
Ruichao
,
S.
,
2020
, “
Attention-Guided Context Feature Pyramid Network for Object Detection
,”
ArXiv
, pp.
1
12
. https://arxiv.org/abs/2005.11475
45.
Kaiming
,
H.
,
Xiangyu
,
Z.
,
Shaoqing
,
R.
, and
Jian
,
S.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Las Vegas, NV
,
June 27–30
, pp.
770
778
. .
46.
He
,
K. M.
,
Gkioxari
,
G.
,
Dollar
,
P.
, and
Girshick
,
R.
,
2017
, “
Mask R-CNN
,”
Proceedings of the 16th IEEE International Conference on Computer Vision (ICCV)
,
Venice, Italy
,
Oct. 22–29
, pp.
2980
2988
. .
47.
Kong
,
T.
,
Yao
,
A. B.
,
Chen
,
Y. R.
, and
Sun
,
F. C.
,
2016
, “
HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection
,”
Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Las Vegas, NV
,
Dec. 9
, pp.
845
853
. 8.
48.
LeCun
,
Y.
,
Boser
,
B.
,
Denker
,
J. S.
,
Henderson
,
D.
,
Howard
,
R. E.
,
Hubbard
,
W.
, and
Jackel
,
L. D.
,
1989
, “
Backpropagation Applied to Handwritten Zip Code Recognition
,”
Neural Comput.
,
1
(
4
), pp.
541
551
.
49.
Shin
,
H. C.
,
Roth
,
H. R.
,
Gao
,
M. C.
,
Lu
,
L.
,
Xu
,
Z. Y.
,
Nogues
,
I.
,
Yao
,
J. H.
,
Mollura
,
D.
, and
Summers
,
R. M.
,
2016
, “
Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning
,”
IEEE Trans. Med. Imaging
,
35
(
5
), pp.
1285
1298
.
50.
Lin
,
T. Y.
,
Maire
,
M.
,
Belongie
,
S.
,
Hays
,
J.
,
Perona
,
P.
,
Ramanan
,
D.
,
Dollar
,
P.
, and
Zitnick
,
C. L.
,
2014
, “
Microsoft COCO: Common Objects in Context
,”
Proceedings of the 13th European Conference on Computer Vision (ECCV)
,
Zurich, Switzerland
,
Sept. 6–12
, pp.
740
755
. 8.
51.
Howard
,
A. G.
,
Menglong
,
Z.
,
Bo
,
C.
,
Kalenichenko
,
D.
,
Weijun
,
W.
,
Weyand
,
T.
,
Andreetto
,
M.
, and
Adam
,
H.
,
2017
, “
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
,”
ArXiv
, pp.
1
9
. .
Preprint
.
52.
Tan
,
M. X.
, and
Le
,
Q. V.
,
2019
, “
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
,”
Proceedings of the 36th International Conference on Machine Learning (ICML)
,
Long Beach, CA
,
June 9–15
, pp.
10691
10700
.
53.
Tan
,
M. X.
,
Chen
,
B.
,
Pang
,
R. M.
,
Vasudevan
,
V.
,
Sandier
,
M.
,
Howard
,
A.
,
Le
,
Q. V.
, and
Soc
,
I. C.
,
2019
, “
MnasNet: Platform-Aware Neural Architecture Search for Mobile
,”
Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Long Beach, CA
,
June 16–20
, pp.
2815
2823
. .
54.
Redmon
,
J.
, and
Farhadi
,
A.
,
2018
, “
Yolov3: An Incremental Improvement
,”
ArXiv
, pp.
1
6
.
55.
Liu
,
W.
,
Anguelov
,
D.
,
Erhan
,
D.
,
Szegedy
,
C.
,
Reed
,
S.
,
Fu
,
C.-Y.
, and
Berg
,
A. C.
,
2016
, “
SSD: Single Shot MultiBox Detector
,”
Proceedings of the Computer Vision—ECCV 2016
,
Amsterdam, Netherlands
,
Oct. 8–16
, pp.
21
37
.
You do not currently have access to this content.