Abstract

In contemporary industrial systems, ensuring the quality of object surfaces has become an essential and inescapable aspect of factory inspections. Cascade regional convolutional neural network (cascade R-CNN), an object detection and instance segmentation algorithm based on deep learning, has been widely applied in numerous industrial applications. Nonetheless, there is still space for improving the detection of defects on metal surfaces. The cascade R-CNN does not currently have good results in metal defect detection, but after improving it by combining some newly proposed modules, cascade R-CNN has a good performance. This article proposes an enhanced metal defect detection method based on cascade R-CNN. Specifically, the improved backbone network is employed to acquire the features of images, which enables more precise localization. Additionally, up and down sampling is combined to extract multiscale defect feature maps, and contrast histogram equalization enhancement is utilized to tackle the issue of unclear contrast in the data. Experimental results demonstrate that the proposed approach achieves a mean average precision (mAP) of 0.754 on the NEU-DET dataset and outperforms the cascade R-CNN model by 9.2%.

Issue Section:
Research Papers
Keywords:
artificial intelligence, data-driven engineering, machine learning for engineering applications
Topics:
Ablation (Vaporization technology), Cascades (Fluid dynamics), Flaw detection, Metal surfaces, Feature extraction, Algorithms, Metals

References

1.
Hao
,
Z.
,
Li
,
Z.
,
Ren
,
F.
,
Lv
,
S.
, and
Ni
,
H.
,
2022
, “
Strip Steel Surface Defects Classification Based on Generative Adversarial Network and Attention Mechanism
,”
Metals
,
12
(
2
), p.
311
.
2.
Xing
,
L.
,
Li
,
T.
,
Fan
,
H.
, and
Zhu
,
H.
,
2022
, “
Defect Detection and Classification of Strip Steel Based on Improved VIT Model
,”
International Federation for Information Processing
,
Catania, Italy
,
June 13–16
, Springer International Publishing, pp.
327
–
335
.
3.
Liu
,
Y.
,
Geng
,
J.
,
Su
,
Z.
,
Zhang
,
W.
, and
Li
,
J.
,
2018
, “
Real-Time Classification of Steel Strip Surface Defects Based on Deep CNNs
,”
2018 Chinese Intelligent Systems Conference
,
Wenzhou, China
,
Oct. 4
, Springer Singapore, pp.
257
–
266
.
4.
Tarafder
,
M.
,
Dey
,
S.
,
Sivaprasad
,
S.
,
Tarafder
,
S.
, and
Nasipuri
,
M.
,
2005
, “
Stretch-Zone Analysis by Image Processing for the Evaluation of Initiation Fracture Toughness of a HSLA Steel
,”
Z. Metallkunde
,
96
(
8
), pp.
924
–
932
.
5.
Bo
,
T.
,
Jian-yi
,
K.
,
Xing-dong
,
W.
,
Guo-zhang
,
J.
, and
Li
,
C.
,
2010
, “
Steel Strip Surface Defects Detection Based on Mathematical Morphology
,”
J. Iron Steel Res.
,
22
(
10
), pp.
56
–
59
.
6.
Zhang
,
Y.
,
Gong
,
W.
, and
Zhong
,
W.
,
2013
, “
Fault Automatic Detection Method of Steel Cord Conveyor Belt Based on Gabor Filter Bank
,”
2nd International Conference on Information Technology and Management Innovation (ICITMI 2013)
,
Zhuhai, China
,
July 23–24
.
7.
Medina
,
R.
,
Gayubo
,
F.
,
Gonzalez
,
L. M.
,
Olmedo
,
D.
,
Gomez
,
J.
,
Zalama
,
E.
, and
Peran
,
J. R.
,
2008
, “
Surface Defects Detection on Rolled Steel Strips by Gabor Filters
,”
3rd International Conference on Computer Vision Theory and Applications
,
Funchal, Portugal
,
Jan. 22–25
, pp.
479
–
485
.
8.
Wang
,
F.
,
Guohua
,
P.
, and
Xie
,
H.
,
2018
, “
Strip Steel Defect Detection Based on Morphological Enhancement and Image Fusion
,”
Laser Infrared.
,
48
(
1
), pp.
124
–
128
.
9.
Zuiderveld
,
K. J.
,
1994
, “Contrast Limited Adaptive Histogram Equalization,”
Graphics Gems.
, P. S. Heckbert ed.,
Academic Press Professional, Inc.
,
Cambridge, MA
, pp.
474
–
485
.
10.
Pizer
,
S. M.
,
Amburn
,
E. P.
,
Austin
,
J. D.
,
Cromartie
,
R.
,
Geselowitz
,
A.
,
Greer
,
T.
,
Zimmerman
,
J. B.
, and
Zuiderveld
,
K.
,
1987
, “
Adaptive Histogram Equalization and Its Variations
,”
Comput. Vis. Graph. Image Process.
,
39
(
3
), pp.
355
–
368
.
11.
Zhang
,
H.
,
Wu
,
C.
,
Zhang
,
Z.
,
Zhu
,
Y.
,
Lin
,
H.
,
Zhang
,
Z.
,
Sun
,
Y.
, et al,
2022
, “
ResNeSt: Split-Attention Networks
,”
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
New Orleans, LA
,
June 18–24
, pp.
2735
–
2745
.
12.
Liu
,
S.
,
Qi
,
L.
,
Qin
,
H.
,
Shi
,
J.
, and
Jia
,
J.
,
2018
, “
Path Aggregation Network for Instance Segmentation
,”
31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Salt Lake City, UT
,
June 18–23
, pp.
8759
–
8768
.
13.
Lin
,
T.-Y.
,
Dollár
,
P.
,
Girshick
,
R.
,
He
,
K.
,
Hariharan
,
B.
, and
Belongie
,
S.
,
2017
, “
Feature Pyramid Networks for Object Detection
,”
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Honolulu, HI
,
July 21–26
, pp.
936
–
944
.
14.
He
,
K.
,
Gkioxari
,
G.
,
Dollár
,
P.
, and
Girshick
,
R.
,
2020
, “
Mask R-CNN
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
42
(
2
), pp.
386
–
397
.
15.
Yuan
,
S.
,
Abe
,
M.
,
Taguchi
,
A.
, and
Kawamata
,
M.
,
2007
, “
High Accuracy Bicubic Interpolation Using Image Local Features
,”
IEICE Trans. Fundam. Electron. Comput. Sci.
,
E90A
(
8
), pp.
1611
–
1615
.
16.
SMITH
,
P.
,
1981
, “
Bilinear Interpolation of Digital Images
,”
Ultramicroscopy
,
6
(
2
), pp.
201
–
204
.
17.
Vu
,
T.
,
Jang
,
H.
,
Pham
,
T. X.
, and
Yoo
,
C. D.
,
2019
, “
Cascade RPN: Delving Into High-Quality Region Proposal Network With Adaptive Convolution
,”
33rd Conference on Neural Information Processing Systems (NeurIPS)
,
Vancouver, Canada
,
Dec. 8–14
.
18.
Oksuz
,
K.
,
Cam
,
B. C.
,
Akbas
,
E.
, and
Kalkan
,
S.
,
2021
, “
Rank & Sort Loss for Object Detection and Instance Segmentation
,”
18th IEEE/CVF International Conference on Computer Vision (ICCV)
,
Virtual
,
Oct. 11–17
, pp.
2989
–
2998
.
19.
Bhatt
,
P. M.
,
Malhan
,
R. K.
,
Rajendran
,
P.
,
Shah
,
B. C.
,
Thakar
,
S.
,
Yoon
,
Y. J.
, and
Gupta
,
S. K.
,
2021
, “
Image-Based Surface Defect Detection Using Deep Learning: A Review
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
4
), p.
040801
.
20.
Liu
,
Y.
,
Rai
,
R.
,
Purwar
,
A.
,
He
,
B.
, and
Mani
,
M.
,
2020
, “
Special Issue: Machine Learning Applications in Manufacturing
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
2, SI
), p.
020301
.
21.
Nand
,
G. K.
,
Noopur
, and
Neogi
,
N.
,
2014
, “
Defect Detection of Steel Surface Using Entropy Segmentation
,”
2014 Annual IEEE India Conference (INDICON)
,
Pune, India
,
Dec. 11–13
, pp.
1
–
6
.
22.
Di
,
S.
,
Dong-bo
,
Z.
, and
Xia
,
L.
,
2014
, “
Based on Gabor and Texture Suppression Scratch Detection for Cell Phone Accessories
,”
Comput. Eng.
,
40
, pp.
1
–
5
.
23.
Liu
,
M.
,
Liu
,
Y.
,
Hu
,
H.
, and
Nie
,
L.
,
2016
, “
Genetic Algorithm and Mathematical Morphology Based Binarization Method for Strip Steel Defect Image with Non-Uniform Illumination
,”
J. Vis. Commun. Image Represent.
,
37
(
May 5
), pp.
70
–
77
.
24.
Yun
,
J. P.
,
Kim
,
D.
,
Kim
,
K.
,
Lee
,
S. J.
,
Park
,
C. H.
, and
Kim
,
S. W.
,
2017
, “
Vision-Based Surface Defect Inspection for Thick Steel Plates
,”
Opt. Eng.
,
56
(
5
), p.
053108
.
25.
Redmon
,
J.
,
Divvala
,
S.
,
Girshick
,
R.
, and
Farhadi
,
A.
,
2016
, “
You Only Look Once: Unified, Real-Time Object Detection
,”
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Seattle, WA
,
June 27–30
, pp.
779
–
788
.
26.
Liu
,
Y.
,
Xu
,
K.
, and
Xu
,
J.
,
2019
, “
Periodic Surface Defect Detection in Steel Plates Based on Deep Learning
,”
Appl. Sci.
,
9
(
15
), p.
3127
.
27.
Akhil
,
V.
,
Raghav
,
G.
,
Arunachalam
,
N.
, and
Srinivas
,
D. S.
,
2020
, “
Image Data-Based Surface Texture Characterization and Prediction Using Machine Learning Approaches for Additive Manufacturing
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
2, SI
), p.
021010
.
28.
Lv
,
X.
,
Duan
,
F.
,
Jiang
,
J.-J.
,
Fu
,
X.
, and
Gan
,
L.
,
2020
, “
Deep Metallic Surface Defect Detection: The New Benchmark and Detection Network
,”
Sensors
,
20
(
6
), p.
1562
.
29.
Liu
,
W.
,
Anguelov
,
D.
,
Erhan
,
D.
,
Szegedy
,
C.
,
Reed
,
S.
,
Fu
,
C.-Y.
, and
Berg
,
A. C.
,
2016
, “SSD: Single Shot Multibox Detector,”
Computer Vision – ECCV 2016
,
B.
Leibe
,
J.
Matas
,
N.
Sebe
, and
M.
Welling
, eds.,
Springer International Publishing
,
Cham, Switzerland
, pp.
858
–
864
.
30.
Girshick
,
R.
,
Donahue
,
J.
,
Darrell
,
T.
, and
Malik
,
J.
,
2016
, “
Region-Based Convolutional Networks for Accurate Object Detection and Segmentation
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
38
(
1
), pp.
142
–
158
.
31.
Wong
,
V. W. H.
,
Ferguson
,
M.
,
Law
,
K. H.
,
Lee
,
Y.-T. T.
, and
Witherell
,
P.
,
2022
, “
Segmentation of Additive Manufacturing Defects Using U-Net
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
3
), p.
031005
.
32.
Wei
,
R.
,
Song
,
Y.
, and
Zhang
,
Y.
,
2020
, “
Enhanced Faster Region Convolutional Neural Networks for Steel Surface Defect Detection
,”
ISIJ. Int.
,
60
(
3
), pp.
539
–
545
.
33.
Konovalenko
,
I.
,
Maruschak
,
P.
, and
Brevus
,
V.
,
2022
, “
Steel Surface Defect Detection Using an Ensemble of Deep Residual Neural Networks
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
1
), p.
014501
.
34.
Cai
,
Z.
, and
Vasconcelos
,
N.
,
2018
, “
Cascade R-CNN: Delving Into High Quality Object Detection
,”
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Salt Lake City, UT
,
June 18–23
, pp.
6154
–
6162
.
35.
Girshick
,
R.
,
2015
, “
Fast R-CNN
,”
IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 7–13
, pp.
1440
–
1448
.
36.
Ren
,
S.
,
He
,
K.
,
Girshick
,
R.
, and
Sun
,
J.
,
2017
, “
Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
39
(
6
), pp.
1137
–
1149
.
37.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2015
, “
Very Deep Convolutional Networks for Large-Scale Image Recognition
,”
International Conference on Learning Representations
,
San Diego, CA
,
May 7–9
.
38.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Las Vegas, NV
,
June 27–30
.
39.
Zhang
,
Y.
,
Xie
,
F.
,
Huang
,
L.
,
Shi
,
J.
,
Yang
,
J.
, and
Li
,
Z.
,
2021
, “
A Lightweight One-Stage Defect Detection Network for Small Object Based on Dual Attention Mechanism and PAFPN
,”
Front. Phys.
,
9
, p.
708097
.
40.
Liu
,
X.
,
Pan
,
H.
, and
Li
,
X.
,
2020
, “Object Detection for Rotated and Densely Arranged Objects in Aerial Images Using Path Aggregated Feature Pyramid Networks,”
MIPPR 2019: Pattern Recognition and Computer Vision, Vol. 11430 of Proceedings of SPIE
,
N.
,
Sang
,
J.
,
Udupa
,
Y.
,
Wang
, and
Z.
,
Liu
, eds.,
Huazhong University of Science and Technology, National Key Lab Science and Technology Multi Spectral Informat Proc; Wuhan Institute of Technology, Automat Assoc Hubei
,
Wuhan
, 11th International Symposium on Multispectral Image Processing and Pattern Recognition (MIPPR) – Pattern Recognition and Computer Vision, Wuhan, China, Nov. 2–3, 2019.
41.
Xie
,
J.
,
Pang
,
Y.
,
Nie
,
J.
,
Cao
,
J.
, and
Han
,
J.
,
2023
, “
Latent Feature Pyramid Network for Object Detection
,”
IEEE Trans. Multimed.
,
25
, pp.
2153
–
2163
.
42.
Gao
,
S.
, and
Gruev
,
V.
,
2011
, “
Bilinear and Bicubic Interpolation Methods for Division of Focal Plane Polarimeters
,”
Opt. Express
,
19
(
27
), pp.
26161
–
26173
.
43.
Rajarapollu
,
P.
, and
Mankar
,
V.
,
2017
, “
Bicubic Interpolation Algorithm Implementation for Image Appearance Enhancement
,”
Ijcst
,
8
(
4
), pp.
23
–
26
. www.ijcst.com.
44.
Jaiswal
,
A.
,
Wu
,
Y.
,
Natarajan
,
P.
, and
Natarajan
,
P.
,
2021
, “
Class-Agnostic Object Detection
,”
IEEE Winter Conference on Applications of Computer Vision (WACV)
,
Virtual
,
Jan. 5–9
, pp.
918
–
927
.
45.
Song
,
K.
, and
Yan
,
Y.
,
2013
, “
A Noise Robust Method Based on Completed Local Binary Patterns for Hot-Rolled Steel Strip Surface Defects
,”
Appl. Surf. Sci.
,
285
(
Part B
), pp.
858
–
864
.
46.
Buslaev
,
A.
,
Iglovikov
,
V. I.
,
Khvedchenya
,
E.
,
Parinov
,
A.
,
Druzhinin
,
M.
, and
Kalinin
,
A. A.
,
2020
, “
Albumentations: Fast and Flexible Image Augmentations
,”
Information
,
11
(
2
), p.
125
.
47.
Redmon
,
J.
, and
Farhadi
,
A.
,
2018
, “
Yolov3: An Incremental Improvement
,”
arXiv, abs/1804.02767.
48.
Lin
,
T.-Y.
,
Goyal
,
P.
,
Girshick
,
R.
,
He
,
K.
, and
Dollár
,
P.
,
2020
, “
Focal Loss for Dense Object Detection
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
42
(
2
), pp.
318
–
327
.
49.
Law
,
H.
, and
Deng
,
J.
,
2018
, “
Cornernet: detecting objects as paired keypoints
,”
Computer Vision. 15th European Conference (ECCV 2018)
,
Munich, Germany
,
Sept. 8–14
, pp.
765
–
781
.
50.
Duan
,
K.
,
Bai
,
S.
,
Xie
,
L.
,
Qi
,
H.
,
Huang
,
Q.
, and
Tian
,
Q.
,
2019
, “
Centernet: Keypoint Triplets for Object Detection
,”
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
,
Seoul, South Korea
,
Oct. 27–Nov. 2
, pp.
6568
–
6577
.
51.
Duan
,
H.
,
Huang
,
J.
,
Liu
,
W.
, and
Shu
,
F.
,
2022
, “
Defective Surface Detection Based on Improved Faster R-cnn
,”
2022 IEEE International Conference on Industrial Technology (ICIT)
,
Shanghai, China
,
Aug. 22–25
, pp.
1
–
6
.
52.
Chen
,
K.
,
Wang
,
J.
,
Pang
,
J.
,
Cao
,
Y.
,
Xiong
,
Y.
,
Li
,
X.
,
Sun
,
S.
, et al
2019
, “
MMDetection: Open MMLAB Detection Toolbox and Benchmark
,”
arXiv preprint arXiv:1906.07155.
Copyright © 2023 by ASME
You do not currently have access to this content.