JCS Cover
Frequency: Monthly
ISSN: 1549-3636 (Print)
ISSN: 1552-6607 (Online)
Research Article Open Access

AI-Guided Anatomical Landmark and Abnormality Detection for Autonomous Endoscopy Examination

Md Shakhawat Hossain1, Md Shakhawat Hossain2, Munim Ahmed2, Md Sahilur Rahman2, Mahreen Tabassum3, Fariha Karim3, Md Aulad Hossain4, Razib Hayat Khan1, Razib Hayat Khan2, M. M. Mahbubul Syeed1,2 and Mohammad Faisal Uddin1,2
  • 1 Department of Computer Science and Engineering, Independent University, Bangladesh, Dhaka, Bangladesh
  • 2 RIoT Research Center, Independent University, Bangladesh, Dhaka, Bangladesh
  • 3 Department of Computer Science and Engineering, American International University-Bangladesh, Dhaka, Bangladesh
  • 4 Department of Gastroenterology, Bangabandhu Sheikh Mujib Medical University Hospital, Dhaka, Bangladesh

Abstract

Endoscopy is the routine medical procedure to observe tumors in the human Gastrointestinal (GI) tract by inserting an endoscope, a thin, flexible, tube-like instrument with a light source and camera. Traditionally, an endoscopist performs the endoscopy, orients the endoscope within these structures and navigates this through the help of familiar anatomical landmarks to reach the abnormalities and mark them. Identifying landmarks and abnormalities is critical for the maneuver and the success of endoscopy, which is related to the patient’s comfort, injury and accurate diagnosis. The manual naked-eye-observation maneuver and examination are highly challenging, take a long time and often cause discomfort to the patients and the endoscopists. As a result, several AI-based landmark detection methods have been proposed recently to facilitate autonomous endoscopy examination. However, these methods lack accuracy and consider only limited landmarks. This study presents a Data-efficient image Transformer (DeiT)-based method to detect anatomical landmarks and anomalies for autonomous endoscopy. The proposed method detected 23 landmarks and anomalies from the entire GI tract with 99% accuracy and precision, outperforming the state-of-the-art (91%). Moreover, this method took only 0.045 sec to identify a landmark. The phi coefficient (0.997) indicated a strong positive association between the proposed method and clinical ground truth. Strong association, high accuracy and rapid speed ensured the reliability of the proposed method for autonomous endoscopy examination.

References

Aabakken, L., Barkun, A. N., Cotton, P. B., Fedorov, E., Fujino, M. A., Ivanova, E., Kudo, S., Kuznetzov, K., De Lange, T., Matsuda, K., Moine, O., Rembacken, B., Rey, J., Romagnuolo, J., Rösch, T., Sawhney, M., Yao, K., & Waye, J. D. (2014). Standardized endoscopic reporting. Journal of Gastroenterology and Hepatology, 29(2), 234–240. https://doi.org/10.1111/jgh.12489
Aliyi, S., Dese, K., & Raj, H. (2023). Detection of gastrointestinal tract disorders using deep learning methods from colonoscopy images and videos. Scientific African, 20, e01628. https://doi.org/10.1016/j.sciaf.2023.e01628
Ayyoubi Nezhad, S., Khatibi, T., & Sohrabi, M. (2022). Proposing Novel Data Analytics Method for Anatomical Landmark Identification from Endoscopic Video Frames. Journal of Healthcare Engineering, 2022(1), 8151177. https://doi.org/10.1155/2022/8151177
Borgli, H., Thambawita, V., Smedsrud, P. H., Hicks, S., Jha, D., Eskeland, S. L., Randel, K. R., Pogorelov, K., Lux, M., Nguyen, D. T. D., Johansen, D., Griwodz, C., Stensland, H. K., Garcia-Ceja, E., Schmidt, P. T., Hammer, H. L., Riegler, M. A., Halvorsen, P., & De Lange, T. (2020). HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Scientific Data, 7(1), 283. https://doi.org/10.1038/s41597-020-00622-y
Bour, A., Castillo-Olea, C., Garcia-Zapirain, B., & Zahia, S. (2019). Automatic colon polyp classification using Convolutional Neural Network: A Case Study at Basque Country. 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), 1–5. https://doi.org/10.1109/isspit47144.2019.9001816
Che, K., Ye, C., Yao, Y., Ma, N., Zhang, R., Wang, J., & Meng, M. Q. H. (2021). Deep learning-based biological anatomical landmark detection in colonoscopy videos. ArXiv, 2108.02948. https://doi.org/10.48550/arXiv.2108.02948
Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21, 1–13. https://doi.org/10.1186/s12864-019-6413-7
Del Moral, P., Nowaczyk, S., & Pashami, S. (2022). Why Is Multiclass Classification Hard? IEEE Access, 10, 80448–80462. https://doi.org/10.1109/access.2022.3192514
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248–255. https://doi.org/10.1109/cvprw.2009.5206848
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ArXiv, 2010.11929. https://doi.org/10.48550/arXiv.2010.11929
Ferlay, J., Soerjomataram, I., Dikshit, R., Eser, S., Mathers, C., Rebelo, M., Parkin, D. M., Forman, D., & Bray, F. (2015). Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012. International Journal of Cancer, 136(5), E359–E386. https://doi.org/10.1002/ijc.29210
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778. https://doi.org/10.1109/cvpr.2016.90
Hirasawa, T., Aoyama, K., Tanimoto, T., Ishihara, S., Shichijo, S., Ozawa, T., Ohnishi, T., Fujishiro, M., Matsuo, K., Fujisaki, J., & Tada, T. (2018). Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer, 21, 653–660. https://doi.org/10.1007/s10120-018-0793-2
Hossain, M. S., Nakamura, T., Kimura, F., Yagi, Y., & Yamaguchi, M. (2018). Practical image quality evaluation for whole slide imaging scanner. Biomedical Imaging and Sensing Conference, 203–206. https://doi.org/10.1117/12.2316764
Hossain, M. S., Rahman, M. M., Syeed, M. M., Uddin, M. F., Hasan, M., Hossain, M. A., Ksibi, A., Jamjoom, M. M., Ullah, Z., & Samad, M. A. (2023). DeepPoly: Deep Learning-Based Polyps Segmentation and Classification for Autonomous Colonoscopy Examination. IEEE Access, 11, 95889–95902. https://doi.org/10.1109/access.2023.3310541
Hossain, M. S., Syeed, M. M., Fatema, K., & Uddin, M. F. (2022). The Perception of Health Professionals in Bangladesh toward the Digitalization of the Health Sector. International Journal of Environmental Research and Public Health, 19(20), 13695. https://doi.org/10.3390/ijerph192013695
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4700–4708. https://doi.org/10.1109/cvpr.2017.243
Iwagami, H., Ishihara, R., Aoyama, K., Fukuda, H., Shimamoto, Y., Kono, M., Nakahira, H., Matsuura, N., Shichijo, S., Kanesaka, T., Kanzaki, H., Ishii, T., Nakatani, Y., & Tada, T. (2021). Artificial intelligence for the detection of esophageal and esophagogastric junctional adenocarcinoma. Journal of Gastroenterology and Hepatology, 36(1), 131–136. https://doi.org/10.1111/jgh.15136
Kaminski, M. F., Regula, J., Kraszewska, E., Polkowski, M., Wojciechowska, U., Didkowska, J., Zwierko, M., Rupinski, M., Nowacki, M. P., & Butruk, E. (2010). Quality Indicators for Colonoscopy and the Risk of Interval Cancer. New England Journal of Medicine, 362(19), 1795–1803. https://doi.org/10.1056/nejmoa0907667
Luo, H., Xu, G., Li, C., He, L., Luo, L., Wang, Z., Jing, B., Deng, Y., Jin, Y., Li, Y., Li, B., Tan, W., He, C., Seeruttun, S. R., Wu, Q., Huang, J., Huang, D., Chen, B., Lin, S. B., … Xu, R. H. (2019). Real-time artificial intelligence for detection of upper gastrointestinal cancer by endoscopy: A multicentre, case-control, diagnostic study. The Lancet Oncology, 20(12), 1645–1654. https://doi.org/10.1016/s1470-2045(19)30637-0
Misawa, M., Kudo, S. E., Mori, Y., Hotta, K., Ohtsuka, K., Matsuda, T., Saito, S., Kudo, T., Baba, T., Ishida, F., Itoh, H., Oda, M., & Mori, K. (2021). Development of a computer-aided detection system for colonoscopy and a publicly accessible large colonoscopy video database (with video). Gastrointestinal Endoscopy, 93(4), 960–967. https://doi.org/10.1016/j.gie.2020.07.060
Nishitha, R., Amalan, S., Sharma, S., Preejith, S. P., & Sivaprakasam, M. (2022). Image Quality Assessment for Interdependent Image Parameters Using a Score-Based Technique for Endoscopy Applications. 2022 IEEE International Symposium on Medical Measurements and Applications (MeMeA), 1–6. https://doi.org/10.1109/memea54994.2022.9856448
Ozawa, T., Ishihara, S., Fujishiro, M., Kumagai, Y., Shichijo, S., & Tada, T. (2020). Automated endoscopic detection and classification of colorectal polyps using convolutional neural networks. Therapeutic Advances in Gastroenterology, 13, 175628482091065. https://doi.org/10.1177/1756284820910659
Pogorelov, K., Randel, K. R., Griwodz, C., Eskeland, S. L., de Lange, T., Johansen, D., Spampinato, C., Dang-Nguyen, D.-T., Lux, M., Schmidt, P. T., Riegler, M., & Halvorsen, P. (2017). KVASIR: A Multi-Class Image Dataset for Computer Aided Gastrointestinal Disease Detection. Proceedings of the 8th ACM on Multimedia Systems Conference, 164–169. https://doi.org/10.1145/3083187.3083212
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the IEEE International Conference on Computer Vision, 618–626. https://doi.org/10.1109/iccv.2017.74
Shakhawat, H. M., Nakamura, T., Kimura, F., Yagi, Y., & Yamaguchi, M. (2020). Automatic Quality Evaluation of Whole Slide Images for the Practical Use of Whole Slide Imaging Scanner. ITE Transactions on Media Technology and Applications, 8(4), 252–268. https://doi.org/10.3169/mta.8.252
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. ArXiv, 1409.1556. https://doi.org/10.48550/arXiv.1409.1556
Suzuki, H., Yoshitaka, T., Yoshio, T., & Tada, T. (2021). Artificial intelligence for cancer detection of the upper gastrointestinal tract. Digestive Endoscopy, 33(2), 254–262. https://doi.org/10.1111/den.13897
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–9. https://doi.org/10.1109/cvpr.2015.7298594
Tan, M., & Le, Q. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the 36th International Conference on Machine Learning, 6105–6114.
Tomar, N. K., Jha, D., Ali, S., Johansen, H. D., Johansen, D., Riegler, M. A., & Halvorsen, P. (2021). DDANet: Dual Decoder Attention Network for Automatic Polyp Segmentation. In A. Del Bimbo, R. Cucchiara, S. Sclaroff, G. Maria Farinella, T. Mei, M. Bertini, H. Jair Escalante, & R. Vezzani (Eds.), Pattern Recognition. ICPR International Workshops and Challenges (1st ed., Vol. 12668, pp. 307–314). Springer, Cham. https://doi.org/10.1007/978-3-030-68793-9_23
Touvron, H., Sablayrolles, A., Douze, M., Cord, M., & Jegou, H. (2021). Grafit: Learning fine-grained image representations with coarse labels. Proceedings of the IEEE/CVF International Conference on Computer Vision, 874–884. https://doi.org/10.1109/iccv48922.2021.00091
Tran, T.-H., Nguyen, P.-T., Tran, D.-H., Manh, X.-H., Vu, D.-H., Ho, N.-K., Do, K.-L., Nguyen, V.-T., Nguyen, L.-T., Dao, V.-H., & Vu, H. (2021). Classification of anatomical landmarks from upper gastrointestinal endoscopic images. 2021 8th NAFOSTED Conference on Information and Computer Science (NICS), 278–283. https://doi.org/10.1109/nics54270.2021.9701513

Journal of Computer Science
Volume 20 No. 8, 2024, 858-871

DOI: https://doi.org/10.3844/jcssp.2024.858.871

Submitted On: 22 April 2024 Published On: 29 May 2024

How to Cite: Hossain, M. S., Ahmed, M., Rahman, M. S., Tabassum, M., Karim, F., Hossain, M. A., Hayat Khan, R., Khan, R. H., Syeed, M. M. M. & Uddin, M. F. (2024). AI-Guided Anatomical Landmark and Abnormality Detection for Autonomous Endoscopy Examination. Journal of Computer Science, 20(8), 858-871. https://doi.org/10.3844/jcssp.2024.858.871

Open AccessCopyright: © 2024 Md Shakhawat Hossain, Munim Ahmed, Md Sahilur Rahman, Mahreen Tabassum, Fariha Karim, Md Aulad Hossain, Razib Hayat Khan, M. M. Mahbubul Syeed and Mohammad Faisal Uddin. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

  • 1,348 Views
  • 370 Downloads
  • 0 Citations

Download

Keywords

  • Endoscopy
  • Anatomical Landmarks
  • Transformer
  • Abnormality Detection
  • Computer Aided Diagnosis