Authenticate images based on their semantic segmentation in deep learning neural networks with their pre-processing with use of filtering methods


Keywords: image authenticity; artificial intelligence; neural networks; ELA; PCA; Inpainting.

Abstract

In modern domestic and international pre-trial practice and legal proceedings, physical evidence used in the form of electronic documents or their digital images. The issue of authenticating such images is hampered by possibilities of using artificial intelligence based editors to fake images making it impossible or significantly complicate the search for changed areas by forensic experts. Research issue considered by the authors is the authenticity assessment of digital images based on the use of their pre-processing (filtering) methods and artificial intelligence technologies for further analysis and determination of edited areas.

This research paper purpose is to develop information technology for finding editing zones based on a combination of imaging techniques and neural network models for use in image authenticity research.

Research paper novelty is to develop a technology to combine several methods of preprocessing images (in particular, ELA and PCA) to create an input stream of a deep learning neural network and to assess effectiveness of identifying editing zones created by the digital painting editor (Inpainting).

Efficiency of 10 editing zone detectors using combination of ELA and PCA methods with different models of neural networks for recognizing editing zones has been developed and investigated. The best results (probability of recognition 0.916) within the used computing resources were obtained by detector based on the EfficientNet model.

IT Effectiveness and related software for assessing authenticity of images based on a combination of image pre-processing methods and models of artificial neural networks in semantic classification and segmentation mode has been developed and evaluated.

References

Blythe, Р., Fridrich, J. (2004). Secure digital camera. Digital Forensic Research Workshop (Baltimore, USA, 11—13.08.2004).
Bobritskii, S. M., Chernyi, S. V. (2010). Metodicheskie aspekty kompleksnogo issledovaniia s tseliu vyiavleniia priznakov montazha v tsifrovoi fotografii [Methodological aspects of a Multidisciplinary Forensic Research in Order to Identify Signs of Montage in Digital Photography]. Teoriia ta praktyka sudovoi ekspertyzy i kryminalistyky. Vyp. 10 [in Russian].
Chollet, F. Xception: Deep Learning with Depthwise Separable Convolutions. URL: https://arxiv.org/pdf/1610.02357.pdf.
Dmitriev, E. N. (2007). Metadannye kak sredstvo individualizatsii tsifrovykh izobrazhenii v sudebnoi fotografii [Metadata as Individualization Means of Digital Images in Forensic Photography]. Teoriia i praktika sudebnoi ehkspertizy v sovremennykh usloviiakh : mat-ly mezhdunar. nauch.- prakt. konf. Moskva [in Russian].
Dmitriev, E. N. (2009). Sudebnaia ehkspertiza fotograficheskikh izobrazhenii: sovremen- noe sostoianie i vozmozhnosti resheniia ehkspertnykh zadach [Forensic Examination of Photographic Images: State of the Art and Possibilities for Solving Expert Tasks]. Sudebnaia ehkspertiza. № 3 [in Russian].
Gurgul, K. (1973). Iz protsessualnoi i kriminalisticheskoi problematiki magnitofonnoi zapisi [From the procedural and forensic issues of tape recording]. Sbornik perevodov. № 213. Moskva [in Russian].
Hollien, Н. (1994). The Acoustics of Crime: The New Science of Forensic Phonetics. Florida.
Kalenskyi, V. H. (2010). Doslidzhennia tsyfrovykh fotoznimkiv z metoiu vyiavlennia oznak montazhu [Research on Digital Photographs Using the Method of Revealing a Sign of Mounting]. Sudova ekspertyza. Suchasnyi stan ta perspektyvy rozvytku : mat-ly mizhnar. nauk.-prakt. konf. Kyiv [in Ukrainian].
Kobozeva, A. A. (2008). Ispolzovanie osobennostei vozmushchenii singuliarnykh chisel matritsy tsifrovogo izobrazheniia dlia obnaruzheniia ego falsifikatsii [Using Peculiarities of Disturbances of Singular Values of a Digital Image Matrix to Detect its Falsification]. Iskusstvennyi intellekt. № 1 [in Russian].
Krawetz, N. (2007). A Picture’s Worth... Digital Image Analysis and Forensics. URL: https:// www.hackerfactor.com/papers/bh-usa-07- krawetz-wp.pdf.
Mirskii, D. Ia. (1982). Sudebnaia fototekhnicheskaia ehkspertiza [Forensic photography]: metod. posob. dlia ehkspertov. Moskva [in Russian].
Problemy kriminalistiki (1971). Varshava. № 90 [in Russian].
Safonov, A. A., Kolotushkin, S. M., Kochubei, A. V. (2005). Kompiuternye tekhnologii v kriminalisticheskoi fotografii: teoreticheskie i prikladnye voprosy [IT in Forensic Photography: Theoretical and Applied Issues] : ucheb. posob. Volgograd [in Russian].
Silkin, P. F. (1979). Sudebno-issledovatelskaia fotografiia [Forensic Research Photography] : uchebnik. Volgograd [in Russian].
Wallace, G. К. (1991). The JPEG Still Picture Compression Standard. IEEE Transactions on Consumer Electronics.

Abstract views: 80
PDF Downloads: 71 PDF Downloads: 31
Published
2022-03-31
How to Cite
Chornyy, S., Brendel, O., & Gratiashvili, D. (2022). Authenticate images based on their semantic segmentation in deep learning neural networks with their pre-processing with use of filtering methods. Theory and Practice of Forensic Science and Criminalistics, 26(1), 125-137. Retrieved from https://khrife-journal.org/index.php/journal/article/view/515