Dynamic Reference Image Selection for Seamless Video Colorization Across Scenes

With the rapid advancement of information technology and the widespread availability of high-resolution videos, the demand for colorizing black-and-white footage from previous eras has increased significantly. However, using a single reference image to colorize a video is a challenging task due to the variations in objects, colors, textures, and lighting across different video frames. This paper proposes a novel method to address these challenges by selecting multiple reference images based on the distinct scenes within a video. The proposed approach consists of two key components. First, a large set of reference images is generated, and their corresponding features are extracted using a custom network that combines ResNet50 with Generalized Mean Pooling (GeM). Second, the method operates in two phases: (1) automatically counting the number of different scenes in the video by calculating the Structural Similarity Index (SSI) between frames, followed by computing the mean and standard deviation to establish a threshold for scene differentiation; and (2) selecting the most suitable reference image for each scene by comparing the features of the scene, extracted via ResNet50-GeM, with those of the reference images. The dot product is used to evaluate feature similarity, with the highest-ranked reference image selected for each scene. The proposed method achieved an accuracy of 99% in identifying scene changes and selecting appropriate reference images, outperforming existing techniques in both accuracy and efficiency