Posted on

Improving Radial Lens Distortion Correction Using Secondary Learning Task

Introduction

Sports image analysis is one of the domains that has significantly benefited from advances in computer vision. With the growing adoption of machine learning methods new products and tools have emerged that make sports image analysis both accurate and accessible for professional athletes, amateur and junior athletes, and fans all around the world. Sports images serve as crucial visual records, aiding in post-game or live analysis and decision-making. However, very often they are affected by radial distortion that hinders their accurate interpretation and utilization.

We propose a deep-learning regression-based method to rectify images containing radial lens distortion, that operates on single independent images. Our method hase been trained on the football domain dataset, and has been fine-tuned specifically for images containing small radial distortion, which is more difficult to rectify accurately. Compared to other deep-learning radial distortion correction methods, we introduce the following contributions:

  • Secondary learning task that learns useful distortion features on random combinations of sample pairs, prevents overfitting, and encourages the feature extractor to learn more general distortion features.
  • Penalty term that encourages better accuracy on low-distortion images

The official GitHub repository for our method can be found at : … URL …

The best trained model and training configuration can be downloaded here – improving-radial-best.tar.gz

 

Results

We have performed a quantitative evaluation of our proposed method on the validation set of the Football360 validation subset, and compared our method with 3 other methods. Our method outperformed all other evaluated methods on all metrics by a significant margin.

MethodSSIMPSNRMDLD
Janos and Benesova (2023)0.85223.075 dB0.0172
Li et al. (2019)0.41710.919 dB1.431
Santana-Cendres et al. (2016)0.71421.647 dB0.1132
Our method0.93330.098 dB0.0088

We have performed a qualitative evaluation of our method on the World Cup 2014 dataset. The WorldCup 2014 images have not been used during the training of neither of the evaluated methods.

Input imagesCorrected by our methodJanos and Benesova (2023)Li (2019)Santana-Cedres (2016)

 

 

 

Posted on

Football360 – Introducing a New Dataset for Camera Calibration in Sports Domain

Introduction

The aim of this dataset is to aid the development and evaluation of computer vision algorithms, primarily the radial distortion correction algorithms, in the sports domain.

This dataset contains 268 panorama images, and was created using the PANONO panoramic camera in 3 football arenas in Slovakia. Each arena was covered from numerous locations on all levels of the tribune, and broadcast camera platforms. The images capture regular football game, pitch maintenance, low/challenging lighting conditions, day and night situations.

ArenaNumber of images
Bratislava111
Trnava104
Dunajská Streda53

You can download the raw images from the following link: football360-raw.tar.gz (7.7 GB)

There is also an official GitHub repository available at: https://github.com/IgorJanos/stuFootball360

 

Raw Images

Raw images are stored as 16384×8192 JPG, they are the direct result of the PANONO stitching service.

 

Exported Sets

There are four exported datasets available for direct download and use. Each dataset contains a collection of images (data) distorted with known distortion coefficients (labels).

Set namePurposeImagesPresetDownload Link
ATraining30,000setA.jsonfootball360-setA.h5 (10.5 GB)
BTraining100,000setB.jsonfootball360-setB.h5 (35.2 GB)
CTraining300,000setC.jsonfootball360-setC.h5 (105.5 GB)
VValidation10,000setV.jsonfootball360-setV.h5 (3.5 GB)

You can see an example of how to load and use these files in a linked notebook – https://github.com/IgorJanos/stuFootball360/blob/main/notebooks/explore.ipynb

Distorted ImageCorrected Image

 

Igor Janos

Posted on

Explainable 3D convolutional neural network using GMM encoding

Abstract:     The aim of this paper is to propose a novel method to explain, interpret, and support the decision-making process of deep Convolutional Neural Network (CNN). This is achieved by analyzing neuron activations of trained 3D-CNN on selected layers via Gaussian Mixture Model (GMM)   …  more

 

Posted on

Automatic Music Transcription using WaveNet

Deep generative models such as WaveNet are surprisingly good at various modelling tasks. We exploit the modelling capacity of WaveNet architecture in a setup that is quite different from the original generative case: for feature extraction and pattern recognition in sake of polyphonic music transcription. The model is trained end-to-end to perform the underlying task of multiple fundamental frequency estimation by processing raw waveforms of digital audio signal.   …  more