Comments on ESRGAN

Introduction

As I in an previous post wrote I have implemented an old original ESRGAN version for upscaling images [1]. For me this version is not outdated, for others this version may be outdated.

Based on the sources I was able to convert old upscaler models to new upscaler models, which can be used with the previous mentioned ESRGAN version.

Sometimes a conversion worked and sometimes not. My tool for printing the Pickle Tensor file content was helpful to understand, what I need and what I have to do [2].

I wrote a tool for analysing the model structure of a given model in a rudimentary way. This small script allows me a fast overview w.r.t. a model, if I can use the model for my purposes [3].

Description

The script reads the raw data of the model. It is not necessary to declare a model first. This allows me in principle to take a look into the structure of each model as long it is in Pickle Tensor format.

The script tells me if I can use the model directly with the mentioned ESRGAN version. It tells me also if I need to convert the model to get it working. And the script is able to tell me if it is necessary to take a look on the data structure.

With this tool I was able to convert some model for the moment by hand. It showed me which keys I had to adjust to get the model converted. Then I checked if my assumption was right.

What I learned so far has to be considered in the next version of my converter.

Example

In the meantime I figured out, how to quick and dirty programme wa working solution for what I introduced as problem while converting. The following images uses the model

RRDB_8xPSNR.pth

RRDB states, that I converted model

8xPSNR.pth

Applying the new converted model results in something like that:

I need some more tests, but it seems, that the conversion works, when the old model has a good or friendly structure.

Side Note

I wrote some scripts, that can analyse RealESRGAN models. The innermost inner internal structure looks like the structure I am working with currently. I found RealESRGAN models, which looks similar, but one needs for each type a special approach for reading the model structure.

Open Issue

I was able to convert a 8x model by hand as described. Using this model it is still upscaling by a factor of 4. I have to take a deeper look in ESRGAN and the theory to understand this behaviour.

Final Words

Have a nice day. Have fun. Be inspired!

Reference

[1] https://github.com/xinntao/ESRGAN

[2] https://github.com/zentrocdot/esrgan_rrbd_converter/blob/main/tools/print_pth_data.py

[3] https://github.com/zentrocdot/esrgan_rrbd_converter/blob/main/tools/analyse_models.py