The perceptional-image-quality-measure- scheme makes use of Human Visual Specialty (HVS). Refer to the reference [2] for more detail. By this scheme, two human visual specialties, lightness shield and special frequency shield, are considered to improve the measure of MSE. Mean lightness and activities in the whole image decide two factors important for PE calculation, which means that PE calculation depends on both subjective and objective measures.

Subjective measure
By now, subjective evaluation by viewers is still a method commonly used in measuring image quality. The subjective test emphatically examines fidelity and at the same time considers image intelligibility. That is to say, when taking subjective test, viewers focus on difference between reconstructed image and the original and, while grading, they notice such details where information loss cannot be accepted.

The representative subjective method is Mean Opinion Score (MOS). It has two kinds of rules. One is absolute and another is relative. Two examples are shown below. In our experiment, we use absolute rule in order to seek the consistence between subjective and objective measures.

Absolute rule 5 Excellent 4 Good 3 Fair 2 Bad 1 Very bad

Relative rule 5 The best in the group 4 Better than the average 3 The average of the group 2 Worse than the average 1 The worst in the group

The standard for quality levels should be established before grading. Then the viewer compares the reconstructed image with the original to decide which level it belongs to and gives the score. The final score is the average of all the viewers’. The number of viewers should be greater than 20.

In order to make scores more accurate, we use the hundred-score system. The rule for scoring is as follows.

90-100: almost distortionless
80-89: a little distortion, which can be ignored
60-79: distortion can be seen evidently but can be accepted reluctantly
40-59: a lot of distortion, which can’t be accepted
0-40: too much distortion to be tolerated

Both professional and amateur viewers are required to finish the subjective test. The former has experience in getting details and their evaluation could be regarded as of more image intelligibility. The latter has no training and their evaluation represent an ordinary sense perception on image quality. To images for a special application, conclusion drawn by the professional is more important than that by the amateur.

Test results
Ten typical images with different scenes are processed by nine compression methods and ninety reconstructed images are obtained. Some test results, gained by both objective and subjective measures mentioned above, are shown below.

Figure 1 is the result from subjective evaluation test, where a group of data dots with the same sign are connected to form a broken line, which indicates one method, applied to different images and the broken lines themselves have no physical meaning. It can be seen from the figure that when a compression method is good enough (MOS is great than 90) the MOS won’t change evidently with different images and keeps relatively steady. But to those whose reconstructed images are not very good, which are usually with high bit ratio or relatively simple algorithms, different images cause changes of the quality of reconstructed images. It shows the necessity to evaluate a compression algorithm with different types of standard images.

Figure 2 shows a contrast between subjective and objective measures on one of those ten images. Explanation to the broken lines is the same as above. We’ve drawn a conclusion based on the tests on the ten images.

Fig.1 Results of subjective measure



Fig.2 contrast between subjective and objective measures

• PSNR can reflect the quality of reconstructed images approximately. Generally speaking, PSNR must be above certain value if the reconstructed image reaches the level of “good”.
• To a sort of compression algorithms based on the same fundamentals, PSNR coincides with subjective evaluation and can be used as a measure to evaluate the quality of reconstructed images. To algorithms based on different fundamentals, PSNR cannot reflect image quality correctly all the time. In other words, PSNR isn’t sufficient to determine whether an algorithm is good or not.
• Perceptional Error can reflect quality of reconstructed images fairly well. Generally speaking, PEm must be under certain value if the reconstructed image reaches the level of “good”.
• Fidelity and resemblity are so obtuse that they cannot show the changes sensitively.

Conclusion
Study on criteria for image quality evaluation is a meaningful but complicated task. The criteria will be used to evaluate the compression algorithm and to guide the design of algorithm as well. We’ve made an approach to the measurement of image quality and drawn some primary conclusions, which indicates that the research method is feasible.

To obtain an accurate and reliable conclusion eventually, we need subjective tests on a large quantity of standard images, including testing samplings of different resolution and different image types, various compression algorithms and different bit ratio, etc. At the same time, objective measures should be studied deeply and improved with the result of a quantitative measure, which will consistently be used as a substitute.

References

• Ahmet M.Eskicioglu & Paul S.Fisher, Image Quality Measures and Their Performance, IEEE Transactions on Communications. Vol.43, No.12, Dec.1995
• Pamela C.Cosman, Robert M.Gray & Richard A.Olshen, Evaluating Quality of Compressed Medical Images: SNR, Subjective Rating, and Diagnostic Accuracy, IEEE, Vol.82, No.6, Jun.1994
• Shanika A.Karunasekera & Nick G.Kingsbury, A Distortion Measure for Blocking Artifacts in Images Based on Human Visual Sensitivity, IEEE Transactions on Image Processing. Vol.4, No.6, Jun.1995