图像Demosaic算法及其matlab实现

87 阅读 0 评论 58 点赞

我是靠谱客的博主默默月饼，最近开发中收集的这篇文章主要介绍图像Demosaic算法及其matlab实现，觉得挺不错的，现在分享给大家，希望可以做个参考。

概述

由于成本和面积等因素的限定，CMOS/CCD在成像时，感光面阵列前通常会有CFA(color filter array)，如下图所示，CFA过滤不同频段的光，因此，Sensor的输出的RAW数据信号包含了3个通道的信息。由于人眼对绿色（大约550nm波长）光更敏感，因此CFA阵列包含1/2的G分量，1/4和R和1/4的B分量。

Sensor输出RAW数据后，需要经过Demosaic模块（ISP中）将其转成RGB图像。作为测试，我们可以对一幅全彩色RGB图进行RGB2Bayer转换，将其转换成RAW数据，再进行Bayer2RGB转换，输出RGB，对比输入和输出，可判断算法是否有效。

Bayer2RGB的转换方法有许多，下面给出基于3x3窗口简单插值的代码，支持BGGR、RGGB、GBRG三种bayer格式。输入一幅RGB图像，先进行RGB2Bayer转换，再进行Bayer2RGB转换。

%% ------------------------------------
% Author    :   lemonHe
% Time      :   20180814
% Function  :   Demosaic 
%%------------------------------------

clc;
clear;
close all;
imSrc = imread('detailTest.bmp');
% imSrc = imread('./bayer_data/bayer_1920x1080_12bit.tif');
figure,imshow(imSrc);

[hei, wid, chan] = size(imSrc);

bayer = uint8(zeros(hei,wid));

%% BGGR
% B G B G B G
% G R G R G R
% B G B G B G
% for ver = 1:hei;
%     for hor = 1:wid
%         if((1 == mod(ver,2)) && (1 == mod(hor,2)))
%             bayer(ver,hor) = imSrc(ver,hor,3);
%         elseif((0 == mod(ver,2)) && (0 == mod(hor,2)))
%             bayer(ver,hor) = imSrc(ver,hor,1);
%         else
%             bayer(ver,hor) = imSrc(ver,hor,2);
%         end
%     end
% end
% 
% figure,imshow(bayer);
% 
% bayerPadding = zeros(hei+2,wid+2);
% bayerPadding(2:hei+1,2:wid+1) = bayer;
% bayerPadding(1,:) = bayerPadding(3,:);
% bayerPadding(hei+2,:) = bayerPadding(hei,:);
% bayerPadding(:,1) = bayerPadding(:,3);
% bayerPadding(:,wid+2) = bayerPadding(:,wid);
% imDst = zeros(hei+2, wid+2, chan);
% 
% for ver = 2:hei+1
%     for hor = 2:wid+1
%         if(1 == mod(ver-1,2))
%             if(1 == mod(hor-1,2))
%                 imDst(ver,hor,3) = bayerPadding(ver,hor);
%                 imDst(ver,hor,1) = (bayerPadding(ver-1,hor-1) + bayerPadding(ver-1,hor+1) + bayerPadding(ver+1,hor-1) + bayerPadding(ver+1,hor+1)) / 4;
%                 imDst(ver,hor,2) = (bayerPadding(ver-1,hor) + bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1) + bayerPadding(ver+1,hor)) / 4;
%             else
%                 imDst(ver,hor,2) = bayerPadding(ver,hor);
%                 imDst(ver,hor,1) = (bayerPadding(ver-1,hor) + bayerPadding(ver+1,hor)) / 2;
%                 imDst(ver,hor,3) = (bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1)) / 2;
%             end
%         else
%             if(1 == mod(hor-1,2))
%                 imDst(ver,hor,2) = bayerPadding(ver,hor);
%                 imDst(ver,hor,1) = (bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1)) / 2;
%                 imDst(ver,hor,3) = (bayerPadding(ver-1,hor) + bayerPadding(ver+1,hor)) / 2;
%             else
%                 imDst(ver,hor,1) = bayerPadding(ver,hor);
%                 imDst(ver,hor,2) = (bayerPadding(ver-1,hor) + bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1) + bayerPadding(ver+1,hor)) / 4;
%                 imDst(ver,hor,3) = (bayerPadding(ver-1,hor-1) + bayerPadding(ver-1,hor+1) + bayerPadding(ver+1,hor-1) + bayerPadding(ver+1,hor+1)) / 4;
%             end
%         end
%     end
% end
% 
% imDst = uint8(imDst(2:hei+1,2:wid+1,:));
% figure,imshow(imDst);

%% RGGB
% R G R G R G
% G B G B G B
% R G R G R G
% for ver = 1:hei;
%     for hor = 1:wid
%         if((1 == mod(ver,2)) && (1 == mod(hor,2)))
%             bayer(ver,hor) = imSrc(ver,hor,1);
%         elseif((0 == mod(ver,2)) && (0 == mod(hor,2)))
%             bayer(ver,hor) = imSrc(ver,hor,3);
%         else
%             bayer(ver,hor) = imSrc(ver,hor,2);
%         end
%     end
% end
% 
% figure,imshow(bayer);
% 
% bayerPadding = zeros(hei+2,wid+2);
% bayerPadding(2:hei+1,2:wid+1) = bayer;
% bayerPadding(1,:) = bayerPadding(3,:);
% bayerPadding(hei+2,:) = bayerPadding(hei,:);
% bayerPadding(:,1) = bayerPadding(:,3);
% bayerPadding(:,wid+2) = bayerPadding(:,wid);
% imDst = zeros(hei+2, wid+2, chan);
% 
% for ver = 2:hei+1
%     for hor = 2:wid+1
%         if(1 == mod(ver-1,2))
%             if(1 == mod(hor-1,2))
%                 imDst(ver,hor,1) = bayerPadding(ver,hor);
%                 imDst(ver,hor,3) = (bayerPadding(ver-1,hor-1) + bayerPadding(ver-1,hor+1) + bayerPadding(ver+1,hor-1) + bayerPadding(ver+1,hor+1)) / 4;
%                 imDst(ver,hor,2) = (bayerPadding(ver-1,hor) + bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1) + bayerPadding(ver+1,hor)) / 4;
%             else
%                 imDst(ver,hor,2) = bayerPadding(ver,hor);
%                 imDst(ver,hor,3) = (bayerPadding(ver-1,hor) + bayerPadding(ver+1,hor)) / 2;
%                 imDst(ver,hor,1) = (bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1)) / 2;
%             end
%         else
%             if(1 == mod(hor-1,2))
%                 imDst(ver,hor,2) = bayerPadding(ver,hor);
%                 imDst(ver,hor,3) = (bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1)) / 2;
%                 imDst(ver,hor,1) = (bayerPadding(ver-1,hor) + bayerPadding(ver+1,hor)) / 2;
%             else
%                 imDst(ver,hor,3) = bayerPadding(ver,hor);
%                 imDst(ver,hor,2) = (bayerPadding(ver-1,hor) + bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1) + bayerPadding(ver+1,hor)) / 4;
%                 imDst(ver,hor,1) = (bayerPadding(ver-1,hor-1) + bayerPadding(ver-1,hor+1) + bayerPadding(ver+1,hor-1) + bayerPadding(ver+1,hor+1)) / 4;
%             end
%         end
%     end
% end
% 
% imDst = uint8(imDst(2:hei+1,2:wid+1,:));
% figure,imshow(imDst);

%% GBRG
% G B G B G B
% R G R G R G
% G B G B G B
for ver = 1:hei;
    for hor = 1:wid
        if((1 == mod(ver,2)) && (0 == mod(hor,2)))
            bayer(ver,hor) = imSrc(ver,hor,3);
        elseif((0 == mod(ver,2)) && (1 == mod(hor,2)))
            bayer(ver,hor) = imSrc(ver,hor,1);
        else
            bayer(ver,hor) = imSrc(ver,hor,2);
        end
    end
end

figure,imshow(bayer);

bayerPadding = zeros(hei+2,wid+2);
bayerPadding(2:hei+1,2:wid+1) = bayer;
bayerPadding(1,:) = bayerPadding(3,:);
bayerPadding(hei+2,:) = bayerPadding(hei,:);
bayerPadding(:,1) = bayerPadding(:,3);
bayerPadding(:,wid+2) = bayerPadding(:,wid);
imDst = zeros(hei+2, wid+2, chan);

for ver = 2:hei+1
    for hor = 2:wid+1
        if(1 == mod(ver-1,2))
            if(1 == mod(hor-1,2))
                imDst(ver,hor,2) = bayerPadding(ver,hor);
                imDst(ver,hor,1) = (bayerPadding(ver-1,hor) + bayerPadding(ver+1,hor)) / 2;
                imDst(ver,hor,3) = (bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1)) / 2;
            else
                imDst(ver,hor,3) = bayerPadding(ver,hor);
                imDst(ver,hor,2) = (bayerPadding(ver-1,hor) + bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1) + bayerPadding(ver+1,hor)) / 4;
                imDst(ver,hor,1) = (bayerPadding(ver-1,hor-1) + bayerPadding(ver-1,hor+1) + bayerPadding(ver+1,hor-1) + bayerPadding(ver+1,hor+1)) / 4;
            end
        else
            if(1 == mod(hor-1,2))
                imDst(ver,hor,1) = bayerPadding(ver,hor);
                imDst(ver,hor,2) = (bayerPadding(ver-1,hor) + bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1) + bayerPadding(ver+1,hor)) / 4;
                imDst(ver,hor,3) = (bayerPadding(ver-1,hor-1) + bayerPadding(ver-1,hor+1) + bayerPadding(ver+1,hor-1) + bayerPadding(ver+1,hor+1)) / 4;
            else
                imDst(ver,hor,2) = bayerPadding(ver,hor);
                imDst(ver,hor,1) = (bayerPadding(ver,hor-1) + bayerPadding(ver,hor+1)) / 2;
                imDst(ver,hor,3) = (bayerPadding(ver-1,hor) + bayerPadding(ver+1,hor)) / 2;
            end
        end
    end
end

imDst = uint8(imDst(2:hei+1,2:wid+1,:));
figure,imshow(imDst);

结果如下所示：
原始RGB图像

RGB2Bayer生成的bayer图像

Bayer2RGB转换后输出的RGB图像

下图为Sensor输出的RGGB格式RAW数据，使用插值的方法对其进行还原
这里写图片描述

这里写图片描述

下面来看看客观指标测试，对于图像复原来说，通过可以根据PSNR和SSIM来评价算法性能。美国Kodak Data公司提供真彩无损图像数据，由专业的全彩色设备采集而来，每个pixel的R、G、B都是通过感光器件得到的，图像大小为512*768，每个通道为8bit。可以对该图像进行降采样，模拟CFA器件，再使用Demosaic算法对采样后的数据进行还原，计算PSNR和SSIM即可评价算法性能。

参考：
[1] https://en.wikipedia.org/wiki/Demosaicing