CVPR2018跟踪算法 STRCF的配置（Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking）

110 阅读 0 评论 73 点赞

我是靠谱客的博主体贴花生，这篇文章主要介绍CVPR2018跟踪算法 STRCF的配置（Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking），现在分享给大家，希望可以做个参考。

CVPR2018 的论文“Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking”已经提供源代码，下面介绍一下它的配置过程。

1、Github下载地址：https://github.com/lifeng9472/STRCF

2、从如下地址下载matconvnet，并解压到external_libs/matconvnet/ 路径

https://github.com/vlfeat/matconvnet

3、从如下地址下载PDollar Toolbox，并解压到external_libs/pdollar_toolbox/路径

https://github.com/pdollar/toolbox

4、从如下地址下载预训练模型imagenet-vgg-m-2048.mat，并放到feature_extraction/networks/路径（注：networks文件夹得自己新建）

Pretrained CNNs - MatConvNet

5、运行install.m，此时需要调用C++编译器，如果出现错误，请参考博客配置好matlab中的C++编译环境：Matlab配置C++/VS2015等编译环境(mex -setup 找不到编译器问题解决)_博博有个大大大的Dream-CSDN博客_matlab mex setup

6、运行demo_STRCF.m得到结果

7、运行深度特征，

注释掉17行results = run_STRCF(seq);

取消注释18行results = run_DeepSTRCF(seq);

8、打开run_DeepSTRCF.m更改67行如下，不使用GPU（因为小本本没有GPU）。

9、再次运行demo_STRCF.m得到深度特征的跟踪结果

2018年10月20日更新STRCF在VOT中使用

1、D:CodeMATLAB2014STRCF-masterSTRCF-masterfeature_extractionload_cnn.m中的相对路径为绝对路径

2、新建DeepSTRCF对应VOT工程（源代码给出了DeepSTRCF的接口，直接使用DeepSTRCF做示例，STRCF可以参考改动测试）

tracker_DeepSTRCF.m设置如下：

复制代码

%error('Tracker not configured! Please edit the tracker_DeepSTRCF.m file.'); % Remove this line after proper configuration

% The human readable label for the tracker, used to identify the tracker in reports
% If not set, it will be set to the same value as the identifier.
% It does not have to be unique, but it is best that it is.
tracker_label = ['DeepSTRCF'];

% For MATLAB implementations we have created a handy function that generates the appropritate
% command that will run the matlab executable and execute the given script that includes your
% tracker implementation.
%
% Please customize the line below by substituting the first argument with the name of the
% script (not the .m file but just the name of the script as you would use it in within Matlab)
% of your tracker and also provide the path (or multiple paths) where the tracker sources
% are found as the elements of the cell array (second argument).
tracker_command = generate_matlab_command('vot_wrapper(''DeepSTRCF'', ''DeepSTRCF_VOT_setting'')', {'D:CodeMATLAB2014STRCF-masterSTRCF-master'});

tracker_interpreter = 'matlab';
tracker_trax = false;
% tracker_linkpath = {}; % A cell array of custom library directories used by the tracker executable (optional)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

%error('Tracker not configured! Please edit the tracker_DeepSTRCF.m file.'); % Remove this line after proper configuration

% The human readable label for the tracker, used to identify the tracker in reports
% If not set, it will be set to the same value as the identifier.
% It does not have to be unique, but it is best that it is.
tracker_label = ['DeepSTRCF'];

% For MATLAB implementations we have created a handy function that generates the appropritate
% command that will run the matlab executable and execute the given script that includes your
% tracker implementation.
%
% Please customize the line below by substituting the first argument with the name of the
% script (not the .m file but just the name of the script as you would use it in within Matlab)
% of your tracker and also provide the path (or multiple paths) where the tracker sources
% are found as the elements of the cell array (second argument).
tracker_command = generate_matlab_command('vot_wrapper(''DeepSTRCF'', ''DeepSTRCF_VOT_setting'')', {'D:CodeMATLAB2014STRCF-masterSTRCF-master'});

tracker_interpreter = 'matlab';
tracker_trax = false;
% tracker_linkpath = {}; % A cell array of custom library directories used by the tracker executable (optional)

3、更改工程目录下DeepSTRCF_VOT_setting.m

设置67行不使用GPU，小本本没有GPU

params.use_gpu = false; % Enable GPU or not

4、重新编写VOT接口

vot_initialize.m 其中E:Datasetsvot2016为我的数据集路径

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
function [images, region] = vot_initialize()
    % Add paths
    setup_paths();
    pathstr='E:Datasetsvot2016';
    video=choose_video(pathstr);
    % read the image file paths
    if ~isempty(video)
        dirs=dir([pathstr video '*.jpg']);
        images = fullfile([pathstr video ''], {dirs.name})';
    end
    
    % read the region
    % 此处对应polygon坐标
    region = dlmread([pathstr video 'groundtruth.txt']);
    region = region(1,:)';
    % 此处对应rectangle坐标    
    %region = read_vot_regions([pathstr video 'groundtruth.txt']);
    %region = region(1,:)';
end

vot_report.m

复制代码

1
2
3
4
5
6
7
8
function vot_report(regions)

for i = 1:numel(regions)
    region = regions{i};
    [~, ~] = traxserver('wait');
    traxserver('status', region);
end;

vot_save.m

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
function vot_save(regions)

fid = fopen('output.txt', 'w');

for i = 1:numel(regions)
    region = regions{i};
    
    if numel(region) == 1
        fprintf(fid, '%fn', region);
    elseif numel(region) == 4
        fprintf(fid, '%f,%f,%f,%fn', region(1), region(2), region(3), region(4));
    elseif numel(region) >= 6 && mod(numel(region), 2) == 0
        fprintf(fid, '%f,', region(1:end-1));
        fprintf(fid, '%fn', region(end));
    else
        error('VOT: Illegal result format');
    end;
end;
fclose(fid);

vot_wrapper.m

复制代码

function vot_wrapper(tracker_name, runfile_name, do_cleanup)

if nargin < 3
    do_cleanup = true;
end

% *************************************************************
% VOT: Always call exit command at the end to terminate Matlab!
% *************************************************************
if do_cleanup
    cleanup = onCleanup(@() exit() );
else
    [pathstr, ~, ~] = fileparts(mfilename('fullpath'));
    cd_ind = strfind(pathstr, filesep());
    pathstr = pathstr(1:cd_ind(end)-1);
    cleanup = onCleanup(@() cd(pathstr));
end

try

% *************************************************************
% VOT: Set random seed to a different value every time.
% *************************************************************
RandStream.setGlobalStream(RandStream('mt19937ar', 'Seed', sum(clock)));

% **********************************
% VOT: Get initialization data
% **********************************

if ~isempty(getenv('TRAX_MEX'))
    addpath(getenv('TRAX_MEX'));
end;
traxserver('setup', 'polygon', 'path');  %'rectangle', 'polygon'  
[images, region] = vot_initialize();

results = cell(length(images), 1);

bb_scale = 1;

% If the provided region is a polygon ...
if numel(region) > 4
    % Init with an axis aligned bounding box with correct area and center
    % coordinate
    cx = mean(region(1:2:end));
    cy = mean(region(2:2:end));
    x1 = min(region(1:2:end));
    x2 = max(region(1:2:end));
    y1 = min(region(2:2:end));
    y2 = max(region(2:2:end));
    A1 = norm(region(1:2) - region(3:4)) * norm(region(3:4) - region(5:6));
    A2 = (x2 - x1) * (y2 - y1);
    s = sqrt(A1/A2);
    w = s * (x2 - x1) + 1;
    h = s * (y2 - y1) + 1;
else
    cx = region(1) + (region(3) - 1)/2;
    cy = region(2) + (region(4) - 1)/2;
    w = region(3);
    h = region(4);
end

init_c = [cx cy];
init_sz = bb_scale * [w h];

im_size = size(imread(images{1}));
im_size = im_size([2 1]);

init_pos = min(max(round(init_c - (init_sz - 1)/2), [1 1]), im_size);
init_sz = min(max(round(init_sz), [1 1]), im_size - init_pos + 1);

seq.s_frames = images;
seq.init_rect = [init_pos, init_sz];

[file_path, file_name_start, file_ext] = fileparts(seq.s_frames{1});
[~, file_name_end, ~] = fileparts(seq.s_frames{end});
seq.path = file_path;
seq.name = 'vot_seq';
seq.ext = file_ext(2:end);
seq.len = length(seq.s_frames);
seq.nz = length(file_name_start);
seq.startFrame = str2num(file_name_start);
seq.endFrame = str2num(file_name_end);

% setup_tracker_paths(tracker_name);

otb_res = eval([runfile_name '(seq, [], []);']);
%convert the results to rectangle format
% otb_res = convert_to_rect(otb_res);

num_frames = numel(images);
for frame = 1:num_frames
    bb = otb_res.res(frame,:);
    sz = bb(3:4);
    c = bb(1:2) + (sz - 1)/2;
    new_sz = sz / bb_scale;
    new_tl = c - (new_sz - 1)/2;
    results{frame} = round([new_tl, new_sz]);
end

% **********************************
% VOT: Output the results
% **********************************
vot_report(results);
vot_save(results);
traxserver('quit');  %退出
catch err
    [wrapper_pathstr, ~, ~] = fileparts(mfilename('fullpath'));
    cd_ind = strfind(wrapper_pathstr, filesep());
    VOT_path = wrapper_pathstr(1:cd_ind(end));
    
    error_report_path = [VOT_path 'error_reports'];
    if ~exist(error_report_path, 'dir')
        mkdir(error_report_path);
    end
    
    report_file_name = [error_report_path tracker_name '_' runfile_name datestr(now,'_yymmdd_HHMM') '.mat'];
    
    save(report_file_name, 'err')
    
    rethrow(err);
end

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
function vot_wrapper(tracker_name, runfile_name, do_cleanup)

if nargin < 3
    do_cleanup = true;
end

% *************************************************************
% VOT: Always call exit command at the end to terminate Matlab!
% *************************************************************
if do_cleanup
    cleanup = onCleanup(@() exit() );
else
    [pathstr, ~, ~] = fileparts(mfilename('fullpath'));
    cd_ind = strfind(pathstr, filesep());
    pathstr = pathstr(1:cd_ind(end)-1);
    cleanup = onCleanup(@() cd(pathstr));
end

try

% *************************************************************
% VOT: Set random seed to a different value every time.
% *************************************************************
RandStream.setGlobalStream(RandStream('mt19937ar', 'Seed', sum(clock)));

% **********************************
% VOT: Get initialization data
% **********************************

if ~isempty(getenv('TRAX_MEX'))
    addpath(getenv('TRAX_MEX'));
end;
traxserver('setup', 'polygon', 'path');  %'rectangle', 'polygon'  
[images, region] = vot_initialize();

results = cell(length(images), 1);

bb_scale = 1;

% If the provided region is a polygon ...
if numel(region) > 4
    % Init with an axis aligned bounding box with correct area and center
    % coordinate
    cx = mean(region(1:2:end));
    cy = mean(region(2:2:end));
    x1 = min(region(1:2:end));
    x2 = max(region(1:2:end));
    y1 = min(region(2:2:end));
    y2 = max(region(2:2:end));
    A1 = norm(region(1:2) - region(3:4)) * norm(region(3:4) - region(5:6));
    A2 = (x2 - x1) * (y2 - y1);
    s = sqrt(A1/A2);
    w = s * (x2 - x1) + 1;
    h = s * (y2 - y1) + 1;
else
    cx = region(1) + (region(3) - 1)/2;
    cy = region(2) + (region(4) - 1)/2;
    w = region(3);
    h = region(4);
end

init_c = [cx cy];
init_sz = bb_scale * [w h];

im_size = size(imread(images{1}));
im_size = im_size([2 1]);

init_pos = min(max(round(init_c - (init_sz - 1)/2), [1 1]), im_size);
init_sz = min(max(round(init_sz), [1 1]), im_size - init_pos + 1);

seq.s_frames = images;
seq.init_rect = [init_pos, init_sz];

[file_path, file_name_start, file_ext] = fileparts(seq.s_frames{1});
[~, file_name_end, ~] = fileparts(seq.s_frames{end});
seq.path = file_path;
seq.name = 'vot_seq';
seq.ext = file_ext(2:end);
seq.len = length(seq.s_frames);
seq.nz = length(file_name_start);
seq.startFrame = str2num(file_name_start);
seq.endFrame = str2num(file_name_end);

% setup_tracker_paths(tracker_name);

otb_res = eval([runfile_name '(seq, [], []);']);
%convert the results to rectangle format
% otb_res = convert_to_rect(otb_res);

num_frames = numel(images);
for frame = 1:num_frames
    bb = otb_res.res(frame,:);
    sz = bb(3:4);
    c = bb(1:2) + (sz - 1)/2;
    new_sz = sz / bb_scale;
    new_tl = c - (new_sz - 1)/2;
    results{frame} = round([new_tl, new_sz]);
end

% **********************************
% VOT: Output the results
% **********************************
vot_report(results);
vot_save(results);
traxserver('quit');  %退出
catch err
    [wrapper_pathstr, ~, ~] = fileparts(mfilename('fullpath'));
    cd_ind = strfind(wrapper_pathstr, filesep());
    VOT_path = wrapper_pathstr(1:cd_ind(end));
    
    error_report_path = [VOT_path 'error_reports'];
    if ~exist(error_report_path, 'dir')
        mkdir(error_report_path);
    end
    
    report_file_name = [error_report_path tracker_name '_' runfile_name datestr(now,'_yymmdd_HHMM') '.mat'];
    
    save(report_file_name, 'err')
    
    rethrow(err);
end

5、附加的.m文件

choose_video.m

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
function video_name = choose_video(base_path)
%CHOOSE_VIDEO
%   Allows the user to choose a video (sub-folder in the given path).
%

	%process path to make sure it's uniform
	if ispc(), base_path = strrep(base_path, '', '/'); end
	if base_path(end) ~= '/', base_path(end+1) = '/'; end
	
	%list all sub-folders
	contents = dir(base_path);
	names = {};
	for k = 1:numel(contents),
		name = contents(k).name;
		if isdir([base_path name]) && ~any(strcmp(name, {'.', '..'})),
			names{end+1} = name;  %#ok
		end
	end
	
	%no sub-folders found
	if isempty(names), video_name = []; return; end
	
	%choice GUI
	choice = listdlg('ListString',names, 'Name','Choose video', 'SelectionMode','single');
	
	if isempty(choice),  %user cancelled
		video_name = [];
	else
		video_name = names{choice};
	end
	
end

6、运行run_test.m，选择需要测试数据

7、再选择一次，技术不精，接口没定义好233333（两次选择同一个数据集）

8、需要等一段时间才能得到结果，鼠标点击进入下一帧，红色为真实值，绿色为DeepSTRCF跟踪结果

9、参考博客：https://blog.csdn.net/qq_17783559/article/details/82146259

10、vot_save.m把跟踪结果保存成VOT的数据格式，可以直接拷贝到vot-toolkit工具中使用。

11、或参考格式先自己运行代码得到数据集结果，再拷贝到vot-toolkit工具中分析也可以。

2021年2月23日更新

论文中公式3推导公式4的过程：

公式3为：

$begin{array}{l} {cal L}(f,g,s) = frac{1}{2}||sumlimits_{d = 1}^D {x_t^d * {f^d}} - y|{|^2} + frac{1}{2}sumlimits_{d = 1}^D {||w cdot {g^d}|{|^2}} \ ;;;;;;;;;;; + sumlimits_{d = 1}^D {{{({f^d} - {g^d})}^T}{s^d}} + frac{gamma }{2}sumlimits_{d = 1}^D {||{f^d} - {g^d}|{|^2}} + frac{mu }{2}||f - {f_{t - 1}}|{|^2} end{array}$ （1）

此处这个公式（1）和论文中的公式3有一点区别，原始公式的第一个变量是 $w$ ，我感觉作者可能写错了，应该是 $f$ ，个人看法哈，不一定对。

在这个公式里面我们是想通过ADMM来求变量 $f$ 和 $g$ 为何值时，使得目标函数取得最小值，因此除 $f$ 和 $g$ 外其余的都可以认为是常量。最小化公式(1)可以等价于：

$begin{array}{l} {cal L}(f,g,s) = frac{1}{2}||sumlimits_{d = 1}^D {x_t^d * {f^d}} - y|{|^2} + frac{1}{2}sumlimits_{d = 1}^D {||w cdot {g^d}|{|^2}} \ ;;;;;;;;;;;{rm{ + }}frac{gamma }{2}sumlimits_{d = 1}^D {||frac{1}{gamma }{s^d}|{|^2}} + frac{gamma }{2}(frac{2}{gamma }sumlimits_{d = 1}^D {{{({f^d} - {g^d})}^T}{s^d}} ) + frac{gamma }{2}sumlimits_{d = 1}^D {||{f^d} - {g^d}|{|^2}} + frac{mu }{2}||f - {f_{t - 1}}|{|^2} end{array}$ (2)

为什么可以等价？

我们的目的是求变量 $f$ 和 $g$ 的值，这一项 $frac{gamma }{2}sumlimits_{d = 1}^D {||frac{1}{gamma }{s^d}|{|^2}}$ 的存在是不影响目标函数最小值的位置的，比如对公式（1）求对 $f$ 偏导和对公式（2）求对 $f$ 的偏导结果是一样的，因为增加的这一项 $frac{gamma }{2}sumlimits_{d = 1}^D {||frac{1}{gamma }{s^d}|{|^2}}$ 对 $f$ 和 $g$ 的偏导值都为0。也可结合论文中公式5来分析，加这一项不会影响ADMM两个子问题的求解。

然后我们令 $h = frac{1}{gamma }s$ ，即可得到论文中公式4的表达:

$begin{array}{l} {cal L}(f,g,h) = frac{1}{2}||sumlimits_{d = 1}^D {x_t^d * {f^d}} - y|{|^2} + frac{1}{2}sumlimits_{d = 1}^D {||w cdot {g^d}|{|^2}} \ ;;;;;;;;;;; + frac{gamma }{2}sumlimits_{d = 1}^D {||{f^d} - {g^d} + {h^d}|{|^2}} + frac{mu }{2}||f - {f_{t - 1}}|{|^2} end{array}$ (3)