welcome to the world of…

Looks like a small bulb used to indicate something unusual, like a malfunction.

OpenCV – convert projection matrix to maps

Filed under: Uncategorized — Tags: , , , , , — admin @ 2013-07-12 11:22

I created an application for a target hit evaluation using a webcam. The camera is never vertical to the target so there is a significant perspective distortion and I need a bird’s eye view. Fixing this with getPerspectiveTransform() and cv::warpPerspective() is easy but cv::warpPerspective() is very slow (at least for my purpose).

Faster perspective correction can be done using cv::remap() if you don’t mind that the maps creation takes a long time. Since the camera has a fixed focus then the maps can be created only once and then just run the remapping. The speedup of cv::remap() compared to cv::warpPerspective() is almost 5x.

Maps creation

The main problem is how to efficiently create those maps. My first try was to calculate the points one by one:

void perspective_to_maps(const cv::Mat &perspective_mat, const cv::Size &img_size,
                          cv::Mat &map1, cv::Mat &map2)
{
  // invert the matrix because the transformation maps must be
  // bird's view -> original
  cv::Mat inv_perspective(perspective_mat.inv());
 
  cv::Mat map_x(img_size, CV_32FC1);
  cv::Mat map_y(img_size, CV_32FC1);
 
  // bird's view point coordinates
  cv::Mat point = (cv::Mat_(3, 1)<double> 0, 0, 1);
  double *ppoint = point.ptr<double>(0);
  // original point coordinates
  cv::Mat orig_point(cv::Size(3, 1), CV_64FC1);
  double *porig_point = orig_point.ptr<double>(0);
 
  for (int y = 0; y < img_size.height; y++)
  {
    ppoint[1] = y;
    float *pmap_x = map_x.ptr<float>(y);
    float *pmap_y = map_y.ptr<float>(y);
    for (int x = 0; x < img_size.width; x++)
    {
      ppoint[0] = x;
      // perspective transformation by matrix multiplication
      orig_point = inv_perspective * point;
      pmap_x[x] = porig_point[0] / porig_point[2];
      pmap_y[x] = porig_point[1] / porig_point[2];
    }
  }
 
  // remap() with integer maps is faster
  cv::convertMaps(map_x, map_y, map1, map2, CV_16SC2);
}

Faster

Previous solution works well but takes a considerable time to calculate the maps even with maximum compiler optimizations. Lately I found function cv::perspectiveTransform() which can do the heavy lifting. Prepare simple XY matrix and apply transformation at once.

void perspective_to_maps(const cv::Mat &perspective_mat, const cv::Size &img_size,
                          cv::Mat &map1, cv::Mat &map2)
{
  // invert the matrix because the transformation maps must be
  // bird's view -> original
  cv::Mat inv_perspective(perspective_mat.inv());
  inv_perspective.convertTo(inv_perspective, CV_32FC1);
 
  // create XY 2D array
  // (((0, 0), (1, 0), (2, 0), ...),
  //  ((0, 1), (1, 1), (2, 1), ...),
  // ...)
  cv::Mat xy(img_size, CV_32FC2);
  float *pxy = (float*)xy.data;
  for (int y = 0; y < img_size.height; y++)
    for (int x = 0; x < img_size.width; x++)
    {
      *pxy++ = x;
      *pxy++ = y;
    }
 
  // perspective transformation of the points
  cv::Mat xy_transformed;
  cv::perspectiveTransform(xy, xy_transformed, inv_perspective);
 
  // split x/y to extra maps
  assert(xy_transformed.channels() == 2);
  cv::Mat maps[2]; // map_x, map_y
  cv::split(xy_transformed, maps);
 
  // remap() with integer maps is faster
  cv::convertMaps(maps[0], maps[1], map1, map2, CV_16SC2);
}

This generator is 5x-10x faster depending on architecture (ARM, x64, …).

Question: can be the XY matrix prepared faster? Any comments are welcome.

1 Comment

  1. Hi
    Thank you for sharing you code ! I have the same issue with remap which is faster than warpPerspective. I was quite surprise that OpenCV don’t have it in the box …
    Your code is what I was looking for ! (Thus, in my case, I don’t need to invert the perspective mat)

    Cheers,

    mhtrinh

    Comment by mhtrinh — 2014-03-13 @ 21:51

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.