CN103985121A - Method for calibrating underwater projector - Google Patents

Abstract

The invention belongs to the technical field of measuring, and provides a method for calibrating a structure of an underwater projector. The method comprises the following steps of shooting image pairs of a waterproof target and a projection target in the at least three positions, wherein the waterproof target and the projection target are arranged in the same plane; under the coordinate system of a camera, calculating a plane normal vector and three-dimensional coordinates of the projection target, adopting a single-view-point model to initially calibrate the projector and structural parameters, and converting the three-dimensional coordinates of the projection target into the coordinate system of the projector to be used as reference values; under the coordinate system of the projector, setting up the incident ray of the projection target and a refracted ray model, and calculating the three-dimensional coordinates of the projection target; setting up a target function according to the distance between the calculated three-dimensional coordinates of the projection target and the reference value, and performing recursive optimization on all the parameters. According to the method, the refraction effect which happens when the ray of the projector is transmitted to the aqueous medium from the air medium is taken into consideration, and under the condition that the optical axis of the projector is not parallel to a normal vector of a window of a waterproof device, the requirement for calibrating the underwater projector can be met.

Description

One is projector calibrating method under water

Technical field

The present invention relates to one projector calibrating method under water, belong to field of measuring technique.

background technology

The structured light pattern that projector is combined with video camera is a kind of important method for three-dimensional measurement and technology in vision measurement.Its principle is to adopt projector architecture optical sensor, projector need to initiatively project feature according to measuring, as dot matrix, grating fringe, coding graticule mesh etc., the image that video camera is caught comprises because object surface appearance changes the projection feature that produces deformation, recovers the three-dimensional information of body surface according to structured light principle of triangulation.The piece image of taking due to video camera can comprise a large amount of characteristic informations, therefore measures application for in-plant underwater 3 D, and projector architecture light technology is a kind of effective means of recovering testee three-dimensional information.

Projector architecture photo measure environment is different from the air dielectric environment of land under water.The projector of underwater environment operation need to be fixed in the water-tight device of sealing, projects feature by the transparent window being positioned at before projector lens to the testee being in water.Due to the use of water-tight device, testee and projector image plane are laid respectively in different medium, be that testee is arranged in aqueous medium, and projector image plane is arranged in the air dielectric of water-tight device, light can produce refraction effect in the time that projector light the heart channel of Hang-Shaoyin is crossed window plane and entered aqueous medium.Now, be based upon light no longer applicable along the single view model on rectilinear propagation basis.

The projection process of projector is contrary with the imaging process of video camera, can regard reverse video camera as.Because making light, refraction there is bending, and inconsistent with projector photocentre position and the actual photocentre position of single view model description.Treibitz etc. analyze and point out at article " Flat Refractive Geometry.IEEE Transactions on Pattern Analysis and Machine Intelligence; 2012; 34 (1): 51-65 ", the distance between testee and video camera is depended in the distortion that refraction produces, if adopt single view model or with this distortion of radial distortion model representation, can produce larger error, therefore single view model can not accurate description projector projects process under water.But situation when this article has only considered that optical axis is parallel with window plane normal vector, when optical axis and window plane normal vector are when not parallel, based on this article, analysis is set up model and is still had deviation.

From above-mentioned analysis, due to the refraction effect of water-tight device window to light generation, and projector optical axis and the uneven impact of window plane normal vector, traditional projector single view model is no longer applicable.Therefore, study a kind of model and scaling method that is applicable to projector architecture light vision measuring sensor under water significant.

Summary of the invention

Technical matters to be solved by this invention is: provide a kind of for the projector parameter of projector architecture optical sensor and the optimization scaling method of structural parameters under water.

Technical solution of the present invention is: one is projector calibrating method under water, it is characterized in that the method includes the steps of:

1.1, adjust the projector architecture optical sensor being positioned under water, ensure video camera can shooting clear image in measurement range and projector in measurement range, can project picture rich in detail, at projector with within the scope of the public view field of calibrating camera, adopt a flat board with waterproof target, projector projects another target to this flat board simultaneously, be called projection target, adjust flat position, until two target unique points are all positioned at public view field scope; Move freely at least three positions of target, position of every movement, takes the image pair that contains two targets;

1.2, extract the image coordinate of waterproof target unique point in image, according to the parameter of calibrating camera, ask for the normal vector of dull and stereotyped place plane under camera coordinate system;

1.3, extract the image coordinate that projects target unique point in image, utilize the normal vector of trying to achieve in step 1.2, by the image coordinate of camera parameters and unique point, calculate projection target and be incident upon corresponding point position in space on the flat board three-dimensional coordinate under camera coordinate system;

1.4, according to projecting target in image coordinate, and the three-dimensional coordinate of its corresponding spatial point of trying to achieve in step 1.3, the structural parameters between the preliminary labeling projection instrument parameter of single view model and projector and video camera utilized;

1.5, use the structural parameters that obtain in step 1.4, normal vector required in step 1.2 is illustrated under projector coordinates system;

1.6, under projector coordinates system, the distance initial value of given projector photocentre along optical axis direction to water-tight device window plane, ray model when setting up light that on projector image, each unique point the is sent air dielectric in water-tight device and propagating, is called incident ray;

1.7, the vector of unit length of setting up refracted ray, goes out scale factor according to Geometric Constraint Solving, determines refracted ray length, solves the three-dimensional coordinate of the lower projection target unique point of projector coordinates system;

1.8, the three-dimensional coordinate calculating in step 1.3 is utilized the structural parameters in step 1.4 be converted under projector coordinates system as reference value;

1.9, given projector optical axis is with respect to the initial value of drift angle and corner between window plane normal vector, set up objective function with the distance between the reference value described in the three-dimensional coordinate described in step 1.7 and 1.8, projector parameter and structural parameters are optimized, and carry out recursive optimization, the structural parameters of trying to achieve are for step 1.5～1.8, until target function value meets setting threshold, threshold value requires to set according to measuring accuracy, is generally 1E-5 to 0.1mm.

2, one according to claim 1 projector calibrating method under water, is characterized in that:

2.1, the waterproof target described in step 1.1 is the plane target drone that waterproof material is made, be fixed on a flat board, waterproof target thickness can be ignored compared with measuring distance, target unique point is the lattice point in plane, between lattice point, the design of distance is determined by actual measurement distance and camera field of view etc., is generally 10～100mm; Described projection target, the image coordinate distance between its lattice point is determined according to the resolution of projector image, is generally 10～100 pixels;

2.2, the waterproof target described in step 1.1 and projection target, video camera rationally make filtering device as the prerequisite of optical filter under, shared region can be overlapping on flat board for two targets; When flat board is during at same position, when video camera does not use filtering device and projector not to project target, video camera is taken and is had to waterproof target image, in the time that video camera uses filtering device and projector projects target, this target and waterproof target have overlapping region, and video camera is taken and had to projection target image;

2.3, the image pair that the shooting described in step 1.1 contains two targets, is characterized in that, in the time using filtering device, two width images are taken in each position, for containing respectively the image pair of waterproof target and projection target.

Advantage of the present invention is:

The first, consider the refraction effect that projector produces when the air dielectric of light from water-tight device is transmitted to the aqueous medium outside water-tight device when operation under water, set up projector light refraction model;

The second, consider situation when projector optical axis and water-tight device window plane normal vector are not parallel, describe this phenomenon with optical axis with respect to drift angle and the corner of normal vector, and draw the value of two angles by optimization method;

The 3rd, rationally use filtering device, adopt waterproof target and projection target that can be overlapping, allow two targets take visual field simultaneously as far as possible, improve stated accuracy.

Brief description of the drawings

Fig. 1 is the light of the projector under water refraction model schematic diagram that the present invention sets up.

Fig. 2 is projector optical axis and the uneven schematic diagram of window plane normal vector under water.

Fig. 3 is projector calibrating method flow diagram under water.

Fig. 4 demarcates the target position schematic diagram that projector uses under water.

Embodiment

Below the present invention is described in further details.The present invention sets up the light refraction model of projector under water, and projector parameter and structural parameters when optical axis and water-tight device window plane normal vector are existed to the anglec of rotation are optimized demarcation, have realized high-precision projector calibrating under water.

Figure 1 shows that projector light refraction model schematic diagram under water.Setting up projector coordinates taking projector photocentre as initial point is O _p-X _py _pz _p, original image coordinate is o _u-uv, (u ₀, v ₀) be the coordinate of principal point.Set up image coordinate system o-xy taking the principal point position on projector image as initial point, have x=u-u ₀, y=v-v ₀.Definition ox||O _px _p, oy||O _py _p.X=[x, y] ^tthe ideal image coordinate of any in expression projector image plane, x _d=[x _d, y _d] ^tfor the actual image coordinate that is subject to lens distortion impact.Point x projector image plane _d=[x _d, y _d] ^tthe light projecting, reflects during through water-tight device window plane, and note incident ray is R _in, refracted ray is R _out.Use n=[w ₁-w ₂0] ^trepresent the unit normal vector of the lower window plane of projector coordinates system, wherein w ₁and w ₂represent that respectively projector optical axis is with respect to the drift angle between window plane normal vector and corner, as shown in Figure 2, and meet represent the angle between optical axis and normal vector.The normal vector of window plane can also be expressed as:

In formula, d represents the distance from projector photocentre along optical axis direction to window plane.

Suppose that center of distortion and principal point overlap, Consideration of Second Order radially and tangential distortion, have following distortion model:

$x_d=\left[\begin{array}{c}{x}_d\\ y_d\end{array}\right]=(1+k_1{r}^2+k_2{r}^4)\left[\begin{array}{c}x\\ y\end{array}\right]+\left[\begin{array}{c}2p_1\mathrm{xy}+p_2(r^2+2x^2)\\ p_1(r^2+2y^2)+2p_1\mathrm{xy}\end{array}\right]---\left[2\right]$

K in formula ₁, k ₂for coefficient of radial distortion, p ₁, p ₂for tangential distortion coefficient, and have:

$r={\left(\frac{x-u_0}{f_x}\right)}^2+{\left(\frac{y-v_0}{f_y}\right)}^2---\left[3\right]$

Wherein f _xand f _yfor projector lens focal length component in the x and y direction.

As shown in Figure 1, the point (x from projector photocentre to the plane of delineation _d, y _d) send incident ray R _in, after superrefraction, light is along R _outdirection continues to propagate until intersect at a P (X with testee surface _p, Y _p, Z _p).Represent incident angle with γ, γ ' represents refraction angle, has according to refraction law:

$\left\{\begin{array}{c}\cos \mathrm{\γ}={\hat{R}}_{\mathrm{in}}\·\hat{n}\\ \sin {\mathrm{\γ}}^{\′}=\frac{\sin \mathrm{\γ}}{n_{\mathrm{water}}}\end{array}\right.---\left[4\right]$

In formula represent the vector of unit length of incident ray direction, represent the vector of unit length of window plane normal vector.

Incident ray R _incan be expressed as:

$R_{\mathrm{in}}=k{\left[\frac{x_d}{f_x}\frac{y_d}{f_y}1\right]}^T---\left[5\right]$

In formula, k is a scale factor.Incident ray R _inmeet with window plane normal vector n:

R _in·n＝1 [6]

Formula [1] and [5] substitution [6] can be solved to scale factor k.Because the thickness of water-tight device window can be ignored than d, the vector of unit length of refracted ray can be expressed as:

${\hat{R}}_{\mathrm{out}}=m_1\hat{n}+m_2{\hat{R}}_{\mathrm{in}}---\left[7\right]$

Wherein m ₁and m ₂for:

$\left\{\begin{array}{c}{m}_1={\cos \mathrm{\γ}}^{\′}-m_2\cos \mathrm{\γ}\\ m_2=\frac{1}{\mathrm{rfn}}\end{array}\right.---\left[8\right]$

The refractive index that wherein rfn is aqueous medium.

Refracted ray continues to propagate, and intersects at a P with testee surface, and under projector coordinates system, the coordinate of some P can be expressed as:

$X_p=\left[\begin{array}{c}{X}_p\\ Y_p\\ Z_p\end{array}\right]=R_{\mathrm{in}}+\mathrm{\β}{\hat{R}}_{\mathrm{out}}---\left[9\right]$

Above formula has represented the process of light transmition, and wherein β is scale factor, has determined the length of refracted ray, and the value of β is determined jointly by the refracted ray of the unique point video camera light definite with being imaged on picture point on camera review.

According to above-mentioned model, the demarcation that the demarcation of projector under water comprises structural parameters (rotation and translation relation) between projector parameter (focal length, principal point coordinate, distortion factor, projector photocentre are along optical axis direction to drift angle and corner between distance and projector optical axis and the window plane normal vector of window plane) and projector and video camera, concrete steps are as follows:

1, utilize the video camera of having demarcated, with projector composition structured light sensor, the focal length, principal point coordinate and the distortion factor that are video camera are known, camera marking method is shown in the article " A Flexible New Technique for Camera Calibration.IEEE Transactions on Pattern Analysis and Machine Intelligence; 2000,22 (11): 1330-1334 " of Zhang.

2, as shown in Figure 4, making waterproof gridiron pattern target (waterproof target) red and white is attached on a flat board, unique point on this target is that angle point spacing is known, and the setting of angle point spacing is jointly determined by camera field of view and operating distance, is generally 10～100mm; The red-black alternate gridiron pattern target (projection target) of projector projects, the coordinate of the angle point on this target under projector image coordinate system is known, is designated as x _p=(x _pi, y _pi), i=1 ..., m, the setting of angle point spacing is determined by the resolution of projector image, is generally 10～100 pixels.Concrete image acquisition step is:

1) waterproof target is placed within the scope of video camera and projector public view field, and takes camera field of view as far as possible.Open the red-black gridiron pattern target of projector projects, adjust flat position, until projection target and waterproof target are all positioned at public view field scope;

2) redness is filtered paper is placed on video camera front, and because redness optical filtering paper can absorb the redness in waterproof target, therefore this waterproof target is rendered as unified background colour in the video camera with redness optical filtering paper.Use camera acquisition image, this image only contains the imaging that projects target, is designated as p _i, (i=1,2,3 ... g);

3) projector is turned off, removed red optical filtering paper, and gather red white gridiron pattern waterproof target image, this image only contains the imaging of waterproof target, is designated as c _i, (i=1,2,3 ... g);

4) repeat 1)～3) step, until gather enough map sheet numbers, need at least gather 3 pairs of images.

Adopting red white gridiron pattern waterproof target and red-black gridiron pattern projection target is one of example of reasonable use filtering device of the present invention, and any demarcation mode of filtering device that uses is in a similar manner all in protection domain.

3, extract image c _ithe image coordinate of middle angle point, according to the video camera of having demarcated, computed image c _ithe normal vector of middle plateform place plane under camera coordinate system , computing method are referring to Hartley R.I., " Multiple view geometry in computer vision, Cambridge University Press, 2004 " that Zisserman shows.

4, extract image p _ithe image coordinate of middle angle point, determines a ray by the image coordinate of camera parameters and angle point, utilizes the plane normal vector of trying to achieve in this ray and step 3, can be in the hope of being incident upon projection target on the flat board three-dimensional coordinate under camera coordinate system

5, according to the projector image coordinate x of projection target angle point _p, and the three-dimensional coordinate of its corresponding spatial point of trying to achieve in step 4 utilize single view model, preliminary labeling projection instrument parameter and projector are to the structural parameters between video camera, concrete grammar is referring to Zhou Fuqiang work " " the gordian technique research that binocular stereo vision detects ", BJ University of Aeronautics & Astronautics's post-doctoral research work report, 2002 ".

6, use the structural parameters that obtain in step 5, represent the normal vector of dull and stereotyped place plane under projector coordinates system

$N_i^p=\frac{R^{-1}\·N_i^c}{1-R^{-1}{T}^T\·R\·N_i^c}---\left[10\right]$

In formula, structural parameters R and T are respectively camera coordinates and are tied to rotation matrix and the translation vector that projector coordinates is.

7, the distance d initial value of given projector photocentre along optical axis direction to window plane, for example, in the time that projector lens is very near apart from window plane, can make d=0; Set projector optical axis with respect to the drift angle w between window plane normal vector ₁with corner w ₂initial value be 0.Utilize image p _iin the image coordinate of angle point, the ray model when being set up light that each angle point the sends air dielectric in water-tight device and propagated by formula [1]～formula [6], is called incident ray R _in.

8, according to every the incident ray R setting up in step 7 _in, set up the vector of unit length of corresponding refracted ray by formula [7] and formula [8] determine the corresponding ray trajectory of each angle point, next needed to determine the scale factor β in formula [9], thereby determined that refracted ray is crossing with flat board wherein.

9, have according to geometrical constraint:

${\left[X_p{Y}_p{Z}_p\right]}^T\·N_i^p=1---\left[11\right]$

By formula [9] substitution formula [11], can solve scale factor β by following formula.

$\mathrm{\β}=\frac{1-{\left(N_i^p\right)}^T\·{\hat{R}}_{\mathrm{in}}}{{\left(N_i^p\right)}^T\·{\hat{R}}_{\mathrm{out}}}---\left[12\right]$

10,, by scale factor β substitution [9] formula of trying to achieve in step 9, can try to achieve the lower projection target of projector coordinates system and be incident upon the dull and stereotyped three-dimensional coordinate X of corresponding spatial point _p.

11, by the three-dimensional coordinate calculating in step 4 utilizing structural parameters in step 6 to be converted under projector coordinates system is

12, will as with reference to value, with the three-dimensional coordinate X of gained in itself and step 10 _pbetween distance set up objective function:

$\underset{i=1}{\overset{g}{\mathrm{\Σ}}}\underset{j=1}{\overset{m}{\mathrm{\Σ}}}\left|\right|X_p-{\tilde{X}}_p(f_x,f_y,u_0,v_0,d,w_1,w_2,k_1,k_2,p_1,p_2,R,T)\left|\right|---\left[13\right]$

To demarcate the projector parameter that obtains and the structural parameters initial value as Optimizing Search in step 5, distortion factor initial value is all made as 0, and other parameter initial value is referring to step 7.Utilize Levenberg-Marquardt algorithm to carry out nonlinear optimization to formula [13], the structural parameters R that optimization is obtained, T is applied to the coordinate conversion of step 6 and step 11, carry out the optimal value of recursive search parameter, until target function value is less than the distance error threshold value of setting, threshold value requires to arrange according to measuring accuracy, is generally 1E-5 to 0.1mm.

Embodiment

Adopt the projector of 3500 lumens, resolution is 1024 pixel × 768 pixels.Projector is packed in water-tight device and fixed, and form structured light sensor with the optical fiber video camera that resolution is 1024 pixel × 768 pixels.Projector and video camera relative position are fixed.

The waterproof target of making is red white gridiron pattern target, and between the angle point of target, distance is 34.75mm.The projection target of design is red-black gridiron pattern target, and the image distance between its angle point is 64 pixels.Utilize image-pickup method of the present invention, structured light sensor is put into experiment pond and gathered 10 groups of totally 20 images.Utilize the method for the invention to carry out projector and structural parameters calibration, the results are shown in Table 1.Table 2 is for reference value three-dimensional coordinate and utilize model to calculate the root-mean-square error between three-dimensional coordinate.

Can find out from the data of table 1 and table 2, the method for projector calibrating under water that adopts the present invention to propose, demarcates the projector parameter and the structural parameters reliable results that obtain, and the stated accuracy of the method can meet qualitative monitoring under water and quantitative measurment task.

Table 1 projector architecture optical sensor calibration result

The root-mean-square error of table 2 three-dimensional coordinate

The above is embodiments of the invention, is not intended to limit protection scope of the present invention.

Claims (2)

1. a projector calibrating method under water, is characterized in that the method includes the steps of:

1.1, adjust the projector architecture optical sensor being positioned under water, ensure video camera can shooting clear image in measurement range and projector in measurement range, can project picture rich in detail, at projector with within the scope of the public view field of calibrating camera, adopt a flat board with waterproof target, projector projects another target to this flat board simultaneously, be called projection target, adjust flat position, until two target unique points are all positioned at public view field scope; Move freely at least three positions of target, position of every movement, takes the image pair that contains two targets;

1.2, extract the image coordinate of waterproof target unique point in image, according to the parameter of calibrating camera, ask for the normal vector of dull and stereotyped place plane under camera coordinate system;

1.3, extract the image coordinate that projects target unique point in image, utilize the normal vector of trying to achieve in step 1.2, by the image coordinate of camera parameters and unique point, calculate projection target and be incident upon corresponding point position in space on the flat board three-dimensional coordinate under camera coordinate system;

1.4, according to projecting target in image coordinate, and the three-dimensional coordinate of its corresponding spatial point of trying to achieve in step 1.3, the structural parameters between the preliminary labeling projection instrument parameter of single view model and projector and video camera utilized;

1.5, use the structural parameters that obtain in step 1.4, normal vector required in step 1.2 is illustrated under projector coordinates system;

1.6, under projector coordinates system, the distance initial value of given projector photocentre along optical axis direction to water-tight device window plane, ray model when setting up light that on projector image, each unique point the is sent air dielectric in water-tight device and propagating, is called incident ray;

1.7, the vector of unit length of setting up refracted ray, goes out scale factor according to Geometric Constraint Solving, determines refracted ray length, solves the three-dimensional coordinate of the lower projection target unique point of projector coordinates system;

1.8, the three-dimensional coordinate calculating in step 1.3 is utilized the structural parameters in step 1.4 be converted under projector coordinates system as reference value;

1.9, given projector optical axis is with respect to the initial value of drift angle and corner between window plane normal vector, set up objective function with the distance between the reference value described in the three-dimensional coordinate described in step 1.7 and 1.8, projector parameter and structural parameters are optimized, and carry out recursive optimization, the structural parameters of trying to achieve are for step 1.5～1.8, until target function value meets setting threshold, threshold value requires to set according to measuring accuracy, is generally 1E-5 to 0.1mm.

2. one according to claim 1 projector calibrating method under water, is characterized in that:

2.1, the waterproof target described in step 1.1 is the plane target drone that waterproof material is made, be fixed on a flat board, waterproof target thickness can be ignored compared with measuring distance, target unique point is the lattice point in plane, between lattice point, the design of distance is determined by actual measurement distance and camera field of view etc., is generally 10～100mm; Described projection target, the image coordinate distance between its lattice point is determined according to the resolution of projector image, is generally 10～100 pixels;

2.2, the waterproof target described in step 1.1 and projection target, video camera rationally make filtering device as the prerequisite of optical filter under, shared region can be overlapping on flat board for two targets; When flat board is during at same position, when video camera does not use filtering device and projector not to project target, video camera is taken and is had to waterproof target image, in the time that video camera uses filtering device and projector projects target, this target and waterproof target have overlapping region, and video camera is taken and had to projection target image;

2.3, the image pair that the shooting described in step 1.1 contains two targets, is characterized in that, in the time using filtering device, two width images are taken in each position, for containing respectively the image pair of waterproof target and projection target.