We have developed a computer simulation to demonstrate the use of a periodic grating structure in the near field of a
telescope to super-resolve objects in the far field. The separation between the telescope pupil and the grating provides a
periodic anisoplanatism for the telescope, modulating the measured brightness of a point source as it moves across the
field normal to the grating. The resulting periodic modulation of an extended source can thus produce a spatial frequency
heterodyning effect, where frequency components outside the diffraction passband are aliased inside the passband and
measured. Using the simulation, we have quantitatively validated the analytically-predicted periodic blur function for the
case of single near-field grating. Further, we have shown the heterodyning effect is observed when the distance between
the grating and pupil corresponds to the Talbot distance, where the grating forms a "self-image" in the plane of the pupil.
By techniques described in this paper, high spatial frequencies in a scene that are beyond the diffraction limit of an
optical system can modulate user-generated low spatial frequency patterns prior to image formation and detection. The
resulting low spatial frequency modulations or "moiré patterns" lie within the optical pass-band and will therefore be
detectable. In favorable and controlled situations the scene's high spatial frequencies can be reconstructed from multiple
images containing these low-frequency modulations and a single super-resolved image is synthesized. This approach to
image super-resolution is feasible and does not violate well-established physical principles.
The paper describes two phases of this ongoing research. In phase one, we investigate active remote imaging methods in
which the low-frequency modulations are produced by controlling active illumination patterns projected onto the scene.
In phase two we investigate passive remote imaging methods in which diffracting structures are interposed between the
scene and the camera to modulate the light fields prior to image formation and detection.
A novel technique for imaging spectroscopy is introduced. The technique makes use of an optical imaging system with a segmented aperture and intensity detector array on the imaging plane. The point spread function (PSF) of such a system can be adjusted by modifying the path lengths from the subapertures to the image plane, and the shape of the resulting point spread function will vary as a function of wavenumber. An image reconstruction approach is taken to convert multiple recorded pan-chromatic images with different wavenumber-varying point spread functions into a hyperspectral data set. Thus, the technique described here is a new form of computed imaging.
KEYWORDS: Point spread functions, Lawrencium, Image sensors, Sensors, Imaging systems, Super resolution, Spatial frequencies, Optical transfer functions, Image acquisition, Signal to noise ratio
Optical imaging systems are often limited in resolution, not by the imaging optics, but by the light intensity sensors on the image formation plane. When the sensor size is larger than the optical spot size, the effect is to smooth the image with a rectangular convolving kernel with one sample at each non-overlapping kernel position, resulting in aliasing. In some such imaging systems, there is the possibility of collecting multiple images of the same scene. The process of reconstructing a de-aliased high-resolution image from multiple images of this kind is referred to as “super-resolution image reconstruction.” We apply the POCS method to this problem in the frequency domain. Generally, frequency domain methods have been used when component images were related by subpixel shifts only, because rotations of a sampled image do not correspond to a simple operation in the frequency domain. This algorithm is structured to accommodate rotations of the source relative to the imaging device, which we believe helps in producing a well-conditioned image synthesis problem. A finely sampled test image is repeatedly resampled to align with each observed image. Once aligned, the test and observed images are readily related in the frequency domain and a projection operation is defined.
We describe a geometric model of high-resolution radar (HRR), where objects being imaged by the sensor are assumed to consists of a collection of isotropic scattering centers distributed in three dimensions. Three, four, five and six point pure HRR invariant quantities for non-coplanar reflecting centers are presented. New work showing invariants combining HRR and SAR measurements are then presented. All these techniques require matching corresponding features in multiple HRR and/or SAR views. These features are represented using analytic scattering models. Multiple features within the same HRR resolution cell can be individually detected and separated using interference-suppression filters. These features can then be individually tracked to maintain correspondence as the object poise changes. We validate our HRR/SAR invariants using the XPATCH simulation system. Finally, a view-based method for 3D model reconstruction is developed and demonstrated.
KEYWORDS: Cameras, Imaging systems, 3D image processing, 3D modeling, 3D image reconstruction, Systems modeling, Visual process modeling, Chlorine, Distortion, Machine vision
Suppose we have two or more images of a 3D scene. From these views alone, we would like to infer the (x,y,z) coordinates of the object-points in the scene (to reconstruct the scene). The most general standard methods require either prior knowledge of the camera models (intersection methods) or prior knowledge of the (x,y,z) coordinates of some of the object points, from which the camera models can be inferred (resection, followed by intersection). When neither alternative is available, a special technique called relative orientation enables a scale model of a scene to be reconstructed from two images, but only when the internal parameters of both cameras are identical. In this paper, we discuss alternatives to relative orientation that does not require knowledge of the internal parameters of the imaging systems. These techniques, which we call view- based relative reconstruction, determine the object-space coordinates up to a 3D projective transformation. The reconstructed points are then exemplars of a projective orbit of representations that are chosen to reside in a particular representation called a canonical frame. Two strategies will be described to choose this canonical frame: (1) projectively simplify the object model and the imaging equations; and (2) projectively simplify the camera model and the imaging equations. In each case, we solve the resulting simplified system of imaging equations to retrieve exemplar points. Both strategies are successful in synthetic imagery, but may be differently suited to various real-world applications.
We will demonstrate for central-projection imaging systems a natural progression of cross-ratio invariant theorems extending from one through three dimensions. In each dimensions there is an invariant quantitative relationship between combinations of geometric entities in image space, and combinations of corresponding geometric entities in object space. In one dimension, when the object points and image points are co-linear, these entities are line segments formed by corresponding paris of object and image points. The 'mother of all invariants' is the invariant relationship between cross-ratios of products of the lengths of these corresponding line segments in object and image. In two dimensions these geometric entities are triangles formed by corresponding triplets of points in the object and in the image. There exists an invariant relationship between cross- ratios of products of areas of these corresponding triangles in object and image. The one- and two-dimensional results are well known. Not so well-known is the fact that for the case of multiple images of 3D scenes and objects the geometric entities are triangles and tetrahedra, and that there exist invariant linear relationships between cross- ratios of products of the areas of image-triangles and volumes of object-tetrahedra. One objective of our paper is to demonstrate that these linear relationships are established by a uniform pattern of algebraic arguments that extends the cross-ratio invariants in a natural progression from lower to higher dimensions. A second objective is to demonstrate that the resulting cross-ratio invariants can be interpreted as metric properties of geometric entities. A third objective is to demonstrate that these cross-ratios of points in the images, which we can observe directly, are equal to the corresponding cross-ratios of points in the objects, which may not be directly accessible. We will use computer simulations to validate the algebraic results we derive in this paper, and 3D graphics to visualize them.
The reference data consists of two or more central- projection images of a 3D distribution of object points, i.e., a 3D scene. The positions and orientations of the cameras which generated the reference images are unknown, as are the coordinates of all the object points. We derive and demonstrate invariant methods for synthesizing nadir and perspective views of the object points, e.g., 2D maps of the 3D scene. The techniques we will demonstrate depart from standard methods of resection and intersection which first recover the camera geometry, then reconstruct object points from the reference images, and finally back-project to create the nadir or perspective views. The first steps in our 'invariant methods' approach are to perform the image measurements and computations required to estimate the image invariant relationships linking the reference images to one another and to the new nadir and perspective views. The empirically estimated invariant relationships can thereafter be use to transfer conjugate points and lines from the reference images to their synthesized conjugates in the nadir and perspective views. Computation of the object model...the digital elevation model...is not required in this approach. In this paper we developed algorithms for invariant transfer of conjugate lines which exploit the synergy of line and point transfer. We validate our algorithms with synthetic CAD models. A subsequent paper will validate the line transfer algorithms with uncontrolled aerial imagery and maps with occasional missing or inaccurately delineated features.
The reference data consists of two or more central- projection images of a three-dimensional distribution of object points, i.e., a 3D scene. The positions and orientations of the cameras which generated the reference images are unknown, as are the coordinates of all the object points. We derive and demonstrate invariant methods for synthesizing nadir views of the object points, i.e., 2D maps of the 3D scene. The techniques we demonstrate depart from standard methods of resection and intersection to recover the camera geometry and reconstruct object points from the reference images, followed by back-projection to create the nadir view. Our approach will be to perform the image measurements and computations required to estimate the image invariant relationships linking the reference images to one another and to the nadir view. The empirically estimated invariant relationships can thereafter be used to transfer conjugate points from the reference images to their synthesized conjugates in the nadir view. Computation of the object model -- the digital elevation model (DEM) -- is not required in this approach. The method also differs from interpolation in that the 3D structure of the scene is preserved, including the effects of partial occlusion. Algorithms are validated, initially with synthetic CAD models and subsequently with real data consisting of uncontrolled aerial imagery and maps with occasional missing or inaccurately delineated features.
We introduce non-standard methods of deriving algebraic invariants and demonstrated two types of applications of these invariants. In model transfer a collection of conjugate points are determined on a set of reference images, and `transferred' to the matching conjugate points on a new view of the 3D object, without prior computation of camera geometry or scene reconstruction. In object reconstruction, general 3D object points are represented as functions of non-coplanar fiducial points and corresponding conjugate points across multiple images. In this application the object points are `reconstructed' once quantitative values are specified for the fiducial points. The methods we introduce for deriving these invariant algorithms are extensible from the linear fractional central projection camera model to weak perspective and certain non-central projection camera models. Stability to adverse geometries and measurement error can be enhanced by using redundant fiducial points and images to determine the transfer and reconstruction functions. Extensibility and stability are indications of the robustness of these methods.
KEYWORDS: Cameras, 3D modeling, Reconstruction algorithms, Visual process modeling, Imaging systems, Model-based design, Machine vision, Computer simulations, Monte Carlo methods, 3D image processing
Invariant methods for object representation and model matching develop relationships among object and image features that are independent of the quantitative values of the camera parameters of object orientation, hence the term invariant. Three-dimensional models of objects of scenes can be reconstructed and transferred to new images, given a minimum of two reference images and a sufficient number of corresponding points in the images. By using multiple reference images, redundancy can be exploited to increase robustness of the procedure to pixel measurement errors and systematic errors (i.e., discrepancies in the camera model). We present a general method for deriving invariant relationships based on two or more images. Simulations of model transfer and reconstruction demonstrate the positive effect of additional reference images on the robustness of invariant procedures. Pixel measurement error is simulated by adding random noise to coordinate values of the features in the reference images.
Invariant relationships have been derived from the mathematical models of image formation for several types of sensors; from the collinearity equations of pinhole camera systems and separately, from the condition equations of strip-mapped SAR. In the present paper, we extend these results by combining the collinearity and condition equations of photographic and SAR systems. The resulting invariants enable us to transfer points and three-dimensional models from multiple photographic to SAR images and vice-versa. Geometric integrity of the different imaging systems is preserved by the technique. The method will facilitate synergistic, model- based interpretation of different sensor types.
We describe in this paper several geometry problems in photogrammetry and machine vision; the geometric methods of projective invariants which we apply to these problems; and some new results and current areas of investigation involving geometric invariants for object structures and non-pinhole-camera imaging systems.
Most current object recognition systems are based on a 3D model which is used to describe the image projection of an object over all viewpoints. We introduce a new technique which can predict the geometry of an object under projective transformation. The object geometry is represented by a set of corresponding features taken from two views. The projected geometry can be constructed in any third view, using a viewpoint invariant derived from the correspondences.
For several useful tasks in photogrammetry and in model-based vision, this paper develops noniterative methods that require only the inversion of systems of linear equations. The methods are based on the theory of projective invariants. The following tasks are addressed: (a) Resection, or determination of parameters of acquisition geometry (requires six control points); (b) Intersection, or determination of the position of an object point from several images; and (c) Transfer, or model matching, which uses the image coordinates of a ground point in two images to predict the coordinates of that point in a third image in the presence of several other tie points in the three images.
Image understanding is a cross-disciplinary field, drawing on concepts and algorithms from image processing, pattern recognition, and artificial intelligence. An integrated system for image understanding may require a variety of capabilities that appear quite disparate, such as image restoration to compensate for degradations detected in the data, followed by logical inference to interpret features extracted from the restored data. The authors establish that constrained optimization provides a uniform formulation for two such apparently disparate problems: restoration of blurred imagery, and logical deduction or mechanized inference. Formulation of these problems in each of these categories as linear programming (LP) problems is shown. The 'deblurred' image is regained by minimizing a linear objective function subject to the constraints imposed by the blur. The degree of truth or falsity of a consequent proposition is established by maximizing a linear objective function subject to the constraints imposed by the premises.
As part of on-going studies of automated techniques for object recognition in imagery, recent experiments in two and three
dimensions have produced promising results. Newly developed methods that exploit projectively invariant relationships in
imagery are able to recognize the same object in images that differ in tilt, scale and rotation. Automatically extracted corner
points are used as the base features in simple two-dimensional objects, and patches of known gray-value are used in threedimensional
terrain perspective views. In both cases, projective invariants are calculated and compared with a catalog of
archetypal values, resulting in successful identification of the objects within experimental error.
The objective of this work is to develop automated techniques for recognizing the same objects in images that differ in scale tilt and rotation. Such perspective transformations of images are produced when aerial images of the same scene are taken from different vantage points. In previously reported work we have identified methods for deriving algebraic projective invariants under central projections. These methods generalize the familiar cross-ratio theorems for single images of finite sets of points on the line and in the plane. The algebraic methods do not utilize the intensity values of the images. Since image features essential for object recognition may be described in terms of derivatives and integrals of the image intensity it is necessary to investigate whether certain differential and integral operators applied to different perspective views of the same object are also invariant under the perspective transformation. We proceed to derive new differential operators and their corresponding integral invariants for curves and planar objects. Extensions to other image formation models such as synthetic aperture radar (SAR) are discussed. These results are steps toward a computational model for perspective-independent object recognition.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.