(Figure 1) This is an outline of my presentation today. The U. S. Army has a dual interest in the use of robots, namely: 1. As a substitute for or an extension of the soldier in the battlefield, and 2. in the factories that make Army materiel, or - as we call it -the the production base. The Production Base can again be divided into three separate segments, i.e., the Army owned and operated facilities or GOG6s, such as Rock Island and Watervliet arsenals, and not to be overlooked, the depot operations. There the Army manufactures gun tubes and other related parts for artillery weapons and repairs and overhauls them. A second category is the Army owned and contractor operated facilities or GOCOs,such as the ammunition plants, the tank plants at Lima, Ohio and Warren, Michigan and the Stratford Engine Plant in Connecticut where gas turbines for helicopter and the Abrams tank are manufactured. The last category covers the industrial base, that is those factories which are not only operated but also owned by the contractor himself also referred to as COCOs. You can see from this description that the Army is supported by a base which produces a diversified line of products. Therefore, the task of technology development and technology insertion is considerably more complex than what one encounters in the average U. S. Manufacturing organization.

For robots to operate effectively in the partially unconstrained environment of manufacturing, they must be equipped with control systems that have sensory capabilities. This paper describes a control system that consists of three parallel cross-coupled hierarchies. First is a control hierarchy which decomposes high level tasks into primitive actions. Second is a sensory processing hierarchy that analyses data from the environment. Third is a world model hierarchy which generates expectations. These are compared against the sensory data at each level of the sensory processing hierarchy. Deviations between expected and observed data are used by the the control hierarchy to modify its task decomposition strategies so as to generate sensory-interactive goal-directed behavior. This system has been implemented on a research robot, using a network of microcomputers and a real-time vision system mounted on the robot wrist.

Increased automation is necessary in future NASA missions. Drivers for automation include constrained funding and physical resources as well as mission capabilities not achievable through conventional means. The application of emerging technology in manipulators and machine intelligence will enable the development of robotic devices remotely commanded by human operators to increase man's productivity in space. The Office of Aeronautics and Space Technology (OAST) has established a program for research in teleoperation and robotics. The program's near-term focus is a Remote Orbital Servicing System (ROSS). The longer range goals include: (1) basic research in autonomous operations, (2) human factors research on the man-machine interface to remote systems, and (3) system integration and analysis of advanced concepts. This paper reviews the current NASA research and technology and considers future work needed to meet the OAST goals.

The Defense Advanced Research Projects Agency (DARPA) is supporting the development of tomorrow's more productive manufacturing processes for the military hardware that will be required at the end of the century, and is establishing technological advances that will support extension and enhancement of military operational capabilities in the combat environment. This paper describes the thrust that has been initiated by DARPA in Intelligent Task Automation (ITA) -- a broad based activity intended to lay groundwork for future developments. Integration of the necessary intelligence for dealing with sophisticated tasks in unstructured environments is specifically addressed. The implied emphasis is on linking computation for an understanding of uncertain environments to mechanical functions.

A historical summary of the Army activities in artificial intelligence and robotics (AI/R) in the last one and one-half years indicates Army interest in AI/R from the laboratories to the Secretariat. Numerous funded and unfunded efforts are now planned by the laboratories even though AI/R technologies lack maturity necessary for autonomous battlefield systems in the 1990's. The potential applications of AI/R systems to Army needs appear to be limitless. DARCOM and TRADOC have prioritized AI/R requirements and plans for five high priority Demonstrators have been prepared. These demonstrators reflect the need to get started with todays technologies. Technological enhancements provided by additional research and development can provide additional autonomy in product improvement phases of the system development cycle.

One hundred applications of artificial-intelligence technology and robotics in Army combat and combat support that may be possible and worthwhile are identified. These possible applications have been divided into ten categories, and one example in each category has been examined in detail. Research and development plans have been developed showing the basic and applied research that would be needed to make the applications possible. Although the number of possible applications is large, the number of key technology elements is relatively small, and many of the same technology elements are required in many different applications.

Robotics technology can be applied to numerous areas of naval operations. Three general areas of applications are discussed in this paper: (1) production, (2) support and (3) operations. Production applications involve manufacturing naval material. Shipbuilding robots are the primary production applications currently of interest to the Navy. Support activities ensure the availability of both expendable and permanent equipment resources at the field units. The most near term support applications of robots are maintenance and repair applications. Operations applications use robots as active parts of the battle process. Robots for naval battlefield operations which could perform such tasks as surveillance, intelligence collection, communications and C3 countermeasures are well within the capabilities of existing technology. This area of Navy applications requires the most advanced technology of all the other areas. The Navy robots which will find the quickest and widest implementations will be those for which there is substantial industrial experience and commercial availability.

Autonomous robots for the battlefield offer a tantalizing alternative to manned warfare. These systems rely upon many well established technological areas. Near term applications of autonomous battlefield robots (ABRs) are possible in such mission areas as surveillance and communications support. Such more advanced applications as weapons and countermeasures control await at least the development of fault tolerant information processing resources. Several open research issues still limit the current realizable performance of ARBs. In spite of these limitations, ABRs should be implemented for near term applications to gain the systems experience necessary to construct more capable ABR systems.

A definition of scale-specific structure follows from the work of Marr and Poggio on zero-crossings in gaussian-bandpass-filtered images. This paper presents several examples which illustrate the potential value of scale-specific structure for recognition, inspection, and correspondence related tasks. The suitability of the approach for practical systems is discussed, and a recent result suggesting the importance of this "hidden" information to human stereo vision is presented.

This paper presents a new method for pattern recognition of 3-dimensional objects. Three dimensional objects are recognized by means of a moire contour difference technique. The technique utilizes the projection of a grating upon the object. The grating is perturbed according to the topography of the object surface; thus the perturbed grating acts as information carrier about the surface topography. The image of a master part which has been illuminated with a projected grating is stored in a negative photographic transparency, and it is used as a transmission filter when inspecting other parts. The transmitted image contains the correlation information of the part under inspection with the master part. If the two parts are identical, an autocorrelation occurs which results in a minimum level of light transmitted. If the inspected part is different from the reference part, a cross-correlation occurs which results in the presence of moire fringes and causes more light to be transmitted. Missing parts are detected by identifying the presence of moire fringes or simply detection of the level of the transmitted light and decision can be made automatically. The feasibility of the method was verified by a laboratory demonstration.

Scale-invariant transforms are those that leave the form of the image representation unchanged over changes in image size. These transforms have the important advantages of simplifying pattern recognition tasks when the distance to the objects is variable, while simultaneously reducing the amount of data that must be processed. The properties of scale-invariant systems are investigated here in the spatial-frequency domain, using an image representation called a "scaled transform". A discrete version of the transform is developed and its properties contrasted with those of the Fourier series. It is shown that unlike the Fourier series representation where the magnitude of the frequency vectors, k, occur at fixed frequency intervals ▵k, the scaled-transform frequencies occur at fixed fractional intervals ▵k/k.

Lens focusing using a hardware model of a retina (Reticon RL256 light sensitive array) with a low cost processor (8085 with 512 bytes of ROM and 512 bytes of RAM) was built. This system was developed and tested on a variety of visual stimuli to demonstrate that: a)an algorithm which moves a lens to maximize the sum of the difference of light level on adjacent light sensors will converge to best focus in all but contrived situations. This is a simpler algorithm than any previously suggested; b) it is feasible to use unmodified video sensor arrays with in-expensive processors to aid video camera use. In the future, software could be developed to extend the processor's usefulness, possibly to track an actor by panning and zooming to give a earners operator increased ease of framing; c) lateral inhibition is an adequate basis for determining best focus. This supports a simple anatomically motivated model of how our brain focuses our eyes.

The computation of a two dimensional Walsh-Hadamard Transform using a two dimensional perfect shuffle is described. A simple, repetitive computational unit for parallel processing, each of which performs a local Walsh-Hadamard Transform, is then combined with an interconnection network which performs a two dimensional perfect shuffle of an N x N array. After log₂N repetitions, the N xN Walsh-Hadamard Transform is obtained.

A moment-based pattern recognition system is most appropriate for providing information on the presence, classification and orientation of an object in the field-of-view of a sensor. Such features are ideal for robotic pattern recognition and parts handling. This paper reviews our hybrid optical/digital system to achieve the above objectives and includes initial demonstration results. We then advance a new optical finite-order cosine-processor that can compute the moments and we discuss how the outputs from this system can easily be corrected for various optical system errors and the design of a reduced size breadboard optical system.

Multi-resolution pyramid structures may be used to compute image properties efficiently and within sets of Gaussian-like windows of many sizes. We define several basic pyramid algorithms which may be applied to a variety of image understanding tasks. These include a multi-resolution low-pass filter (the Gaussian pyramid), a multi-resolution band-pass filter (the Laplacian pyramid) and a multiscale window function (the Hierarchical Discrete Correl-ation). To illustrate these algorithms we present procedures for computing local edge density and spectral energy estimates for texture discrimination, and procedures for computing local correlation and gradient based estimates of pattern displacement for motion analysis. Pyramid computations are shown to be particularly low in both cost and complexity.

The piecewise gradient image modeling algorithm provides a multi-level relational graph representation of an image where the nodes describe regions of constant intensity gradient and the arcs provide structural information about the image. A set of statistical attributes is associated with each node to further describe the image characteristics. The paper describes the algorithm's six major steps, presents a set of examples, and discusses its, usefulness in two major vision problems: classification and interpretation of scenes with 3-D objects.

A new method for selecting features for object recognition based on training data is proposed. This method avoids overspecifying or selecting too many features by using the criterion of minimal representation, which penalizes the representation complexity of features. The presented approach can be used to search for high level structural features such as relations or production rules.

The synthetic discriminant function concept together with its modifications of maximum common information filters and decorrelation transformations are reviewed. We then advance a unified procedure for determining the coefficients for such linear combination filters for recognition of objects in different orientations and from different aspect views. Our formulation utilizes only deterministic techniques and a correlation matrix observation space. This formulation is most attractive for the realization of shift-invariant filters for use in correlator architectures. We then advance the highlights of our initial results on the performance of this new type of generalized shift-invariant filter.

In a recent paper [Lowry 81], we described an architecture for a computer vision rectangular processor array that is suitable for VLSI implementation. In this paper we will review that architecture and discuss extensions to it and present results of an array simulator applied to vision algorithms. We will also present an algorithm for re-routing an array with bad processors into a working subset of the array, making it feasible to implement a large array on one wafer-sized chip.

The advance of semiconductor technology has made monolithic implementations of commonly used imaging system functions feasible and affordable. This is particularly true of high-throughput functions such as low-pass filtering, edge detection, and automatic gain control that are typically applied to an entire image to enhance the imaging sensor output for display or further processing. This paper describes the integrated-circuit implementation of three important image-processing functions: a general-purpose programmable neighborhood operator, a Sobel edge-detection circuit, and a histogram-modification circuit.

Visual servo control offers advantages for flexibility and interactive performance of robotic systems. Design of such systems requires formal analysis of both the image processing requirements and the control strategy for multi-degree-of-freedom systems. Image features may be extracted and interpreted to provide estimates of relative camera position in three-dimensional space. Image features in themselves often maintain continuity during sequences of time-varying images, and that continuity provides a basis for use of features as control variables. Analysis and simulation of model reference adaptive control structures have shown that with proper attention to regimes of operation and priority among features, image-based control of multi-degree-of-freedom systems may provide improved dynamic performance due to reduction in image interpretation computational load and estimation noise.

This report presents a procedure for processing real world image sequences produced by relative translational motion between a sensor and environmental objects. In this procedure, the determination of the direction of sensor translation is effectively combined with the determination of the displacements of image features and environmental depth. It requires no restrictions on the direction of motion, nor the location and shape of environmental objects. Extensions to other cases of motion analysis are considered.

This paper presents a bionic approach to pattern classification entitled Neural Analog Processing (NAP). NAP systems are based upon information processing principles discovered by neural modelers, but are not themselves neural models. To set the stage for a discussion of how NAP systems work, the theory of a particular type of local-in-time template-matching classifier -- the Generalized Nearest Neighbor (GNN) classifier -- for general time-varying patterns (imagery, spectra, tactile signals, etc.) is reviewed. The definition and function of the fundamental NAP structure -- the slab -- is then presented and it is shown that a GNN classifier can, in principle, be implemented using slabs. The embellishments necessary to allow NAP systems to be realized in hardware are then described. Finally, a summary of NAP system characteristics is presented.

The use Fourier descriptors and two-dimensional moments for rapid recognition of shapes is discussed. Both algorithms and architectures for these methods are considered. Some example recognition experiments are shown including aircraft identification and industrial parts inspection.

We discuss the time-varying imagery analysis problem, contrasting low level and high level feature correspondence algorithms. Low level techniques are shown to have lower sensitivity to signal-to-noise level changes. High level techniques are shown to be more computationally attractive for complex scenes because of reduced feature data set sizes and heuristic matching methods. We go on to describe our high level motion and depth analysis system based on graphs composed of curvature segments called half chunks.

The authors have previously reported on the absolute distance measuring systems being developed at Lockheed Missiles and Space Company for use in space on large erectable satel-lites. One, a two-color CO₂ laser system, has a precision of 0.03 μm (RMS) for aligning optical elements. The second, a HeNe phase modulated system, has a precision of 25 μm for the measurement and control of large antennas. Also, previously reported, a multi-channel relative distance measurement system originally conceived as a vibration sensor for the active (dynamic) control of large space structures. The application of these measuring systems to the positioning and control of robots is discussed. With a single laser measurement system, the control of several robotic stations may be handled and real-time position-ing of both robot arms and workpiece fixtures in three dimensions, with great precision, is possible. Through computer control, the system would lend itself to batch processing with a variety of functions underway at the different work stations. By controlling the position of the tool, minimum precision is required for the robot.

This paper examines some computational issues in robotics. The concept of a robot-based manufacturing cell is outlined by viewing its behavior at three levels: machine level, cell integration level, and integration with higher level functions. Computational requirements associated with the first two of these three levels are discussed, and the need for very high computation rates at the machine level of a manufacturing cell, particularly for the real-time control of a robot arm and for robot vision is pointed out. The paper also explains the need to be able to manage multiple processes at the cell integration level. A unified solution to these computational needs which requires that the computation structure used to control and manage a manufacturing cell be designed as an object-based computer architecture is then proposed. The hardware /software boundary in the implementation of the objects comprising the overall architecture is transparent from a logical viewpoint but not from a timing viewpoint. Real-time constraints are met by implementing time-critical process objects directly in hardware. This "object level" design provides the hardware /software transparency necessary to formulate a unified system specification for these real-time embedded computer systems. Timing considerations can then be used determine the hardware /software boundary.

The priority associated with U.S. efforts to increase productivity has led to, among other things, the development of Electronic Vision Systems for use in manufacturing automation requirements. Many such systems combine closed circuit television cameras and data processing equipment to facilitate high speed, on-line inspection and real time dimensional measurement of parts and assemblies. These parts are often randomly oriented and spaced on a conveyor belt under continuous motion. Television imagery of high speed events has historically been achieved by use of pulsed (strobe) illumination or high speed shutter techniques synchronized with the camera's vertical blanking to separate write and read cycle operation. Lack of synchronization between part position and camera scanning in most on-line applications precludes use of this technique and dictates that another approach be utilized. Unique inject-inhibit capability of General Electric Company's Charge Injection Device (CID) imager will, when interfaced appropriately with Xenon strobe illumination, provide asynchronous stop motion imagery and facilitate dimensional measurement at rates of up to 900 parts per minute. OPTOMATIONIM Electronic Vision Systems incorporating this capa-bility have been operating in factory applications for now, in excess of three years. Operation of the CID Solid State imager will be reviewed in this context along with the system equipment, and possible future developments will be discussed.

Optical techniques have been a key ingredient in greatly improving the quality control and product assurance of solar panel manufacturing. In fact, major breakthroughs in this area have been in progress over the last two years. These techniques involve electro-optical devices used for on line sensing and, in some cases, controlling production. Optical sensor displays assist the operator and inspector to assure maximum quality control. The sensors output is simultaneously, along with other pertinent data, recorded for documentation and stored for future reference. In this age of very large, lightweight, folding solar arrays, the individual solar cell bonding to the circuit is most critical. The bond must be strong mechanically, good electrically and introduce no undesirable side effects such as puncturing the junction, cracking the cell, or melting the circuit material. One related problem is mislocating the bond such that edge effects or unwanted insulation material interfere with the bonding, reducing the strength and size of the bond. This alignment problem was solved by using a high resolution, high contrast color TV camera with high contrast capability allowed detection of the low contrast insulation material. Color effects immediately after bonding, giving the inspector, bond quality and bond shape data, as well as revealing any circuit melting. Since bonding takes only a few milliseconds and there can be more than 20 variables involved, a sensor controlled bonder was required. This was solved by means of an electro-optical bond temperature sensor that automatically controlled the bonder to a preset bond temperature. Another inspection technique utilized was an electro-optical sensor which consisted of a custom designed videoized near infrared microscope that permitted crack inspection after bonding and gives infrared bond footprints. By inserting crossed polarizers, inspection of the residual stress patterns left in the silicon was possible. In addition, a robotic solar cell bubble inspection concept is also presented in the last part of this paper.

The Normal Incidence Spectrophotometer (NIS) Measurement Tool is used to automatically measure the thickness of transparent films on silicon wafers. Under the control of an IBM. System 7 computer, both, wafer handling and the thickness measurement are performed. automatically. Wafers are transported through the tool on a covered airtrack, and after posi-tioning in a vacuum, chuck, are moved under the measurement head to pre-programmed measurement sites. Reflectivity data from the wafer surface, as a function of wavelength, is used by a software algorithm to calculate film, thickness. This tool is used on advanced manufacturing lines at IBM in both. East Fishkill, N.Y. and Burlington, Vt.

Many products in the food and drug industry are sold in heat-sealed translucent film packages. Automatic inspection of these packages before shipment to the customer is an important step in assuring quality. This paper describes a laser scanner and associated electro-optical and electronic system for inspecting blister packages which also serve as reaction vessels, identifying and classifying defects for process control in an on-line situation. This system is a practical application of a coherent light scanner which utilizes spatial filtering and a transform plane array of optical sensors for performing some of the signal processing necessary for defect detection and classification. Automatic registration in 2 dimensions is incorporated, thus relaxing the positional accuracy requirements of the product handling system.

The objective of this paper is to describe the Naval Surface Weapons Center (NSWC) Program for developing high performance, simple, rugged, cost effective magnetoelastic force feedback sensors for robots and machine tools. Recent advances in magnetoelastic materials technologies have paved the way for corresponding improvements in the state-of-the-art in force feedback sensors for robots and machine tools. Also, NSWC has designed magnetic circuits which are easily adapted to force feedback sensors. In this paper, magnetoelastic materials are described along with the properties that make them potentially such outstanding force feedback sensors. Following this, the Naval Surface Weapons Center Program is detailed including advances in materials research, in simple, low cost electronic and magnetic circuits, and designs for force feedback sensor modules. The results are in the public domain.

A compact, portable dual manipulator robot has been constructed to solve Rubik's Cube^TM. The robot system reads the color pattern from the scrambled cube faces, computes a solution, and executes the manipulations required to return each cube face to a uniform color. The vision, mechanical manipulator, and microprocessor control subsystems will be discussed in conjunction with the solution algorithm.

There are a number of Robot vision/image processing systems on the market at the moment. Basically these split into two categories, either they aim at a totalhardware/software system with direct robot interaction to be used in a wide variety of environments, or they consist of a software package to perform some of the more commonly used image processing functions. Neither of these two solutions seems to work particularly effectively in practice being, on the one hand too general and intolerant of working conditions, and on the other too unrelated to robotic inspection and assembly. To develop a more flexible and versatile vision system, a number of industrial problems will be studied, with the aim of finding the differences and similarities between them. Eventually a library of routines will be built up to cope with the different parts encountered. Presented here is one such study into the assembly of a power diode unit, comprising mainly of circular components.

A general purpose image analyzer can be programmed to read 5 to 10 strings of alphanumeric characters a second by proper restrictions of information content, and by doing some of the processing in fast outboard hardware.

This paper demonstrates an example of utilization of machine intelligence in automation. A system was developed to perform precision part inspection and automated workpiece handling. This system consists of a robot which is utilized to perform simple pick-and-place function, a MI VS-110 machine vision system which provides a vision library and the functions of masking and programmable image overlay, and an X-Y-θ table. This setup demonstrates a simplified approach to machine vision and automation. In this complete sensorimotor system, BASIC was the programming language to develop and integrate the control software for the inspection process by using the MI DS-100 machine vision development system. By calling vision functions, X-Y-θ table commands and simple robot commands, the task of parts" inspection under high- and low-resolution cameras, sorting, as well as disposition is shown to be easy to conceptualize and implement. This robot system can perform tasks without the necessity of prealigning or jigging workpieces. Numerous other applications can be accomplished by adopting a similar methodology.

CONSIGHT is a computer vision based system developed by General Motors which is able to locate and identify parts on moving conveyor belts. Two different factory applications are presented which illustrate CONSIGHT's capabilities and which bring out some of the issues of developing production systems. The first system discussed employs a single vision station and a series of pneumatic kicker devices to sort passing parts into one of 16 bins. The second system employs a single vision station and multiple robot stations to load large castings from a conveyor into shipping containers.

The defect and measurement evaluation system represents a successful computer-microscope partnership that has enhanced the visual inspection of LSI wafers during the manufacturing process. Fabrication and process control of small geometry, semiconductor devices demand high quality visual inspection systems and real-time analysis of pattern width measurements, defects and surface characteristics. The defect and measurement evaluation system tool has coupled a high quality, wide-field microscope system to an IBM 5100 computer. This provides an integrated tool controller, real-time data collection and analysis system. With the inspector in mind, the design and packaging of the tool was human-factor-engineered for ease of operation and minimum fatigue.

In this paper, we propose the use of an incoherent optical image subtraction technique for automatic micro-circuit board inspection. We believe that this proposed technique would have profound effect on automatic inspection scheme in the application of faulty detection and identification of micro-circuitries. The technique would provide the capability of rapid identification, inspection, and possibly utilizing for synthesis and fabrication. Experimental simulation of the IC chip inspection is provided.

The matching of synthetically generated images of known solid objects with "real world" scenes is of fundamental importance in advanced robot vision systems. An efficient "hierarchical convolution" algorithm is presented. A scene contained in a quadtree is convolved with a matched filter quadtree derived from a synthetic image. During algorithm operation, a series of filtered images is generated at increasingly finer resolution. This allows for strategies based on coarse-to-fine matching. The algorithm is derived for both binary and gray scale images. In the gray scale case, the new algorithm is shown to require one-third fewer multiplications than conventional cross correlation.

We describe an apparatus for obtaining three-dimensional surface information that may be used to recognize objects and determine their position and orientation. A lightweight camera and a light-stripe projector using an infrared laser diode are mounted in the hand of a robot manipulator. Image-processing routines locate the stripe in the camera image, and homogeneous coordinate transform techniques are then applied to solve for the three-dimensional coordinates of points illuminated by the stripe. We describe the hardware, the equations for coordinate measurement and the procedures for accurately calibrating the apparatus.

We consider the problem of reconstructing a function from a finite collection of its line integrals. We discretize the function on a logarithmic polar grid. Assuming a special data collection scheme, we derive a large system of linear equations in the unknown discretized function values. This system has a very nice structure, which is used to decompose it into a number of reasonably small systems. After regularization, they can be solved using standard direct methods of numerical linear algebra. Results of experiments are shown.

Correlation techniques for obtaining depth profiles from stereo pairs have been studied extensively yet continue to have problems with generality and computational efficiency. A new method is presented which develops the camera model constraints required to ensure that binocular disparity shifts only shift in the x direction. An optimal mapping algorithm is then applied which gives a globally optimal depth profile for a given line of the stereo pair. The algorithm performs well in cases which are difficult for correlation based algorithms namely, repetitive patterns, depth discontinuties and large, relatively featureless regions. With currently available IC chips this algorithm could be implemented in hardware to achieve video rate performance.

Moire is a relatively old technique used for years in the field of non-destructive testing. A system is described here using the Moire fringe projection technique for 3-D volume and geometry characterization. A special purpose software package has been written to receive these measured data in a three-dimensional data bore.