For centuries, optical imaging system design centered on exploiting the laws of the physics of light and materials (glass, plastic, reflective metal, ...) to form high-quality (sharp, high-contrast, undistorted, ...) images that "looked good." In the past several decades, the optical images produced by such systems have been ever more commonly sensed by digital detectors and the image imperfections corrected in software. The new era of electro-optical imaging offers a more fundamental revision to this paradigm, however: now the optics and image processing can be designed jointly to optimize an end-to-end digital merit function without regard to the traditional quality of the intermediate optical image. Many principles and guidelines from the optics-only era are counterproductive in the new era of electro-optical imaging and must be replaced by principles grounded on both the physics of photons and the information of bits. This short course will describe the theoretical and algorithmic foundations of new methods of jointly designing the optics and image processing of electro-optical imaging systems. The course will focus on the new concepts and approaches rather than commercial tools.

This course provides an overview of the motivation, theory and basic examples of joint design of optics and image processing in computational sensing and imaging systems in which image processing performs significant, non-trivial role in creating the final digital output. 1) The course will review the history of optical imaging and its four revolutions: 1) forming a real image with lenses and curved mirrors through basic physical optics, 2) fixing the image with silver halide through photography, 3) capturing the image using CCDs and CMOS image sensors through digital photography, and 4) deep integration of optics and signal processing through computational imaging, where an image or image estimate is not simply captured but instead computed. 2) Joint design of imaging systems based on traditional lenses and linear signal processing, including electro-optical compensation for manufacturing variations. 3) Lensless computational sensing and imaging based on diffraction (rather than refraction or reflection). 4) Joint design of application-specific sensors, in which the output is not a two-dimensional digital image but instead a numerical output or discrete decision based on the input scene.

This course is an introduction to the application of computer vision and image analysis to problems in art and art history, specifically realist art. Realist paintings are a rich source of information, both of the scene portrayed and the techniques the artist used to render that scene. Students will learn the principles of perspective and how to apply perspective analysis to paintings to infer vanishing points, locate perspective inconsistencies and to determine whether the artist used perspective constructions or tools. Students will learn how to infer the number, color, and position of light sources based on position, color and blur of cast shadows and highlights along occluding boundaries. Students will learn how to estimate sizes of depicted objects based on perspective and fiducial or reference objects or relationships. Students will learn how to estimate "camera parameters" of the artist (or imaging system), such as the effective magnification, focal length and in some cases aberrations. Some of these methods require no more than ruler and pencil, others require commercial software (e.g., Adobe Illustrator), others were adapted from their use in forensic analysis of digital photographs and require powerful commercial image processing packages (including ones based on C++, Matlab, Mathematica), and yet others require researchers to write special code. This course will be excellent introduction and background for research presented in symposium EI122, "Computer image analysis in the study of art."