Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

Claims

20 · 2 independent · depth 4

20 granted claims

›1.independentA method of synthesizing a higher resolution image from a plurality of images captured from different viewpoints, comprising:nnormalizing a set of ima…+18

›2.↳ 1The method of claim 1, wherein:nthe calibration data indicates mappings between addresses of physical pixels in imagers and logical addresses within a…d2

›3.↳ 1The method of claim 1, further comprising performing normalization of the color planes of the set of images using an upstream pipeline processing modu…d2

›4.↳ 1The method of claim 1, further comprising performing a process to adjust the set of images using an upstream pipeline processing module, where the pro…d2

›5.↳ 1The method of claim 1, further comprising performing temperature normalization using an upstream pipeline processing module based upon a temperature m…d2+1

›7.↳ 1The method of claim 1, wherein detecting and metering parallax comprises determining the parallax that yields the highest correlation between pixels f…d2+2

›10.↳ 1The method of claim 1, wherein aligning portions of images captured by different imagers to compensate for parallax using an image pixel correlation m…d2

›11.↳ 1The method of claim 1, wherein obtaining a higher resolution image having a resolution that is higher than the resolutions of the images in the set of…d2+3

›14.↳ 1The method of claim 1, wherein obtaining a higher resolution image having a resolution that is higher than the resolutions of the images in the set of…d2

›15.↳ 1The method of claim 1, further comprising generating a depth map using the parallax confirmation and measurement module.d2+1

›18.↳ 1The method of claim 1, further comprising:ncapturing the set of images using an array of imagers;wherein each imager includes a spectral filter select…d2

›19.↳ 1The method of claim 1, wherein the address conversion module, the parallax confirmation and measurement module, the image pixel correlation module, an…d2

›20.independentA method of synthesizing a higher resolution image from a plurality of images captured from different viewpoints, comprising:nnormalizing a set of ima…

A method of synthesizing a higher resolution image from a plurality of images captured from different viewpoints, comprising:nnormalizing a set of images using calibration data stored in a storage device using an address conversion module, where the set of images comprises a plurality of images that are:ncaptured from different viewpoints and include sampling diversity;include different occlusions sets;wherein the occlusion set of a first image is the portion of a scene visible in a second image that is occluded in the first image; and the set of images includes:images captured using the same imaging characteristics;images captured using different imaging characteristics;a first image that captures pixels around an edge of a foreground object that is visible in a baseline image and is in the occlusion set of a second image captured using the same imaging characteristics as the first imager;detecting and metering parallax using a parallax confirmation and measurement module, where detecting and metering parallax comprises ignoring pixels in the images that are in an exposed occlusion set;aligning portions of images captured by different imagers to compensate for parallax using an image pixel correlation module based upon the detected and metered parallax and the stored calibration data;obtaining a higher resolution image having a resolution that is higher than the resolutions of the images in the set of images using a super-resolution module, where color information around the edge of the foreground object that is visible in the baseline image and in the occlusion set of the second image is reconstructed in the higher resolution image using the pixels captured by the first image; andgenerating a depth map for the higher resolution image using the parallax confirmation and measurement module;wherein the higher resolution image is obtained by:nfusing the aligned image portions; andperforming super-resolution processing on the fused image portions to synthesize a super-resolution image.

Classifications

10 codes

IPC · International Patent Classification

Section H — Electricity

H04N5/357
H04N5/265
H04N9/083
H04N5/225
H04N5/262

USPC · US Patent Classification

348/218.1348/290348/294348/335348/340

Claim changes

Soon

Coming soonHow the claims changed between publication and grant

See which claims were amended, added or cancelled during examination, with every added and removed word marked.

AmendedAddedCancelledUnchanged

The published claims of this patent are not paired with the granted ones in what we hold.

File wrapper

Pendency

0.5 y

188 days filing → grant

Office actions

none on record

Responses

1 RCE

Examiner

Hung Lam

art unit 2661 · TC 2600

Law firm

Law firm · Log in to unlock

Citations: 474 back · 0 forward

See the full prosecution history — every USPTO and applicant action on this file, in order.

Term & fees

See the term timeline — pendency span, in-force span, the maintenance fees paid and both computed expiry dates.

Priority chain

2 priority documents

Priority

20 Nov 2009

earliest claimed

›Priority documents — 2

Type	Document	Date
provisional	US 61281662	20 Nov 2009
related publication	US 20140354853 A1	4 Dec 2014

Validity challenges

See the validity challenges on record — reexaminations, IPRs and PGRs, with their institution decisions and outcomes.

Citations

See every patent this one cites and every patent that cites it back — publication, assignee, and how each one was found.