"One has to be an expert in the Sigma-delta ADC and signal processing theory to be able to understand the benefits and limitations of this method when applied to the imaging. Yes, the sigma-delta approach can be easily modified to be able to capture low-lights. Also, it can be easily modified to remove reset noise, threshold offsets and any pixel or column-level FPN.
Here are some measurement results:
Input referred read noise at 30fps= 37.8microV (nothing like this has been reported previously). This easily translates into <1e- read noise
Intra-scene linear dynamic range = 91dB
FPN less than 0.5%
Power at 30fps 10 x lower than APS and other digital approaches
Allows real time low power compression (inluding JPEG and JPEG 2000), motion blur correction, white balancing, ....
The only drawback is increased memory size required for the Sigma-delta approach."
As a matter of fact, the previous part DPReview discussion has been deleted from the forums there, but saved as a copy here, here and here.
Update: Another Jake Ignjatovic post in DPReview forum adds more statements:
"To stop further speculations on the Sigma-delta imager:
- there are only 3 minimum size transistors per pixel - fill factor is as high if not higher than in the APS.
- the design scales well with the technology since it relays on transistor speed and not the precision - another unique feature.
- the design is not affected with power supply voltage reduction in more advanced CMOS technologies
- and finally, yes we can and we will be utilizing PIN diodes.
At least some of these new statements are questionable. For example, many modern APS with 4T shared pixel have 1.75 transistor per pixel or even 1.5T per pixel. 3T per pixel is not very competitive for small pixels.
Update #2: On the product schedule Jake Ignjatovitc says:
"ADVIS, Inc has plans to release its first commercial HD sensor based on the Sigma-delta technology in Q2 2010."
On the possibility to see the results in the paper Jake says:
"The "real" paper is coming out soon. We just have to make sure that certain people interested in slowing down the imaging science progress are not the reviewers."
Update #3: Below is ADVIS sigma-delta pixel noise spectrum that Jake referred in the comments to this post:
Jake posted again to say "ADVIS, Inc has plans to release its first commercial HD sensor based on the Sigma-delta technology in Q2 2010."
ReplyDeleteI like EF's post: "If it is real, great!"
Where is EF's post? This guy Jake is clueless. He says: "yes we can and we will be utilizing PIN diodes." He doesn't know what he is taking about. How is a PIN diode useful here? A PIN diode is P-I-N, where the I is intrinsic. What he'd like to say is that he can use a buried diode in which the surface is brought to the substrate potential. Some people call that pinning the surface. This is not a PIN diode. There is nothing to see here.
ReplyDeleteEF's post is at http://forums.dpreview.com/forums/read.asp?forum=1000&message=32809091
ReplyDeleteEF made several comments there 3 years ago, all of which look sound. Jake and Bocko, no so much.
It is straightforward to go from 3T to 1.75T or 1.5T designs with Sigma-delta imager (readout and reset T simply shared between four pixels). I thought that was obvious and not worth mentioning.
ReplyDeleteOK Anonymous, we see that you are smart and know the difference between PIN and pinned. Of course, I was referring to pinned diode.
Anyway, stay tuned and you'll have a chance to see the paper and the commercial sensor soon
J
J, could you answer on few technicalities?
ReplyDelete- Talking about 30fps speed, for what sensor resolution it is achieved?
- What is your sigma-delta ADC resolution at the above 30fps?
- I'm still puzzled how you can share readout and reset transistor between photodiodes. If photodiodes just muxed, it leaves part of the charge on floating diffusion node. Does this destroy charge conservation in your sigma-delta loop?
It is a shame that Jake Ignjatovic has decided to make this personal with several (implied) slams at me on a world-wide forum (DPR). Especially after not responding to a series of technical points/questions I made 3-1/2 years ago until just recently.
ReplyDeleteNevertheless, I remain open to the possiblity that this reported sensor performance is real and commercially viable. The proof will be in the commercial success and worldwide adoption of the technology.
Meanwhile, a copy of the main ADVIS patent 7,466,255 is available thru www.pat2pdf.org, among other sites and it can be easily discussed. I don't have easy access to the paper referenced in the patent however.
Fig 10 of the patent shows the charge injection circuit to the photodiode. I still think there is noise injected here but maybe Jake can clarify this for us all. I also could find no anticipation of the use of a pinned photodiode nor moving from 3T to 1.75T or 1.5T designs which Jake says is obvious.
I also note that I have some old, practically ancient, experience with sigma delta ADC with CMOS APS. The first paper dates from 1993 and was led by Suni Mendis:
http://www.ericfossum.com/Publications/Papers/Design%20of%20a%20low%20light%20level%20image%20sensor%20with%20on%20chip%20sigma%20delta%20ADC.pdf
The second paper if one of several papers published on a current mode device that Junichi Nakamura and others of my group explored. It was published in 1997.
http://www.ericfossum.com/Publications/Papers/1997%20Program%20Design%20of%20an%20Image%20Sensor%20with%20On-Chip%20Oversampling%20Analog%20to%20Digital%20Conversion.pdf
The person that got sigma delta conversion going in our minds at JPL was Bill Mandl formerly of Aerojet and then AMAIN. At least Jake et al reference his work in their patent.
I am very interested in seeing a histogram of the output signal under low light. I guess with sub electron read noise, as claimed, we should see a series of peaks spaced 1 e- apart.
The use of electronic shutter is also suggested in the patent app. I am still trying to figure out how it works but probably Jake can explain it for all of us - at least what is openly published in the patent office docs.
Lastly for this post, I wonder how Jake got to this line in his patent application:
"Although the parameters such as sensitivity, dynamic range and linearity of the CMOS imagers are inferior to CCD imagers...." (written in 2006).
There is a very small chance that Jake has produced something worthy of this much attention. Who really thinks that data conversion is the biggest issue for progress in CIS?
ReplyDeleteEric,
ReplyDeleteTalking about ADVIS noise histogram, sigma-delta loops (or delta-sigma, according to IEEE naming convention) normally have limited cycle oscillations, so called tones. This is especially true for first order loops. So, I expect the 1-pixel noise histogram to be completely bizarre, with many discrete peaks spaced apart by some strange spaces.
And if we make the whole array histogram, we need to take into account 0.5% FPN that Jake talks about. I'm not sure where such a huge FPN comes from, may be Jake can explain, but it should average all the individual pixel histograms with random (?) FPN offsets.
Like you mentioned, sigma-delta sensors is not a new approach. Probably Amain remains the only company still committed to it, other than ADVIS. In terms of commercial success of sigma-delta sensors, Pixim takes the crown, selling about half a million of such sensors a year. But even Pixim is rumored to switch over to 4T in recent years.
Pixim has never used a sigma-delta converter in any product. They use a ramp with a comparator in the pixel, perhaps shared. And is the volume really that high?
ReplyDeleteI wish a have more time to devout to this discussion. I'll try to answer to as many questions as I can in a few days:
ReplyDelete" So, I expect the 1-pixel histogram to be completely bizarre, with many discrete peaks spaced apart by some strange spaces"
This is true for conventional first order Sigma-delta, but not true for our modified SD. Take a look at the typical spectrum in dark ( I assume you were referring to the 1-pixel spectrum and not the histogram):
http://i28.tinypic.com/1z2gj00.jpg
Jake
"Who really thinks that data conversion is the biggest issue..."
ReplyDeleteWell, of course if you are talking about analog readout methods, data converters are not an issue since they are at the end of the signal acquisition chain (signal can be amplified before it hits the ADC so that the ADC noise becomes irrelevant). However, it's becoming more difficult to design low-noise analog front-end in more advanced technologies.
For fully-digital imagers (and everyone in their right mind believes that this is the way to continue), the ADC noise performance is critical. Pixim and its Nyquist rate ADC does not stand any chance in this race (low-noise performance) since they also rely on the transistor precision in a similar way the APS does. Actually, in Pixim's case, this problem is even more emphasized. Basically, their comparator noise is killing their performance and it is not easy to remedy that problem.
In our case comparator is in the loop of the SD and its noise is shaped to high -frequencies and removed by the decimation filter.
Jake
"It is a shame that Jake Ignjatovic decided to make this personal..."
ReplyDeleteUnfortunately, ADVIS had troubles at the beginning (for almost 3 years) securing investment for its further development and that happened thanks to speculative comments from a few experts. Comments that were based on their limited experience and understanding of the Sigma-delta approach. Some of them were smart enough to keep it private, but some weren't and decided to go public on www even though the whole thing was under the NDA (frankly we were discussing this with our lawyers and were about to go after them, but being nice guys and knowing that we do have something extraordinary we decided to simply take our time). However, I did took this very personally and will enjoy discrediting them hoping that some of them will learn their lesson.
In recent years, ADVIS was fortunate to meet people that would take experts' opinions with the grain of salt and secure funding. Again, the product will hit the market soon.
Jake
And the last thing for today's discussion:
ReplyDeleteThe main ADVIS patent as mentioned above is not really our main patent. That one simply claims that one does not need an analog amplifier at the pixel. Furthermore, one does not even need entire comparator at the pixel (only its input transistor, row select T and reset T). I do apologize if you guys have problem understanding the patent. Frankly, sometimes I get confused and do not understand it the way its written. I guess patent lawyers like to write it that way to justify their price.
The main patent is still pending.
Jake
"I still think there is noise injected here but maybe Jake can clarify this for us all. I also could find no anticipation of the use of a pinned photodiode nor moving from 3T to 1.75T or 1.5T designs which Jake says is obvious."
ReplyDelete"The use of electronic shutter is also suggested in the patent app. I am still trying to figure out how it works but probably Jake can explain it for all of us - at least what is openly published in the patent office docs."
Dear Dr Fossum,
I have a nice suggestion for you. You can sign up for my spring term course on Advance Analog IC in sensor applications and then you'll be able to see and learn it all. It's always good to refresh one's knowledge. I would be happy to help you get a discount on the tuition.
Nice Jake. You have probably strongly impressed many people with your posts.
ReplyDeleteYou have implied that I would want to slow image science progress and would not be able to do a fair review of a technical paper. This is a rather serious allegation but remarkably ridiculous under the circumstances. Furthermore now you have implied that I broke some NDA. I have never signed any NDA related to ADVIS technical information. I only responded to a thread in DPR about your company's announcement. Actually, your approach was published by the patent office in 2005 and available publicly to all. (US PTO 2005/0073451 Pub. April 7, 2005)
Guys - I think better to take this offline, or agree to disagree. Mails sent in a moment of irritation are almost always regretted, and it's too easy to say something in a mail/post that would be so difficult or not appropriate to say in person.
ReplyDeleteIf work was sport we'd shake hands at the end of the game regardless of what had happened and then all move on... I vote a virtual shake of hands of mutual respect, of professional people playing on the same field.
Dear Dr Fossum,
ReplyDeleteI am deeply sorry that you feel this way. It was not my intention to upset you and I never associated your name to any of the comments I made.
I know that you are respected scientist and your contribution to the field is remarkable. However, I am sure you agree that such a respected scientist should be careful when making comments on other people's work because they may have a profound impact on their future work. Please let us "small fish" survive in this world, as well.
I agree with the previous post. We should agree to disagree. I suggest we shake hands and move on. Once again, I have a deep respect to your work and I would be thrilled to have a person like you on my side.
Jake
I've been away of Internet for a couple of days and see quite a lot of activity I've missed.
ReplyDeleteFirst, it's good to see the personal side of things is settled somehow. I think most questions could be easily cleared if we have more information about ADVIS technology.
Jake, why not to publish the most recent papers on your company site? No reviewer can prevent you from doing this. Then we all would be in sync with your recent work and can have more intelligent discussion.
Regarding the histogram of the sigma-delta noise, I meant that tones appear not only in spectrum, but also in histogram. Here is the explanation:
If the photodiode current is very low, the sigma-delta feedback is mostly zero, whith sparse ones appearing once every many cycles, let's say once per million cycles. This mostly zero bit stream enters the decimation filter, having limited history length. FIR filters have very well defined history length, while IIR history are less well defined, but still very distant events have little influence on IIRs. Actually, the history length is better to be not much longer than the filter output sampling interval, otherwise it would appear as image lag. So, when one pops up in the bit stream, the filter output gets higher value, while most of the time the filter output is close to zero. Looking on the histogram of the filter output, it would have peaks at zero and some other value - this is what I meant. In some other cases there might be few more peaks, but in many practical cases there should be a histogram quantization of some sort.
By the way, in the sigma delta noise spectrum you posted the PSD is almost flat from fs/2 to fs/32, going down only by 6db or so. This seems to suggest either that duty cycle of sigma-delta loop is low, or there is some other factor making the loop noise spectrum be non-classic. This brings another question I've asked before, what is the ADC resolution at 30fps you mentioned? And what is the pixel array resolution?
I believe the paper can give us insight to what you are doing and we can see what we are missing.
Regarding Pixim being non-sigma-delta - Anonymous is right, I just mixed up Pixim technology with sigma-delta work made by Abbas El-Gamal's group in Stanford. Pixim sales of 0.5M sensors a year are nice, especially considering a price premium the company is charging for its HDR. However, its digital pixel still remains a niche product.
Jake says that an amazing product will be available in 7-10 months from now. Let's check back in 10 months on this claim. If data conversion is the only expertise of this group, there will be a lot of things they will get to learn before they run out of money learning them. It's probably better that they release their real papers after the amazing part is released so as to not waste good people's time in helping them see why the product will not be as amazing as they think. Nothing to see here.
ReplyDeleteI am going surprise many and say something positive about ADVIS.
ReplyDeleteI think data conversion in the pixel has always been something to strive for. Right now I think it should be for photoelectron counting (aka "jot") but between here and there, exploring other architetures for on-chip data conversion has always been of interest to me. See 1989 - Architectures for Image Processing for example.
Or see 1994 - On-Focal-Plane ADC
The column-parallel architecture with signal-dependent feedback to the pixel has been used by several companies and ADVIS is applying it for ADC. I obviously have concerns about some of the details, but it is not a waste of time to consider alternative approaches like this to imaging. You never know when the approach, changed a little bit, suddenly becomes an important idea, so understanding it is useful. Plus, it would not be so bad if people, myself included, made some positive suggestions on how to make their approach work better. So I would love to see a real paper (not necessarily peer reviewed yet) that we could all constructively analyze and comment on and help the ADVIS guys build a successful niche product.
Eric, Nobody is asking for your help, in fact many don't want it. ADVIS would be much happier not to have your opinion unless it is biased positive. Getting on here and posting your own references and trying to imply that you are thinking of these things first is self-fulfilling. The ADVIS guys are going to need a lot of time, just let them have it. Try picking it up again in 3 years.
ReplyDelete> trying to imply that you are thinking of these things first is self-fulfilling.
ReplyDeleteThis is not true. If you re-read Eric's commnent, he says that the idea first came from Bill Mandle. The first paper was written by Suni Mendis. I doubt this can be called self-fulfilling.
To me, every opinion has a value. As much as I'm interested to see Jake's comments and read about his ideas, I'd be interested to know what others are thinking about it, including Eric.
Thanks Image Sensor dude. The sigma-delta references were given in response to the comment that one needed to be an expert is sigma delta to understand what they were doing and why it was good. The ADC architecture references were to support that I have been interested in data conversion for image sensors for some time. Certainly others have made prior significant contributions and many are referenced inside those references. So, Anon, give me a break.
ReplyDeleteIn this thread are comments, perhaps all by the same Anon person, maybe not, that data conversion is not that important, and that looking at the ADVIS idea is a waste of time. This is a myopic point of view and perhaps a consequence of the silo-ization of image sensor producers. People are put into narrow focus jobs (the pixel modelling dept., the data conversion dept. etc.) and don't care about the other areas. I hope some people can step out of their silo and look at the big picture of what needs to be done in imaging over the next dozen years and how are we going to get from here to there. We need a new generation of big-idea technologists in our field.
Second, as image sensor technologists, we have a common bond that in many ways is stronger than company affiliation. Many of us work on image sensors even as we move from company to company. I know a few people who have moved to a different country just so they could continue to work on image sensors. So while competitive information needs to be kept confidential and not shared, there is still a need and a place to talk about common issues and future directions. That is we try to foster at the IISW (yeah, that is sort of a plug) and that is also what I have been told is a desired outcome of this blog and comments. I am going to continue to comment and to try to help out my fellow image sensor technologists until Image Sensor dude asks me to stop.
"ADVIS, Inc has plans to release its first commercial HD sensor based on the Sigma-delta technology in Q2 2010."
ReplyDeleteWe are in Q2 2010. Any sign of this commericial HD sensor?
Is the commercial HD sensor from ADVIS available yet? It was supposed to be released by Q2 2010. There were a lot of claims made about the performance, so it will be interesting to know if there is any basis for them.
ReplyDeleteWe are in Q4 2010. Any sign of this ADVIS commericial HD sensor?
ReplyDeleteNo. They are out of funding and were not able to do what they claimed.
ReplyDelete