CMOS vs CCD for spectroscopy applications

Design, construction, tuning of spectroscopes
Information and discussion about softwares (telescope remote, autoguiding, acquisition, spectral processing ...)
Christian Buil
Posts: 1431
Joined: Mon Sep 26, 2011 6:59 pm
Contact:

CMOS vs CCD for spectroscopy applications

Post by Christian Buil »

CMOS can be used for spectrography observation?

I have pleasure to present some elements of the confrontation on this paper:

http://www.astrosurf.com/buil/CMOSvsCCD/index.html

I compare ATIK460EX, ATIK414EX, ASI290MM and ASI1600MM performances.

Some subject are developed, primary parameters of detectors:

Image

Fringing problem (note a short discussion about Lhires III and flat-field acquisition):

Image

Observation result concerning a peculiar type of stars, the chromospheric active stars
(a future and exiting topic of discussion, I prepare elements). Note the detection of chromospheric
emission on the core of Ca II lines:

Image

(note that science is possible with a relatively inexpensive CMOS sensor).

And more...

The winner is presently the ATIK460EX CCD camera for faint light spectroscopy, but the CMOS are not
so far... The main problem is finally dark signal and the operating temperature of CMOS detectors.
A minor difficulty I think.

Good reading!

Christian B
Thierry Lemoult
Posts: 136
Joined: Thu Sep 29, 2011 9:08 am

Re: CMOS vs CCD for spectroscopy applications

Post by Thierry Lemoult »

Hello Christian

Very impressive work. I really love your article. Thank you.

In case of a spectro with fiber, we can put in in a fridge. Perhaps you can try to put a CMOS camera at -20°C ambiant (in a fridge) to check the dark of detector at @-45°C. And also check is the camera still work at such temperature. Some electronic component stop to work at low temperature.

Regards

Thierry
Christian Buil
Posts: 1431
Joined: Mon Sep 26, 2011 6:59 pm
Contact:

Re: CMOS vs CCD for spectroscopy applications

Post by Christian Buil »

Thanks Thierry,

Given the importance of temperature for the performance, yes, I actually planned to make tests at temperatures much inferior to see if everything goes as planned.

Important because I think that the ASI1600MM camera with its low read noise and its large surface could be a good challenger for a échelle spectrograph...

Christian
Thilo Bauer
Posts: 37
Joined: Sun May 31, 2015 4:45 pm
Contact:

Re: CMOS vs CCD for spectroscopy applications

Post by Thilo Bauer »

Christian Buil wrote:The winner is presently the ATIK460EX CCD camera for faint light spectroscopy, but the CMOS are not
so far... The main problem is finally dark signal and the operating temperature of CMOS detectors.
A minor difficulty I think.
Hi Christian,

Very interesting article. One question: I'm really interested in how you did measure the quantum efficiency. Is this based on flux calibration of alpha Lyr or another light source? Comparing those values is important for such comparison.

One remark from my side: I'm not quite sure what will be the winner if the goal to be achieved is not well defined.

Without doubt the quantum efficiency of the classic CCD reaches values above current CMOS sensors. Also current CCDs have a larger full-well capacity, which means a larger dynamic range. As you already pointed out, there are different use cases to consider.

When computing a usable dynamic in the range from the low level of 3 sigma of the RON to the low photon flux measured of a low light level source, this will change everything. Why? Because, lower RON means, you are more safe to detect and measure low light levels. The CCDs presented in your table have a RON that is at least 3 times larger compared to the fantastic low RON of any of the CMOS sensors. For someone who will detect even the very low light level of an emission line nebula, the winner should be the CMOS sensor. Detecting only a few photon evens seems much easier compared to the CCD. In other words: Which one of the compared sensors will be winner depends on the application and light level, you are interested in. Probably, in the domain of deep sky imaging and faint object spectroscopy CMOS sensors push the limits.

Best regards,

Thilo
Christian Buil
Posts: 1431
Joined: Mon Sep 26, 2011 6:59 pm
Contact:

Re: CMOS vs CCD for spectroscopy applications

Post by Christian Buil »

I have not measured the absolute QE (not easy task and incertain presently). I indicate on the page: the given QE is relative to the estimated ATIK460EX QE, i.e. if ATIK460EX QE values change, all the values for other change in proportion. My measure of ZWO camera for example, is the result of the observed signal for these camera and for the ATIK460EX used in parallel.

You are right Thilo, my analysis concern faint light spectroscopy (or narrow filter deep sky imagery).

I have added more info about measured dark current in function of temperature:

Image

Note the result for ASI1600MM.

Also, more precise measure concerning the response linearity (note the presence of a possible digital command problem
of the exposure time on the ASI290MM and the « superlative" linearity of ASI1600MM camera sensor (Panasonic)):

ASI290MM:

Image

ASI1600MM:

Image

Image

The most deviation to linear law for the ASI1600MM concern only the first 4 LSB of the dynamic!

Update and more detail on my page: http://www.astrosurf.com/buil/CMOSvsCCD/index.html

Christian B.
Thilo Bauer
Posts: 37
Joined: Sun May 31, 2015 4:45 pm
Contact:

Re: CMOS vs CCD for spectroscopy applications

Post by Thilo Bauer »

Hi Christian,

Again thank you for this information. Indeed the ASI 1600 is a very interesting camera. And I quickly learned: Cool it down to the lowest possible temperature. This is interesting to note because I used uncooled DSLR for a while being quite happy.

Regarding your comments on non-linearities in the least significant bits, CCD and CMOS have one thing in common: Digitalization errors due to technical limitations in the electronics of the analog sensor electronics and also analog-digital conversion unit (ADC). I see these effects everywhere looking to the low intensities of an image histogram of a digital imaging sensor. It is no surprise. The error appears as a comb-like structure of a single(!) image histogram. As the number of changing bits is large for the least significant bits (due to noise) in astronomical imaging the effect is more obvious when scaling the image to the Gaussian noise level of the low image intensities. But it is also present with the high bits which don't switch often between 0 and 1. The correct source and interpretation often depends on the electronics of the camera device used and sometimes environmental conditions with unknown source (see my report on this below).

The error (non-linearity) occurs because of two major reasons: (1) non-linearities in the ADC unit (non-linear or deviating resistance steps for the bits to be digitized), and (2) coupling between the power supply and number of bits registered at the ADC. The latter seems a bit more tricky for CMOS imagers having all on one single chip. However, it is interesting to note: Even the CCDs in the 1990ies with first class ADCs showed such effects. Since the 1990ies nothing changed. With several DSLRs, that I used, the effects seems to vary with time, the comb in histogram is not fixed to special bits of the digital numbers and also seems to depend on content and width of its noise distribution. In astronomy use case we find many numbers scattered around a mean background value. This is not the case with conventional photography, where the effect is not so obvious.

There is no ideal ADC. If power supply of the amplifier stage and ADC are not well separated and stabilized, the number of bits set to one or zero may affect the analog amplifier part. So it is typical to find a voltage drop, if many digital "1" bits are set by the ADC drawing more current. Drop in power supply affects pixel intensities, which will vary coupled to the digital numbers. In such a way statistical pixel intensities are coupled to the number of 0 and 1 of the ADC, if power supply of the stages are not clearly separated. As CCDs have decoupled ADCs (but not ideally decoupled voltage supply of the stages) the effect might differ with the designs of the ADC, and the capabilities of the camera manufacturer to decouple power supply of the CCD and digital electronics. For typical monolithic CMOS you have it all on a single chip. I'm actually wondering, why this works so well with modern CMOS imagers. :D

Anyway the effect is common, for both, CMOS and CCD. There is not much difference compared to CCD to overcome the effects of these digital non-linearities. The effect quickly disappears when averaging (or just adding) a few images together. With averaging images the nonlinearities and comb in histogram disappear quickly due to statistical reasons. This is true for both, CCD and CMOS.

From a quick look on the few image histograms of my rough sky tests taken in the last nights, the ASI 1600MM-Cool provides less digitalization errors seen in histogram compared to a Canon DSLR. However, I expect the ASI to have similar problems like any sensor, CCD or CMOS. And the ASI might show this effect a bit later, when photon count and Poisson noise dominates the very low read-out noise of the Panasonic sensor. In other words, if you have strong background with broader statistical deviation of pixel intensities the effect may change as the prominent bits and distribution may change. This is how you can separate a fixed non-linearity (fixed error in the resistance stage of the ADC) from a statistical influence of the number of 1 bits measured (or 0 in case of inverse logic, where 0 means high current).

It is also interesting to note: Almost every effect documented in about 1 TB of imaging data taken in the last 20 years with CMOS (DSLR) or CCD may vary over time, like the non-fixed position of warm or hot pixels, or digitalization errors. I have no good explanation for this - except saying it is "wear" or runout of the silicon structures -, but it is a fact. Therefore, I prefer to take darks every start of the night and end, as I learned it using CCD twenty years ago. Use of calibration frame libraries, as often discussed in amateur astrophotography, is a method to quickly run an auto-guiding camera. Its a no go for measuring. Don't do it when measuring stellar parameters like in spectroscopy.

People not familiar with the behavior of silicon sensors should read the fundamental work of Craig D. Mackay "Charge Coupled Devices in Astronomy" (1986). And, hey! The author already mentioned CMOS in 1986! Wise guy. So, I'm wondering, why it is still not popular in astronomy and we still discuss about differences between sensors. It is like religion. http://adsabs.harvard.edu/abs/1986ARA%26A..24..255M

Yes, CCD and CMOS are different. And the difference is hidden in its similarities.

Use Mackay's work as reference and follow the guidelines for calibration of the devices. It is fundamental work now applied in many interdisciplinary fields of science, where imaging with silicon detectors is common practice. Then you are on the safe side with both types: CMOS and CCD. Although. a few problems with flat field are left out and still discussed in literature. This is related on how the non-flat field is caused by your optics, however.

I don't know if someone had a similar problem at the OHP star party 2017: I noticed unusual count of hotpixel and also digitalization errors in the digital numbers of my DSLR at the OHP for almost every image taken starting from the second day. First I thought my camera sensor is going to be dead soon. Back home everything was fine again with exactly the same setup. I guess this was a combination of high environmental temperature, irradition from an anonymous power source like RADAR, the many mobile devices on the base, more powerful WLAN - hey, this year it worked like a charm! -, or whatever. At least my bad image quality collected at the OHP 2017 is a fact. I've never seen this before with my DSLR. The first time I saw such problems was in the early 1990ies when we faced problems with our CCD at the Hoher List observatory. In certain nights we could trash 10% of our images taken with a pro-CCD. The reason was beleived to be caused by a strong local airforce base RADAR sometimes firing in direction of the observatory. We didn't know this for sure, but the image artifacts happened with two different CCD cameras used at the 1m dome of the observatory. Mobile phones or wireless computers were not invented at that time. Long cables connected the PC in the office with the CCD of the dome with the bits crawling slowly over parallel lines and carefully buffered by robust TTL logic drawing high current on the lines. Today we read out 12 Million pixels in a few seconds transferred by USB 3 serial lines. Many people walking around the observation site with a pocket computer called a cellular phone. Nobody knows.

I don't think we should discuss this as typical for this or that sensor or camera. It is typical for both kinds of cameras, which I ever used in the past and future, which we also use in astronomy. As such it is how it is. Sometimes it is impossible to clarify the true source of the effects measured. We are talking about photon-counting case (even if it is denied). Hence, we are talking about very low energy amplified and converted to digital numbers. Hey, that's experimental physics. In Germany we have a proverb: "Wer misst, misst Mist." Who starts measuring will also measure garbage.

Would you agree with what I describe having found as these non-linearities with the CMOS?

Best regards,

Thilo
Last edited by Thilo Bauer on Wed Aug 30, 2017 9:46 am, edited 3 times in total.
Peter Somogyi
Posts: 420
Joined: Sun Jul 13, 2014 8:56 am

Re: CMOS vs CCD for spectroscopy applications

Post by Peter Somogyi »

Great job, Christian!

About the ASI 290mm: the cooled version on TS is advertised as "14 bit Output (14 bit ADC)" (it's a copy/paste from there). Would need a confirm if it's a real 14 bit.
Its actual price would push me today to the ATIK 414exm again, given its sensor size for the actual spectroscopes / usable scope diameter combination really requiring at least a 1.5x bigger size. Maybe a very good focal reducer + resolution could help out here (I did already try, a cheap 0.85x focal reducer does work with my LHires, but not using due to the fear of later scientific use concerns).

The ASI 1600 would be attracting if it had enough dynamic range... But since Alpy / LISA already works optimal in that range, the ATIK 414 has already a sufficient sensor size, you wouldn't pull out much more benefit. Maybe LHires + 600/mm could pull out its optimal use, avoiding H-alpha due to dynamic difficulties.

I also expect troubles using it with Alpy 600: the IR part requires order filter, and I'm aware of no serious use other than faint SN's CaII triplet (coming up later).
And, the biggest use of Alpy 600 is its UV, where the 12 bit will be very limiting at flat taking (I'm shooting the flat to fullwell for a wellknown reasons).
With LISA, flat is less concern but as told the ATIK 414 chip size is already sufficient (same price!) but then have no trouble with 12 bit.

I will review your deripple method later (I finally ended up with the ATIK 428 EXm that working fine, its characteristics very similar to ATIK 460).

- Peter

EDIT: buffered readouts serie should stabilise the exposure time, I guess... also interesting what happens with linearity close to bias level, that requires a very clean distribution of noise, assume it is not checked yet. But, that's exactly the spectroscopic use, we always have lot of noise in short exposures and when summing all the photons even from the low SNR areas...
Peter Somogyi
Posts: 420
Joined: Sun Jul 13, 2014 8:56 am

Re: CMOS vs CCD for spectroscopy applications

Post by Peter Somogyi »

2 more thoughts:
- for the H & K line (or for <4000A in general), the LHiresIII internal flat (from Shelyak cal.upgrade) is unusable, takes up only parasite light. External flat is better, targeting that <4000A wavelength otherwise producing internal reflections.
- for me, the LHires internal flat DOES produce "similar" ripples for ATIK 414, and it is very stable (same after 1 year...) and "somewhat" similar to the ext. flat. Only problem is, the internal flat's ripple is totally inappropriate (shifted etc... and makes things worse). This means your internal flat might need a check for parasite light (I did put lots of tape below slit, as Shelyak cal.upgrade pdf suggested). For me, there is no more light leakage "around" the slit after wrapping with tapes.

- Peter
Thilo Bauer
Posts: 37
Joined: Sun May 31, 2015 4:45 pm
Contact:

Re: CMOS vs CCD for spectroscopy applications

Post by Thilo Bauer »

Hi Peter,

this explains well the "parasitic" light found in my first image shown here: http://www.spectro-aras.com/forum/viewt ... f=8&t=1832
I didn't see this before because the field of view of my DSLR is small compared to the image taken with the ASI.

Thank you for the tip to fix this by placing tape around the slit. Probably, it will be a good idea to think about use of black paint on the backside of the slit of my guiding unit. This should be a considerably good improvement for the Alpy in general, while looking at the first images of my ASI 1600. Think about patent fees to Shelyak reading our discussion. ;-)
Peter Somogyi wrote: I also expect troubles using it with Alpy 600: the IR part requires order filter, and I'm aware of no serious use other than faint SN's CaII triplet (coming up later).
And, the biggest use of Alpy 600 is its UV, where the 12 bit will be very limiting at flat taking (I'm shooting the flat to fullwell for a wellknown reasons).
With LISA, flat is less concern but as told the ATIK 414 chip size is already sufficient (same price!) but then have no trouble with 12 bit.
I would suggest: (1) A filter is not required, as long as you don't exceed spectral analysis beyond 800 nm (as stated in the manual). So far I didn't find the second order spectrum in my images taken with Alpy 600 + ASI 1600. It seems to be reasonably suppressed due to the (blazed) grating used in the Alpy 600. (2) If you need to exceed your analysis beyond 800 nm, use of any optical yellow, red or IR low pass filter starting to block at 500 - 700 nm is a good idea. Probably a simple wratten filter is enough. In case of doubt, use a high quality low pass interference low-pass cut filter. If there is still doubt, or you are taking large series of images to be averaged with large dynamic range, use two filters. Filter curves multiply with each other and thus blocking capabilities.

For UV part, your suggestions above sound like a great idea. Clever! Thanks!

A further improvement of the Alpy 600 is a larger cut out at the back side at the camera thread. I found the free aperture (hole) being too small to place a 1,25" filter in front of the ASI 1600 camera when screwed to the Alpy 600. I will think about how to solve this by increasing the aperture using a lathe at the back side thread of the Alpy. Having the opportunity to place a filter here sounds like a really great improvement (François, again, be aware of patent fees. :-)).
Peter Somogyi wrote: EDIT: buffered readouts serie should stabilise the exposure time, I guess... also interesting what happens with linearity close to bias level, that requires a very clean distribution of noise, assume it is not checked yet. But, that's exactly the spectroscopic use, we always have lot of noise in short exposures and when summing all the photons even from the low SNR areas...
So far I didn't measure linearity towards the low intensities with the ASI sensor. From my experience with DSLRs in general, non-linearties are important at high intensities close to saturation level. It is not important at low intensities close or below low intensity level, where photon noise dominates mixed with low read-out noise. From image series of 200+ images I couldn't find non-linearities at the low energy levels, but good photometry. I would think most silicon sensors like the ASI camera could behave like that.

Such non-linear effects at the very low intensities have been reported in literature from analysis of certain astronomical CCD sensors, however. Some of them are reported to work better and more linear, when flashed before the exposure. So, far I didn't find such effects especially with the CMOS sensors.

Above mentioned digital non-linearities caused by interference of digital electronics with analog amplifiers are typical for both sensor types, however. I wouldn't exclude these effects to be the root cause of the reported non-linearities of CCDs found in literature, which have been solved by flashing. Flashing with arbitrary low portions of light means to "move" the average background intensities towards an intensity range where the comb-like non-linearity of digital bits (remember: the number of 1 bits in digital numbers could impact measured analog intensity and seen as comb in digital histogram) could disappear in histogram of the low intensities. Probably this is exactly the effect seen in my images taken this year at the OHP, where higher environmental temperatures could have moved and caused the effects seen in the histograms of my images.

Cheers,

Thilo
Christian Buil
Posts: 1431
Joined: Mon Sep 26, 2011 6:59 pm
Contact:

Re: CMOS vs CCD for spectroscopy applications

Post by Christian Buil »

Now, test of ASI1600 camera on an echelle spectrograph (the large surface is here a clear advantage for the cost).

A true visual color image of the sun light captured by using my VHIRES-MO spectrograph and an ASI1600MC-Cool CMOS camera
(color version of ZWO ASI1600) :

Image

Always spectacular and informative!

Now by using the B&W version (ASI1600MM-Cool), some science - for example Be star phi Per:

Image

An important point here: because the small sized pixel (3.8 microns) the original spectrum is very oversampled (13 pixels / FWHM !!!).
But this also opens possibilities ... For example, although I am usually very careful with these methods, amplifier noise, also telegraph noise, cosmic impacts, cosmetics defaults in general... are well filtered in this example by using 5x5 median kernel without reducing spectral resolution and scientific content ( the spatial frequency characteristic of the noise is not that of the science data). This is an advantage of small pixels, and a method for effectively use CMOS sensors for spectrography. In a second step, I also realized a 4X software binning along the spectral axis during the processing for increase SNR. The final sampling is near 3.2 pixel/FWHM.


The Be star gamma Cas:

Image


A bright star (Capella):

Image

alpha Per (F5Ib)

Image


and a more difficult example for a R = 48 0000 spectrograph (+ a fiber link to the telescope), the symbiotic star AX Per (V=10.6):

Image

Christian Buil
Post Reply