Chapter 12 · Lesson 3 of 4 · about 18 minutes
Rates from flight data and published cases
A higher error count can come from a storm, from a new operating mode or from a counter that now sees more events. A quiet interval may include time when the monitor was disabled or the downlink was missed. Counts are compared only after checking how long the hardware and the detector were live and what each record actually counted.
What you need to know
A flight rate is a count divided by the valid monitored time, and both halves need work. The counts are sorted by what was counted (corrected bits, physical events, interrupts), by the mode and configuration, and by the orbit region and the environmental interval. A count that doubled after a software update may only mean that the counter now sees events it used to miss. The time is the live time: the hours during which the function was monitored and the detector enabled, outside telemetry gaps and recovery dead time, with counter rollover and saturation accounted for. The exact Poisson interval of chapter 8 lesson 1 applies to the count, and the prediction is compared with the rate and its interval. An observed count far from the predicted mean says something about the observation, the exposure or the model. The three are examined starting with the easiest to check: the counter before the environment, and the environment before the model. Agreement within a factor of two is generally the achievable standard for a well-characterized part in a well-known environment. Closer agreement on one mission is partly coincidence.
Published cases are useful as long as the lesson drawn from them stays tied to what was actually established. The TDRS-1 attitude-control RAM upsets during the October 1989 solar particle events were compared with a model that combined proton and heavy-ion contributions, and the published analysis reports observed and modeled counts close together. That comparison supports the reconstruction for that part and that event. It does not establish how accurate the model is in general. Multiple-bit upsets observed in flight DRAMs showed that clusters happen in flight and are not an artifact of the beam. Long-term solid-state recorder records have given rates over years and over solar-cycle phases, and models have been checked against them. Operational reports that trace a safing event to a bit flip while leaving the cause of the flip open show the discipline of lesson 12.2 at work: the functional path can be known while the initiating particle is not. Before any of these cases is applied to a new mission, the construction, the environment, the operating state and the observability have to be compared.
The way events get into the record shapes what the record contains. If only severe events trigger high-rate telemetry, the sample is conditioned on consequence. If recovery disables the monitor, clustering is suppressed. If the quiet intervals coincide with telemetry gaps, the rate comes out too high. A count model is fitted only after the censoring and the detection limits have been written down, and absent telemetry is not counted as zero events.
Enter the count, the monitored days, the live fraction and the predicted rate; the table gives the observed rate with its exact interval and the tail probabilities under the model.
Worked example 12.3. Thirty days of U2 corrected errors against the prediction
In its first 30 days the recorder reported 2,130 corrected single-bit errors, with 92 % of the time monitored and the rest lost to gaps and reconfigurations. The observed rate is 77.2 per day with a 95 % interval of 73.9 to 80.5. The prediction of chapter 9 was 140 per day, 1.8 times higher, and under the model the probability of a count this low is negligible. The discrepancy is examined starting with the observation. The first question is whether the counter counts events or bits: the scrubber reports corrected words and the prediction counted upsets, so a cluster inside one word is one count. Next comes the exposure. The thirty days fell in the declining phase of the cycle, AP-8 MIN is a solar-minimum model, and the anomaly passes were all monitored. Last comes the model. The saturated 200 MeV proton cross section times the flux above 15 MeV is a first estimate, and it generally over-predicts when the cross section is still rising at 50 to 100 MeV, where most of the trapped flux lies. A factor near two in the over-predicting direction is within the stated uncertainty of the proton term. The record is kept as evidence for the next revision of the proton model. It is not a reason to change the scrub interval.
For comparison, the TDRS-1 case as reported had 243 upsets observed against 248 modeled (176 from heavy ions and 72 from protons). The 95 % interval on 243 runs from 213 to 276, so the agreement, close as it is, would have been declared for any model mean between those limits.
Exercise
For every flight rate you report, give the count, the valid live time, the definition of an event and the limits of the detection together, as in worked example 12.3.
For each published case you cite, keep the observation, the analysis and the attribution status separate, and give a direct link and the exact location in the source.
Going deeper
Selection can bias the apparent mechanism
If only severe events trigger high-rate telemetry, the sample is conditioned on consequence. If recovery disables the measurement, clustering of events can be suppressed in the record. A count model is fitted only after the censoring and the detection limits have been defined. A bound on the events that were not observed needs a defensible account of what the telemetry could see during the gap; interpolating across a missing period gives none.
The rate is the count per unit of valid live time, with the uncertainty that follows from that live time, and absent telemetry is not counted as zero events.
A good case does not calibrate every mission
The TDRS-1 agreement supports that reconstructed case and says nothing about the accuracy of the model elsewhere. Cassini's functional path is known, and that does not establish a cosmic-ray origin. The K2 explanation is qualified by the mission manager who gave it. A case transfers to a new mission only after the construction, the environment, the operating state, the observability and the alternative mechanisms have been compared.
The common mistake
Dividing by live time does not correct for a detection efficiency that changed, for events that flip more than one bit, or for a susceptibility that depends on the operating mode.
A published operational account can be preliminary, and it is not the same thing as a complete failure investigation.
Takeaway
- A bias in how events were observed, left uncorrected, can look like a change in the environment or hide a failure mode that matters.
- Real flight cases show how a fault propagates through a system and how carefully the limits of an attribution should be stated.
Check yourself
The corrected-error count triples after a flight software update. What is checked first?
The counter: whether the update changed what is counted (bits against words, a previously masked bank, a changed scrub interval that now reports more corrections per pass) and whether it changed the live time. The exposure and the model come after that.
Why does the TDRS-1 agreement not validate the model for the recorder's U2?
It validates that reconstruction for that RAM, that orbit and that event. U2 has a different construction, environment and operating state, so any transfer has to be argued for U2 rather than assumed.
A quiet interval shows no U1 events in 60 days. What rate does it support?
An upper bound of about 0.05 per day (18 per year) at 95 %, from zero counts in 60 days of live time. A prediction of 4.6 × 10⁻⁶ per day is far below that bound, so the quiet interval does not test it.
The U2 corrected-error rate came in at 77.2 per day against 140 predicted, a factor of 1.8. A colleague calls the prediction wrong and asks for the scrub interval to be lengthened to match. What does the lesson say?
Agreement within a factor of two is generally the achievable standard for a well-characterized part in a well-known environment, and a factor near two in the over-predicting direction is within the stated uncertainty of the proton term. The discrepancy is examined in order: the counter first (the scrubber reports corrected words while the prediction counted upsets, so a cluster inside one word is one count), then the exposure (a declining-phase month against a solar-minimum AP-8 MIN model), then the model (a saturated 200 MeV cross section times the flux above 15 MeV generally over-predicts when the cross section is still rising at 50 to 100 MeV, where most of the trapped flux lies). The record is kept as evidence for the next revision of the proton model; it is not a reason to change the scrub interval, which is checked against the anomaly pass and the event week rather than the average day (lesson 10.1).
The 95 % interval on the 30-day count, 73.9 to 80.5 per day, looks precise, and the project wants to adopt it as the recorder's flight rate for the whole mission. What limits that number?
The interval is the statistical uncertainty of a count of 2,130 over 30 days at 92 % live time, and nothing more. The count is of corrected words rather than upsets, the 30 days fell in one phase of the solar cycle, and the live time has to be the hours during which the function was monitored and the detector enabled. If the quiet intervals coincide with telemetry gaps the rate comes out too high, and if recovery disables the monitor, clustering is suppressed. A flight rate is reported with its count, its live time, its event definition and the limits of its detection together, and absent telemetry is not counted as zero events.
The TDRS-1 comparison reports 243 upsets observed against 248 modeled. Why does the course call agreement closer than a factor of two "partly coincidence", and what does figure 12.3 show?
Because the 95 % interval on a count of 243 runs from 213 to 276, so the comparison would have been declared consistent for any model mean between those limits; the near-match carries no more information than that interval does. The figure shows the probability of each count if the model mean were exact, with the observed count and its interval marked. Agreement can also come from model errors that cancel (lesson 12.4), and the comparison supports that reconstruction for that RAM and that event, not the accuracy of the model elsewhere. Before a published case is applied to a new mission, the construction, the environment, the operating state and the observability are compared.
Further reading
- D. C. Wilkinson, S. C. Daughtridge, J. L. Stone, H. H. Sauer and P. Darling, "TDRS-1 single event upsets and the effect of the space environment," IEEE Trans. Nucl. Sci., vol. 38, no. 6, pp. 1708-1712, 1991.
- G. M. Swift and S. M. Guertin, "In-flight observations of multiple-bit upset in DRAMs," IEEE Trans. Nucl. Sci., vol. 47, no. 6, pp. 2386-2391, 2000.
- C. Poivey et al., "In-flight observations of long-term single-event effect (SEE) performance on Orbview-2 solid state recorders (SSR)," IEEE Trans. Nucl. Sci., vol. 50, no. 6, pp. 2169-2174, 2003.
- J. W. Adolphsen, J. L. Barth and G. B. Gee, "First observation of proton induced power MOSFET burnout in space: the CRUX experiment on APEX," IEEE Trans. Nucl. Sci., vol. 43, no. 6, pp. 2921-2926, 1996.
- D. L. Hansen et al., "Correlation of prediction to on-orbit SEU performance for a commercial 0.25-µm CMOS SRAM," IEEE Trans. Nucl. Sci., vol. 54, no. 6, pp. 2525-2533, 2007. doi:10.1109/TNS.2007.908787.
- R. Harboe-Sørensen et al., "Observation and analysis of single event effects on-board the SOHO satellite," IEEE Trans. Nucl. Sci., vol. 49, no. 3, pp. 1345-1350, 2002.