How an answer is scored.

What one answer has to serve.

The obvious design is one score: how well you did, used for everything. It does not survive contact with the four questions the app actually has to answer, because they are not the same question and a single number cannot be right for all of them at once.

Two of those read a discounted number and two read the raw one, which is the single most confusing thing on this page and also the reason the design works. A score answers a question about the answer. A schedule answers a question about your memory. Paying the same number into both would mean either telling you that you got something wrong when you did not, or telling the scheduler that a coin-flip guess was a recall.

The two halves of a score.

A graded answer is carried through the engine as a pair, not a scalar (ItemScore, mastery/mod.rs). The raw half is what you are shown. The other half is what the scheduler is shown, and it is never above the raw one.

Where the raw number comes from

Nine of the fourteen formats are graded on your device, deterministically. Five need a model, because there is no way to compare prose to prose without one.

Where the scheduler's number comes from

The raw score is capped by how much of that format you could have collected by guessing (chance_rate and clamp_fsrs_for_kind, mastery/mod.rs). One formula, one input:

cap = 1 − 0.5 × c

where c is the share of that format a random answer collects. The reasoning is that a cap should represent the most durability evidence one correct answer of that format can possibly carry, and the quantity that fixes it is how much of the answer could have been luck. A score already below its cap is untouched, and the stored attempt is never modified — only the copy handed to the scheduler.

Two entries in that table deserve their own sentence. Every format where you pick from what is on screen tops out at Good — there is no perfect multiple choice, as far as the scheduler is concerned. And the flashcard is uncapped even though its number is a self-report, which looks like an oversight and is not: capping it would be treating it as a recognition format, and it is not one — you retrieve the answer from memory before you rate yourself. The problem with a flashcard is that we cannot verify you did; that is a problem about what it should be allowed to certify, not about what grade it should earn, and it is listed under the limits below.

Four grades, and the five bands you land in.

The capped score is turned into one of four grades (grade_of). Separately, a different constant decides whether the answer counts as a successful retrieval for durability (CORRECT_THRESHOLD, 0.80). That second line falls inside the third grade, so there are four grades and five bands.

The Easy boundary is set at 0.95 rather than 0.90 on purpose: with automatic grading a clean answer hits exactly 1.0 constantly, and a lower boundary plus the large Easy bonus over-extends intervals, which shows up as points quietly going un-reviewed. That is a calibration decision, not a finding.

What a grade actually buys.

The scheduler keeps two numbers per learning point: a stability, which is roughly how many days it takes for your chance of recalling it to decay to the target, and a difficulty, which governs how fast stability can grow. The grade enters in two places.

• On the first successful retrieval, the grade seeds both numbers. Left alone, the seeds are 0.40, 1.18, 3.17 and 15.69 days for Again, Hard, Good and Easy. They are then clamped to one day on a freshly introduced point (HOT_FIRST_INTERVAL_DAYS), because a concept met minutes ago sits on the steepest part of the forgetting curve and one recall straight after teaching does not earn a two-week gap. The grade still seeds the difficulty, so the signal is kept and shapes every later interval — the first interval is the one the app takes out of the algorithm's hands, and only the first.
• On every later review, the grade scales the growth. Stability grows by an increment that depends on the current difficulty, the current stability and how likely you were to recall it at that moment — and that increment is then multiplied by 0.23 for a Hard, 1 for a Good and 2.99 for an Easy (next_stability_recall).

Which is abstract until you put days on it. Take a point sitting at a stability of one day, reviewed exactly when it was due — so your chance of recalling it was 0.90, which is what "due" means here:

That is the whole of "an easy format is worth less to your schedule", in two rows. A flawless matching set and a flawless typed answer, on the same point, on the same day, differ by a factor of about two and a half in when you next see it — not because one is punished, but because the matching set could have been one fifth guessed and the typed answer could not.

One point, from first sight to its second review.

A single learning point — say osmosis — in a course where the terminal objective puts it at a retention target of 0.90. Nothing here is illustrative: the numbers are the ones the engine produces for these inputs.

What this does not buy.

Five of these are real gaps rather than modesty, and two of them we are actively arguing about.

• The judge's number has no arithmetic we can show you. For the five formats that need a model, everything on this page downstream of the score is exact and auditable, and the score itself is a model's judgement. We can tell you it is graded per submission on its own text, that it always comes with an explanation, and that the same answer re-graded may not land on the same number. We cannot show you a formula, because there is not one.
• We know the direction of one correction and not its size. Several exercises on one point in one sitting are now combined into a single observation rather than kept-first-and-discarded, which is the repair this page used to promise. The combination deliberately makes each further answer on the same point count for a little less than the one before. That shape is not a preference: in test theory a bundle of items that share an object is called a testlet, and the measured consequence of treating such a bundle as independent answers is that the value comes out roughly right while the precision comes out overstated. Our two numbers split exactly along that line — the grade and the interval are the robust half, the evidence total is the half that would be inflated. So the discount corrects a bias whose direction is established. Its magnitude is not: the published way to size a testlet effect is to estimate it from a population of learners answering the same items, and a private, single-learner app has no such population. We use a fixed rate borrowed from the adjacent literature and we would rather say so than imply we measured it.
• The caps are principled in shape and approximate in size. That a format's cap should fall as its guess rate rises is an argument. That the coefficient is one half is a calibration notch. What we defend is the ordering of the caps, which is pinned by a test; the absolute values are due a revisit against real answer data.
• There is no difficulty per question. Nothing in the app knows that one question on a point is harder than another. The only difficulty in the engine belongs to the memory model and describes the point, not the item. So two questions of very different hardness on the same point produce scores the scheduler reads identically.
• One format cannot be verified at all, and we have not finished deciding what that should cost it. A flashcard is graded by the button you press. It is a genuinely demanding format — you have to produce the answer before you can rate yourself — and it is also the one place where a generous self-rating stretches your own intervals. What bounds it today is only that a single self-reported Easy seeds a stability below the durability threshold, so it cannot on its own reach "mastered". Whether it should be allowed to count towards durability at all is an open question, and it is ours, not the algorithm's.
• The confidence you report is collected and unused. Some exercises ask how sure you were. It is stored, it is put beside the result on the exercise you have just answered, and no decision reads it. It is the only signal in the app that distinguishes two people with the same score — succeeding while certain and succeeding while guessing come out identical otherwise — and right now we are not spending it. Either it gets wired into something or it should stop being asked for.
• The 0.80 line is ours, not a result. Where "this counts as a successful retrieval" falls is a decision about how strict the durability claim should be. It is not derived from anything, and it sits inside a grade band rather than on a boundary, which is why the first question in the list below is the first question everybody asks.

Questions.

Sources.

What each one is doing here, so you can tell which claim rests on which work. Every one of these is a published paper you can look up.

• Ye, J., Su, J., & Cao, Y. (2022). A stochastic shortest path algorithm for optimizing spaced repetition scheduling. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 4381–4390.

  The primary paper behind the scheduler this page describes, deployed at scale on millions of review logs. Its optimisation goal is stated in the authors' own words — "to minimize the memorization cost for users to form long-term memory" — and their algorithm is named for it. So a scheduler of this family does not maximise how much you remember: it minimises how much reviewing it takes to hold a chosen retention target. That is why a grade here is a measurement and not a reward. It is also where the engine's notion of difficulty comes from, and that difficulty describes a memory state rather than how hard a question is.

• Rowland, C. A. (2014). The effect of testing versus restudy on retention: A meta-analytic review of the testing effect. Psychological Bulletin, 140(6), 1432–1463.

  Why the formats where you produce the answer are not capped while the formats where you pick it are. The format of the initial test moderates how much the test helps: producing an answer beats recognising one. We cite it for that ordering only.

• Kang, S. H. K., McDermott, K. B., & Roediger, H. L. (2007). Test format and corrective feedback modify the effect of testing on long-term retention. European Journal of Cognitive Psychology, 19(4–5), 528–558.

  The condition that decides which way the ordering runs, and the reason it runs our way: with corrective feedback a constructed answer beats a multiple choice, and without feedback it is the other way round. Ignia always returns feedback, which is what puts us in the first branch — and means the day a format stops explaining itself, its ranking flips.

• Carlson, J. E., & von Davier, M. (2017). Item response theory. In R. E. Bennett & M. von Davier (Eds.), Advancing Human Assessment: The Methodological, Psychometric, and Policy Contributions of ETS (pp. 133–178). Springer. Open access.

  Where the name for what a session does to one point comes from, and where the consequence is reported. A set of items sharing one object is a testlet, and its answers are not independent. The chapter records what happens when that dependence is ignored: Wainer and Wang, on TOEFL data, found difficulty estimated well either way while discrimination and the guessing floor were biased; Bradlow, Wainer and Wang found the resulting precision bias to be a function of the bundle's design; Li and colleagues found little difference in the estimates and more in the information. That split — value robust, precision overstated — is why the evidence total is discounted and the grade is not. We name this review rather than the three studies because this is the text we read.

• Pelánek, R. (2016). Applications of the Elo rating system in adaptive educational systems. Computers & Education, 98, 169–179.

  Where the shape of the discount comes from: an update weight that falls as the number of answers rises, fitted on real system data. We borrow the function and not the model — the model it belongs to needs a difficulty value per question, which we do not have, and a population of learners on the same items, which a private single-learner app cannot produce. The rate we use is therefore an imported constant, which is the limit stated on this page rather than hidden in it.

• Wilson, R. C., Shenhav, A., Straccia, M., & Cohen, J. D. (2019). The eighty five percent rule for optimal learning. Nature Communications, 10, 4646.

  Here as the rule the app does NOT apply, because this is the page where you can see why. It is derived for two-choice tasks with continuously adjustable difficulty. The caps below exist precisely because 85% on a true-or-false and 85% on a typed answer are not the same evidence, so a single target success rate has nothing to attach to.

You will find effect sizes in these papers. We deliberately do not print them on this page: an average across studies does not predict what any one person will get, and a number quoted without its conditions reads as more precise than it is. In the papers they come with those conditions.