Can ChatGPT score my IELTS essay accurately?

What we actually measured

We scored a fixed set of reference scripts four times over, holding out a different quarter of them each time so no script was ever marked against itself. Without a calibrated reference ladder in the prompt, our scorer came out 0.29 bands high on average, with a 95% confidence interval of +0.05 to +0.52. That interval excludes zero, which is the statistical way of saying the generosity was real and not noise.

Half a band matters. It is the difference between a 6.5 and a 7, and a 7 is what most universities and immigration streams ask for. A tool that tells you that you are ready when you are not has not been kind to you.

So we built the obvious fix: show the model a ladder of already-marked scripts and let it place the essay against them. It worked in the experiment and it was wrong in production. The ladder overcorrected. Measured on its own, it marked 0.21 bands LOW, and because its band was the one being published, the whole scorer went from a generous lean to a harsh one. Seven consecutive weekly runs had sat within a few hundredths of zero. The first run after the ladder shipped came in at 0.19 bands low, and the share of essays landing within half a band of the reference fell from 100 per cent to 79.

We reverted it. The full marking pass owns the band again, and the ladder still runs to give you a fast first number. As of the 31 October 2026 run, the measured lean was 0.02 bands low.

That episode is the actual argument of this page. Anyone can ship a change that looks like an improvement. The question worth asking of any tool that scores you is whether it would notice if the change had made things worse, and whether it would say so. We noticed because we measure the same scripts every week and publish the result, including this one.

General-purpose AI against a dedicated scorer

The last row is in the table on purpose. Our reference set is in-house, descriptor-labelled, and no script in it has been marked by a certified examiner. Any comparison table that gives its own product a clean sweep is selling you something.

Who marked the reference scripts

The band descriptors were applied by qualified teachers, not generated by the model being tested. Delta is the Cambridge Diploma in Teaching English to Speakers of Other Languages, the qualification above CELTA. Between them they have 13 years of IELTS classroom experience.

What a general assistant is genuinely good at

The useful version of this page is not the one where the chatbot loses every round. It does not lose every round. The most careful published look at what a general assistant actually says about a second language essay is Saricaoglu and Bilki, in the Annual Review of Applied Linguistics, volume 45, 2025. They put 35 argumentative essays to ChatGPT-4 across the same four dimensions IELTS uses, then checked every individual comment it made for whether the comment was true.

Almost all of them were. Accuracy of its observations came out at 99.4 per cent on Coherence and Cohesion, 98.3 per cent on Task Response and on Lexical Resource, and 95.9 per cent on Grammatical Range and Accuracy. When a general assistant tells you something is wrong with your essay, it is very rarely inventing it.

Where it thinned out was specificity. Only 36.7 per cent of its Task Response comments were specific rather than general, and 46 per cent on Coherence and Cohesion, against 79.7 per cent on grammar. Relevance was weakest on Lexical Resource at 73.5 per cent, and the authors noted it sometimes filed a grammar error under vocabulary.

Read plainly, that is a fair picture of the tool. A general assistant is a good reader and a vague marker. It will tell you truthfully that your second body paragraph drifts off the question, and then not tell you which sentence did the drifting.

Things it does well enough that we use it ourselves:

• Explaining a band descriptor in your own first language until it makes sense.
• Generating practice questions, endlessly, on any topic you are weak on.
• Rewriting one of your own paragraphs two ways, so you can see what a higher band move looks like on your sentence rather than on a model answer.
• Arguing with you about ideas when you have run out of them halfway through a body paragraph.
• Drilling collocations and topic vocabulary, and telling you why a word is wrong rather than only that it is.
• Reading as many drafts as you want. Our free tier stops at three writing scores. That is a real advantage and it belongs on this list.

None of those is marking, and none of them is a reason to stop.

What happened when a researcher marked 55 real IELTS scripts with it

There is one peer-reviewed study aimed squarely at this question, and it is more generous to the chatbot than the answer at the top of this page. Osama Koraishi, Language Teaching Research Quarterly, volume 43, 2024, pages 22 to 42. Fifty-five Writing Task 2 scripts drawn from publicly available official sources, each already carrying an official band, re-marked by ChatGPT-4 in November 2023.

The means matched exactly. Both sets of marks averaged 6.027. Agreement at the level of the individual script was strong: an intraclass correlation of 0.814, 95% CI 0.702 to 0.887, and a weighted kappa of 0.811, 95% CI 0.726 to 0.896. We are not going to pretend that says something it does not say.

What it also does not say is the part worth reading. Every caveat below is the author's own, printed in his paper rather than added by us:

• Fifty-five scripts is small, and he names the sample size as the first limitation of the study.
• He pasted a rubric into the prompt and used the identical prompt every single time. That is not what most people do at midnight the week before a test.
• It was one model in one month in 2023, and he writes that there is no guarantee a later model reproduces the numbers.
• Nearly every script sat between band 4 and band 8, which he argues is representative of the real candidate population but does mean the extremes went untested.
• Individual scripts still deviated markedly, and his conclusion is that it should not replace human judgement.

That last one is not a technicality, and it is the whole reason a mean is the wrong number to shop by. A perfect average across 55 essays is completely compatible with being a band high on yours and a band low on somebody else's. You are not submitting 55 essays. You are submitting one, and the question you are really asking is how big the error is on that one, not what the errors average out to across a set that happens to include it.

It is also why we report a mean deviation with a sample size and a confidence interval rather than a percentage. A percentage hides the shape of the errors. Our figure, the method behind it and the limits on it are on the quality page, re-run every week, published in the weeks it gets worse.

Why consistency is the property to test, not accuracy

Accuracy is what everyone argues about. Consistency is what decides whether a tool is any use to you.

Say you are trying to move from 6.5 to 7. That is one half band. If your measuring instrument moves half a band on its own between two readings of the same unchanged essay, your improvement is inside the noise and you cannot see it. You will read a rise that is not there, or miss one that is, and every week spent that way is a week of practice you cannot evaluate.

So test it. On us, and on anything else you are considering. Take one essay, score it, then open a brand new chat or a new session, wait an hour, and score the identical text again. If the two bands differ, that tool cannot detect a half band change no matter what its accuracy claim says.

Our side of that test is a design decision rather than a promise. The first full score for a piece of text against a given task is stored under a hash of three things: the normalised text, the normalised task prompt, and the pinned model version. Submit the same text against the same task again and the stored result comes back, byte for byte. Change the model and the hash changes, which retires the old cohort instead of quietly rewriting scores people have already seen.

The honest limit on that: consistency is not correctness. A scorer can be perfectly repeatable and repeatably wrong. What repeatability buys is that the error becomes measurable at all, which is the precondition for publishing it. It is necessary and it is not sufficient, and a tool that sells you the first as though it were the second is overselling. More on why two tools disagree in the first place is in why two AI tools give your essay different band scores.

What a purpose-built scorer actually adds

Not better judgement. The same class of model does the reading in both cases, and anyone telling you their wrapper has better taste than a frontier model is guessing. What a dedicated scorer can do is take the parts that never needed judgement away from the model, and hold the parts that do steady between runs. Four concrete examples, all of them things you could verify by feeding the tool a deliberately broken essay.

1. Counting happens in code, not in the modelWord count, paragraph count, sentence count, unique words, lexical diversity, average sentence length and complex clause markers are all computed before the model is called, then handed to it as figures. A model asked to count the words in a 300 word essay will sometimes be wrong, and every criterion that depends on length inherits the mistake.
2. An under-length script is penalised where IELTS penalises itThe penalty lands as a ceiling on Task Response, scaled by how far below the word floor you came in, rather than as a cut to the overall band. Pulling Grammatical Range down because an essay is short describes the wrong problem to the person reading the report. This one is on the list because it was broken here first: a 75 word Task 2 once came back at overall 8.0 with nothing on the live path enforcing the ceiling that the code comments promised.
3. Off topic is checked before the model sees the essayBy keyword overlap against the task prompt. A well written essay about the wrong subject should not collect partial credit for being well written, and a general assistant handed only your text has no prompt to compare it against unless you remember to paste one.
4. On Academic Task 1, the marker sees the same figure you didThe chart on screen is drawn from the same string the scorer reads, numbers intact. Paste a Task 1 report into a chat window on its own and the assistant has never seen your chart: it cannot tell you that you picked the wrong key features, described a fall as a rise, or quoted a figure that is not on the graph. On Task 1 that is most of Task Achievement. This is the clearest case on the page where a general assistant is not a weaker substitute but the wrong instrument.

Notice what is not in that list. None of it is a claim to read an essay better than a frontier model reads an essay. It is a claim about the parts that can be made not to vary, which is a smaller claim and a checkable one.

Speaking is the widest gap, and not for the reason you would guess

A chat window marks what you typed. The speaking test marks what you said, once, out loud, with no backspace key, while somebody waits. Typing a Part 2 answer into any assistant and reading its band is measuring a skill that is not on the exam. You will always sound better in writing, because in writing you get to think.

Our speaking path records your audio, transcribes it, and scores the transcript against the four speaking criteria. The value there is mostly the conditions: you had to produce the answer aloud, in one pass, against a prompt you had not seen.

And the limit, which you should hear from us rather than find out later. Pronunciation is inferred from the transcript, not measured from the audio, and the scoring prompt says so in its own text. Fluency, range and coherence largely survive transcription. Pronunciation only partly does. We publish a weekly measurement for Writing and we do not publish one for Speaking, and we are not going to let the first quietly stand in for the second. The longer version of what software can and cannot hear is in how accurate AI IELTS speaking scores are, and the practice itself is at IELTS speaking practice.

Before you paste an essay into any chat window

Thirty seconds, once. Whether a conversation you type into a general assistant can be used to improve that model depends on the plan you are on and on a setting inside the product, and the wording changes over time. We are not going to quote somebody else's policy at you second hand and be wrong about it in three months. Open that product's own data controls page, read it, and decide before you paste work you would rather was not stored.

What we publish about our own handling is short and it is on the privacy page. Your writing and your speaking transcripts go to Anthropic through the API for scoring and are not used to train those models. We do not publish anything you submit, on any tier. Account deletion is a hard delete and it asks you to type a confirmation phrase, because a deletion that happens on a stray click is a different kind of problem.

Frequently asked questions