Harvard's latest study: why is it more difficult for applicants to be admitted when AI documents are well written?

An application document that is almost unspeakable and well-structured may now pose an unexpected risk: the admissions officer suspects that the article does not present the applicant's own voice.

Three academics from Harvard and New York University recently analyzed 7462 applications from a highly competitive public policy master's program in the United States, spanning six application seasons from 2020 to 2025.This is one of the few studies to date that has been able to put AI use, clerical quality and actual admissions into the same set of data.。

The result is not simply "AI useful" or "AI useless". AI has made the paperwork more silky and with fewer errors, but it has not made it easier for applicants to gain admission. Once the admissions officer feels that an instrument is like a AI work, the evaluation of it will be significantly reduced.

Nearly 60% of applicants have submitted at least one suspected AI document.

The research involved a project that required each applicant to submit five documents ranging from 250 to 500 words, covering motivation, personal characteristics and professional background, and how to participate in public service in the future. The project requires applicants to complete the materials independently in English, and the use of generative AI is also explicitly prohibited from 2023.

Researchers useGPTZeroIdentify "high-confidence, mostly AI-generated" text and use another detectorPangramand more relaxed thresholds for review. From 2020 to 2022, before the release of the ChatGPT, almost no documents were marked as AI generated. Of all applicants in 2025, 56% of at least one document was judged by the detector to be mainly generated by AI with high confidence. When 2025 applicants are separated by status, 69.3 per cent of international applicants have at least one document so marked; among U.S. applicants, the proportion is 38.6 per cent.

Since the study only takes into account texts that are "mainly generated by the AI" and with a high degree of confidence in the detector, and only partial rewriting or modification by AI is not fully counted, these 56 per cent, 69.3 per cent and 38.6 per cent should be understood as conservative estimates under the test caliber, not the AI usage rate reported by the applicant.

Between 2020 and 2025, the percentage of applicants with at least one paper judged to be predominantly AI-generated; the red and blue lines represent U.S. and international applicants, respectively, and the dotted line represents all applicants.

AI improvement is mainly in language, less touching on content

The researchers evaluated the instruments from seven dimensions: grammar, style, clarity, whether to respond to the topic, personal voice, commitment to public service, and leadership. They first let the model score, and then asked the program's five admissions staff to rate the same batch of samples. The correlation coefficient of the two sets of scores is 0.70; the closer the coefficient is to 1, the more consistent the ranking of the two scores. This suggests that model scores are strongly correlated with admissions officer judgments, but not identical; this step is used to calibrate automated assessments, rather than using model scores directly as admissions officer judgments.

The correlation coefficient between the comprehensive score of the model and the score of the admissions staff is 0.70; the higher the score, the higher the quality of the document.

After the launch of the ChatGPT, the overall quality of application documents has improved significantly, especially for international applicants. However, the improvement mainly comes from mechanical aspects such as grammar, style and clarity. In terms of whether to really respond to the topic, whether to retain personal voice, whether to show lasting investment and leadership, the improvement range is obviously smaller.

This is one of the study's most useful reminders for applying families.AI are very good at making an ordinary sentence smooth, but they don't produce weighty details out of thin air. If the original experience, observation and judgment are already empty, they will only become more fluid after AI modification.

After the advent of the ChatGPT, the grammar, style and clarity of the instrument improved significantly, and the quality of the substantive content improved less.

The quality of paperwork has improved, but the acceptance rate has fallen.

During the 2023 and 2024 filing seasons, people who marked at least one instrument for AI writing had lower actual acceptance rates than unmarked people. There may be systematic differences in the grades, experiences and other materials of the two types of applicants, and the researchers therefore useDouble MLMethods, age, standardized examination and hundreds of characteristics extracted from the whole set of application materials into the control.

After controlling for the applicant's age, standardized examination and other observed characteristics, the statistical model estimated that the probability of admission was 1.5 percentage points lower on average for each additional AI document marked compared with applicants with similar other conditions and no documents marked.The "percentage point" here is the absolute difference in the probability of admission: for example, from 20% to 18.5, it is a decrease of 1.5 percentage points.

Among applicants who had at least one instrument marked, there was an average of 2.32 instruments marked. Based on an estimate of 1.5 percentage points per article, the overall probability of admission for such applicants is estimated to be 3.48 percentage points lower. When the statistical model further incorporates the quality of instruments into the control, the estimated variance corresponding to each marked instrument expands to 2.6 percentage points; for an average of 2.32 applicants, the estimated variance is about 6 percentage points. Both sets of results are statistical estimates, not fixed deductions published by the school.

These figures cannot be read as a fixed "AI points deduction" that applies to all universities ". The data came from only one master's program in public policy, and observational studies cannot rule out all unmeasured differences. What it shows is a direction worth taking seriously: the improvement of language quality does not mean that the enrollment results will be better.

At that time, the school did not use the AI detector system to screen the application materials, nor did it establish a set of processing procedures for automatic deduction of points after detection.Therefore, the admission difference is more likely to come from readers' judgments about the style and quality of the text, rather than a pre-set technical penalty.This explains why the researchers then tested whether the admissions officers themselves could recognize AI text.

Different statistical models estimate "the number of AI documents and the probability of admission"; negative numbers represent the more marked documents, the lower the estimated probability of admission.

Can the admissions officer see the AI paperwork?

The program's five admissions staff evaluated a total of 100 documents, 50 of which were tested as human writing and 50 as AI-generated. Without knowing the test results, they used a six-level scale to determine whether each instrument was more human or AI-like.

Identification of the experimentAUC is 0.70. Intuitively, if a random sample of an instrument detected as AI-generated and an instrument detected as human-written, admissions officers have about a 70% probability of judging the former as more like a AI; the benchmark for random guessing is 50%. Therefore,Admissions officers do recognize some of the AI writing, but it's a long way from being accurate.

According to the six-level judgment group of the admissions staff, 70.4 per cent of the documents classified as "more human-like writing" were also GPTZero classified as human writing. Of the documents classified as "more AI-like", 67.8 per cent were also GPTZero classified as AI generation. The two percentages here, the denominator, are the two sets of instruments that admissions officers judge as partial human and partial AI, respectively.

Admissions officers and AI detectors determine whether instruments are AI generated with high relevance

More noteworthy is the rating. The more the admissions officer believes that an instrument is generated by AI, the lower the quality score given; the instrument judged "very likely to be generated by AI" has the largest decline. This negative correlation persisted even when the researchers controlled for the quality of the instruments assessed by the model, whether the instruments were actually detected as AI generation, and the academic and demographic backgrounds of the applicants.

The more the admissions officer considered the document to be like a AI work, the lower the quality score given; a "6" represents "very likely to be generated by a AI"

Detection tools cannot be the basis for conviction

This study does not prove that AI detectors can reliably adjudicate on individual applicants. The researchers chose a high confidence threshold and checked for false positives with materials from the three application seasons before the ChatGPT came out, with the aim of observing trends in a large sample. This is two things with the school taking a test score to judge a student for cheating.

Detectors are subject to false positives and false negatives, and non-native English speaker texts have been found to be at risk of being mislabeled. This study did not find a higher differential false alarm rate for international applicants or language test filers in its own data, but nor did it describe the detector as error-free.

For universities, the more important question than "whether they can catch the AI" is: what exactly should the application document measure.If it is used to observe the applicant's experience, way of thinking and self-knowledge, the focus of the review should be on whether the information is true and whether the expression can be the responsibility of the person, rather than taking an uncertain technical score as a conclusion.

Different universities have different boundaries for AI.

American universities do not have a national set of rules for AI instruments.Joint Application System(Common App) requires applicants to confirm that the materials were completed by themselves, the facts are true and honestly presented, and that submitting content generated by others or tools as their own work may touch the application fraud policy.

Yale's rules are more specific: you can use AI to check grammar and spelling, or you can seek general advice or topic inspiration at the beginning of writing. Submitting substantive content generated by the AI platform as your own application materials is considered fraudulent and may lead to revocation of admission or withdrawal from school.

Brown UniversityIt is also emphasized that the content of the application must be a true reflection of the applicant. Kaplan's 2025 survey of admissions officers shows that more and more schools are clarifying AI boundaries, but the exact statement is still not entirely consistent. Applicants cannot only deduce that the target school is allowed by "other schools are allowed", but should be subject to the official website, application commitment and topic description of the year.

How Chinese Students Use AI Without Losing Self

For Chinese applicants, the easiest AI help is to reduce grammatical errors, check whether a sentence is difficult to understand, or point out where the text is repeated. These jobs are similar to asking teachers or classmates to proofread, provided that the school policy allows them and the revised sentences still express the applicant's own experience and meaning.

The risk often comes in the next step: students throw the entire first draft to the model for "full optimization" or simply let it produce a more conflicting and moving story. Such texts are prone to vocabulary that applicants would not normally use, overly neat rhythms, vague sublimation of values, and even fictional experiences in details.

The safer method is: first, the students themselves determine the theme, list the specific time, scene, characters, actions and reflections, and complete the first draft that can be clearly explained orally; If you use AI, only let it point out the problem and not let it replace the experience and conclusion. Finally, check sentence by sentence to ensure that every sentence can be explained face to face and every fact can stand checking.

Applicants also need to be informed about the third-party advancement advisor or clerical service model. Even if it is written by a third party or rewritten by AI, it is the students who ultimately make a commitment to the authenticity and originality of the application materials. "I don't know AI the consultant used" usually does not eliminate the applicant's responsibility.

When fluent writing becomes cheap, truthful information becomes more valuable

This study looked at only one master's program, which is not directly representative of U.S. undergraduate enrollment, nor can it tell us how each university will handle AI documents. It records a change that has already begun to affect applications: when correct grammar, clear structure and smooth expression can be obtained by tools at low cost, this is no longer a scarce signal.

Admissions officers will pay more attention to information that is difficult to mass-produce: what the applicant did in what scenario, why he made that choice, what small and real resistance he encountered, and how his or her understanding changed after the experience. These contents may not be gorgeous, but they can help the reviewer observe a living person.

AI can be a proofreading tool in the writing process and can also help applicants find gaps in their expression. But the hardest step in applying for a paperwork is never to write a sentence like a publication, but to decide which experiences are worth writing about, why they are important, and how you are willing to be responsible for them. If you give this step to the machine, you will lose not only your personal tone, but also the reason for the existence of the application document.

Reference material:

https://arxiv.org/abs/2609.22549

https://admissions.yale.edu/ai-policy

https://admission.brown.edu/apply/how-apply/integrity-application-process

https://www.commonapp.org/files/Common-App-Fraud-Policy.pdf

https://kaplan.com/about/press-media/college-admissions-officers-survey-2025-essays-ai