AI providers as criminal essay mills? Large language models meet contract cheating law

ABSTRACT

Many jurisdictions have passed very broadly drafted laws to tackle academic integrity issues, criminalising the provision or advertising of contract cheating or essay mills, such as the Skills and Post-16 Education Act 2022 in England and Wales. Recently, AI models such as chatGPT have amplified academic concerns. Here, we look at the intersection between these phenomena. We review academic cheating laws, showing that several may apply even to general-purpose AI services like chatGPT, without knowledge and intent. We identify a range of illegal adverts for AI-enhanced essay mills, and illustrate how difficult it is to draw the line between writing an essay and supporting it, such as by generating bone fide references. We also outline the consequences for intermediaries hosting these ads or providing these services, which may be significantly affected by these primarily symbolic laws. We conclude with a series of recommendations for policymakers, legislators, and education providers.

KEYWORDS:

• Contract cheating
• essay mills
• large language models
• AI
• academic integrity

Introduction

Providing or advertising essay mills and contract cheating has been a criminal offence in England and Wales since the Skills and Post-16 Education Act 2022 (the Skills Act). This follows similar legislation passed in multiple over the last two decades, amidst calls from organisations such as the Council of Europe for these very laws.¹

Also in 2022, OpenAI released chatGPT, a large language model stimulating academic integrity concerns. They were rapidly followed by other providers such as Google (Bard), Anthropic AI (Claude), and Meta (LLaMA). These technologies can provide varied assistance to individuals completing assignments – from research and editing to substantial or complete authorship.² The education sector has rushed to create policies and practices in the wake of this disruption.

These two events were not linked, but they interact.

Essay mills and contract cheating services rarely advertise explicitly as such. They typically claim to offer general educational support. Consequently legislative frameworks have often cast offences widely to attempt to bring services within scope. This often has led to few if any intent requirement, and additional offences relating to advertising.

A wide net has difficult to delimit edges. ‘General purpose’ AI language models do not make claims to specific functionality but can be deployed in a wide variety of ways by end users, including for the creation of assessment materials. Like essay mills, their providers typically have commercial motivations. This leaves a distinction between the two difficult to draw, and yet to be explored. In this paper, we explore this possibility of liability, and its practical consequences for AI providers and varied intermediaries, across multiple jurisdictions.

After a background on essay mill laws, and a review of legislation across jurisdictions, we outline the emerging AI business model in light of these regimes. Analysing the tensions we find, we conclude with directions for policymakers and thoughts on the future of these two developing trends.

Background: contract cheating and essay mills

Contract cheating can be defined as students submitting work for academic credit, for which a third party provided non-permitted assistance, typically for payment.³ Student cheat for many reasons, from academic pressure, a lack of capacity for full-time study due to financial needs,⁴ or limited language skills and influential online advertising,⁵ threatening degree standards and the integrity of the higher education sector.⁶ Such firms on occasion even blackmail and extort students with the threat of reporting their misconduct.⁷ To seek to escape liability, essay mills often state in their terms and conditions that work should only be used as a point of reference.⁸ Opportunities within programmes for misconduct also grew following the growth of at-home examinations during and after Covid-19.⁹

Emergence of contract cheating law

Essay mills have long been of legislative concern. California addressed essay mills in its Education Code in 1976, and other states followed. Many of these laws are decades old.¹⁰ Regulation in other common law jurisdictions came later, including New Zealand, Australia, Ireland, and most recently, England and Wales. Other jurisdictions, such as Canada, have seen legislation proposals, as early as 50 years ago, which failed to materialise.¹¹ Some civil law countries, such as Austria, have also recently in 2021 addressed essay mills in their code of education, although as an administrative rather than criminal offence. We look at this legislation below.

Analysis of legislation

Starting with the early examples, we find the United States of America. In the US, education is mostly a state matter. 17 out of 50 states address contract cheating as part of a legal regime.¹² These states are Massachusetts,¹³ Pennsylvania,¹⁴ Nevada,¹⁵ New Jersey,¹⁶ Colorado,¹⁷ Connecticut,¹⁸ Maryland,¹⁹ Oregon,²⁰ Washington,²¹ Virginia,²² Texas,²³ New York,²⁴ Florida,²⁵ California,²⁶ North Carolina,²⁷ Illinois,²⁸ and Maine.²⁹ As examples, we will look at California, Maryland and Florida. We draw on select components of the remaining laws later in the analysis where relevant.

From 1976 onwards, the California Education Code prohibited any distribution, preparation and selling of, as well as offering or causing to prepare, any written material for compensation at higher education institutions.³⁰ The person must have known or reasonably known that the material would be submitted for academic credit. If a person makes or disseminates any statement, regarding the above-mentioned acts, with the intent to induce any other person to enter any obligation relating thereto, they will incur liability.³¹ Courts may grant adequate relief.³²

The offence in Maryland has a significantly wider scope. Its base offence states that ‘[a] person may not sell or offer for sale any assistance in the preparation, research, or writing of an academic paper if he knows that the buyer intends to submit the academic paper substantially unchanged’.³³ This breadth, which includes preparation and research, rather than just writing, will be explored more later in this paper.

The offence in the Florida Statutes is similar but classified as a second-class misdemeanour which can be punished with a prison sentence of up to sixty days (or a fine of up to 500 USD).³⁴ It adds further offences of advertising, where the seller or advertiser knew or reasonably should have known the assignment was intended for submission by a student, and explicitly excludes assistance.³⁵

Beyond the United States, in New Zealand it is an offence under the Education and Training Act 2020 to provide or advertise cheating services. This offence was first stated as part of the now-repealed Education Act 1989, based on an amendment in 2011.³⁶ There are no substantive differences between the two versions. Yet in contrast to many other regimes studied there is a general intent requirement to give a student an unfair advantage, which has been criticised, including in the UK House of Lords where it formed the basis of an (unsuccessful) 2017 proposed law.³⁷ Liability in New Zealand is limited to a maximum fine of 10,000 NZD.³⁸ The New Zealand Qualifications Authority (NZQA) has standard investigatory powers, such as entering premises,³⁹ and can issue compliance notices,⁴⁰ but has no direct prosecuting powers. This offence has seemingly been applied only once in 2014.⁴¹ Proceedings were initiated after the NZQA received allegations that a company, Ateama Limited, was selling assignments to Chinese students. The case was resolved through a court confiscation to seize the property of a contract cheating company for further investigation.⁴²

In Ireland, the relevant legislation criminalising essay mills and contract cheating is the Qualifications and Quality Assurance (Education and Training) Act 2012, s 43A. The amendment was inserted in November 2019 by the Qualifications and Quality Assurance (Education and Training) (Amendment) Act 2019 (32/2019), s 15, S.I. No. 540 of 2019. It is based on the New Zealand offence⁴³ but different in that it has a variation without an intent requirement, with differing penalties. In Ireland it is similarly illegal to advertise or publish an advertisement relating to any of the acts above. Intermediaries may be liable, subject to any liability shielding, e.g. awareness.⁴⁴ Quality and Qualifications Ireland (QQI), the quality-assurance body, can initiate prosecutions.⁴⁵ Those convicted can be fined up to 100,000 EUR and/or imprisoned for up to five years.⁴⁶

In Australia, the Tertiary Education Quality and Standards Agency Amendment (Prohibiting Academic Cheating Services) Bill 2019 amended the Tertiary Education Quality and Standards Agency Act 2011. New criminal offences were to provide or advertise an academic cheating service on a commercial basis.⁴⁷ The penalty is 2 years imprisonment or 500 penalty units, or both.⁴⁸ Imprisonment is only an available penalty if the arrangement had a commercial purpose.⁴⁹ Proof of provision to a specific student is not necessary, limiting the evidential burden.⁵⁰ Advertising academic cheating services is prohibited with the same penalties.⁵¹ The Tertiary Education Quality and Standards Agency (TEQSA) is responsible for administering the law. It can take out injunctions against overseas websites, pursue prosecutions and gather intelligence.⁵² In the 2018–2019 budget, the government provided one-off 1.1 m AUD and 660,000 AUD in ongoing annual funding to respond to cheating activity.⁵³ The authors could find no other jurisdiction where a dedicated budget was provided to combat academic cheating activity.

This regime has seen enforcement activity, primarily by blocking access to the websites attracting the most traffic.⁵⁴ TEQSA monitors about 600 websites and blocked access from Australia to 40 websites which commanded about half a million visits every month in 2022. Even though the legislation was enacted in 2019, compliance action increased immensely in 2022. TEQSA was able to block websites without court action because of protocols negotiated with Australia’s major internet service providers (ISPs) through their representative association.⁵⁵ This was significantly more effective than relying on court orders. In October 2021, there was one injunction against an essay mill, likely in India.⁵⁶ The injunction applies until 2026, is binding on many Australian ISPs and requires them to block access to the domain names, internet protocol (IP) addresses or uniform resource locators (URLs).⁵⁷ TEQSA also exchanges information about contract cheating with Ireland.⁵⁸

Austria introduced an offence for ‘ghostwriting’ in higher education in 2021 as an amendment to the university law.⁵⁹ It is an offence to produce a work for another person, whether for a fee or without, or to make it available to another person if the provider knows or can assume based on the circumstances that it will be used in part or in full for any paper, exam or thesis (including artistic work). In effect, the offence contains a knowledge requirement with a low threshold. Fines can reach 25,000 EUR,⁶⁰ or 60,000 EUR and a maximum of four weeks imprisonment where there is intention of generating ongoing income.⁶¹ The lower penalty is an administrative offence (Verwaltungsübertretung) rather than a criminal one, prosecuted by the local district administrative authority.⁶² However, the scope of the offence excludes non-universities, even excluding universities of applied sciences (Fachhochschulen), private universities and teacher training colleges.⁶³ Reported effectiveness appears low, with the same ghostwriter agencies appearing in 2023 in in Austrian Google search results,⁶⁴ although some providers refusing clients with Austrian email addresses.⁶⁵

Most recently, in England and Wales, the provision and advertisement of relevant services was criminalised under the Skills and Post-16 Education Act 2022 (the Skills Act). Liability is not on students⁶⁶ but on entities commercially providing⁶⁷ relevant services – completion of all or some of a student’s work such that it can no longer be considered that student’s work.⁶⁸ The jurisdictional scope is a little confusing due to devolution of education but not criminal law – offences can be committed in either England or Wales but only in relation to students at English institutions.⁶⁹ The provisions were not initially in the Skills Bill, but added in the Lords as a concession after debate, following the similar issue raised in a Private Member’s Bill by Lord Storey,⁷⁰ as well as campaigning by the Quality Assurance Agency for Higher Education (QAA).⁷¹

The scope of this regime differs in important ways and as such we present it in more depth. Providing material to students in connection with the assignment where the material could be used to complete a part of or the whole assignment is in scope.⁷² This wider conception of ‘relevant services’ is different to other legislation we analyse in this paper. The material must either have been prepared in connection with the assignment or not previously published generally.⁷³ This means generally available without payment, or available in general published educational material.⁷⁴ This is aimed at excluding tutoring services and legitimate support.⁷⁵ Relevant assignments are both those which need to be completed as part of the student’s course or those needed to receive a qualification for the course.⁷⁶ Subject to conviction, liability will lead to a fine.⁷⁷

It may be a defence to prove that the provider of relevant services did not and could not have known with reasonable due diligence that the student would either use the material, had to complete the assignment personally or that assistance was not permitted.⁷⁸ However, this cannot be proven simply by a statement or agreement that the student may not submit the work.⁷⁹ As such, blanket exclusions of liability in terms and services are insufficient.

Advertising a relevant service, even if the advert is also shown to persons other than students, is a separate offence.⁸⁰ If an incorporated company is seen to engage in these services and a director or manager has consented to these acts, they will be liable.

Any investigations and prosecutions fall to the police and the Crown Prosecution Service (CPS),⁸¹ a body likely less incentivised to prosecute essay mills than a dedicated education regulator, particularly given there is no clear body to investigate (unlike in New Zealand or Australia).⁸² Even informal referrals are likely to remain low as the QAA has recently stepped away from being the Designated Quality Body (DQB) for higher education in England, as it could not apply England’s lower standards of assessment and transparency to English institutions while remaining an internationally accredited regulator.⁸³ The role currently falls to the Office for Students as a fallback.⁸⁴

The essay mill aspects of the Act are regarded as mostly symbolic deterrence by some. It was designed as a supporting element to any pre-existing measures of policing essay mills and contract cheating by universities or otherwise.⁸⁵ It clarifies the ethical problems of cheating to students⁸⁶ and prevents essay mills from claiming they are not illegal to use.⁸⁷ The advertising aspects of the Skills Act may be more influential, as such entities might be easier to identify and prosecute. However, large-scale prosecutions seem unlikely, and, contrary to Australia’s approach, foreign providers are not dealt with.

In terms of the effect of the prohibition on advertising, since 2018 the QAA has engaged in efforts to ban essay mills and contract cheating providers from using services like Facebook, YouTube, and Google or advertising through them.⁸⁸ The QAA successfully lobbied PayPal to establish plans to remove essay mills from using its site.⁸⁹ Consequently, intermediaries may find themselves more explicitly steered to avoid such content. They may be liable, particularly if they are made aware of such advertisements, and lose their liability shielding under the UK intermediary liability regime.⁹⁰

AI and essay mills, intertwined

Capabilities of large language models post a range of challenges to academic integrity. Models can create plausible text relating, with claims they can pass some fields’ examinations.⁹¹ Students can ‘cognitively offload’ onto models,⁹² which may autocomplete, check, suggest or structure substantive content of their work in a way that it may no longer be fully their own.⁹³

Students can access models in varying ways. These ways reflect different configurations of the AI supply chain which structures this emerging industry and enmeshes many actors in its functioning.⁹⁴ We will look at these from least intermediation to most intermediation.

Students may seek direct access to models by downloading and using them. Some ‘open-source’⁹⁵ models like Meta’s LLaMA 2⁹⁶ or BigScience’s BLOOM⁹⁷ can be downloaded and run on users’ own computers, with advances in both hardware and model design making this increasingly feasible,⁹⁸ and apps serving as easy to use ‘wrappers’ around the more technical challenges of doing this.⁹⁹

Most students are however currently likely to interact directly or indirectly through cloud application programming interfaces, or APIs.¹⁰⁰ Models are hosted on another service, and apps or websites query that service. Students can directly create accounts to do this with models they download or pay the low-level providers directly for (making it similar to downloading them on their own device), but they are also likely to interact with these models through intermediaries. Model providers can serve models through cloud services include Amazon Bedrock, Google PaLM API, OpenAI Platform, Hugging Face Inference Endpoints, or Microsoft Azure AI.

Other app providers can also build services – including essay writing services – on top of these services, tailoring them to specific customers and needs, and market these directly to students. The best-known direct consumer facing service is chatGPT from OpenAI, which queries its GPT-4 model, but other providers also provide versions of chatGPT both within the same website (including tailored for essay writing), and on other apps. AI essay writing services will typically consist of a high performance model from a major model provider, perhaps tweaked by the application user, hosted on a major cloud platform, being queried on-demand on the basis of student requests with parameters designed by the app provider to give desired results. Some of these apps may even be extremely household – such as Microsoft’s intention to integrate GPT-4 and similar technologies into Office 365.

Figure 1 illustrates some of the data flows between an individual (on the left) querying a model, and an individual getting a response to their query (right), amidst the many actors in the AI ecosystem. When we talk of an ‘AI provider’, we might be talking about the services that provide the mode, that finetune and deploy it, that provide an interface to it alongside multiple models and services, or a mixture of the above.

Figure 1. Diagram illustrating the networked nature of data flows and actors within the AI supply chain. Adapted from Cobbe, Veale and Singh (2023). Source: Jennifer Cobbe, Michael Veale and Jatinder Singh, ‘Understanding Accountability in Algorithmic Supply Chains’, Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery 2023) <https://doi.org/gsb98p>.

These supply chains are likely to be configured in varied ways in relation to services undermining academic integrity. Essay mills may themselves be AI providers, pretending to sell essays that are written by humans, but created with AI. Plugins and software designed to allow students to cheat on at-home exams proctored or invigilated using ‘lockdown browsers’ or webcams may emerge (although effective methods which defeat these techniques, such as running browsers within virtual machines, already exist). Specialised models and interfaces which accept documents, PDFs, presentations and similar make writing references and integrate with the content uploaded already exist and will improve. These technologies may be available through apps or web interfaces for a cost or may be downloadable, through free and open-source software, and/or interface directly with the APIs of model providers.

To help us look forward, we can look at what is being advertised and offered today. In the following section, we review a wide array of companies that offer services online that seemingly integrate the essay mill business model within an AI business model.

Companies facilitating AI academic cheating services

A staggering array of services, both free and paid-for, claiming to use AI currently present themselves to students. We researched and categorised an array of existing services in October 2023 and describe them below. To do so, we drew on both searches in using conventional search engines and also the advertising archives required of large platforms by the EU Digital Services Act (DSA).¹⁰¹ When queried with terms such as ‘essay’, ‘university’, ‘coursework’, ‘thesis’ and ‘dissertation’, a wide array of providers are shown which could be then examined both on that platform and elsewhere.

AI essay writing services

Some firms operate similarly to existing contract cheating or essay mills, but claim to use AI. This may initially seem like a downside, but firms appear to use this to differentiate on speed and price. EssayAid claims to write an essay ‘in 24 hours with the power of AI’. There is scarce information on its website, although it advertises on TikTok and is registered in the United Kingdom. Its domain name is registered by an English company, and therefore would likely be in breach of both the relevant service provision and advertising provisions of the Skills Act (Figure 2).

Figure 2. EssayAid claims to be registered in the United Kingdom, and advertises on TikTok. Source: TikTok Ad Library <https://library.tiktok.com/ads/detail/?ad_id=1761732337777713> accessed 4 September 2023

.

Some services are more interactive. EssayGenius (Figure 3) offers an interface which allows individuals to enter a prompt and receive an essay in return, and to restructure and rephrase parts of the essay.¹⁰²

Figure 3. EssayGenius asks individuals for a prompt, and writes a short essay on the basis of this. It offers more functionality to paying members, including the ability to suggest, complete and rephrase terms.

AITaskWizard is a similar service, advertising itself as ‘perfect for A-level and GCSE students’. It promises an ‘inbuild humanizer’ which ‘can be useful for avoiding plagiarism’. It charges 3.99 USD/month for a premium plan. It does not have terms and conditions, a location, and it shields its WHOIS domain registration entry. However, it does advertise to the UK on TikTok, where it claims to be registered in the United Kingdom. The website is simplistic and has several idiosyncratic features, suggesting that this is not a corporate effort, but may be an individual simply placing a wrapper around a large firm’s AI API and repurposing it with a user interface, and potentially some post-processing.

While all aforementioned are websites, several apps also exist. This introduces a new intermediary, the app store, which plays a significant role as a de facto contemporary regulator.¹⁰³

One app is Friday: AI Essay Writing, produced by the Swiss company Sekterra GmbH. This firm heavily advertises on TikTok (Figure 4), mainly to indicate that its texts pass plagiarism detection that others do not. The adverts heavily use images of classrooms, assignments, and teachers detecting chatGPT-written scripts.

Figure 4. AI Task Wizard’s simplistic website, centre, with one of several advertisements that ran on TikTok, left. Source: (TikTok Ad Library <https://library.tiktok.com/ads/detail/?ad_id=1758267600437297> accessed 4 September 2023) Right: an advert for Friday: AI Essay Writer. Source: (TikTok Ad Library <https://library.tiktok.com/ads/detail/?ad_id=1768865302800401> accessed 4 September 2023).

Plagiarism camouflage services

Some services advertise primarily that they help students rephrase texts in a way that will confound detectors such as Turnitin, rather than firms which claim their AI-generated texts are resistant to detection by default.

Quilbot, a company owned by Course Hero (trading name of Learneo Inc., which also owns CliffNotes), sells a service with which students can use to re-write AI-generated texts. This service, it appears, assists students in avoiding plagiarism detectors – their premium version (100 USD a year) resells the services of CopyLeaks, which is a plagiarism detector like Turnitin, allowing students to check whether their text would trigger a detector or not. The service is based in Illinois, which has laws prohibiting essay mills,¹⁰⁴ and advertises in the United Kingdom (Figure 5).

Figure 5. Left: Quillbot allows individuals to paste text and have it rewritten, in a ‘more scholarly way’. It offers a premium service for 100 USD a year. Centre: Quillbot advertising on Meta services with a ‘smart start’ feature to ‘create an outline’ that is AI generated, allowing you to write essays in ‘under an hour’. Source: (Meta Ads Library https://www.facebook.com/ads/library/?id=169125769545892 accessed 4 September 2023). Right: Quillbot advertising on Google services as an ‘article rewriter’. Source: (Google Ads Transparency Center, <https://adstransparency.google.com/advertiser/AR15286630488773492737/creative/CR18265641898590863361?region=GB> accessed 4 September 2023).

It is worth noting that the advertising offence would appear to extend to Google and Meta, where liability may accrue when they are notified and if they refuse takedown. While Quillbot offers entire writing services, including services to rewrite and cite sources, the re-writing feature is worth considering. Neither English nor Illinois law explicitly contemplates services which attempt to fool attempts to detect plagiarism. However, under English law, this service can be conceived as completing ‘part’ of an assignment in a way that ‘could not reasonably be considered’ to have been completed by the student, as students needing to rephrase to avoid plagiarism detectors are unlikely to be able to be considered that they are completing the work independently.

If Quillbot avoided saying exactly what it was to be used for, Phrasly.ai (Figure 6) has no such qualms. They have published blogs with the names ‘How to Bypass Turnitin’¹⁰⁵ and ‘How to Write an AI-resistant essay’.¹⁰⁶ The latter claims, somewhat confusingly, that such essays, which they recommend generating with AI and then running them through their own AI system, ‘represent academic integrity, assuring authenticity and originality’. The firm even offers a refund if their content is detected by a detector such as Turnitin.

Figure 6. Right: Phrasly.ai states that it is designed to ‘transform AI-generated content’, ‘boost grades’, and cites the specific assessment of a ‘Doctorate’ in its writing style menu. Left: Phrasly advertising on Google. Source: (Google Ads Transparency Center <http://adstransparency.google.com/advertiser/AR06266791351438802945> accessed 7 September 2023).

The firm Phrasly LLC is based in Delaware, which to our knowledge does not have relevant legislation. However, they advertise in the United Kingdom using Google (Figure 6), claiming to ‘Bypass AI Detectors’, triggering English law in the same way as stated above.

A similar company to Phrasly, EasyEssay.ai (Figure 7), offers similar generation and plagiarism detection services, and offers citation services within its generation tool. It claims to be operated by a Hong Kong company, ShannonAI Technology HK Limited (2772858). No adverts could be found for this firm.

Figure 7. EasyEssay.ai’s interface.

Interestingly, all the above firms have relatively generic, boilerplate terms and conditions, and did not at the time of writing disclaim the use of their services for the submission of assessments as other essay mills are reported to do.

Writing and referencing services

Several services encountered placed a premium on being able to integrate genuine sources, citations and references into their essays. This is important as educators often anecdotally believe that AI systems are incapable of detecting accurate references, allowing real work to be distinguished from generated work.

Words and Paper is an app that claims to integrate web searches to generate referenced essays that pass plagiarism checks. It specifically namechecks Quillbot, discussed above, as being unable to do so to distinguish itself on TikTok. TikTok notes its registered location as the United Kingdom, although its privacy policy and terms and conditions on the Apple App Store link to a shared Google Doc. It advertises itself as being able to generate an outline, which users edit, and which it expands using AI-generated text. To that end, it appears to resemble systems like AutoGPT, which integrate the querying of large language models with the introduction of new text as a result of the language model itself directing a Web search (Figure 8).¹⁰⁷

Figure 8. TikTok advertisements for Words and Paper. Source: (TikTok Ad Library <https://library.tiktok.com/ads/detail/?ad_id=1763589432655922> accessed 4 September 2023).

Scite (Figure 9) sells a tool that claims to allow researchers to see more about the context in which certain work was cited. However, its ‘Assistant’ tool, similar to Words and Paper, claims to check a wide range of academic sources by generating search terms, then querying papers, and generating a reference list and in-essay citations to match. The firm advertises explicitly to students on Meta¹⁰⁸ and on Google,¹⁰⁹ although with no explicit mentions of essays.

Figure 9. Left: the interface of Scite Assistant, with output to the prompt ‘Write an essay on plagiarism, essay mills and the law in different jurisdictions’, featuring references that can be examined in context on the right hand side. Right: advertising copy from Scite on its website, suggesting its use in writing an essay.

BrainstormGPT is a Singaporean company (Cyberspace Imagination Pte. Ltd.) which offers meeting summarisation and reporting tools on its website. However, their advertisements tell a different story (Figure 10), claiming to ‘get an essay in 20 minutes’ and provide ‘reliable sources’ and ‘fresh ideas’, providing ‘more than just essay structure’.

Figure 10. Advertisements for BrainstormGPT on Twitter and on Meta (Instagram).

Finally, Source.ly invites users to copy and paste work they have generated using AI tools into its system which will return a list of online references which they can use to make it appear that they have referenced those works to create the essay. They state on their website that their ‘mission’ is to ‘save students hours of research’.¹¹⁰ Their service, like many others, features a plagiarism checker with a report score that can be viewed before an assignment is submitted. The location on their LinkedIn profile is set to New York, and their website lists the logos of many universities as users (Figures 11 and 12).

Figure 11. Advertisement on TikTok for Sourcely. Source: (TikTok Ad Library https://library.tiktok.com/ads/detail/?ad_id=1757088495907889 accessed 4 September 2023).

Figure 12. Page listing Source.ly’s users. Source: (Sourcely <https://www.sourcely.net/old> accessed 4 September 2023).

Interim conclusions

In essence, we see a significant number of providers of AI assisted services that could be considered contract cheating or essay mills. Many of these explicitly advertise and market themselves in this fashion, although not all do. Some services explicitly offer essays, and even advertise themselves as being able to dodge AI detectors (e.g. Source.ly, Words and Paper, Phrasly, Friday or AITask Wizard). Some, such as Quillbot or EasyEssay, are a little more subtle, offering plagiarism detection checks, and in the case of Quillbot, advertising to create outlines, write in a scholarly manner and write faster, but do not go so far as saying they are designed to avoid such checks. However, they offer the same service. Others (e.g. Scite, BrainstormGPT) advertise to students, and showcase their capabilities at writing an ‘essay’, but seem to aim themselves at a research or enterprise market, and do not engage with the misuse of their tool, even though they design the system to reference in the style of an academic paper.

However, it is worth noting that most of these tools appear to be wrappers around the language models trained by others. It appears they pass or rephrase prompts, or use chains of prompts to get different results, but these are all techniques that users can do with general purpose providers alone, particularly with the help of prompting guides and similar. As we will see below, given the intent requirements are loose in some jurisdictions’ laws, this raises a challenging question. If these tools should be considered academic cheating service providers – then where is the line exactly drawn?

Tensions

It is likely that students will increasingly use AI for academic cheating rather than essay mills, because of the lack of risk for blackmailing as well as the free availability of many increasingly sophisticated AI models. Commentators have argued that in the future essay mills may seek to differentiate themselves against AI, machine-written services, as students may only be willing to pay for higher quality services.¹¹¹ However, as we have seen, it also seems likely that essay mills incorporate AI services. Consequently, the two may become more blurred than distinct. While essay mills may not become irrelevant, cheating employing AI models may become the predominant form of third-party cheating.

We will analyse the issues of addressing academic cheating using AI through current essay mill legislation, keeping in mind there remains an ongoing discussion about the extent to which AI should be classified as academic cheating. We, therefore, focus on times when the substantive use of AI has been expressly forbidden.

This section will contain a categorisation of the main tensions between AI systems and the various legal regimes, indicating the relevant jurisdictions.

The main tensions which emerge are:

1. relevant services;

2. knowledge and intent;

3. commercial nature;

4. extraterritoriality;

5. intermediaries/advertising; and

6. enforcement.

Scope of relevant services

What counts as a relevant service differs by instrument, which we examine here in context of the empirical work on the market presented above.

Students may use an AI model for high level tasks such as outlining or searching for resources. They might use the assignment prompt, potentially with ancillary information and documents, to write the essay to receive an outline for an assignment or to search for source material, i.e. for high-level tasks. They may use an AI model to write the entire assignment by giving some input with desirable information (collated by the students themselves) or even just the assignment prompt, letting the language model write everything. A student may co-write an essay with AI by continuously amending each output and the student writing many elements themselves. An AI model may also be used merely to improve the clarity of an assignment by cross-checking whether the essay accurately corresponds to the question. For the least substantive, and perhaop, they may improve phrasing, grammar and style, much as spell-checkers built into word processors have long facilitated.

AI systems may facilitate one or more of these use cases. Under what conditions, if any, could this fall into the scope of the legislation outlined above? As we will see, this differs widely.

Support or assignment production?

Even high-level assistance may be a relevant service in some jurisdictions. We categorise jurisdictions by their scope in this regard below, and summarise in Table 1.

Table 1. Maximal material scope of the legislation considered.

Metaphor	Most broad nature of the work implicated	Jurisdiction
Preparation, research or assistance	research results or substantial material […] will be used	Massachusetts
	any assistance in the preparation of an assignment	Oregon
	preparation, research, or writing	Florida (see discussion), Maryland
Reasonably considered to constitute part of	reasonably regarded to be or form (substantial) part of a task	England and Wales, Australia
	work undertaken in whole or in part	Ireland
Partial assignment	used in part or in full to prove achievements	Austria
	in whole or substantial part	Connecticut, New Jersey, Pennsylvania
	(whole) assignment, answers, papers, dissertations	New Zealand, Maine, North Carolina, Nevada
Full assignment	work not substantially altered	Florida (see discussion), Illinois.
	work able to be/intended to be submitted	California, Colorado, New York, Texas, Virginia, Washington

Preparation, research or assistance

Maryland specifically prohibits sale of assistance in ‘preparation, research, or writing’.¹¹² Oregon widens this to ‘any assistance’.¹¹³ Massachusetts includes in the concept of the work that could be submitted ‘research results’ or ‘substantial material therefrom’.¹¹⁴ All of these imply an extremely wide scope; however, they are all tempered by having a knowledge requirement (discussed below). Florida has a potentially wide scope, although parts of the provision seem internally contradictory. The offence itself states that the scope should be offering work that the seller or advertiser should have known was intended for submission without substantial alteration. However, exempt services do not include those which include ‘the preparation, research or writing’ of work (emphasis added) – widening the scope from simply the creation of final or near-final content.¹¹⁵

Partial, or reasonably consider to be part of

A ‘part’ could be understood as a ‘part’ of a document, such as a chapter or section, or as a ‘part’ of a task (as Australia’s legislation explicitly does). In assessment, the task being assessed is often the process not just the final piece alone – hence the restrictions on how writing can be produced, with the artifact serving to evidence the task. Connecticut, New Jersey and Pennsylvania explicitly include occurrences where a ‘substantial part’ of the assignment has been obtained. Ireland’s legislation supports a similar construction, where a person is ‘undertaking in whole or in part’ an assignment or other work. Austria’s legislation is narrower focussing on ‘achievements’, although it is plausible a process might be part of this. However, as Austrian legislation talks oof producing or making available an entire piece of work (ein Werk) for another person, this may exclude more distant preparation support.

Both Australia and England and Wales relax the definition of ‘part’ further by adding a test of reasonable perception, with England and Wales discussing where ‘all or part of an assignment on behalf of a student’ where an assignment in that way ‘could not reasonably be considered to have been completed personally’ by that student; and Australia the stricter requirement that the service can ‘reasonably be regarded as being, or forming a substantial part’ of that assessment ‘task’.¹¹⁶

Submittable, full assignments

Other legislation seems to rule out such high-level guidance or components. While California includes any ‘written material’, it must be able to be submitted as the assignment itself. Colorado, New York, Texas, Virginia and Washington have similar constructions. In New Zealand, the services are limited to those where assignments are ‘completed’, ‘provided’ or ‘arranged’. The limit to, in varying wording, the whole assignment can also be found in Maine, North Carolina and Nevada. This does not rule out AI-created assignments but does narrow the scope to where systems produce finished or near-finished pieces, rather than used interactively and/or iteratively, such as ‘co-writing’ tools or outlines. Florida, if read restrictively without reference to the mentioned exemptions, and Illinois also require the creation of work not substantially altered. While judicial interpretation may choose to include components of the work, these statutes appear more restrictive than others we examined.

Availability exemptions

In cases where the scope is wide, legislation often includes exemptions concerning the availability of work. Exempting a mass-produced study guide is one obvious logic behind such an exemption, but as AI systems are also ‘available’, might they benefit from this?

In England and Wales, it would need to be considered whether the material has been published generally or available generally without payment. The structure of the offence is unusual, as not being generally published material is a condition of an example, rather than a clear exception to the definition of service – however, it seems designed to narrow the offence.

Considering paid-for AI models, the only relevant exemption is that it would be ‘included in a publication’, which a private API response is not. Consequently, we turn to consider ‘free’ (potentially ad-supported or similar) systems.

While a free AI system is ‘available generally’, the content it produces is not, as it will typically differ each time (even with the same prompt) due to its stochastic nature, regular updates, and different starting ‘seeds’. While it could be argued that the material was ‘available generally without payment’, in the sense that an individual did obtain it, without exclusivity or payment, the Bill’s explanatory notes indicated that ‘[t]he offering of a menu of available essays which must be paid for, for example, would not be considered material published generally since it does not include other educational or training material’.¹¹⁷ AI systems similarly lack such other material.

Florida clarifies the law does not stop ‘any person or educational institution from providing [..] information [..] unless this service includes the preparation, research or writing of a report or paper as [outlined in the offence].’ This exemption is a little circular in nature, but the intention seems to be similar to the England and Wales exemption.

Commercial nature

England and Wales requires that relevant services be provided commercially. Florida’s law applies to ‘sellers’. This excludes free help from family members, and some might argue would exclude freely available AI language models like chatGPT, and only apply to paid versions. However, Internet law has long had to deal with firms that indirectly make revenue from users, with EU and UK law’s definition of an information society service as one ‘normally provided for remuneration’, but which has not limited the scope of intermediaries to exclude advertising-supported firms.¹¹⁸ ‘Open’ and ‘free’ AI language models are similarly deeply enmeshed in technology giants’ business strategies.¹¹⁹

Ireland’s, California and New Zealand’s laws lack the requirements for commercial circumstances and would therefore apply to both paid-for and freely available AI models in all interpretations, while in Australia and Austria, sanctions can differ.¹²⁰

In sum, constraints on the services being provided commercially do not seem to fully exclude free-to-access AI services in any jurisdiction examined, given their current business models, and in any case continue to include paid-for AI services.

Extraterritoriality

After Australia and Ireland criminalised essay mills, many reportedly left or closed down.¹²¹ Some German-language mills reportedly refused Austrian emails, but this is trivially circumvented.¹²² In an informationalised world, local restrictions matter little if you can obtain from an overseas service.

Regimes differ in their territorial extent and effect. Australian legislation includes specific powers to issue Web blocking injunctions (e.g. to local ISPs) in the context of academic cheating services while other jurisdictions have to rely on generic powers where they exist. Australia also applies ‘category D’ extended geographic jurisdiction, meaning that this offence applies whether or not the result of the conduct occurs in Australia.¹²³ In contrast, in England and Wales, offences must be committed in England and Wales in relation to English institutions,¹²⁴ while Ireland, New Zealand or Austria omit territorial discussion in their laws.

Most AI model companies are located in the United States. It would be politically difficult to block general-purpose services, and such a block could be challenged on grounds including freedom of expression or to conduct a business. It may also create economic damage domestically for businesses that rely on such APIs. Regulatory co-operation, such as the choice of OpenAI to stop serving Italy following an order by the Italian Data Protection authority, seems preferable for such issues, but requires regulatory leverage.¹²⁵ However, for narrow, illegal businesses, such as many essay mills, such a block may be able to be targeted with few externalities. Often however, such blocks are just whack-a-moles, as firms can easily change both their domain names and their IP addresses.

Advertising

Several jurisdictions analysed above include an offence criminalising the advertisement of relevant academic cheating services, as defined in their respective acts. Advertising offences in England and Wales, Ireland, Australia and New Zealand differ slightly but broadly apply to those advertising in connection with the in-scope services. Australia implemented an integrity unit to prosecute and make use of the federal injunction powers to block advertising, also through the aforementioned Web blocking powers.¹²⁶ Austria’s law is scoped slightly differently, and thus has less potential to apply to intermediaries.

Intermediaries, like Google, other search engines and other platforms may commit a criminal offence if they have relevant advertisements on their platform, particularly if they are made aware of it and do not remove it, as they typically then lose shields present in intermediary liability law.¹²⁷

Many of these platforms now make some form of ‘ad archive’ available, particularly following the requirements on Very Large Online Platforms now in force following the passage of the Digital Services Act.¹²⁸ These can be useful for research and accountability purposes.¹²⁹ A cursory look through these ad archives with relevant search terms shows adverts for a wide array of traditional essay mills, in addition to the adverts for AI-powered mills described in the previous section (Figure 13).

Figure 13. A wide array of adverts from the Meta Ad Library advertising ‘classic’ essay mills.

In England, after the passing of the Skills Act, the Skills Minister called on search platforms to crack down on the illegal advertisement of essay mills.¹³⁰ In reality, it appears to be the Advertising Standards Authority that has been the most active in investigating essay mill ads under the Committees of Advertising Practice (CAP), Advertising Code. Before the passing of the Skills Act 2022, the ASA investigated essay mill ads in 2019 for being misleading. Websites proclaimed that students could submit work as their own without risks, arguably in breach of the CAP Code (Edition 12) 3.1 and 3.3.¹³¹ Similarly, in February 2023 the ASA challenged whether an ad by BrilliantMinds, an essay mill, misled students by implying they could submit an essay they had bought as their own.¹³² Any ASA rulings have the result that the ads may not appear again in the form complained against. Usually, no fines result and services typically agree to amend the ads. No prosecutions against intermediaries were pursued by the CPS under the Skills Act, though this may have been because the companies were in Scotland and offering services to Scottish students.¹³³

Conceptually, applying the rules on advertising with this legislation is complex. That is because the often broadly drafted nature of these offences makes the service provided contingent on how it is being used, particularly because some jurisdictions, as discussed above, loosen or remove intent requirements. Yet advertising is all about intent – describing a service such that individuals wish to use or purchase it. This creates a difficult legal question – should an advertiser who advertises for a general-purpose service be held liable for it? Intuitively, the answer should be yes – these provisions were typically designed to capture essay mills even if they were advertising (as they do, and as they contractually seek assurance of) services which were different from the student’s intended use, such as practice essays, rather than submittable work.

In sum, if relevant services in such laws encompass general purpose AI systems, insofar as they do, they would seem to also prohibit advertising of such systems, regardless of how they were advertised. This could present a challenge for commercial activity of these firms, and the intermediaries involved in publicising them.

Enforcement

Most regimes are only directly enforceable by normal prosecuting bodies. Some of them allow direct referral by universities (e.g. Illinois), but this too is rare. Given the nature of the offence, this will likely lead to questionable levels of enforcement, particularly without dedicated enforcement bodies with knowledge of the higher education sector. In England and Wales, the CPS has not engaged in enforcement action, although the ASA has taken action against advertisements in the past. Of the few dedicated enforcement bodies, the Australian body, TEQSA appears the most active, with a dedicated yearly enforcement budget of 660,000 AUD.¹³⁴

Private enforcement and platform law

Direct enforcement is not all. Other laws interact to create interesting enforcement pathways. In particular, the EU’s Digital Services Act (DSA) and the UK’s Online Safety Act (OSA) place obligations on certain intermediary internet services designed for (typically) user-to-user content sharing.

Model providers can be intermediaries in a variety of ways. They might intermediate users and search results, thus acting somewhat as a search engine does (and as both Bing Chat and Google Bard do). Providers might host finetuned versions of their model which are illegal, such as finetuned versions designed for writing academic essays. For example, OpenAI hosts ‘Academic Assistant Pro’ as one of their top custom GPT models.¹³⁵

Two main categories of obligation are relevant for the interaction between essay mill law and platform law. The first concerns direct obligations that relate to illegal content. The two regimes differ a little in this respect; the OSA has obligations to use proportionate measures to detect and remove ‘priority illegal content’ and provides a notice and takedown regime for ‘illegal content’. However, it seems unlikely that the offences in the Skills Act will qualify as a model for consideration even as base-level illegal content, as the offence does not have individual victim(s) or intended victim(s).¹³⁶ The advertising offence might be understood this way, as the ‘victim’ is the individual being advertised to. The DSA has a much wider concept of ‘illegal content’, and national law essay mill law such as the Irish law examined above would seem to fall within its remit.¹³⁷

The DSA however has a somewhat lower set of obligations that relate to such content, and we will focus on these alone due to the scope of the OSA. It provides that platforms must create a mechanism by which they can be notified of suspected illegal activity, which in turn, if confirmed, would remove their liability shielding and mean that they may become liable for hosting this content, where the offence allows. Unlike the OSA, intermediaries do not find themselves directly liable under the DSA for failing to remove certain categories of content; their liability continues to flow from the underlying law that would make the content illegal. If they are unafraid of prosecution for hosting a general-purpose AI system that may be illegal under the above regime, then they are unlikely to remove it on these grounds due to the legal risk seeming acceptable. This is particularly the case given that general-purpose AI systems have not yet been targeted by the regimes outlined above. It is also not particularly new, as the main innovation of the DSA in this space is to require an effective way to provide notice to platforms in order to lead their liability shield to fail – notice and takedown regimes have been around for over two decades in the EU, and are the norm around the world.¹³⁸

Both regimes contain provisions which in effect oblige (some) intermediaries to proportionately enforce their terms and conditions.¹³⁹ AI providers often use contractual terms to try and govern downstream use.¹⁴⁰ This is a typical legal technique to manage supply chain risk. OpenAI, the producer of chatGPT, state that it is disallowed to use its services for illegal activity in its Usage Policies.¹⁴¹ Other AI actors include model marketplaces (e.g. Hugging Face), which store models uploaded by others and provide them as a service, or systems such as Google’s AutoML, which allows third parties to customise and tailor models.¹⁴² Model marketplaces like Hugging Face are the main location for the distribution of open source, powerful language models such as Meta’s LLAMA and Eleuther’s GPT-J, and in turn, are clear examples of internet intermediaries. Marketplaces such as Hugging Face have content moderation policies, which also forbid the hosting of illegal content.

Typically and historically, terms of service have been sporadically and inconsistently enforced. However, intermediaries can be held directly liable for failing to enforce terms proportionately, including terms which state that they forbid illegal content. Whether such liability is possible depends on the nature of the service and the scope of the OSA and DSA in relation to it. In these cases, intermediaries will have to interpret the laws above. Platforms will neither have recourse to courts to aid their interpretation nor to challenge any interpretation made by firms, were they to make take-down decisions. This illustrates how vague law with symbolic purposes may create unchallengeable and difficult-to-clarify legal effects.

We have already seen how intermediaries such as ISPs, PayPal and Facebook have been and can be targeted in relation to essay mills and advertising them. It seems that intermediaries may, again, be a place where these skirmishes play out, particularly due to their cross-jurisdictional nature.

Essay mills themselves using AI systems

In most cases, essay mills will allocate an individual employee who will write an original piece of work for the student in exchange for payment. As mentioned in the parliamentary debates by Lord Lucas, essay mills and contract cheating services also make use of AI.¹⁴³ In that context, he referred to the use of AI to disguise what essay mills ‘are creating based on existing sources’ to evade detection by cheating software. This practice is clear in our reviewed services.

The essay mill itself naturally falls within the listed laws. What about the AI service being utilised? It seems unlikely that the upstream AI providers themselves would be the first point of liability unless they were well-aware of the nature of the service being provided to the user. Given that the structure of such services is typically to remain ignorant of the detailed activities of their users, and to deal with them at arms-length through automated systems, there does not appear to be a significant current legal risk for these services. However, such analysis may change in the future, if these provided systems become enmeshed in more and more criminal activity, and laws and attitudes change.

Knowledge and intent

As indicated in the discussion in the House of Lords in 2017,¹⁴⁴ proving intent to provide an unfair advantage or any other intent or knowledge requirement is already difficult for essay mills and contract cheating providers. A knowledge requirement makes it easier for essay mills to hide behind a contractual clause stating that provided services should never be handed in as original academic work. A 2017 study analysing 26 websites offering such services found these types of clauses or disclaimers present on all websites studied.¹⁴⁵

Such clauses are similarly common in general-purpose AI providers’ contracts. For example, OpenAI’s usage policy includes a prohibition against ‘[f]raudulent or deceptive activity, including: [..] academic dishonesty’.¹⁴⁶ These clauses are part of broader attempts to govern AI systems and APIs through contractual means.¹⁴⁷ We found no AI essay mills with such clauses.

Though difficult in practice, it may be possible to understand the intent of essay mills and contract cheating providers through investigative methods, disclosure of correspondence, and testimony from employees. In contrast, the business model of general-purpose AI providers is typically one that is distant from the user, with the primary interaction being programmatic, through APIs. Unlike essay mills, language model providers are aiming in practice at a wide variety of users, with some likely to be using the system for illegitimate or illegal ends.

Consequently, for general-purpose providers with contractual exemptions, under standard factual conditions reflecting the AI-as-a-service industry today, an intent element present will likely shield them from liability.

Legislation without knowledge or intent elements

The offence in England and Wales lacks an intent element, is purely a strict liability offence and avoids any issues of proof around the offence.¹⁴⁸ This means that if the relevant services element is satisfied AI services, including general-purpose AI services, could likely fall within the Skills Act offence on academic cheating. In Australia, strict liability applies to the physical element of circumstance for the first two offences outlined, concerning the definition of academic cheating service.¹⁴⁹ Provision of work for students, in circumstances where the work forms or could reasonably be seen to form a substantial part of an assessment task that students are required to personally undertake, will lead to liability.

Both instruments make it difficult to distinguish between general-purpose AI providers and AI services advertising as essay mills, due to the lack of intent requirement. Given that AI essay mills seen above typically are a loose wrapper around general-purpose AI APIs such as those in the algorithmic supply chain, this creates significant problems for the law’s scope.

The Irish provisions have two sections on academic cheating, one with an intent element and one without. The section without an intent element applies primarily to undertaking ‘an assignment or any other work’ or ‘sitting an examination’ in ‘the enrolled learner’s stead’ or providing ‘answers for the examination’. ‘Undertaking’ an assignment contrasts with the intent required offence of ‘providing or arranging the provision of an assignment’ – indicating that the target of the first offence are people who are replacing individuals in actual examinations, and likely limiting the scope to exclude AI.

Legislation with knowledge or intent elements

Some legislation requires the provider to intend to give students an unfair advantage. This will be difficult to prove with AI providers. Both Ireland¹⁵⁰ and New Zealand¹⁵¹ have such intent elements in parts of their legislation – unsurprisingly, as the Irish Act was modelled after the New Zealand legislation.¹⁵² This might be proven for essay mills promising students a first, but it will be difficult to prove with general-purpose AI providers. However, it may be possible if they are specified resellers and advertise their services as such. Nevertheless, the difficulties of proof contributed to the removal of the intent requirement in England and Wales, as discussed.

Knowledge requirements are easier for AI providers to meet in principle but still pose challenges. Austrian law contains a knowledge element. It is required that you know or can assume based on the circumstances that a piece of work will be used in part or in full as a seminar, examination, or final thesis (bachelor thesis, scientific or artistic work). This is also the case for California and Florida, among other US states, including those with the wide scopes described above such as Maryland, Massachusetts and Oregon. However, AI providers receive prompts as inputs, and would presumably argue that they have no way of knowing whether this was just a prompt to do genuine research in authorised ways, to produce practice essays to study or examine (for example, on previous years’ questions), or illegitimately submit. Cost seems to play an indirect part here. It may be reasonable to ‘assume based on the circumstances’, as Austrian law indicates, that an essay will be for submission where a student is paying a significant sum for it. Where that sum is small, because the activity is fully automated, it seems plausible that they are just doing so for research, information or knowledge.

In sum, so far England and Wales and Australia look to have constructions that are amenable to application to general-purpose AI providers, based on scope and the lack of intent or knowledge elements. The section of Ireland’s offence without intent elements seems too specifically aimed at individuals fraudulently sitting exams. Austria and several US states have a knowledge, rather than an intent, requirement, which could be met by AI providers, but will likely prove difficult.

Interim conclusions

Australia’s and England and Wales’ legal regimes are most likely to apply to general purpose AI services which do not explicitly advertise as contract cheating or essay mill services, because they have at least one in-scope offence with strict liability, without a knowledge or intent requirement. It becomes hard to distinguish between firms that advertise as being able to write essays, and firms, such as OpenAI, Google and similar, that provide services that in practice have the same or similar functionality, but do not advertise in this way. This is made more challenging still as these regimes do not care whether the use of their technologies in this way is contractually forbidden through terms of service. As a result, it is clearly arguable that general purpose AI providers seem incompatible with some jurisdictions’ contract cheating laws.

In the bulk of the jurisdictions, the rules would apply to services that seem to advertise to students, claim to be able to write essays, or similar, depending on the scope of the assistance in the legal regime. Some providers simply look like high-tech essay mills, making it natural for them to fall under laws designed to target essay mills. But this is interesting because some tools go beyond essay mills or contract cheating to provide technical tools and services, and the distinction between research, outlining, co-writing and other functions is blurred. Effectively, familiar tools like spelling or grammar checkers expand to the substance of the work being undertaken. Drawing a line between form and substance is an ongoing task and remains unclear. Some regimes delegate that boundary management to the educational institutions, others to the legal system.

On top of this, enforcement regimes differ. The majority of regimes only allow prosecution by a normal prosecuting body such as a state attorney, or the CPS, although some regimes give some prosecution or investigative ability to regulators, or referral ability to institutions. The law in these areas may be understood as more symbolic than coming with the rigorous expectation of enforcement. However, making something a criminal offence interacts with other regimes, particularly online. In particular, it interacts with (1) the terms of service of general-purpose AI APIs; (2) the contractual licenses attached to ‘open-source’ models, such as the OpenRAIL licenses; and (3) obligations placed on platforms under laws such as the OSA and DSA, among others. These may prove to be an impactful set of interactions, even without direct enforcement.

Discussion

In a wider context, we will briefly discuss issues of attaching responsibility to tech companies more generally and offer a possible explanation for doing so. We will briefly explore why intermediary liability and recommender systems failed to attract liability for content on their platforms and compare why AI services’ liability for offering contract cheating on their platforms is different to that factual scenario. This may change the assessment of liability under the legal regimes. With these considerations in mind, several amendments are suggested to existing legal regimes which could make contract cheating legislation more effective in addressing AI. Because of the differences between AI and essay mills, limitations remain.

Takedown requests

Many of the discussed AI essay mills and advertisements outlined are manifestly illegal in many jurisdictions. Takedown requests can be made by any organisation under most intermediary liability laws. We are unaware of any actors arranging takedown requests as there are in areas like child abuse imagery, and as mentioned, few regimes contain regulators. As we have indicated, private enforcement and intermediation has underexplored potential to enforce these regimes.

Higher education institutions and their representative bodies are candidates to take this action, as are regulators. Takedown requests might be directed at advertising intermediaries, social media companies, payment processors, upstream AI providers or other online organisations. Higher education institutions could share funding to organise individuals to monitor advertising archives and other services for essay mills, and report these to prosecutors as well as issue take down requests. Reporting should be wide, including to payment service providers, who may be able to stop money-flows, and to AI service providers.

Internalisation of responsibility

Policy discussions around the responsibility of technology companies necessitate determining what we base responsibility on, and what level of control, awareness and agency we link to liability. For long periods, intermediaries like search engines broadly escaped liability for activities which they may have had technical control to curtail or monitor.¹⁵³ That norm has been changing in recent years, both in the case law relating to existing bodies of law, as well as in new legislative proposals and statutes.¹⁵⁴

General-purpose AI services may not qualify as intermediaries under existing regimes such as the DSA.¹⁵⁵ They may not benefit from immunities, but also will not be subject to new proactive obligations, as other intermediaries have been. Despite this, scholars have noted the importance of misuse monitoring obligations to the governance of AI-as-a-service more generally.¹⁵⁶ Trends in law towards enforcing terms and conditions seem like they will be of importance, especially as the governance of AI through private law, such as licensing and contractual obligations, is becoming a main pillar of global governance.¹⁵⁷ Insofar as essay mills operate illegally, they are likely in breach of standard API terms of service and model licensing.

However, the effectiveness of this as a governance mechanism is questionable. As essay mills already operate in the shadows, and operate illegally, they will be difficult to find and drag to court for misuse of model IP out of step with the terms specified in the license – although their access to APIs may be more easily cut off.

Jurisdictions should explore creating obligations for AI providers to enforce their terms and conditions, similar to obligations placed on intermediaries under the DSA and OSA. This would create an avenue to cut off professionalised essay mills once services are notified or investigated.

Clarification and enforcement

The blurred lines we have highlighted between legal and illegal services under academic cheating laws are unlikely to get much clarification from a court due to the limited number of prosecutions being brought. Determination of illegality in practice is likely to be left primarily to platform companies to determine, with limited clarity and accountability. This is far from ideal. It is even less ideal because such platforms themselves would not support a broad reading of a law which could capture AI services they themselves offer. There is a significant moral hazard in allowing a platform to define the scope of regulation in practice, where that regulation may if defined broadly, encompass their own firms’ products.

There must be avenues for judicial clarification of these rules or further statutory elaboration by regulators. Amending the laws to give regulators explicit powers to both prosecute and provide guidelines on the changing and uncertain boundaries is the first step. Australia and New Zealand are the furthest ahead in this regard. England and Wales have no statutory body that can take on this role, and without the QAA only the Office for Students is available to potentially, voluntarily, take up this role.

Jurisdictions should create or empower regulators, or as a second resort, empower higher education institutions to refer cases to prosecutors. They should issue guidelines on the interaction between essay mill law and AI to all relevant bodies. Finally, more regulators create the potential for an international regulatory forum, for coalescing around standards, joint investigations across borders and shared enforcement and capacity on cross-border cases. This might be modelled after the European Data Protection Board or similar organisations.

How to earn a safe harbour?

Essay mills and contract cheating are companies whose sole purpose is to facilitate academic cheating. The legislation addressing such services works because it targets services that only fulfil that purpose. General-purpose AI providers are different. They fulfil multiple purposes and have not been designed to facilitate academic misconduct. It is therefore questionable, whether it is desirable to criminalise the acts of a service that only inadvertently facilitates academic cheating. The strict liability approach of many jurisdictions, like Australia, is appropriate for purpose-specific essay mills and contract cheating. It is less so for AI services.

In many jurisdictions analysed in this policy paper, the legislation around essay mills and contract cheating is largely not enforced. The symbolic element of legal regimes was successful because it targeted a specific industry and made a strong public policy statement against such services. Liability of general-purpose AI services will not deter students from using AI for academic cheating, because AI services are unlikely to be treated as criminal entities like essay mills. If anything, governments are eager to court these entities, hoping – often with little evidence – for national investment and economic prosperity related to facilitating their services.

However, drawing the line is difficult. Adding intent or knowledge requirements allows bone fide essay mills to flourish – hence their removal in recent laws, such as those in England and Wales. In those cases, it may be worth considering whether certain due diligence requirements on general-purpose AI services could exempt them from consideration as essay mills, and the liability that may result. Essay mills would be unlikely to cooperate in this manner, and so this could effectively draw a line between the services. This could include technologies and obligations such as the following:

Integration of watermarking

Watermarking is a technique designed to place a difficult-to-remove signal, such as a statistical pattern in the frequency of words, within generated content to indicate its artificiality. Good watermarks are easy to detect but difficult to remove, and indeed may be needed for ‘AI detection’ tools to function at all on text, given current failures and biases (e.g. against non-native speakers) in this space.¹⁵⁸ Watermarks do not prevent all issues, particularly for cognitive offloading, or were students to significantly rephrase. While watermarking is welcome, care should be taken that it does not create proprietary power for detection companies like Turnitin or Copyleaks. Furthermore, as we have shown, detection tools are often even built into the latest array of AI-powered essay mills, so legislation may be required to ensure such tools are only provided to bone fide institutions.

Detection and retention of essay querying

While more hands-on, there may be a possibility for collaboration between AI providers, educational institutions, and plagiarism checkers. Providing examination questions securely to AI providers through a consortium may allow them to retain the result of queries that relate to those questions on their servers for a certain limited time. Those materials could in turn be securely queried by educational institutions, running their submitted assignments through them, and retrieving results if any similar content was detected. Firms already retain data on queries for certain of their services. OpenAI retains query and response data to chatGPT, although claims not to retain data for the same models queried professionally through its API, as essay mills relying on it will likely use. Such arrangements will be intricate but may be worth exploring. Given that the queries would remain on the AI firms’ servers, this would also frustrate students’ abilities to run their text through commercial plagiarism detectors to understand usage. However, it may place a significant infrastructural burden on AI providers, and this may require statutory intervention to incentivise.

Due diligence and cooperation

AI service providers could be obliged to provide practical assistance with enforcement. For example, when an AI essay mill is discovered by a regulator, they may wish to enter an example question and put AI providers on notice to monitor their servers for this question, so they can understand which model(s) are behind the intermediary, and inform them so they can disable the relevant account(s). This comes with potential data retention and protection challenges, but these do not seem insurmountable, particularly if co-operation is sought with sectors facing related challenges.

Challenges of open-source models

A challenge with all these concerns relates to open-source models, many of which can already generate text very capable of breaching academic integrity. The above provisions assume an AI essay mill is querying a service live, through an API, and can then be monitored and cut-off. Where models are uploaded and run on local or third-party hardware, monitoring becomes much more difficult. While models can be run locally on users’ own computers, as it stands, the most powerful models require powerful graphical processing units (GPUs) which are not economical to own for occasional use. While API providers can in principle monitor and intercept queries before they are transmitted to their models, generic cloud providers will find it more difficult and much more invasive and risky-to-security to intercept such queries from within a virtual machine, as they will likely be encrypted in transit and effectively require the provider to compromise their clients’ code. Detecting the systems used by apps or website will also be tricky, because students’ devices might query them indirectly through apps or websites, masking the IP address of the cloud hosting service. Because of this, more sophisticated essay mills may adopt the strategy of hosting their own models to limit their potential to be shut down and/or interrupted. It further seems likely that if AI-facilitated crime is to increase, out-of-jurisdiction datacentres and computational facilities will operate in countries with limited interest in cooperation, extradition or the execution of warrants, such as Russia.

Concluding remarks

Legal tensions around AI-facilitated academic cheating illustrate crucial insights into the intricacies of holding technology companies liable through legal frameworks not intended for novel technologies. This discussion is also part of a larger conversation around imposing liability on corporate actors. Academic integrity is crucial for successful higher education institutions. AI – perhaps even more so than essay mills – throws into doubt both our original conceptions of academic integrity and our ability to combat illegal services.

Essay mills are increasingly turning to AI systems and selling tools directly to students. While these are likely covered by existing law, they also stress it, blurring the lines between digital support tools and the completion of an assignment further. Enforcement against essay mills using these tools will require cooperation with AI providers, and the lack of regulators or associated powers currently scuppers this. Instead, the wide criminalisation of essay mills risks delegating this law to private enforcement, including by AI providers themselves, who are unlikely to define the scope of these laws widely to avoid being implicated themselves. The regulation specifically shaping the private enforcement of AI-as-a-service firms is likely to be needed to limit misuse in this sector and beyond. However, not all enforcement should be delegated – there remains an important role for regulators here, yet few jurisdictions have named them or given them appropriate powers to enforce these laws.

Legislation should be amended to give general-purpose AI systems safe harbour from criminal consideration as an essay mill, insofar as they meet a series of criteria designed to lower their risk in this regard. We propose watermarking, regulatory co-operation, and time-limited data retention and querying capacity based on queries provided by educational institutions, as mechanisms to consider.

In sum, essay mill and contract cheating law is not a good way to regulate general-purpose AI systems, but as it stands, it is regulating them, at least in some jurisdictions, and on paper. Australia and England and Wales stand out most in this regard. Legislation in all jurisdictions needs updating to draw clearer boundaries, provide capacity for ongoing navigation and negotiation and support real, rather than symbolic, enforcement. Consideration should be given to how to exempt general-purpose AI providers from these regimes while obliging them to cooperate and mitigate the damage to academic integrity that their tools are facilitating. Safe harbours may be one way to achieve this.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work is supported by the UKRI [grant number EP/V00784X/1] (https://tas.ac.uk) Trustworthy Autonomous Systems Hub, and the UCL ChangeMakers fund.

Notes

1 Platform on Ethics, Transparency and Integrity in Education (ETINED), South-East European Project on Policies for Academic Integrity (Council of Europe 2018) 56.

2 See generally Jürgen Rudolph, Samson Tan and Shannon Tan, ‘ChatGPT: Bullshit Spewer or the End of Traditional Assessments in Higher Education?’ (2023) 6 Journal of Applied Learning and Teaching 342.

3 The Quality Assurance Agency for Higher Education, ‘Contracting to Cheat in Higher Education’ (QAA, 20 September 2022) <https://www.qaa.ac.uk/docs/qaa/guidance/contracting-to-cheat-in-higher-education-third-edition.pdf> 3.

4 QAA (n 3) 10–11.

5 HC Deb 10 February 2021, vol 689, col 350; Tracey Bretag and others, ‘Contract Cheating: A Survey of Australian University Students’ (2019) 44 Studies in Higher Education 1837.

6 QAA (n 3) 9–10.

7 HC Deb 10 February 2021, vol 689, col 350; QAA (n 3) 9. It is worth noting that this may further heighten the desirability for AI, where blackmail risk may be easier to avoid. Pablo Arredondo, Sharon Driscoll and Monica Schreiber, ‘GPT-4 Passes the Bar Exam: What That Means for Artificial Intelligence Tools in the Legal Profession’ (Stanford Law School, 19 April 2023) <https://law.stanford.edu/2023/04/19/gpt-4-passes-the-bar-exam-what-that-means-for-artificial-intelligence-tools-in-the-legal-industry/> accessed 2 September 2023.

8 Michael Draper and Philip M Newton, ‘Using the Law to Tackle Essay Mills’ (HEPI, 26 September 2018) <https://www.hepi.ac.uk/2018/09/26/using-law-tackle-essay-mills/> accessed 2 September 2023.

9 Guzyal Hill, Jon Mason and Alex Dunn, ‘Contract Cheating: An Increasing Challenge for Global Academic Community Arising from COVID-19’ (2021) 16 Research and Practice in Technology Enhanced Learning 1; HM Government, ‘Skills and Post-16 Education Bill: Policy Summary Notes’ (November 2021) 56.

10 Dates where they passed can be found in the database maintained by Mary McCormick and Hunter Whaley, Term Paper Mills: Statutes and Legislative Information (Florida State University College of Law Research Center, 2019) <https://guides.law.fsu.edu/termpapermills/statutesandlegislativeinformation> accessed 2 September 2023.

11 Sarah Elaine Eaton, ‘Contract Cheating in Canada: A Comprehensive Overview’ in Sarah Elaine Eaton and Julia Christensen Hughes (eds), Academic Integrity in Canada: An Enduring and Essential Challenge (Springer 2022).

12 Philip M Newton and Christopher Lang, ‘Custom Essay Writers, Freelancers, and Other Paid Third Parties’ in Tracey Bretag (ed), Handbook of Academic Integrity (Springer 2016); McCormick and Whaley (n 10).

13 Mass. Gen. Laws ch 271, § 50 (current through 2022).

14 18 Pa. Cons. Stat. § 7324 (current through 2022).

15 NV Rev. Stat. § 207.320 (current through 2022).

16 NJ Rev. Sta.t § 18A:2-3 (current through 2022).

17 CO Code § 23-4-101 et seq (current through 2022).

18 Conn. Gen. Stat. § 53-392a et seq (current through 2022).

19 Md. Code, Education § 26–201 (current through 2022).

20 Or. Rev. Stat. § 165.114 (current through 2022).

21 Wash. Rev. Code § 28B.10.580(1) et seq (current through 2022).

22 Va. Code Ann. § 18.2-505 (current through 2022).

23 Tex. Penal Code § 32.50 (current through 2022).

24 N.Y. Educ. Law § 213-b (McKinney current through 2022).

25 Fla. Stat. Ann. § 877.17 (West current through 2022).

26 Cal. Educ. Code § 66400 et seq (West current through 2022).

27 N.C. Gen. Stat. § 14-118.2 (current through 2022).

28 110 Ill. Comp. Stat. Ann. 5/1 (current through 2022).

29 Me. Rev. Stat. tit. 17-A, § 705 (current through 2022).

30 Cal. Educ. Code § 66400.

31 Ibid § 66401.

32 Ibid § 66402.

33 Md. Code, Education § 26-201(2).

34 Fla. Stat. Ann. § 775.082(4)(b), 775.083(1)(e).

35 Ibid § 877.17(1–2)

36 Education Amendment Act 2011 (New Zealand) s 42, inserting Education Act 1989 (New Zealand) s 292E (repealed). The offence was recently restated as Education and Training Act 2020 (New Zealand) s 393(1)(a).

37 Michael J Draper and Philip M Newton, ‘A Legal Approach to Tackling Contract Cheating?’ (2017) 13 International Journal for Educational Integrity 1. See Qualifications and Quality Assurance (Education and Training) (Amendment) Bill 2018, 17; HL Deb 25 January 2017, vol 778 col. 776.

38 Ibid s 393(2).

39 New Zealand Qualifications Authority, ‘Factsheet – Education Amendment Act 2011’ (NZQA, December 2011) <https://www.nzqa.govt.nz/assets/About-us/Our-role/factsheet-education-amendment-act-2011.pdf> accessed 3 September 2023.

40 Ibid.

41 Commissioner of Police v Li [2014] NZHC 479.

42 Pursuant to the Criminal Proceeds (Recovery) Act 2009 (New Zealand).

43 Dáil Deb 12 June 2019, vol 983 col 5.

44 Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act) OJ L 277/1, art 6.

45 Anna McKie, ‘Irish Law to Clamp Down on Essay Mills “Could Be Model for UK”’ (Times Higher Education (THE), 9 August 2018) <https://www.timeshighereducation.com/news/irish-law-clamp-down-essay-mills-could-be-model-uk> accessed 3 September 2023.

46 ibid.

47 Tertiary Education Quality and Standards Agency Act (Australia) 2011 s 5 (hereafter TEQSA Act).

48 A penalty unit describes a financial penalty, with the units themselves set to increase with inflation. As of July 2023, a penalty unit was 313 AUD. They are indexed following a formula in the Crimes Act 1914 (Australia) s 4AA.

49 TEQSA Act 2011 (Australia) s 144A(3).

50 Ibid s 114A(5).

51 Ibid s 114B.

52 Australian Government, Department of Education, The Higher Education Standards Panel (HESP), Tackling contract cheating <https://www.education.gov.au/higher-education-standards-panel-hesp/tackling-contract-cheating> accessed 7 September 2023

53 Australian Government Response, More support for academic integrity in higher education Australian Government response to recommendations of the Higher Education Standards Panel

54 John Ross, ‘Australia Blocks Access to Biggest Contract Cheating Websites’ (Times Higher Education (THE), 5 August 2022) <https://www.timeshighereducation.com/news/australian-regulator-forces-mass-blocking-cheating-websites> accessed 3 September 2023.

55 Ibid.

56 John Ross, ‘First Blood for Australian Contract Cheating Law’ (Times Higher Education (THE), 8 October 2021) <https://www.timeshighereducation.com/news/first-blood-australian-contract-cheating-law> accessed 3 September 2023.

57 Ibid.

58 QQI, ‘Clamping down on Academic Cheating in Ireland’ (EOLAS, December 2019) <https://www.eolasmagazine.ie/clamping-down-on-academic-cheating-in-ireland/> accessed 3 September 2023.

59 Universitätsgesetz 2002 (UG), Bundesgesetz über die Organisation der Universitäten und ihre Studien, BGBl I Nr 120/2002, amended by BGBl I Nr 93/2021 (Austria) § 116a.

60 Ibid § 116a(1).

61 Ibid § 116a(4).

62 The örtlich zuständige Bezirksverwaltungsbehörde; ibid § 116a(7).

63 Theo Anders, ‘Schlupfloch für Ghostwriter: Faßmann will trotz Warnungen nichts ändern’ (Der Standard, 23 March 2021) <https://www.derstandard.at/story/2000125235403/schlupfloch-fuer-ghostwriter-fassmann-will-trotz-warnungen-nichts-aendern> accessed 3 September 2023.

64 Aschendorff Medien, ‘Ist ein Ghostwriter-Verbot sinnvoll?’ (Westfalen-Blatt, 6 March 2023) <https://www.westfalen-blatt.de/freizeit/ratgeber/ist-ein-ghostwriter-verbot-sinnvoll-2718027> accessed 3 September 2023.

65 ‘So umgehen Ghostwriter-Agenturen die UG-Novelle’ (Kronen Zeitung, 29 September 2021) <https://www.krone.at/2518166> accessed 3 September 2023.

66 Skills Act, s 27(1)(6). Some jurisdictions do sanction students rather than providers, such as in Germany, where civil penalties follow through the Hochschulegesetz. See generally ‘Wenn Geister schreiben’ (FOCUS online, 9 June 2023) <https://www.focus.de/magazin/archiv/hochschulen-wenn-geister-schreiben_id_195936431.html> accessed 3 September 2023; in Nord-Rhein Westfalen, Hochschulgesetz 2004 – HG 2004 (NRW) vom 14 March 2000, in der Fassung vom 1 September 2023 (Nordrhein-Westfalen, Germany) § 92(7)(a)(b). In France, such academic integrity issues can be penalised as un delit; see generally République française, ‘Que risque-t-on en cas de fraude au bac ?’ (Service Public, 13 September 2022) <https://www.service-public.fr/particuliers/vosdroits/F22211> accessed 3 September 2023. Montenegro also enacted regulation in 2019 mostly targeting those using the work; Law on Academic Integrity (Zakona o Akademskom Integritetu) (Official Gazette of Montenegro, 17/2019).

67 ibid s 26(5)(a).

68 Skills Act s 26(2); Explanatory Notes to the Skills and Post-16 Education Bill (as brought from the House of Lords on 26 October 2021, Bill 176), para 156.

69 HM Government, ‘Skills and Post-16 Education Bill: Policy Summary Notes’ (November 2021) 57–58.

70 Thomas Brown, ‘Higher Education Cheating Services Prohibition Bill [HL]’ (House of Lords Library, 16 June 2021) <https://lordslibrary.parliament.uk/higher-education-cheating-services-prohibition-bill-hl/> accessed 3 September 2023; Anna McKie, ‘Peer Sniffs “Real Chance” of Success on UK Contract Cheating Law’ (Times Higher Education (THE), 2 July 2021) <https://www.timeshighereducation.com/news/peer-sniffs-real-chance-success-uk-contract-cheating-law> accessed 3 September 2023.

71 The Quality Assurance Agency for Higher Education, ‘QAA Welcomes Ban on Essay Mills in England’ (QAA, 28 April 2022) <https://www.qaa.ac.uk/news-events/news/qaa-welcomes-ban-on-essay-mills-in-england> accessed 3 September 2023.

72 Skills Act s 26(3)(a).

73 ibid s 26(3)(b)(i).

74 ibid s 26(4)(b)(ii).

75 Explanatory Notes to the Skills and Post-16 Education Bill (as brought from the House of Lords on 26 October 2021, Bill 176), para 156.

76 Skills Act s 26(7)(a–b).

77 ibid s 27(2).

78 ibid s 27(4).

79 ibid s 27(5).

80 ibid s 28(1–2).

81 Policy Summary 57

82 Tom Williams, ‘Judge Contract Cheating Law “on Culture Change, Not Prosecutions”’ (Times Higher Education (THE), 26 April 2022) <https://www.timeshighereducation.com/news/judge-contract-cheating-law-culture-change-not-prosecutions> accessed 3 September 2023.

83 The Quality Assurance Agency for Higher Education, ‘QAA Demits DQB Status to Focus on Sector and Students in England’ (QAA, 20 July 2022) <https://www.qaa.ac.uk/news-events/news/qaa-demits-dqb-status-to-focus-on-sector-and-students-in-england> accessed 3 September 2023.

84 Jack Grove, ‘OfS Takes England’s Quality Role after QAA Delisted’ (Times Higher Education (THE), 30 March 2023) <https://www.timeshighereducation.com/news/ofs-takes-englands-quality-role-after-qaa-delisted> accessed 3 September 2023.

85 Department for Education, Skills and Post-16 Education Bill, Policy Summary Notes, November 2021, 56, 58

86 QAA (n 3) 6.

87 Williams (n 82).

88 The Quality Assurance Agency for Higher Education, ‘QAA Calls for Online Companies to Stop Essay Mills in Their Tracks’ (QAA, 6 December 2018) <https://www.qaa.ac.uk/news-events/news/qaa-calls-for-online-companies-to-stop-essay-mills-in-their-tracks> accessed 3 September 2023.

89 The Quality Assurance Agency for Higher Education, ‘PayPal Says No to Essay Mills’ (QAA, 3 April 2019) <https://www.qaa.ac.uk/news-events/news/paypal-says-no-to-essay-mills> accessed 3 September 2023.

90 The Electronic Commerce (EC Directive) Regulations 2002.

91 For a review, see Rui Mao and others, ‘GPTEval: A Survey on Assessments of ChatGPT and GPT-4’ (arXiv, 23 August 2023) <http://arxiv.org/abs/2308.12488> accessed 4 September 2023.

92 Tricia Bertram Gallant, Navigating the Era of Outsourcing: Rethinking Higher Education in the Age of GenAI and Contract Cheating (International Center for Academic Integrity, 16 May 2023) <https://academicintegrity.org/resources/blog/113-2023/may-2023/437-navigating-the-era-of-outsourcing-rethinking-higher-education-in-the-age-of-genai-and-contract-cheating> accessed 7 September 2023

93 Mike Perkins, ‘Academic Integrity Considerations of AI Large Language Models in the Post-Pandemic Era: ChatGPT and Beyond’ (2023) 20 Journal of University Teaching & Learning Practice <https://ro.uow.edu.au/jutlp/vol20/iss2/07>.

94 Jennifer Cobbe, Michael Veale and Jatinder Singh, ‘Understanding Accountability in Algorithmic Supply Chains’, Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery 2023) <https://doi.org/gsb98p>; Petros Terzis, ‘Law and the Political Economy of AI Production’ [2024] International Journal of Law and Information Technology <https://doi.org/10.31219/osf.io/q593j>.

95 David Gray Widder, Sarah West and Meredith Whittaker, ‘Open (For Business): Big Tech, Concentrated Power, and the Political Economy of Open AI’ (17 August 2023) <https://papers.ssrn.com/abstract=4543807> accessed 29 August 2023.

96 Hugo Touvron and others, ‘Llama 2: Open Foundation and Fine-Tuned Chat Models’ (arXiv, 19 July 2023) <http://arxiv.org/abs/2307.09288> accessed 4 September 2023.

97 BigScience Workshop and others, ‘BLOOM: A 176B-Parameter Open-Access Multilingual Language Model’ (arXiv, 27 June 2023) <http://arxiv.org/abs/2211.05100> accessed 4 September 2023.

98 Compare to the growing availability of on-device image models through model marketsplaces; see generally Robert Gorwa and Michael Veale, ‘Moderating Model Marketplaces: Platform Governance Puzzles for AI Intermediaries’ (2024) 16 Law, Innovation and Technology <https://doi.org/k5kf>.

99 See e.g. for image apps, local apps such as DiffusionBee, and for language models, see Ollama.

100 Jennifer Cobbe and Jatinder Singh, ‘Artificial Intelligence as a Service: Legal Responsibilities, Liabilities, and Policy Challenges’ (2021) 42 Computer Law & Security Review 105573.

101 Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act) OJ L 277/1 (hereafter DSA), art 39. Companies have typically extended the scope of this archive to include the United Kingdom despite the United Kingdom not being subject to the DSA.

102 We could not determine this firm’s location.

103 Joris van Hoboken and Ronan Ó Fathaigh, ‘Smartphone Platforms as Privacy Regulators’ (2021) 41 Computer Law & Security Review 105557; Josh Cowls and Jessica Morley, ‘App Store Governance: The Implications and Limitations of Duopolistic Dominance’ in Jakob Mökander and Marta Ziosi (eds), The 2021 Yearbook of the Digital Ethics Lab (Springer 2022).

104 Illinois law creates an offence for an advertiser, preparer or seller to purposefully engage ‘in a course of conduct which he reasonably should have known would result in the submission of such academic papers, substantially unchanged’ to accredited higher education institution in the state. 110 Ill. Comp. Stat. Ann. 5/1 (current through 2022).

105 Phrasly, ‘How to Bypass TurnItIn’ (15 July 2023) <https://blog.phrasly.ai/blog/how-to-bypass-turnitin> accessed 4 September 2023.

106 Phrasly, ‘How To Write An AI Proof Essay?’ (22 August 2023) <https://blog.phrasly.ai/blog/how-to-write-an-ai-proof-essay> accessed 4 September 2023.

107 ‘Significant-Gravitas/Auto-GPT’ (GitHub, n.d.) <https://github.com/Significant-Gravitas/Auto-GPT> accessed 4 September 2024.

108 Meta Ads Library <https://www.facebook.com/ads/library/?id=950153349388736> accessed 4 September 2023.

109 Google Ad Transparency <https://adstransparency.google.com/advertiser/AR17899575067084324865/creative/CR08035784511172116481?region=GB> accessed 4 September 2023.

110 ‘About’ (Sourcely) <https://www.sourcely.net/about> accessed 4 September 2023.

111 Thomas Williams, ‘Essay Mills “under Threat from Rise of ChatGPT”’ (Times Higher Education (THE), 2 March 2023) <https://www.timeshighereducation.com/news/essay-mills-under-threat-rise-chatgpt> accessed 5 September 2023; Thomas Lancaster, ‘Cheating with Artificial Intelligence – Addressing The Consequences’ <https://thomaslancaster.co.uk/blog/cheating-with-artificial-intelligence-addressing-the-consequences/> accessed 5 September 2023.

112 Md. Code, Education § 26-201(b) (current through 2022).

113 Or. Rev. Stat. § 165.114(2) (current through 2022).

114 Mass. Gen. Laws ch 271, § 50 (current through 2022).

115 Fla. Stat. Ann. § 877.17 (West current through 2022).

116 TEQSA Act 2011 (Australia) s 5.

117 Explanatory Notes to the Skills and Post-16 Education Bill (as brought from the House of Lords on 26 October 2021, Bill 176), para 159.

118 The Electronic Commerce (EC Directive) Regulations 2002, reg 2.

119 David Gray Widder, Sarah West and Meredith Whittaker, ‘Open (For Business): Big Tech, Concentrated Power, and the Political Economy of Open AI’ (17 August 2023) <https://papers.ssrn.com/abstract=4543807> accessed 29 August 2023.

120 In Australia, commercial purposes will lead to a criminal sanction of 2 years imprisonment or 500 penalty units, or both. On the contrary, the same act without commercial purposes will lead to a civil penalty of 500 penalty units — currently up to 156,500 AUD. Austrian penalties are higher if the action is to generate ongoing income. See Universitätsgesetz 2002 (UG), Bundesgesetz über die Organisation der Universitäten und ihre Studien, BGBl I Nr 120/2002, amended by BGBl I Nr 93/2021 (Austria) § 116a(4).

121 Anna McKie, ‘Essay Mills Quit Australia as UK Falls behind but Covid a Threat’ (Times Higher Education (THE), 18 November 2020) <https://www.timeshighereducation.com/news/essay-mills-quit-australia-uk-falls-behind-covid-threat> accessed 4 September 2023.

122 ‘So umgehen Ghostwriter-Agenturen die UG-Novelle’ (Kronen Zeitung, 29 September 2021) <https://www.krone.at/2518166> accessed 3 September 2023.

123 TEQSA Act 2011(Australia) s 12; Criminal Code Act 1995 (Australia) s 15.4.

124 HM Government, ‘Skills and Post-16 Education Bill: Policy Summary Notes’ (November 2021) 57–58.

125 Pier Giorgio Chiara, ‘Italian DPA v. OpenAI’s ChatGPT: The Reasons Behind the Investigation and the Temporary Limitation to Processing’ (2023) 9 European Data Protection Law Review 68.

126 McKie (n 121); John Ross, ‘Australia Blocks Access to Biggest Contract Cheating Websites’ (Times Higher Education (THE), 5 August 2022) <https://www.timeshighereducation.com/news/australian-regulator-forces-mass-blocking-cheating-websites> accessed 5 September 2023.

127 In some cases, platforms may benefit from liability shielding due to unawareness of hosting such adverts, even where they have done so for remuneration. See e.g., in relation to the EU and the UK, Joined Cases C-236/08 and C-238/08 Google France ECLI:EU:C:2010:159 (where Google was found to benefit from liability shielding in relation to advertising on their AdWords service, as long as they were not drafting the message themselves and met the other conditions of the liability shielding, such as taking down rapidly upon being notified).

128 Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act) OJ L 277/1, art 39.

129 Paddy Leerssen and others, ‘News from the Ad Archive: How Journalists Use the Facebook Ad Library to Hold Online Advertising Accountable’ (2023) 26 Information, Communication & Society 1381.

130 Alex Burghart, ‘Essay Mills Are Now Illegal – Skills Minister Calls on Internet Service Platforms to Crack down on Advertising’ (Department for Education: The Education Hub, 28 April 2022) <https://educationhub.blog.gov.uk/2022/04/28/essay-mills-are-now-illegal-skills-minister-calls-on-internet-service-providers-to-crack-down-on-advertising/> accessed 4 September 2023.

131 Advertising Standards Authority, Committee of Advertising Practice, ‘ASA Ruling on Person(s) Unknown t/a Proacademichelp.co.uk (A19-564582)’ (ASA, 11 September 2019) <https://www.asa.org.uk/rulings/person-s-unknown-A19-564582.html> accessed 4 September 2023.

132 Advertising Standards Authority, ‘ASA Ruling on Brilliant Minds Ltd t/a EssayMills (A23-1188070 Brilliant Minds Ltd)’ (ASA, 19 April 2023) <https://www.asa.org.uk/rulings/brilliant-minds-ltd-a23-1188070-brilliant-minds-ltd.html> accessed 4 September 2023.

133 Advertising Standards Authority, ‘ASA Ruling on Home of Dissertations’ (ASA, 19 April 2023) <https://www.asa.org.uk/rulings/home-of-dissertations-a23-1188078-home-of-dissertations.html> accessed 4 September 2023.

134 Australian Government, ‘Australian Government response to the advice of the HESP on student academic integrity and cheating’ (18 December 2018) <https://www.education.gov.au/higher-education-standards-panel-hesp/resources/australian-government-response> accessed 5 September 2023.

135 Seemingly made by the owners of https://awesomegpts.vip/.

136 Online Safety Bill, cl. 59(5)(b).

137 Digital Services Act, recital 18.

138 It is worth noting that at least two of the regimes described above, in Florida and in California, would not have these problems in relation to intermediaries hosting models. The US’s ‘Section 230’ intermediary liability regime would not protect the deployers of AI services directly, it would protect intermediaries regardless of whether or not they received notice. See generally Matt Perault, ‘Section 230 Won’t Protect ChatGPT’ (2023) 3 Journal of Free Speech Law 363.

139 Digital Services Act, art 14(4); Online Safety Bill (23 July 2023) HL Bill 164 (as amended on Report) cl. 73(3)). The Online Safety Bill only requires this of ‘category 1’ services; the DSA requires it regardless of size (i.e. not just for the Very Large Online Platforms (VLOPs) or Very Large Online Search Engines (VLOSEs).

140 Michael Veale, Kira Matus and Robert Gorwa, ‘AI and Global Governance: Modalities, Rationales, Tensions’ (2023) 19 Annual Review of Law and Social Science.

141 OpenAI, ‘Usage Policies’ (23 March 2023) <https://openai.com/policies/usage-policies> accessed 2 September 2023.

142 See generally Gorwa and Veale (n 98).

143 HL Deb 19 July 2021 vol 814 col 61.

144 HL Deb 25 January 2017, vol 778 col. 776.

145 Michael J Draper, Victoria Ibezim and Philip M Newton, ‘Are Essay Mills Committing Fraud? An Analysis of Their Behaviours vs the 2006 Fraud Act (UK)’ (2017) 13 International Journal for Educational Integrity 1.

146 OpenAI, ‘Usage Policies’ (23 March 2023) <https://openai.com/policies/usage-policies> accessed 2 September 2023.

147 Danish Contractor and others, ‘Behavioral Use Licensing for Responsible AI’, 2022 ACM Conference on Fairness, Accountability, and Transparency (ACM 2022) <https://dl.acm.org/doi/10.1145/3531146.3533143> accessed 23 June 2022.

148 Skills and Post-16 Education Act 2022 (England and Wales) s 26–30.

149 TEQSA Act 2011, s 114(2)(a), 5.

150 QQA(ET) Act 2012 (Ireland) s 43A(3).

151 Education and Training Act 2020 (New Zealand) s 393(1)(a).

152 Dáil Deb 12 June 2019, vol 983 col 5.

153 See generally Lilian Edwards, ‘“With Great Power Comes Great Responsibility?”: The Rise of Platform Liability’ in Lilian Edwards (ed), Law, Policy, and the Internet (Hart Publishing 2019); Uta Kohl, ‘Google: The Rise and Rise of Online Intermediaries in the Governance of the Internet and beyond (Part 2)’ (2013) 21 International Journal of Law and Information Technology 187.

154 Daithí Mac Síthigh, ‘The Road to Responsibilities: New Attitudes Towards Internet Intermediaries’ (2020) 29 Information & Communications Technology Law 1.

155 Philipp Hacker, Andreas Engel and Marco Mauer, ‘Regulating ChatGPT and Other Large Generative AI Models’, Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery 2023) <https://dl.acm.org/doi/10.1145/3593013.3594067> accessed 14 June 2023.

156 Seyyed Ahmad Javadi and others, ‘Monitoring Misuse for Accountable “Artificial Intelligence as a Service”’, Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (Association for Computing Machinery 2020) <https://doi.org/10.1145/3375627.3375873> accessed 29 August 2022.

157 Veale and others (n 140).

158 Weixin Liang and others, ‘GPT Detectors Are Biased against Non-Native English Writers’ (arXiv, 18 April 2023) <http://arxiv.org/abs/2304.02819> accessed 6 July 2023.