Why AI Can’t Take Over Creative Writing – AIhub
In 1948, Claude Shannon, the founder of information theory, proposed a model for language based on the probability of the next word in a sentence given the preceding words. While Shannon’s probabilistic language models were initially criticized, most notably by linguist Noam Chomsky who dismissed the ‘probability of a sentence’ as useless, they have now formed the foundation for modern language models like ChatGPT.
Fast forward to 2022, a staggering 74 years after Shannon’s initial proposal, and we see the emergence of ChatGPT, a large language model (LLM) that captivated the public’s imagination, with some touting it as a stepping stone to super-human intelligence. The delay between Shannon’s proposal and ChatGPT’s arrival can be attributed to the previously unimaginable amounts of data and computational power required.
ChatGPT, trained on a vast repository of internet text, predicts the next word by considering the context—like a prompt and the previously generated words. It resembles drawing words from a hat, where words with higher predicted probabilities have more entries. Consequently, ChatGPT can produce text that appears intelligent.
However, the use of such tools has sparked considerable debate regarding their influence on learning and practicing creative writing. As a computer science professor with a wealth of AI knowledge, including its social impact through LLMs, it’s crucial to understand how these models operate to assess their limits and potential roles in supporting ‘creative’ writing.
There is a needed distinction between “creativity” in LLMs and human creativity. For individuals who had low expectations of computer-generated content, it’s easy to attribute creativity to machines. Others remained skeptical, like cognitive scientist Douglas Hofstadter, who noted “a mind-boggling hollowness hidden just beneath its flashy surface.” Similarly, linguist Emily Bender and colleagues labeled these models as stochastic parrots, repeating data with randomness from their training corpus.
Understanding why a particular word gets generated is informative; it has a relatively high probability because it’s frequently found in similar contexts within the training data. Thus, creating text with LLMs could be equated to a form of plagiarism, assembling new content word by word based on existing material.
Human creativity involves conveying novel ideas and thoughts. With generative AI, users input ideas into prompts, generating text, images, or sounds. If users are indifferent to the output, the prompt content is inconsequential. But for those who value the generated output, alignment with personal creativity is vital.
An LLM attempts to replicate what a random author might have produced with prefixed text. However, creative writers typically seek uniqueness and often desire tools that help articulate their specific style and intention. Without a comprehensive corpus from a specific author, LLMs must fill in details which may diverge from the intended narrative.
Drawing parallels with software development can be insightful: given a defined idea, developers construct code (analogous to natural language text). Like writers, software engineers produce relevant outputs from LLMs trained on dual corpuses encompassing natural language and code. The resulting text varies with context.
Writers can thus glean lessons from software developers’ experiences. LLMs excel at small, frequently accomplished projects such as database queries or standard letter drafting, and portions of larger projects, like interface pop-up boxes. For extensive projects, however, developers should generate multiple outputs, refining the one closest to their goal. Historically, the challenge in software development has been specifying requirements accurately; coding is the simpler task.
Crafting effective prompts has been termed “prompt engineering”—an artistic endeavor where techniques are suggested to enhance LLM output. For example, asking for an outline first, then building on it for detailed text generation. Another involves chain of thought techniques, prompting the LLM to display reasoning steps as part of its output.
Yet, such guidance is likely temporary. Once a prompt engineering method proves effective, it will likely integrate into future LLM versions, automating desired effects without explicit user intervention. Recent iterations of models have already incorporated step-by-step reasoning improvements.
Joseph Weizenbaum, a computer scientist who developed the ELIZA program in the mid-1960s, observed an intense emotional engagement and anthropomorphism from users interacting with the program. Though the technologies have evolved, the human tendency to anthropomorphize remains unchanged.
Amidst an era rife with misinformation, it’s vital to critically assess often self-serving hyperbolic claims. There is no mysticism in generative AI, only extensive data informing potential word choices. Ultimately, creativity should transcend mere repetition of others’ written ideas.