Thoughts on AI in code and writing

Since Large Language Models (LLM’s) were introduced in November 2022, there has been a near constant buzz about their potential, limitations, and real-world consequences. When ChatGPT 3.0 was first released, I remember bringing my laptop around to colleagues in my master’s program, explaining how this was the tip of the iceberg of something new. While our cohort was just about to graduate, and therefore unlikely to have our trajectories altered too heavily by this new gravitational pull, the year after us could use this new tool to radically change how they went about learning the basics of concepts in statistics and epidemiology.

I didn’t anticipate the rapid acceleration in the versions to follow (3.5 and 4.0) as well as the addition of competing models from a range of companies. What started as just text generation has now morphed into complete data analysis of tabular data, image generation, image recognition, and code generation and validation. While these newer models can perform tasks quicker, with more accuracy, and over a wider range of subjects and circumstances, they still get things wrong and always will. The key parameter is the appropriate error rate that we are willing to tolerate, coupled with what control mechanisms will be put in place to ensure correction and accountability.

In academia, one main concern is that students will no longer develop the capacity to think critically about problems, and will instead pass on this labour to a system that does not tire. With an approaching deadline and a word limit that seems miles away, the use of LLM’s may prove too tempting for many students to resist. A counter-argument is that denying students the ability to use a tool that will no doubt become commonplace in the workforce in the decades to come is just stunting potential, and that those who are able and willing to use the tool will leave the purists in the dust.

For myself, I have graduated and so no longer submit work done for evaluation under the same constraints. However, I still write and code in similar ways and publish things as my own work. On this blog, for instance, I want to communicate clearly with readers about what is and what is not my own original work, and I carry over many of the habits used during my schooling days, such as linking to sources or referencing things when appropriate. In my published Excercism exercises, I have tried to highlight that the solutions that I publish may have been aided by AI to some extent (either through in-line suggestions within the editor or asking questions of ChatGPT about concepts). Through doing so, I have learned that it can be difficult to accurately convey that AI has been used, while also conveying that there was still a great deal of human work that went into the final product.

For example, let’s say that as I was typing this sentence, the AI suggested that I finish it with these three words. Agreeing with them, I hit tab to accept. Does this mean that this entire piece was aided by AI? Potentially, but that use case was quite close to a grammar checker that suggests a rephrasing of a single sentence. I think most observers would conclude that the AI had a negligible role in the final composition or the crux of the argument.

With an easier example out of the way, what if instead I used AI to generate the next paragraph? Using AI to generate the next paragraph brings about interesting considerations. As we navigate this era of advanced language models, it becomes imperative to delineate the boundary between augmentation and automation in creative processes. While AI can undoubtedly streamline certain aspects of content creation, the nuanced interplay between human ingenuity and algorithmic assistance demands a more discerning evaluation. Employing AI for efficiency should not diminish the essence of individual contribution. As we explore these collaborative possibilities, transparency becomes paramount. Acknowledging the AI’s role in the creative process ensures an honest portrayal of intellectual endeavor. Striking the right balance between leveraging technological advancements and preserving the authenticity of human expression is an ongoing challenge—one that prompts us to redefine the dynamics of authorship in an age where our digital counterparts contribute to the tapestry of our ideas.

The cadence and prose sticks out quite drastically from my normal writing (I have found ChatGPT often straddles both sides of an argument and doesn’t usually write strongly opinionated sentences). It did introduce a new concept, that of transparency and acknowledgment, something that coincidentally I had planned to include in this piece. Should I write it off as just rephrasing an idea that I already had? Convincing myself that I had those thoughts swimming around in my mind anyways? Sometimes, it may be hard to tell the difference. What if, for instance, the writer sees a suggested prompt but doesn’t accept it? Surely that would nudge the thinking even a little, even if not completely consciously. Personally, the acceptance of a suggestion which represents a material alteration to the argument presented by a written piece is where I draw the line, and label such content as “assisted by AI”.

Moving from inline to web interface interaction

So far I have written mostly about AI in writing, but this article is also about AI in code. I mainly practice coding on exercism and have been constantly debating about how much to involve AI assistance when solving the exercises. On one hand, AI assisted coding is no doubt the future of the industry, and so it makes sense to both practice coding itself and also how to work with AI to solve problems. Asking a developer to write boilerplate code to solve something like determining the nth prime isn’t going to happen very frequently. On the other hand, it is still vitally important to understand the essentials of a language and coding in general.

One of the methods that I tried when doing the nth prime exercise myself was removing the in-line suggestions within neovim, and instead asking questions of ChatGPT within the web interface. I would explain where I had gotten to, the general objective, and what I knew was wrong with my current implementation. I had written a solution that worked, but was very far from optimal (it assembled a list of all prime numbers up until 10 million, and then was constantly checking against that list). The AI helped to develop a mechanism to assemble the smallest list possible to check each number, as opposed to using 10 million for each.

Some will say that this is still cheating yourself, and the only way to truly master the art is to do it all yourself. I’m sympathetic to this, but the realist in me thinks that those that choose this path may ultimately just end up using AI in the workplace to avoid getting beat by competitors. Even if you know exactly what you are doing, typing out 50 lines of code takes time, and the company is going to want you to use the best tools available rather than manually typing it all out. Using the web interface as opposed to constantly waiting for a good tab suggestion to appear seems like a healthy balance of still thinking critically while embracing this modern reality, and I try to be as transparent as possible by listing “exercises may have been assisted by AI” in the bio of my public exercism profile.

Transparency is key

On this blog, I outline three different labels for written content: no-ai-used, assisted-by-ai, and made-with-ai. I don’t think I’ll ever have a post that was entirely written by AI, because it doesn’t feel genuine or worthwhile for the reader. Most posts will be labelled with “no-ai-used”, but this one, for instance, would get the “assisted-by-ai” label due to the example text used. To represent these three states, I went to ai-label.org which contains some handy labels to download under the creative commons license. There are many other websites that have similar badges to use, and I think that using any of them is a great way to communicate with readers, employers, or users about how something came to be. AI can be used when we are exploring things, or reducing mundane labour, however in high stakes situations it should only be used with clear supervision and must be used transparently.

Open source models and avoiding monopolistic control

One incredibly valid worry is that the use of large language models is going to cement further power with the big tech giants, who will then continue to dominate the market in years to come due to the predicted exponential growth of the technology. Through the use of open models, this power balance can be shifted towards a more distributed structure. I don’t have the hardware to run a lot of models locally, but am constantly on the lookout for new developments that make it easier to boot up a model and go.

Looking forwards

I am sure that the landscape for Large Language Models will change drastically in 2024 and 2025, and that we are just now starting to figure out how we want these tools to fit into society. Already, I’ve changed the way that I think about this topic, and will likely do so again many times. Like all tools, AI is neither intrinsically good nor bad. It will present each of us with profound options for how we go about learning, interacting with one another, and presenting our own versions of our identity. Through thinking out loud, perhaps we can all help each other to make better sense of the challenges and opportunities that may lie ahead.