Since the beginning of last year (2023), I have been using ChatGPT for nearly a year, transitioning from version 3.0 to 3.5, and subsequently to 4.0. From the initial statement of amazement to later confusion, and presently, as I approach and use this tool with a sense of equanimity, it has been an unforgettable journey. This article shares some of my insights and experiences garnered during this period.

 

 

Just as many have described, in the initial encounter, ChatGPT was truly astounding. Three aspects particularly surprised me.

 

First, regardless of whether it operates in Chinese or English, the language it employs exhibits exceptional naturalness and fluency. Initially, I misconception it was just an upgraded search engine, simply integrating search results better. However,  it transcended the mere integration of search outcomes. ChatGPT successfully passed the Turing test, providing deceptively human-like conversations. In our daily lives, we often receive sales or survey calls from automated systems. After experiencing many of them, it's easy to discern the non-human origin of the voice on the other end, prompting us to hang up without a second thought. In the case of  ChatGPT, the exchange feels like talking to a real person. The letter "T" in its name stands for Transformer—inviting one to imagine how the Transformers engaged in the movie.

 

Secondly, its conversations are "generative" in nature, producing original content. This aligns with the "G" in its name, signifying Generative. The sentences and paragraphs it generates are not pre-existing, and cannot be traced online. To test this aspect, I conducted the following experience:

 

• I asked ChatGPT to write a fund application centered on a specific topic, and it swiftly produced one within a matter of seconds. Temporarily setting aside the considerations of quality, the output conforms to the prescribed format of a project proposal, with a comprehensive structure and coherent logic.

 

• As I was tutoring my little son with his homework, I engaged ChatGPT to write a piece on "A Happy Winter Holiday" at a third-grade level. In the blink of an eye, the task was accomplished. I shared the piece with my child, discussing their strengths and weaknesses.

 

• I also tasked ChatGPT with writing poetry, such as one titled "Xietang Lantern Festival," where it wrote:

            In Xietang's lantern-lit embrace,

           The spirit of tradition finds its place.

           A timeless celebration, a vibrant display,

           The Lantern Festival in Suzhou, a wondrous array.

           

           It reads quite evocatively. The Chinese version is even more amazing:

       

           追忆缅怀，怦然心动。

           蹉跎转眼，即是华灯。

 

• I was teaching "Environmental Statistics" to third-year students, a particular assignment tasked them to write an analytical report on environmental science data, including the data analysis process within the R language. I assigned this task to ChatGPT, and it completed it in a matter of seconds. Compared to the reports written by the students, the output from ChatGPT exhibited an above-average quality.

 

• When assessing student submissions, it undergoes a plagiarism check against existing literature, online resources, and previous submissions from our university. However, due to the originality of the text produced by ChatGPT, checking for plagiarism becomes very challenging, although the AI-generated work has been integrated into the Turnitin system. This could pose a significant hurdle for us in evaluating the academic performance of students in the future.

 

The third remarkable quality lies in its coherent conversation, with the ability to self-correct. Unlike search engines which provide results without the facility for further inquiry within those results. In contrast, ChatGPT provides a continuous interaction. After receiving an answer to a question, we can continue to ask follow-up questions for elaboration or correction without the necessity of repeating the initial inquiry. This particular feature provides a delightful surprise. In teaching, sometimes students email me unclear questions. The unclear articulation of the question leads to well-intentioned individual responses that may not precisely align with the original question. The iterative process of rephrasing the question and re-answering can result in a loss of patience for both parties. However, with ChatGPT's coherent conversation, this issue is easily resolved, as AI does not become impatient (at least not at present). For example, I had the following coherent exchange with ChatGPT 3.5:

 

Round 1:

 

 

Round 2:

 

 

Its explanation in this response surprised me, as I did not specify why I did not want a nested loop, yet its explanation precisely mirrored my underlying rationale — it anticipated my thoughts!

 

Round 3:

 

 

Round 4:

 

 

This dialogue was replicated in GPT 4.0, which gave similar answers.

 

Throughout this interaction, I gradually realized that initially, I held a hope to solve the problem using an existing function, but I was not even aware of this intent. Through the back-and-forth dialogue with ChatGPT, I refined the question, gradually understanding what I truly desired.

 

 

However, upon further scrutiny, it becomes evident that these three aspects conceal pitfalls at every turn.

 

ChatGPT exhibits a natural flow of language, creating an illusion of omniscience. However, engaging with it can kill one's time. Particularly for those who suffer from social anxiety, interacting with ChatGPT may be effortless, possibly leading to addiction. As long as the system is unoccupied, one can unwittingly engage in endless conversation, unaware that it is akin to a talkative person who, despite appearing articulate, exhibits numerous fallacies in discourse. This issue is precisely what I am about to address.

 

While ChatGPT’s output is quite inventive; the creation of intriguing content is accompanied by numerous errors, blurring the line between truth and falsehood, making it difficult to discern. For instance, ChatGPT 3.5, in drafting a project proposal complete with references, includes citations to non-existent sources. The authors of these references may appear genuine, the journals may be real, and the formatting may adhere strictly to academic norms, yet these articles do not actually exist. This reminds me of a joke:

 

 

ChatGPT acts as A, capable of performing a task, but not necessarily with precision, adept at deceiving people. New versions of GPT have improved this a lot, but this issue should be always taken into mind, and double-checking is always recommended.

 

ChatGPT possesses the ability for self-correction, but at times, it may overcorrect. I once observed a case about GPT 3.0 as follows:

 

 

GPT 4.0 gave similar answers, but with some humor sense:

 

 

The exclamation mark was funny.

 

I conducted a similar test by fabricating a place name and inquiring about its existence. ChatGPT initially claimed it did not exist. However, upon notifying it of my knowledge of the place’s actual existence, ChatGPT then acquiesced, stating, "Then it exists." This incident highlights a perceived lack of principles. As its name begins with Chat, it is fundamentally a conversational tool. Viewing it as an articulate individual with a penchant for hyperbole, one can comprehend its behaviors.

 

 

Xi’an Jiaotong-Liverpool University carried out a series of discussions, beginning in February 2023, for all faculty members with the topic being the impact of ChatGPT on higher education. The discussions were lively and diverse, with a wide range of voices contributing to the discourse. During the segment of the session, faculty members voted on the incorporation of ChatGPT into students' academics. One colleague presented an interesting analogy: our approach to students using ChatGPT is similar to the way we handle sex education for adolescents— they will learn sooner or later, and the consideration now is who should educate them and when. I believe that instead of letting students rely on potentially unreliable sources, we should guide them using proper educational methods.

 

Another topic of the discussions centered on the design of student assignments that cannot be completed with the help of AI. This task is quite simple for our school (School of Science): conducting experiments such as analytical chemistry titration operations, or field observations and investigations, which require physical presence and are currently beyond the capabilities of AI. However, this approach may not be suitable for other disciplines. Some assessment methods involve writing reports or essays that may be influenced by ChatGPT, making it challenging to discern the extent of AI assistance. A prospective solution considered involves increasing the weightage of oral presentations, debates, and group discussions as assignments. For the environmental statistics course that I teach, I have allowed students to use pre-existing datasets from the R language package for analysis to save time. In the future, I will replace this with data files from my own research, which cannot be uploaded to ChatGPT, effectively minimizing AI assistance.

 

Instead of avoiding AI, it may be better to adapt and design assignments that permit AI assistance, followed by a distinct evaluation of students' performance. This could be a direction for us to consider in the future. Currently, we are carrying out a project supported by the Teaching Development Fund (TDF), which aims to integrate AI into the marking process. Rather than positioning AI as an adversary, we can approach it from a higher perspective, viewing AI as a tool—akin to calculators instead of manual calculations and search engines instead of dictionaries for our benefit in enhancing educational practices.

 

 

I believe ChatGPT is particularly suited for specific contexts:

 

• Creation from scratch. ChatGPT excels in generating initial drafts or outlines when contents are challenging to start writing. The ability to produce a rough skeleton that we can then refine and flesh out makes the process much easier. Additionally, its fluent language is a great aid for non-native English speakers, serving as a personal secretary.

 

• Educational assistance. ChatGPT's process of improving code to list prime numbers has taught me step-by-step how to improve code. In helping children with their homework, it can ease parents' anxieties.

 

So, in which scenarios is it not suitable?

 

• What involves your own ideas or your original work? If you have Nobel Prize-worthy ideas, do not seek its input, nor allow it to modify the grammar of the description of these ideas. ChatGPT is like a very gossipy person; you never know to whom it might repeat what you've told it.

 

• What involves professional matters? Laypeople may find ChatGPT serious, but professionals will see it as nonsensical. Of course, ChatGPT itself is a general-purpose conversational AI, so we cannot expect too much. I believe tools of this kind will eventually become more specialized in various fields.

 

 

Overall, the user experience of ChatGPT is astonishing. Such AI tools are the wave of the future, for better or for worse. Whether we rejoice or worry about them, AI remains indifferent. Let us prepare early for their comprehensive arrival.