All is not well in AI world… at least for today.
As I’ve been working my way through using AI to create an entire article, I find it impossible to stay completely on track. Each step of the way, I’m finding things that exhaust my capacity, divert my attention, and slow down my writing process. Here’s a quick summary of the challenges from the past two posts.
Getting the illustrations right was surprisingly much easier than I expected. The AI created some examples based on my prompt, I selected from them, made a few minor changes, and got great results I couldn’t have had any other way. Kudos for that!
Generating lists of potential topics was a breeze. After a couple of tries, I had my lists, which I copied into my note-taking tool. This sort of brainstorming assistance seems to be a sweet spot for ChatGPT. I thought the rest of the project would be a simple exercise in documenting the process. My confidence outran my competence…
I have a huge list of potential “Rules of Thumb” I want to evaluate, but the day was winding down, so I decided to return later to finish.
Starting in a fresh session the next day, I began to immediately hit seemingly small obstacles. First, none of the preferences I had set up before the previous session had been retained. Re-establishing my preferences each session is a small distraction that I’m certain won’t be needed soon.
Then, I found that ChatGPT couldn’t read the format of my notes. So, I had to export them to four massive .pdf files and then convert them file to text files. However, the files were far too big to put into the chat window since it had a limitation of about 4000 tokens (roughly 4k words). So, I needed to break down my notes into sections and manually cut and paste them into the chat window. How’s that for intelligence? This tedious and very repetitive process can’t be automated in ChatGPT (yet).
I decided to try a different AI called NotebookLM, which could directly use my notes as a source. There’s a learning curve to master the new AI, so that took a couple of hours to set up and restart with my notes now being direct sources for the NotebookLM.
This AI can’t read any of the prompts I made for ChatGPT, but those weren’t working all that well yet anyway. Let’s create a new set of prompts… Wait, what? Illegal word error?? Maybe I made my requests too complex? I’ll start with a simple index of the words in my notes. I had to split my notes between 4 different, very long pages to create the .pdf files that were then converted to text files. I get an index, but it only includes words from the first source file. Simple fix… just don’t forget to check the little box next to each source file name. Okay, I’m all set… I’ve checked the boxes and added just one tiny thing, “please also list the source page name with each item.”
Drum roll, please! … Wait? What!? You can’t provide page names because the pages aren’t numbered?? Okay, number the pages, resave them, and try again… Now it looks promising until I realize there are a LOT of words in this index that aren’t part of the source files at all… I tried the obvious and asked the AI why those words were in the index, and it made up an implausible explanation that boiled down to “They seemed like good words to put into an index.”
Okay, I think I’ve got this. These are the “hallucinations” they mention in the warnings. Rather than tell you about only your source materials, the AI includes things it has gathered from other sources.
By this point, I’ve spent about 4 times longer than normal on this article and stretched it out over three weeks. I’m almost giddy with all the distractions and having a bit of a hard time remembering what my original objective for this article was. I found a few things the AI did amazingly well and a lot of things that require much more careful instructions to the AI than I can deliver on the fly.
I gave up on using AI end-to-end for my workflow for now. I’ll use it more as I get more familiar with what works well and what to do on my own.
Here’s my bullet point run-down:
Astonishing illustrations!!
Topic and idea list wizard!
Distractions abound.
Error messages
Size limits
File type constraints
Bogus instructions and completely hallucinated answers
Errors compound and get replicated until the next session
Partway through a session, the AI “forgets” what’s already been done and has to be refreshed.
Massively entertaining and almost good enough - until you look closely
It can’t explain its logic or reasoning coherently.
MASSIVE TIME SUCK
Damn, it’s a lot of fun though!
Is AI worthwhile? For some things, the answer is HELL YES!
For others, at least for this week, (Things are changing at incredible speed.) the answer is HELL NO! but that will change.
In the meantime, next week’s post will have (mostly) human-generated content, and I’ll use AI to assist with illustrations and the things it does better than me.
Lol! At least the frustration was entertaining.
Thank you