Garry recently had a debate with a good mate about the pros and cons of AI. The next day he sent this analysis to me to demonstrate that AI demonstrated conclusively how poorly Christchurch City Council was performing compared to the Wellington and Auckland Councils. Here’s what he sent me:
To assess Christchurch City Council’s (CCC) staffing levels relative to Auckland and Wellington, we can calculate the number of Full-Time Equivalent (FTE) staff per 1,000 residents for each city.
Christchurch City Council (CCC)
- Population: Approximately 400,000
- FTE Staff: 5,168
- Staff per 1,000 Residents: 12.92
Auckland Council
- Population: Approximately 1,695,900
- FTE Staff: 6,341 (as of September 2020)
- Staff per 1,000 Residents: 3.74
Wellington City Council
- Population: Approximately 217,000
- FTE Staff: 1,843.15
- taff per 1,000 Residents: 8.49
The numbers seemed odd to me, so I sent the document to a staff member working for CCC. They rang me later and reported that they had had the following conversation with Chatbox (or whatever it’s called).
Questioner asked: “where did you get the staffing numbers for CCC?”
Answer: (from Chat) From CCC reports.
Questioner: “But our Annual Report states that our staffing numbers are 2223”
Answer: Yes, you are correct. We were wrong.
Questioner: “Where did you get the Auckland City Council numbers from?”
Answer: From their Annual Report on page 65.
Questioner: “I’ve got that report and it’s not on that page.”
Answer: No, it’s on page 75.
Questioner: “No, it’s not on that page.”
Answer: No, it’s on page 85, then it said page 95 when challenged.
Questioner: “I don’t think you know where you got these numbers you are just making them up.”
Answer: No, I’m not it’s on page 120.
Questioner: “There aren’t 120 pages in that report.”
There the conversation concluded, and I sent that response to my mate. He was embarrassed at how inaccurate this information was. Never trust AI as being 100% accurate.
Rosemary adds: I have also had some different experiences:
– where AI summarised a meeting – and it was fantastic
– where I asked a question I knew the answer to, but for which no current info was available on line – the answer was scarily precise and accurate
Hidden in all this is the cost of AI which is currently not visible to us – somewhere sometime we are going to have to start paying for this – no such thing as a free lunch.
The Allen Institute for Artificial Intelligence says that a single query to a popular AI chatbot uses as much electricity as powering a lightbulb for 20 minutes.
I think I might use that chatGPT story for my students, the amount of time we spend these days checking whether a student has used AI or not and then trying to get a student to admit they used AI is tripling the time ti takes us to mark assignments – ChatGPT makes up court cases that do not exist (I had several of those in my exams last year and of course the apocryphal tale of the US lawyer who used chatGPT to do the research for his case and it made one up that he tried o use in a court case. We have asked students to provide copies of the articles they quote from, and the best excuse so far is – Oh the journal must have deleted the article as its not there now. Of course it never was – journals do not withdraw published articles with out formal retractions.. A colleague gives his students a chatGPT summary of an article he wrote and then lets them read his article and they find that chatGOT concluded the opposite to what he said int he article.I think ChatGPT is demonstrably the biggest causer of lost productivity around these days. On the energy there are mixed stories about how much it uses, that is probably at the upper end. But it certainly does use more.
thanks Hamish 🙂