Limitations of AI

While you are welcome to use AI in this class to help with assignments, you should be aware of possible issues with doing so.

As recently as January 2025, to demonstrate these limitations, I've asked AI to generate pictures representing Waikiki in the year 1500. This is a question for which there is no answer on the web.

I started by asking this AI to create a picture of a cat, and I got a perfectly good picture of a cat. It looked like a photograph rather than an AI-generated picture, but overall I asked for a cat and got a cat, so no problem with this simple question.

I next asked:

create an image of Waikiki in the year 1500

I got back a very accurate disclaimer,

It's difficult to say exactly what Waikiki looked like in the year 1500, but here are some images that depict a plausible scene.

And this image:

There are a few problems with this image:

• The people in the canoes appear to be made of wood.
• The canoe outriggers are missing.
• The sails in the canoes resember modern sailboat sails rather than traditional two-masted Hawaiian-style sails.
• Diamond Head is missing.
• The water is much calmer than it normally is in Waikiki.
• The buildings and structures do not resemble what would have been in Waikiki in the year 1500. The 3-story building does resemble buildings in Waikiki in the early 1900s.
• While the coconuts are likely correct, the remainder of the vegetation resembles monkeypods, which would not have been found in Waikiki in the year 1500. Also, the coconuts have no fruits, which is normal in modern Hawaii but would not have been 500 years ago.

In other words, we get a picture with some of the right ideas (canoes, Tikis, water, trees, hills), but with very incorrect details. Anyone thinking that this is accurate, even if they could overlook the wooden paddlers, would be substantially misled.

I make this information available because if you are relying on AI to generate code, you have to be aware of its limitations. It may do a good job with simple tasks, such as a picture of a cat, but may fail when you ask more challenging questions.

At first I thought I could get the AI to mend its ways by asking refinement questions (the AI had told me "If what you see doesn't quite match what you had in mind, try adding more details to the prompt. The more specific you are, the better [this AI] can create images that reflect your vision."). Unfortunately, I had just as little luck with refinement.

I had recently read a book that described the streams that used to flow through Waikiki, so I asked:

create an aerial image of waikiki in the year 1500

I got back the usual disclaimer:

It's difficult to say exactly what Waikiki looked like in the year 1500, but here are some images that depict a plausible aerial view.

Anyone familiar with Waikiki will notice that Diamond Head has been placed far inland. Also, there are no streams. This image is better than the earlier one, at least the structures on the shoreline are in the style of Hawaiian Hale. The open fields are not quite taro, but could be. The sails on the canoes are from the 1800s, and the outriggers are still not clearly shown.

I tried to refine the question to at least fix the placement of Diamond Head, and this is what I got.

create an aerial image of waikiki in the year 1500 showing the streams that used to flow there and the beach ending at diamond head

Here is an image of Waikiki in the year 1500, showing the streams that used to flow there and the beach ending at Diamond Head.

This picture shows a grid of streets and a long dock, neither of which is plausible for the year 1500. The streams fail to show Manoa and Palolo valleys, and both streams lack continuity all the way to the shore. The mountains are similarly disorganized. And of course, no Diamond Head.

this image does not accurately show the streams in Waikiki in the year 1500. Can you do better?

Here is an image of Waikiki in the year 1500, showing the streams that used to flow there and the beach ending at Diamond Head.

Again, the river flowing between the two non-existent beaches is implausible. Diamond Head is backwards (taller peak towars the mountains). And the closer we look, the more problems we find, including the river never reaching the ocean and waves coming from two opposite directions.

refine the image you gave as response to the prompt "create an aerial image of Waikiki in the year 1500" to align the beach and diamond head as they are in reality

This is a lot closer, but again no streams, as is true for modern Waikiki but not for Waikiki in 1500. The buildings do not look like they would have looked in the year 1500.

refine again to show a hawaiian village of the year 1500

This one lost all the improvements that the previous picture gained! In other words, refinement sometimes makes the result worse.

The problem isn't with AI -- AI is doing its job of learning what it was fed and combining it in interesting ways, and along the way, certainly creating beautiful pictures. AI can be very useful when used properly. However, AI can also be very harmful if its output is taken as being "correct", or "the truth".

So if you ask an AI to generate code, please be aware that the AI has no concept of whether its answer satisfies your requirements. It is up to you to make this determination. And the less common your requirements are, the less likely it is that the program will satisfy them.

This page was created in Spring 2025. It is very likely that technological progress will make things different in the future.