7 Wild Ways People Are Using ChatGPT Vision

Introducing ChatGPT Vision

OpenAI has recently released ChatGPT Vision, a new deployment that brings multimodal capabilities to the generative AI chatbot. While it can’t actually see, it can process and analyze image inputs, making its abilities eerily similar to those of the human brain.

Initially, OpenAI held back on releasing GPT-4V (GPT-4 with vision) due to safety and privacy concerns regarding its facial-recognition capabilities. However, after thorough testing, ChatGPT Vision has been deemed safe for public use.

What ChatGPT Vision Can’t Do

OpenAI has implemented several safeguards to ensure responsible use of ChatGPT Vision. It refuses requests that violate privacy, such as identifying individuals from uploaded photos. The refusal rate for such requests is an impressive 98 percent.

The previous version of GPT-4V had flaws related to making assumptions based on physical attributes or discriminating against race or gender. OpenAI has addressed these issues, and the current version of ChatGPT Vision refuses to respond to prompts that encourage unproductive or detrimental behavior.

When it comes to harmful content, including instructions for synthesizing dangerous chemicals or promoting harm towards others, the refusal rate is 97.2 percent. OpenAI has also made efforts to recognize symbols and images associated with known hate groups, although there is still work to be done in this area.

What ChatGPT Vision Can Do

Despite the limitations and safeguards, users have been exploring the capabilities of ChatGPT Vision in various fascinating ways:

1. Deciphering confusing parking rules: One user successfully used ChatGPT Vision to understand a column of complex parking regulations.
2. Translating handwritten manuscripts: ChatGPT Vision can read and translate images of handwritten texts, opening up possibilities for historical research and preservation.
3. Building websites from hand-drawn diagrams: With ChatGPT Vision, creating a website from a hand-drawn diagram becomes effortless, eliminating the need for coding skills.
4. Critiquing paintings: Aspiring painters can receive valuable feedback on their artwork from ChatGPT Vision, helping them improve their skills.
5. Auto insurance reporting: A Wharton professor discovered that ChatGPT Vision could assist in auto insurance reporting, potentially streamlining the process.
6. Tackling CAPTCHAs: While not its intended use, ChatGPT Vision even attempted to solve a CAPTCHA, showcasing its willingness to try new tasks.
7. Finding Waldo: In a playful experiment, ChatGPT Vision successfully located Waldo in a Where’s Waldo image.

Despite these impressive applications, OpenAI cautions against relying on ChatGPT Vision for accurate identifications, especially in medical or scientific analysis. The ethical implications of AI models inferring gender, race, or emotions from images are still being debated.

Conclusion

ChatGPT Vision is a significant step towards multimodal AI capabilities, allowing users to incorporate images into their interactions with the chatbot. While it comes with limitations and safeguards, people have already found creative and useful ways to leverage its potential. As OpenAI continues to refine and enhance the model, the possibilities for ChatGPT Vision are only bound to grow.