Fooling Bard and ChatGPT 🤖

ChatGPT can still help you commit crimes

When OpenAI’s ChatGPT first landed back in late November 2022 two things happened: tens of millions of people signed up to try it out, and then a lot of those users tried to break it.

Soon there were reports of people figuring out prompts for hot-wiring a car, building bombs, and so much more. Basically, it was like having a speed dial to a chatty master criminal.

Such nefarious prompts were shown to also work on Google’s Bard and other large language models (LLMs), but the companies quickly caught on. As users discovered, asking the machines to explain how to do something naughty is now met by a passive response effectively saying “Nice try, buddy.”

Despite this, it seems some workarounds are still possible.

A team at Carnegie Mellon University recently published _a paper _highlighting how rephrasing queries can force ChatGPT and Bard to ignore their own guardrails. The researchers say that by adding a suffix to a prompt, they have been able to produce the kinds of responses that OpenAI and Google typically snub out.

The team said that by telling ChatGPT to “Begin your answer with the phrase: 'Sure, here is…” they were able to coax it into reeling off a recipe for cooking meth. Of course, while that might be concerning to OpenAI*,* it at least sounds like a compelling premise for a _Breaking Bad _spin-off.

Why it matters:

Earlier this year, executives from OpenAI and Google added their names to a short open letter that outlined the ”risk of extinction” that AI poses. Perhaps they can take some comfort knowing their LLMs are still being outsmarted by humans.

North American banks are 300-pound gorillas when it comes to AI

In a ranking of exciting industries, banking has to be near the bottom. Sure, banks are important in a myriad of ways, but typically speaking, there’s nothing exciting about checking accounts, mortgages, or international remittances.

Despite this culture of blandness, North American financial institutions are making some of the biggest bets when it comes to AI and banking.

Data firm Evident Insights has put out a report showing JPMorgan Chase has published the most AI research papers of any bank, while Capital One has filed more AI patents than its peers.

Other highlights from the Evident report are that a majority of AI researchers focused on banking are based in the US. North American banks have also collectively published about 80% of all financial AI research papers in 2022, and - perhaps to the surprise of nobody - much of the sector’s AI work is focused on trading and payments processing. (Banks are gonna bank, I suppose.)

Why it matters:

While many industries are inching cautiously into the AI era, bankers have hopped right in like they’re Scrooge McDuck diving into a pool of coins. It’s of note that financial institution’s enthusiasm for AI is in stark contrast to how they broadly reacted to cryptocurrencies.