This is the one question the Microsoft CEO asks to test AI's smarts

• Microsoft CEO Satya Nadella likes to test AI using a poetry prompt.
• Nadella will ask the AI to translate the work of Persian poet Rumi.
• He said GPT-4's answer "preserved the sovereignty of poetry across two language boundaries."

Imagine you're the CEO of Microsoft, and you're testing out an early version of OpenAI's most powerful AI model, GPT-4.

What do you ask it?

Well, now we know, thanks to a new interview Microsoft CEO Satya Nadella recently gave to Wired.

It turns out Nadella turns to a specific question to test the prowess of a new AI model, and it has to do with the poet Jalaluddin Rumi.

"There is one query I always sort of use as a reference. Machine translation has been with us for a long time, and it's achieved a lot of great benchmarks, but it doesn't have the subtlety of capturing deep meaning in poetry," Nadella told Wired.

"Growing up in Hyderabad, India, I'd dreamt about being able to read Persian poetry — in particular the work of Rumi, which has been translated into Urdu and then into English. GPT-4 did it, in one shot," he added. "It was not just a machine translation, but something that preserved the sovereignty of poetry across two language boundaries. And that's pretty cool."

So what's Nadella getting at here?

Jalaluddin Rumi, or Rumi as he's most commonly referred to, was a Persian poet who lived during the early 13th century. His work is widely celebrated around the world — the BBC called him the "best-selling poet in the US" — from people on social media to celebrities like Coldplay frontman Chris Martin, Madonna, and actress Tilda Swinton.

But translation can be fickle, especially in poetry — Robert Frost famously said that "poetry is what gets lost in translation." This difficulty is further compounded when translating a work that already has been translated.

You run the risk of ending up with something like the game of telephone, something that's just ... off. And while technology like Google Translate is impressive, human translators specialize in ensuring the original work's nuance is preserved as much as possible.

For Nadella, this is a proper test of an AI model: seeing whether GPT-4 would simply spit out a "machine translation" or preserve the "sovereignty," as he put it, of Rumi's work.

The Microsoft CEO told Wired that seeing GPT-4 accomplish this early on was a "mind-blowing experience."

You can read Nadella's full interview over at Wired's website.