It is the voice that triggers it. It wouldn’t be easy to make that connection under normal circumstances: they are a long way from Minrathous, and a Tevinter intelligence operative should have better places to be than Lowtown.
Varian Skye has a long way to go to jump into Byerly’s mind. But that voice is so dreadfully distinctive - the accent idiosyncratic amongst even Tevenes. And Byerly has a talent for learning things by ear.

His smile doesn’t change. His posture remains the same. But his gaze sharpens, and he lets his hand move (bit by bit, through apparently unconnected gestures) towards his dagger sheath.

“It’s sensible to me,” says Byerly. “I’ll even let you borrow a few fine things that’ll let you blend in. If you cut us both in, of course.” He winks at Cassian. “I think our friend has earned something for his trouble.”

“Get bent,” Ingrid says. But at long last, she sheathes her knife, her silent and meek compatriot following her lead and doing likewise. Then she eyes Cassian. “You new in town? Vint, right? You need a crew to hook into?”

(Whatever her rough language and poor manners might imply, it seems she’s impressed by Cassian’s poise.)