So when I explain that I don't have a phone I put it in context and give it some history.
I explain that I am a minimalist, I practice voluntary simplicity. I tell them how I got rid of it back in 2014. I tell them how the salesman at the phone store that I bought my home phone from said: "You'll be back in a week to get your smart phone back!"
So it becomes very believable. Easy to believe.