Like the singer, some people get restless and uncomfortable when they're doing what's expected and the "status quo" like going to a work or school everyday, paying bills, being responsible, etc. It's technically "the right thing" by society's or the government's or maybe the "Jones'" standards, but for him, it just feels wrong. Horrible. Almost painful.
What unfortunately does feel right though is when he's "doing the wrong thing". Being unhealthy, overeating, drinking, doing drugs, rebelling, being a non-conformist, keeping odd hours, staying up all night & sleeping all day, ditching work or school or whatever should be done to do what he shouldn't be, like gorging on fried food, sweets, spirits, and substances.
Later he reiterates this concept with " everything that kills me makes me feel alive" and " everything that drowns me makes me feel alive".
All the things that feel the best are often the worst for you if you want to live a long life, etc.