I always thought that a toerag was a rag used for cleaning the toe-caps of your shoes. I was wrong. A toerag is a substitute for a shoe or sock. Unshod tramps would wrap a rag around their toes to keep them from falling off in the cold.
Nobody is quite sure where the word rag comes from. It could be related to rug. It could be a backformation from ragged. Anyway, it seems to have been the ragged nature of the syncopated rhythm that gave us ragtime, first recorded in the title of Mississippi Rag by W.H. Krell in 1897 (Krell followed it up the next year with Picaninny Rag or Shake Your Dusters).
Newspapers have been referred to as rags since 1734, when a fellow called Roger North wrote:
Would any one expect in Print, upon tolerable Paper, and a clear Character, such Malice and Knavery as lies here, scarce fit for Midnight Grubstreet Rags.
Rags To Riches was a play by E.E. Rose and came out in 1897. Rag-and-bone men wanted bones to make glue.
But nobody knows why you lose your rag when somebody gets your goat. I assume the two are attached.
The Inky Fool's career as a couturier