Mushishi/WMG

l

The Reason noone cares about Ginko's strange clothes, or pale looks, or glass eye, is that he's a Mushishi
I mean really, just like you wouldn't expect a wizard to look like a normal person (AT THE LEAST they need a Badass Longcoat...)), Mushishi commune with the strange and invisible and always introduces himself as a Mushishi. If he looked normal, people would feel they weren't getting their money's worth.

The series takes place not in the past, but in a post-apocalyptic world.
This would explain the more modern appearance of Ginko's Western clothing. (Were this taking pace in the 19th century, he'd be wearing clothing of a different fashion.) Obviously something happened to cause Japan to abandon technology and regress back to a Feudal age. (This "something" could have very well been a disaster of some sort.) The western-style clothing that Ginko wears could have been scavenged from a disaster site or just found, lying abandoned somewhere. (Ginko wears such clothing as a way to symbolize the strange condition that separates him from the rest of his fellow citizens.)
 * Or maybe he just time-traveled with the help of a Mushi.
 * At least one other person (the little boy who helps the girl with Mushi in her eyes) wears modern clothes, but this was taken out of the anime.
 * Jossed. The author, Yuki Urushibara, explained that his clothes are remnants from when the series was going to be set in the modern day. I guess she couldn't be bothered to change them.
 * However, the jossing doesn't make it any less of a cool interpretation. I mean, it explains a lot about the world of Mushishi, not just Ginko's appearance. For instance, perhaps the Mushi aren't natural organisms but genetically engineered lifeforms Gone Horribly Right, a'la Nausicaa of the Valley of the Wind.

Midichlorians are actually a form of blood-based Mushi that grant their users super powers.
Oh wait. This belongs in the Star Wars WMG section....

Ginko only visits one city in the whole series.
Everyone looks the same because it's the same people in every episode. Ginko lost touch with reality after losing his memories and his eye. He wanders in circles and thinks he has come to different villages. In reality, he has entered a large city from multiple directions. The city is cursed, and constantly plagued by Mushi. No one recognizes Ginko because they all have undiagnosed mild amnesia from some unidentified Mushi. Those who don't have amnesia play along with Ginko, such as his doctor friend. No telling what a crazy man would do when his delusions are broken. Especially one who controls invisible magic creatures. Thus, this city has the following features: A mountain range, a dry waterfall, a large beach, a rocky peninsula that appears to be an island when approached from the sea, a bamboo forest, a tree forest, large rice farming grounds, and a river.
 * Perhaps he's entraped in a Lotus Eater Machine by Mushis.
 * Ginko attracts Mushi; he could be the reason the one city has a Mushi problem to begin with.

Ginko is actually Sanji in another life.
He still retains his appearance and smoking habit, but he's older and wiser and, because he no longer roams the sea freely, he takes to land instead.

Ginko is high the whole time
He seems to smoke fairly persistently, but what he's actually smoking is a fairly mild hallucinogen. The mushi aren't real, and the reason he manages to heal people is because he stumbles upon the actual medical method for curing their problems while believing he's dealing with mushi. Everyone just goes along with it, because he has a reputation as an amazing doctor even though it is all fluke.
 * I'd have to second this, just because it amuses me.
 * Even if it isn't All Just a Dream, he's so… Mellow. I'm thinking this could easily be canon.
 * The mushi might be real, and he could be high out of neccessity: who knows what herbs you need to put together to make mushi-binding smoke?