Well, it all goes back to Greek. The Greek word for "Christ" is Χριστος, so as early as 1,000 years ago, it was abbreviated as Xp. You may recognize this symbol, the labarum, from Catholic or Orthodox churches, which came from this Xp abbreviation:
So use "Xmas" as much as you want! Think of it as a more "historical" (and time-saving) way to say "Christmas"!
Merry Xmas Eve, all!