I am not understanding the variance question. Yes, the variance of x is 1. Ergo, the variance of the mean of x in a sample of size N is 1/N. There is no contradiction. The "variance of the mean" would be better called the variance of the average; we are calculating the variance of (x1+x2+...+xN)/N.
As for the mean being 0, I just assumed that to save myself some math; there's nothing you can look at in X'X to see that. In my business, it is not uncommon for us to calculate not X'X, but Z'Z, where z_i = x_i - mean(x_i). My business is statistical software and calculating z'z is a more numerically accurate way to obtain X'X. One could add back in the mean after making the z'z calculation to obtain X'X, but there's no reason to do that because most formulas you want to calculate are simpler and therefore easier to calculate accurately when you use Z'Z. Anyway, I skipped all that and just said "assume ..." so I could make my point, and I just assumed everyone would understand that we could get easily calculate a matrix with these properties and thus I had made my point without loss of generality. Sorry.