I agree, if you don't want to use their loyalty card, have the purchase delivered, or pay by cheque (which many in Canada won't even accept these days). Maybe there is something to be said for paying for everything with cash, or by anonymous postal order or bank draft for mail purchases (which can be costly, unless you're over 60).
Everyone has to make their own judgement given the circumstances. I too use loyalty and reward cards, some of which are tied to credit cards, but I'm beginning to rethink the whole thing.