Declining consumer confidence among American consumers in general, as well as affluent consumers in particular, could signal a slowdown in affluent spending which will have an outsized impact on the retail economy. Weak holiday retail results may be the first sign of retail trouble brewing.