Towards Robust Speech Representation Learning For Thousands Of Languages