Personally, I feel that it doesn't really matter if one does not come from an 'elite' primary school. so long one works hard, one (rich or not) can make it into an 'elite' secondary school, and from there if one works sufficiently hard, one can make it to top JCs, and subsequently get into a good university. I have many friends in an 'elite' secondary school who come from a 'non-elite' primary school. Moreover, they do not come from a rich family and require financial assistance, yet they excel just as well as the others from rich families. My point is, money is not that important as what it seems, to excel academically. If there is a need to, why not apply for a scholarship or a bursary to get into a university!
And also, being in an 'elite' primary school may result in the child being complacent.
From what I've seen, those who excel are those that work hard, not whether they are rich or not.