I used to work for the government but now in a finance-related field. (VC/PE) took me a while to get there, through a combination of luck and lots of perseverance. Here's what I did.

1) I did my CFA and took a distance learning MBA.
2) I look for jobs within the government sector that gave me relevant 'business' experience or anything that requires me to deal with the private sector directly.
3) I did not complete my distance learning MBA as the degree was not recognized and hardly matter to my promotion and future job prospect. It did however, gave a good grounding on the key business knowledge that I lack as a non-business admin graduate. More importantly, it helped me fine tune what I was looking for.
4) I managed to switch to a relevant sector, did quite well and was sponsored to do another MBA from a better university in US. (hint: at Boston)
5) I eventually work for a large bank based out in HK.

my only advice to you is to clarify what you are looking for. Buy-side or sell-side? do you want to be an analyst? investment banker? private banking? private equity? debt? quantitative analyst? risk management? etc If you start with a vague idea and simply being drawn by the allure of the finance/banking sector, chances are, you will be wondering in the dark for a while and makes it more difficult to get there.

All the best!