There is a general perception that the young should start out in the private sector first to gain more experience. However, I think it really depends on what you want in the long term, which in your case seems to lean towards the private sector.

If your long term goal lies in something like policy work, starting out in public makes sense. You can still learn stuff in public side, just that the things you are learning are different from the private side. For example, the public sector work may be more support roles and it can take a few years before you get to do any decision making or solutioning, while in private you may be tasked to create solutions and produce results from the get go. This is also affected by the stricter hierarchical structures in the public side. What you want to see being created may take forever to be rolled out with all the checks and balances, whereas in the private side, you can perhaps see your results faster.