That is because you are brainwashed by the "passion" ******** America was sending out in the 90s. The "Dawn of the New Age Thinking" or "The Secret" and all that nonsense. Look around research papers about passion and success and the results are roughly the same. Being passionate about the job has very little correlation with your success at the job. Only a few miserable percent of people found success with their passions. There are a few reasons.

1. Passion is fluid. You may love your job this week, but hate it next week.
2. People who follow their passion tend to choose jobs with high failure rates. For example, musicians or fashion designers, etc...
3. No correlation between passion and competence. Just because you like to wear brand clothes does not mean you can design and make them.

Findings actually noted that people at the top of their game tend to find success first, then came their passion for their job. So are you able to go through the grind, and learn the skillsets that edges you ahead of your peers? If not, anywhere you go, it will be the same, you lose passion and then want to leave.