Did I make the wrong choice to major in physics?

You remind me so much of myself because i also wanted to study physics and had a deep passion for it but just like you my brain was not really adept with mathematics, i was more theoretical and philosophical than mathematical, but thankfully i had decided to pursue a different career and i strongly suggest you change majors. There's no problem in having a passion for something like physics, but it's not all theory, you have a lot of complicated math courses that go with it and even the theoretical physics part of it is never devoid of mathematics, so if mathematics is not your strong suit, then im sorry to say but you're in the wrong place and i think by asking this question you already feel that way too.

We have a lot of parallels in our lives! My senior year of high school is also when I fell in love with physics and astronomy. That made me consider studying it in college. But what I ended up doing was doing my liberals initially, and dipping into different areas of study to see what kind subjects I clicked with.
My introductory physics class was okay, I got a B in it and I discovered some of the extent of the mathematics that high level physics requires. I ended up not taking the next level course. I realize that I also had a knack for theoretical physics rather than experimental, but the stuff that I was interested in what extremely high level, like relativity and other big theoretical subjects. I wasn't interested in any of the small intricate workings of the science, which was required to learn to actually pursue a degree in physics.
There's a quote I heard that goes "Minor in what you love, major in what'll get you a job", which is why I went into business instead. Although I'm not minoring in physics, I still enjoy reading about physics news and keeping up with space exploration, It's like learning about it by yourself for your own enjoyment and I am satisfied with that.
I hope I gave you some helpful insight!

You share a problem with many students internationally and a lot of dreams and motivation are conjured for the destination and not the journey. If the dream is worth it to you, you will suck it up and take it. However, if it's too much and you fear you'll hate your future job, I suggest maybe becoming an author where you write about your dreams. You can find more idealistic and radical majors such as Neuroscience (bio+psych) or you can even create your own major. Many schools offer that and if not, suggest it to your university. Research some more on that - it's very fun. :)

Wow for me physics is almost a religion. My hope is to be in a field where I can take human biometric systems and use them in robotic applications. So far my math skills are pretty good for my age, but I am getting some special tutoring for a summer calculus class that I will be taking to give me a head start. If you are really passionate about physics then you should shake off that I can't or it's too hard shit and study the math. Good luck and I think you made the right choice, now just don't fuck it up by backing out.

You did the right choice! Listen to your heart! It is the most important, believe me!