There are two biggest factors before you purchase a upgrade to your GPU card.
1- PSU Unit installed.
Generally Pre-Built OEM PCs doesn't have very powerful PSUs. You need to get a GPU card that requires a Minimum PSU of at least 50-100 watts greater than the GPU card minimum power needs. Plus sometimes HP installs propriety PSU & Motherboards that can't be replaced by 3rd Party PSUs.
2- Sounds like you need a Low Profile GPU card to be able to fit in your HP PC case. There aren't as many choices for Low Profile GPU cards. Most are Regular size.
At PCPARTPICKER I inputted Low Profile GPU cards. They only list 9 models to choose from: https://pcpartpicker.com/products/video-card/#sort=-length&page=1
If you PC case is large enough for a regular but smaller Non-Low Profile GPU card then just click on Video Cards and it will show almost 6000 models to choose from.
EDIT: This Tech site lists the best 2022 Low Profile GPU cards: https://www.wepc.com/reviews/best-low-profile-graphics-card/
This tech site list 8 Low Profile GPUs for 2022: https://diamondlobby.com/review/best-low-profile-graphics-card/
NOTE: You noticed that all Low Profile GPUs have a maximum of 4GB of vRAM. You need to find out which GPUs are recommended for your VR set and how much vRAM is recommended for a GPU card to have.
