Where do you live? I'm in the UK and I find shopping at Indian/ Caribbean grocery stores and local green grocers actually surprisingly much cheaper for fresh produce than in the big chain supermarkets. I can get huge bunches of dill, methi, spinach, mustard greens, etc. for offers like 3 bunches for £1. Potatoes and onions are very cheap, and calorie for calorie I also find african yam, jamaican sweet potato, plantain and taro very good value. Some of the local green grocers near me do big bags or bowls of produce all measured out so that one is £1. This always turns out to be great value as it tends to be seasonal stuff that needs to be sold fast every day.
I also found that it is cheaper for me to buy 2 half dozen boxes of egg from my favourite Indian grocer than it is to buy a dozen eggs from the supermarket. And this is even though both are free range! In fact, the Indian grocer sells higher quality eggs...
If you like to use coconut oil and palm oil, these are also cheaper from Indian, African and Caribbean stores. They may not be organic oils, but you have to pick and choose your battles if you are on a budget.
For me cheap cuts of meat are as follows:
*Shoulder of lamb - depends on the season really but lately my butcher has had some gorgeous stuff for less than £5 a kilo
Fish wise, sprats are very cheap year round where I am. So good for you too. Sardines are very cheap when I can get them, which is a lot of the year tbh.
I don't really eat chicken so not sure how that stacks up - to get non battery farmed chicken I'd probably have to pay too much, so I find red meat more ethically pleasing and also delicious.