There is no magic here, if you are really eating 4000 calories and not gaining weight then, uh, you need to eat more. Since you have an aversion to dairy, you are going to have to find some other food to eat. Or, you could just drink some milk and eat some real carbs for 8-12 weeks and then abstain. You are caught up in the dogma (where reality doesn't exist).
So, you are eating 4000? It ain't enough. This isn't rocket science and you're not a unique snowflake. Eat 5000, eat 6000. At some point you will start to gain weight. YNDTP
Milk + Squats for the win.
"The problem with quoting someone on the Internet is, you never know if it's legit" - Abraham Lincoln