It would take a high wire fence to keep the deer out. Or you can get a big barky dog. We fenced our 8 acres with the 2"x4" non climb horse wire. Not even the chickens can get through it. Our dogs stay IN and everybody else's stay OUT. Coyotes stay out too. We have two Great Pyrenees to keep the varmits away. They keep the deer away too, although a couple of does figured out that the horse pasture was safe and they stashed their fawns in it. They walked mere feet away from the back yard fence, unafraid of the big white dog barking a frothing frenzy on the other side of the fence.
You do have a lot of work ahead of you. Is there any kind of fence at all? What kind of fence will you put up? Can I recommend fencing the outer parameter in sheep and goat fence, getting some goats to underbrush it and a LGD to keep them safe? You have a lot of FREE feed on your 20 acres. Get meat goats, sell the young ones for meat. They will also eat the trees that you cut in clearing for buildings. Why let it go to waste? As you transition to pasture for your cows, you can sell out the goats.
SC here, although I have lived here and there many places through the past 39 years.
I can teach you what NOT to do.
