Festive first showed up in the mid-seventeenth century as something specifically related to a feast, with the holiday-merry meaning not coming along until the end of the eighteenth century. It comes from the classical Latin festivus (for the rest of us!), with the same meaning, and festum, which is just festival. And that word comes from festus, the origin word for feast.
Dang. They’re not related after all. Fester showed up in the late fourteenth century from the Old French fester, a pus-leaking sore and no I’m not making that up. Festre actually comes from the classical Latin fistula, which means pipe or ulcer and is even an English word that means "a narrow passage formed by disease or injury".
Infest showed up in the late fifteenth century from the Middle French infester and classical Latin infestare, to attack or disturb, and infestus, aggresive or dangerous. The in- prefix isn’t a stranger around here—it means opposite of in this case. Aggressive certainly sounds like the opposite of a joyous, festive occasion, however that is probably not where that festus comes from. Infestus is actually from infensus, which is related to -fendere, fight or strike, the origin word for defend.
The final -fest word is manifest. These days we mostly know it as a cargo list, but it’s original and now rarely used meaning is evident or easily perceived. Cargo manifest didn’t show up until 1706, but the other manifest came about in the late fourteenth century. It comes from the Old French manifestand classical Latin manifestus, which has about the same meaning. It’s root words are manus, hand, and like infest, the festus that means struck.
TL;DR: Infest is related to defend, not festive. Fester and festival aren’t related to them or each other.