I went in with two Daeodons eight megatheriums a yutyrannus and one megalania to kill the alpha version. I climbed to the ceiling of the cave with the megalania and blasted it with rockets from the roof while the megatherium tanked it and the yuty gave the Megas a buff and the daeodon healed
When i first fought lystrix, i was over paranoid about how strong she'd be, so i bred and tamed 5 spinos, and 5 rexes. It was an absolute pain to get them to an obelisk, but when it came to the battle, it only lasted around 5-7 seconds. Might have been my settings, might have been my choice of dinos.
You actually don't need any rexes in this boss fight. for the gamma I think about 10 high level megatheriums a yutyrannus and a daedon will work. For higer difficulties just get more megas and maybe a bother daedon. Before entering, set the daedon(s) and the yuty to ingnore group whistles, then make tame ordering groups with like 3 to 5 megas each. When you enter the boss fight set the daedon passice healing on (make sure it has a lot of food on it and upgrade food because healing uses its food) and tell the megas to attack. They will all get stuck behind each other so only 4 or so will be dealing damage. Then choose a tame ordering group and make them move to the side of the broodmother,then make them attack. They will start attacking from the side and won't get stuck up front.Keep doing that until your megas surround the boss. The tame you will want to mount is the yutyrannus and then spam the crouch button for the courage buff. Remember minions will ignore its fear debuff. Good luck
From my experiments it seems that a good team to take down the broodmother is: three carnos, two daeodons, four megatheriums, all levelled equally in damage and health, some food for the daeodons, as well as some speed for the carnos, and a single yuty levelled the same with extra stamina, the carnos are mainly distractions and keep the broodmother and her spiderling's attention off of the megatheriums, the daeodons are healers, to keep the others alive, the megatheriums as pure damage, and the yuty to cheer them on with roars for extra resistance, damage, ect
Hands down the easiest boss even on alpha difficulty all you need is 19 imprinted rexs with good ascendant saddles and a yuty and you will destroy the brood... btw I play on official for anyone wondering