I was listening to a speech online about orthodox achdus and I think the main point the speaker was making is that it’s very hard for all the orthodox groups to unify because each group disagrees and wants to protect those in a specific group. Both the modern groups and chareidi groups are concerned about the other's views being either too modern or too extreme. But the thing they all have in common is that they are all trying to do what they feel is best for the Jewish people. The different groups may disagree on what is best for the Jewish people, but they all care about doing what's best and that’s something to be respected of all the groups.
I like what the speaker said at the end “We can fight and oppose and yet still accord love and respect.” Meaning we can disagree but still respect each other's good intentions and each other.