With the Aztecs or Nazis, one can understand that they had some sort of ideas about how the world worked - a hungry sun god, vast and wicked Jewish conspiracies - that led them to do the terrible things they did. With Sparta or the conquistadors, there is a sort of comprehensible, if brutal, self-interest present in their decisions.
The Imbangala lack much of that. They are so extreme that on my first reading of them, I assumed that I had to be reading the lies made up by their enemies to insult them. But as far as I can tell, the scholarly literature generally agrees, and there are signs visible even to my layman's perspective - e.g. descendant groups doing similar things - that yes, the Imbangala were pretty much the worst people, ever.
Dr. T.J. Desch-Obi, a specialist in "the historical ethnography of pre-colonial Africa and the African Diaspora with a focus upon martial arts, physical culture, religion, sport, historical linguistics, and military history", wrote in 2009 in Fighting For Honor: The History of African Martial Art Traditions in the Atlantic World:
[T]he Imbangala success was based on their martial culture, which included disciplined military training and their adherence to the yijila, strict and sometimes brutal codes of behavior including the ritual practice of infanticide and cannibalism, which had revolutionary military repercussions.
So what I am about to say is, in fact, not just what their enemies had to say about them. It is also, insofar as the scholarly consensus, what they were actually like.
The Imbangala were an army without a state, militarized bands with their own unique culture which exercised military power due to extreme levels of training, discipline, and organization relative to all their neighbors. Economically, they existed solely as parasites, never producing food of their own, and only settling into roles as actual rulers when they abandoned the Imbangala way of life, such as the Kasanje Kingdom (formed c. 1620), eventually merging into the surrounding cultures. While they were still Imbangala, though, they would come into a place, make a camp, cut down palm groves there to make wine, and then move on once they had cut down all the trees, a process which took a few months. They worked as mercenaries, and lived as raiders. Andrew Battell noted "For they will not sow, nor plant, nor bring up any cattle, more than they take by wars."
Imbangala did not reproduce normally: any child born within the walls of the fortified camp (kilombo) would be killed. Women could give birth outside the kilombo, but the child would be treated as any other, and not become a proper member of Imbangala society until they were initiated. They would wear a collar throughout and after initiation, until they had killed their first person.
Of course, the entire point of keeping children from being born inside the kilombo was that Imbangala society was not based on kinship, but entirely on initiation. They could - and did - regenerate their forces by capturing, initiating, and training as many new recruits as were needed. They would take children old enough to have teeth, anoint them in an oil made from human infants (maji a samba), and train them vigorously in their way of war. Some teeth of the upper row (I have seen it as middle two, middle four, and an upper canine) would be removed, and only when they presented the head of a slain foe would they actually be considered a proper member of Imbangala society.
Cannibalism was also an extremely common part of Imbangala society. Virtually any time they are noted as killing enemies in battle, it is followed up by "and then they ate their corpses." The victims of the infanticides noted earlier are eaten, of course. Disgraced soldiers who died in battle are eaten. Infants are also crushed with mortar and pestle into the maji a samba, which was believed to provide supernatural protection.
Understanding Imbangala as a Culture
Many of the things the Imbangala did make some sense, assuming your goal is to create the endless death blob they became (a very strange goal, one has to admit):
Cannibalism was a taboo among most of the cultures they engaged with. It made them scarier, evoking a sense that they were supernatural horrors to be feared. It also played a role in the initiation, and by extension, forcibly separated the children from their home culture.
The refusal to allow children born in the kilombo to be allowed to live prevented a reversion to a more normal culture. By requiring children to be captured from other peoples, you force your own people to keep up the killing and plundering forever. (Or, at least as long as they keep to that.)
Because you do not sow, plant, raise cattle, etc, you can devote all of your labor-hours to military training. Since you spend all your labor-hours training your military skills, you get very, very good at it, allowing you to avoid sowing, planting, raising cattle, etc by just stealing from other people who do that, since they can't defend themselves.
Using ground-up babies as oil to grant yourself invincibility is just wacky, though. At best we can justify it as more supernatural terror, playing on the ideas people have about how something that is costly in some way must also reap great rewards.
A Brief Historical Overview
The origin story of the Imbangala, according to Cavassi (writing started 1668, finally published 1687) is that they were a group of Jaga people, descended from Tembandumba, a Jaga queen who was the daughter of Ndonji and his concubine, Musasa. Tembandumba was noted to have many concubines, killing them when she grew tired of one and then moving onto the next. She had a child, which she killed when he was three years old, grinding him in a mortar and then anointing herself in his blood, claiming this would give her invincibility; other mothers did much the same. Tembandumba herself would be killed by her concubine Kulumba, who had some popularity among her kingdom's people and had noticed a pattern with previous concubines. (Also, Tembandumba was ugly, while he was handsome. So, possibly he was looksmatching.)
Modern scholarship disagrees. The more modern belief is that the Imbangala instead were descended from the followers of the Lunda kinguri (a title), who migrated into the region of what is now eastern Angola, where they met the male initiation warrior society of the Ovimbundo called the kilombo. Since the kinguri's band were migrating, they were already divorced from traditional ancestry, and the kilombo provided them with a way to expand and develop their manpower, bringing strangers into this band. By this point, the kinguri's band was already viciously violent, noted to be (from Joseph C. Miller's 1976 Kings and Kinsmen: Early Mbundu States in Angola):
As the Bihe told the story, Kinguri’s band reached the river Luhando after many years of fighting their way through hostile peoples and strange lands in the east. They practiced customs ('kesila laws') involving extensive cannibalism and killed many humans as sacrifices to the spirits represented by their chiefs. These spirits required so many victims, in fact, that the band could not settle long in a single spot without driving away or killing most of the neighbouring peoples. This repeatedly forced them to move on in search of new populations to devastate in order to satisfy their leader’s cruel demands.
By borrowing the initiation rituals of the kilombo, and merging culture with them, the kinguri transformed his band of followers into the Imbangala, and proceeded to wreak havoc across the region of the Congo, depopulating everywhere they went, creating an orgy of slaughter and violence. They were, at times, employed by the Portuguese by mercenaries against the Ndongo, such as Bento Banha Cardoso in 1615, but they were difficult to control - a group of them, who broke off and headed south to what is now Baixa de Cassanje, are the origins of the modern Imbangala ethnic group, who do not do all this insane shit. They played a vital role in the development of Ndongo and Matambe, particularly given the life of Nzinga Ana de Sousa Mbande (also Romanized as "Njinga," among others), who was first pushed out of Ndongo by the Portuguese, then allied with an Imbangala band led by a man known as "Njinga Mona" (Njinga's son), before conquering Matambe and Ndongo both.
The Imbangala didn't really "lose" in military terms - they conquered Mbundu states and became kings, eventually adopting local cultures entirely and fading into the broader society. Their lifestyle was just psychotically unsustainable, and those who refused to adapt to the changing circumstances and keep up the old way of life lost military conflicts against relatively "moderate" Imbangala who played various roles of the local states.
Further Reading
If you are interested in reading in more detail about the Imbangala, here is a primary, near-contemporary source writing on them (by the time he is writing, most of the actual Imbangala bands have either settled down or been absorbed/annihilated), available legally and free online. He probably exaggerates at times, and his opinion is as predictably polemic as one might imagine based on who he is writing about, but it is a quite intimate and detailed outline of what they were actually like. The translator's notes also provide some information on how scholarship has shifted from perspectives common at Cavazzi's time, e.g. that the Giaga/Jaga are a different people-group from the Imbangala.
Other books are more difficult to get your hands on, but you could check out Joseph C. Miller's Kings and Kinsmen: Early Mbundu States in Angola (1976), and T.J. Desch-Obi's Fighting For Honor: The History of African Martial Art Traditions in the Atlantic World (2009) for some further details.