It was arguably the English Aristocracy.
They were pretty horrendous in their dealings with the Irish during the potato blight, they also cleared the highlands of Scotland of people in a most brutal fashion. Both were the result of the aristocracy (Large numbers of “absentee landlords”) taking maximum profit from the land and treating the people worse than livestock. They cleared people and burned the thatch on their home in both countries, sentencing people to death by starvation, or forcing emigration to the colonies. Wales was systematically stripped of its resources, and the population working there were subject to pretty grim conditions in industry like slate and coal mining.
I was about to say the same. In Ireland, the ascendency class was pretty much exclusively english. Even the Irish born ascendency class were to all intents and purposes, English - schooled and often raised there before going on to be absentee landlords.
Many Liverpudlians are descended from impoverished Irish people who were forcibly shipped from their country by landlords who wanted rid of them. The Irish Famine itself is a misnomer. There was potato blight but Ireland was still exporting food to England as its people starved to death. Grain was imported to Ireland throughout the famine but chiefly used to feed livestock.
And how did the absentee landlords respond to starving, poor tenants? They evicted thousands and destroyed their homes so they could graze animals in their place.
Sorry, this was unintentionally a rant and not belonging in this thread. But, yeah, the English aristocracy were the carriers of the British Empire.