I think you'll find that the Conservative Group is, and always has been, the largest in the House of Lords, although obviously not as large as in the days of hereditary peerages. Bear in mind also that the Tories have won recent elections with about 42% of the votes in a turnout of less than 70%, so in effect have had the backing of less than 30% of the electorate - hardly "overwhelming"!