D&D is a melting pot of various fantasy tropes. D&D's parents include Tolkien (honestly, I don't care to argue about to what degree or how much Gygax did or didn't like LOTR), Conan, Fafhrd and the Grey Mouser, Elric, Vance's stories, weird sword and planet stories, Lovecraft and other horror as well as folklore and mythology.
When D&D was being assembled, there wasn't much thought about narrowing down influences to make sure a "pure" theme could be distilled. It was the first, so it's pretty much got a bit of everything within it. That said, having influences that range that far does not mean that it doesn't have recurring themes that are native to the game itself.
When I was young, reading through settings like Greyhawk, Mystara, and the Forgotten Realms, I got a very specific narrative. That narrative was not "the adventuring company are people from all walks of life that have arisen to take on a great evil." The narrative that was most consistent was "adventurers are a known thing in society; sometimes people want to hire on as mercenaries, bodyguards, or try their hand at tomb robbing in order to make their way in life, and sometimes those folks turn out to be heroes the save the world, sometimes they die young, and sometimes they retire rich and famous."
One of the reasons that the Forgotten Realms is often a contentious setting is that the best selling, longest running books of the setting, the Drizzt novels, are at odds with this basic principle. Drizzt and his companions are much more epic fantasy types that just happened to rise to defeat great evil, and then great evil keeps presenting itself to them. They didn't become adventurers to stave off boredom, to get away from a peasant's life, to shirk responsibility, or to forge their own destiny. They all had settled in Icewind Dale and were friends that banded together to fight the menace of the Crystal Shard.
That's all well and good, but it seemed like most of the rest of the adventurers in the Realms were people looking for fame, fortune, and a life free of the expectations of more constrained society. They gained patrons in the nobility or various clandestine power groups, might be manipulated into doing one thing or another, but overall, it wasn't about the One Evil that Arises Every Generation. It was about surviving and maybe dealing with local evils once in a while.
I am certainly not a gamer that thinks Dragonlance ruined D&D with it's vile storylines and metaplot, but I do think it was a departure, and it was an intentional departure from standard D&D tropes. Even then, the Heroes of the Lance were originally . . . mercenaries. Even the noble future knight. They were actually adventurers before the Great Evil returned to the world,
The main reason this strikes me is that no matter what D&D has done right over the years, when it comes to advice on how to run D&D campaigns, whomever is at the helm of D&D tends to be reluctant to flat out explain the fact that D&D mainly assumes that you are probably selfish, rebellious ruffians. D&D is not really a game about noble knights defending their home kingdom, nor is it a game about religious orders making the world safe for the adherents of their faith. Its not even a game about navigating a city as a member of the thieves guild.
It's become a joke, and people talk about it like it's a bad thing, but do you know why so many D&D adventures start in a bar? Because adventurers go to bars for work, and to meet other adventurers. People on a grand quest may be assembled by a meeting of the great powers of the world, or in a temple to the great god of light, but adventurers find job postings and hear rumors and rub shoulders with other adventurers, and there is nothing really wrong with that.
A lot of angst over how character classes and alignments work in a party might actually be avoided if we took a moment to consider that some things exist in the game not expressly for player use. I know, even with the idea that adventurers were kind of outlier ruffians from earlier source material, I felt like people should at least try to be "good." But that's not really true of adventurers. Adventurers don't need to be bloodthirsty killers, and in fact I hate the term "murder hobo," but they do tend very much to be neutral in their outlook. Sure, you have things you love and would sacrifice for, but you probably also want to be rich, famous, and at the very least to live your life on your own terms.
Even when somebody wants to play a lawful good paladin in a "traditional" D&D game, the obvious choices for gods really aren't the best choices for adventurers. Tyr, god of justice? Torm, god of duty? What are you doing wandering around the countryside poking into tombs? But maybe with a little guidance, a paladin of Lathander can justify being there to heal his friends, and potentially shape them into heroes and nudge them in the right direction if they do happen to trip over that Great Evil that sometimes shows up and needs to be put down.
In the earlier days, when D&D was the beginning of the RPG industry, it may have been hard not to see it as the tool to emulate fantasy across the board. Now that there are more games that are much more aware of what they are actually modelling, and not trying to paint on too broad a canvas, perhaps it's time to realize that D&D may not be the "everything high fantasy" RPG, and might be more rewarding if it's focus was expressly narrowed to "adventurers that may or may not become great heroes."
I'm not saying you cannot make D&D work to emulate the Lord of the Rings or your epic fantasy of choice, I'm just saying that you need to make sure you know this isn't really the default setting of the game, and that everybody needs to be onboard, so that you don't have half the party comprised of LOTR style heroes and the other half comprised of Fafhrd, Mouser, and Conan, and everyone ends up disappointed.