A good number of fantasy works, and in some cases science-fiction, have accumulated a huge corpus of background information that is not directly part of the plot or even relevant to it but still a source of endless fascination to dedicated fans. This isn’t new and has been a big thing in RPGs since the 90s. But it has gained an increased prominence in longer running videogame series and some of them, like The Elder Scrolls as perhaps the most famous example, deliberately focus a considerable amount of the development work on this aspect. And huge numbers of fans love it and enjoy going hunting for clues and trying to figure out the connections between seemingly unrelated bits and pieces of information.
I’ve long been wondering how this stuff really works and how one could deliberatey build it into the worldbuilding for books and RPGs as well. And I believe what’s really going on with all this lore information is that the creators are telling secondary stories within the gaps of the main stories. It’s not uncommon for writers to overdo it with the worldbuilding and tell the audience about things that are ultimately irrelevant and don’t connect to anything else in meaningful ways. But bodies of lore are different. The small pieces of information that are scattered around to be discovered do not stand by themselves for their own sake. They are puzzle pieces for the audience to collect and assemble into more or less complete stories. Often very small stories with very simplistic plots, but it’s the act of finding the pieces and interpreting them that makes these background stories interesting and compelling for the audience.
An extreme case are the Dark Souls games, in which the lore of past events is all the story the players get. The actions of the player character are really insignificant to the story and the hero has absolutely no agency to influence anything. In Dark Souls games the whole story has already happened. What the player does is playing with fun combat mechanics and discovering the backstory, which really is the main plot. The actions of the player character are only a minor footnote or an epilog to the story.
When writing books or creating s world for an RPG with the desire to give the audience lots of background lore to discover (as the A Song of Ice and Fire books do very well), I think it is probably best to focus on content that constitutes stories. People are unlikely to care much about manufacturing, agriculture, or the legal system of a fantastic world. People respond primarily to stories snd stories are also the most suitable content for letting people fill in the blanks with their imagination or deduct the missing pieces from details that are already known.
If you want to create a world with interesting background lore that draws the audience in, focus on stories that happened in the past.