The Lost Art of the Web Directory
In the modern web world, most of us are used to just typing what we want to look for on a search engine of our choosing and getting results. The quality of the results can vary; sometimes, it can be good, while other times, you'd have to dig deep into the trash results to find something good. Before the time of search engines, however, users found websites through a listing of sites called a web directory.
What Is A Web Directory, Exactly?
The concept of a web directory is simple: it is a listing of websites divided into certain topics. (For example, sites about arts may have its own section, and it may be divided into its own subsection, i.e. visual arts, musical arts, prose, etc.) Think of it as the Internet equivalent of the Yellow Pages. Directories could come in different types, be they free or paid, curated or community-maintained, and so on
Perhaps the most famous example would be the Yahoo directory. In 1994, Jerry Yang and David Filo would establish what was then called "Jerry and David's Guide to the World Wide Web". Then a year later, they would change the name to Yahoo, and the rest was history. Though with the rapid growth of the Internet, Yahoo would refocus on their own search engine as early as 2002, and would eventually shutter the directory at the end of 2014.
The other "big", now shuttered directory out was DMOZ. Unlike Yahoo, which required sites to pay to get in (and therefore cause sites with more money to be emphasized, now that I think about it), DMOZ was a community-maintained directory, which meant volunteers were in charge of maintaining and adding new entries. While the original DMOZ does not exist anymore, in spirit, Curlie lives on.
So, why don't we see web directories anymore?
It can be argued that the commonly-cited reason why web directories aren't as popular anymore is the search engine. On the user side, it's a lot more convinient to type something and get what you are looking for rather than figure out where the topic you're looking for is in the directory's hierarchy. On the website owner's side, you don't need to submit your website to even get listed, and specific pages can be show instead of having to link to an entire website.
Do web directories make sense in today's Internet?
On one hand, the near-abolition of "general" web directories may seem like it would have solved some problems with visibility. However, in today's search-engine powered Internet, this isn't always the case. Search engine optimization algorithms means that websites nowadays have to race for "first". Unfortunately, this can lead to low-quality results being first in a search, leading to lesser-known, but well-written websites to be buried underneath.
It can be argued that niche web directories could make a good counterpoint to wider-spanning search engines. Take the now-shuttered Yahooligans, for example. As a kid-friendly directory, it placed focus on educational and otherwise kid-friendly content. The Anime Web Turnpike (or Anipike) is another good example of such a directory; as its name implies, it was laser-focued on anime and manga websites. I feel like niche directories like these could be useful. For example, we could have directories dedicated to topics like fandoms, technology, literature, and so on. In this case, though, human moderation will be necessary to maintain the integrity of the directory. This could potentially make updating these directories slow, though it does keep high-quality sites in focus on these directories.
My thoughts on web directories
I admit, despite the convenience of search engines, there was something fun about going to a web directory and looking for sites on a certain topic. You could discover some cool stuff, and the chances of being deraied by an irrelevant search result was lower. As I mentioned earlier, despite search engines seemingly making "general" web directories obsolete, niche web directories could still have a place in the modern web by focusing on certain topics and emphasizing high-quality results. In the end, I can see web directories still having a purpose, albeit in a more niche way compared to search engines.