I wanted to ask a technical questions, maybe high level, on why sites may have bad search and what the bottlenecks might be in almost never updating such in years. Was there something in the original development of the stack that is affecting progressive updates around the feature, how should one approach “Search” then in this case? Or is it simply a management issue.
I would think there are plenty of “off the shelf” systems they could buy for this purpose, but they’re too cheap to do it. I mean all the Reddit post and comment data is in a database to begin with.
There are “off the shelf” systems, for a sufficiently broad interpretation of “off the shelf.” But they are not cheap (requiring probably a dedicated team just to properly configure and maintain, and probably also requiring significant rearchitecturing of your application’s data), and are usually still quite shitty even after all that.
Search is just very, very hard. Much harder than even experienced devs who have not worked in the area appreciate.
Source: I am a dev on a major search engine. No, not that one, but one you have definitely used many times.
I know it’s cool to shit on Reddit (and I dislike them too!), but this really is a technical issue. Stuff being in a database doesn’t mean that you can magically do good searching without anything. Off the shelf systems exist for off the shelf products, problem with those solutions is that once you differ significantly from the target type of project, it costs more and more to integrate. And since Reddit is pretty unique (if you also account for its scale), it doesn’t make financial sense to make a product that’s optimized for Reddit.