At AuntBertha.com, the core of our product was search. Before a user can take any action, they have to find a program they want to act on. I’ve spent 3 years continuously analyzing and refining search relevancy, through UI and algorithm improvements. Over my tenure our search relevancy has improved month over month, and our zero results has consistently dropped.
I had to build a deep understanding of which programs were most relevant and why, as well as where we were failing to deliver relevant programs. I started by investigating and determining the cause of every search with zero results in our system. After making improvements there that addressed low hanging fruit, we saw our zero results rate fall from 7% to 1.5%, on average.
At that point I moved on to analyzing low relevancy searches. I identified 5 qualities that make a highly relevant program, and we added those into our search algorithms as a boost. Now we are promoting programs that are very nearby, have verified their listing, have openings, respond quickly to inquiries, and are more likely to serve you (“serve everyone” if we don’t know much about you, match your personal qualities if you’ve told us about yourself).
The next major obstacle I identified is that people searched using terminology that didn’t appear in program descriptions. For example, “I am hungry” wouldn’t be listed in the description of any of our programs. I designed a database of relationships between searched terms, and their relationships to our services taxonomy. At first we tried a direct mapping, but we found that the results were missing a lot of information. For example, people searching “help with pills” might need drug addiction help, medical education, or prescription financial assistance. We implemented a match scoring system. Now, when someone searches “I am hungry”, our system can say “Would you like a meal, a food pantry, or help applying for government benefits?”
I’m continuing to fine tune search, the next major issue we’ll be tackling is our taxonomy of tags. 70% of our searches are based on tags, so a poorly worded, organized, or defined tag has a huge impact on our search relevancy. We’ve hired an analyst to work with me and identify the tags most in need of an update.