daniel@home:~$

Stract, an open-source search engine

In recent years, using Google has gotten worse, as search results seem to be overrun with SEO content, AI-generated results, and websites loaded with affiliate links and ads. In response, a programmer has launched a new open-source search engine, partly as a reaction to the overwhelmingly corporate and homogeneous search ecosystem of the web. The new search engine, called Stract, runs on a server in the basement of its developer’s office, is highly customizable, and, according to user feedback on the project’s Discord, is improving rapidly.

The project stemmed from founder Mikkel Denker’s master’s thesis at the Technical University of Denmark, which focused on helping people search their own files and documents, as he told me in an online chat. Next week he finishes his master’s degree and will be working on Stract full-time.

“Most of our searches go through the same handful of entities (Google, Bing, Yandex),” Denker told me. “Even other search engines like DuckDuckGo use Bing for their results. It felt really weird to me that there was no way to browse the web in an open way. So that’s what I’m trying to build.”

There are a handful of independent search engines out there, and some are open source. But Stract is particularly notable because it’s highly customizable, fast, uses its own web index, and is very easy to use, despite having been built mostly by one person in his free time over the past few months (Denker says a friend helped a lot early on, but is now busy with a PhD program). Developer Rohan Kumar, who has spent three years maintaining an incredibly exhaustive and up-to-date list of search engines with independent indices, wrote on his website that Stract is “my favorite generalist engine on this page.” Denker said that since appearing on Kumar’s site list and getting some attention on Hacker News, the site has been handling about a thousand searches an hour.

Stract’s most interesting feature is called “Optics,” which are essentially customizable instructions you can give the search engine to help search for specific types of content or to restrict searches to specific websites.

For example, there are currently Optics that allow users to search only websites that regularly appear at the top of Hacker News, search only forums and message boards, search only websites federated on the Fediverse, and search only independent blogs. Since the project is open source, any web developer can write their own Optic or take and modify any existing Optic (whose code is also open source). Stract also allows users to “like,” “dislike,” or “block” specific websites, as well as downrank websites containing third-party trackers or heavy advertising. The feature is similar to Brave’s Goggles, but Optics are more customizable: “The idea is to put as much control over search into the user’s hands as possible,” Denker says. List of currently available Stract Optics

“Part of Stract’s advantage comes from the fact that Google makes money from web pages that contain Google ads through its AdSense program, and is therefore incentivized for those results to rank higher in search results, whereas Stract can focus purely on which results are best for the user,” he said on the project’s Discord. Stract’s About page notes that “if we ever turn evil (maybe by changing our motto), please take our code and build a competitor. The fact that you have this ability ensures our values will always be aligned with our users.”

Stract currently isn’t monetized in any way, but its website says it will eventually feature contextual ads tied to specific search terms without tracking its users, similar to the system DuckDuckGo uses. Stract also plans to offer ad-free search to paid subscribers.

As we’ve discussed in several recent articles, using Google has gotten increasingly frustrating over the past few years. I asked Denker if he thought Stract could avoid being overrun by AI-generated content and SEO. He said it’s a hard problem to solve, but his goal is to allow users to customize their own rankings using Optics, “so everyone can decide for themselves what kind of content should be prioritized.”

Stract is the latest entrant in a suddenly revitalized search market that includes Kagi, a paid subscription search engine I’ve been tinkering with over the last month (and I’ll write about the experience in more detail sometime in the next few weeks). I like Kagi so far, but it operates as a search aggregator that combines results from Google, Yandex, Mojeek, Brave, and other major web presences like Wikipedia, Wikimedia Commons, TripAdvisor, and Flickr to generate its search results.

Stract, on the other hand, is building its own index, which is a monumental undertaking. “The internet is a huge place, and most people know it’s not exactly trivial to build a search index,” Denker explains. “That being said, it’s also not quite as hard as you might think.”

Obviously, a search index is crucial to a search engine’s usability, and Stract felt a bit limited when I messed around with it. I couldn’t find some of the websites I searched for, likely because they haven’t been indexed yet (though results for some of my searches were satisfactory). When I typed “New York Times,” for instance, the New York Times website didn’t come up. The good news is that Stract’s index is growing rapidly, and users on the project’s Discord have noted that their experience is regularly improving.

“I expect search results to improve quite a bit over the next year. Stract is currently in beta, but as the index grows and our ranking models improve, search results should follow suit,” Denker says. “The index currently sits around 250 million pages, and as it grows we hope to get better search results. In particular, I’m trying to figure out how to best handle these navigational queries.”

Translated from https://www.404media.co/this-guy-is-building-an-open-source-search-engine-in-real-time/


Automatically translated from the Spanish original.

Publicado en: Blog

Tags:

Want to subscribe to new posts?