ɴᴇᴏɢʀᴏᴋ

Neogrok is a frontend for zoekt, a fast and scalable code search engine. Neogrok exposes zoekt's search APIs in the form of a modern, snappy UI.

This page describes the functionality of this neogrok site. To see the source code and more detailed documentation on configuring and deploying neogrok, check out the project on GitHub.

Code search

The main/home page of the site is the code search page. Here you can enter a search query in the zoekt syntax, which is a complicated enough topic that it gets its own page.

One of the selling points of neogrok and zoekt is speed. So, by default, search is live: every character you type into the search inputs will execute a new query. If you prefer to not have live search, there is a button in the search form that toggles between live and manual search. When manual search is enabled, the search query can instead be executed by pressing the enter/return key, and any search inputs that have pending changes are highlighted in yellow.

Search results

Search results consist of N matches for the query across M files, where N ≥ M. Matches can occur not only in file contents, but in file paths as well. As a search engine, zoekt attempts to rank results by a relevancy score: it will return the highest scoring file first. The scoring mechanisms of zoekt are opaque and unspecified.

Unlike some other search engines, zoekt does not paginate search results. To prevent both neogrok and your browser from being overwhelmed, the number of results returned from zoekt to neogrok are limited by the files and matches inputs. Zoekt collects the top N matches across the top M files, stopping whenever it meets either of these two limits, and sends the truncated results to neogrok.

The search results header thus describes how many files and matches for the query zoekt found, and how many it sent to neogrok. It will show which of the files and matches inputs are the current limiting factor on the truncated results by highlighting the relevant count in the header. You can get more results by increasing the relevant limit in its input.

Other search inputs

Finally, there are a few more inputs in the search form that control how search results are displayed:

  • In addition to the live/manual toggle button, there is another toggle button that affects the ordering of matches within a file. Zoekt not only ranks files in search results, but also ranks matches within each file. By default, neogrok ignores this ranking and displays matches ordered by line number, but you may use this button to switch to ordering matches by score.
  • You can set the number of lines of context around matches in file contents; this has the same semantics as grep's -C flag.

Repositories list

The repositories list page tabulates all of the repositories indexed in the backing zoekt instance, including a variety of data about them.

Note that the search input on this page has the same semantics as the search input on the main search page: you are writing a full zoekt query, but instead of getting normal search results, you get repositories that contain any results matching the query. So, r:linux filters the table to repositories with "linux" in their name, while linux filters the table to repositories with linux in their contents.

To improve page performance on deployments with large numbers of repositories, there is a repos input that limits the number of displayed repositories in the same way that the files and matches inputs on the search page do.

Repository stats

The tabulated data includes links to the repository and its indexed branches, the times the repository was last indexed and that it was last committed to, and data about the index shards and their contents. The table can be sorted by clicking on column headers: the first click will sort in ascending order, the second in descending, and the third will restore the status quo.

Shards are what zoekt calls the files emitted from its indexer, and they're all that's used by the zoekt-webserver backing neogrok to handle neogrok's API requests; they contain the above-described repository metadata, indexes used to quickly search repository content, and the repository content itself (file names and contents). Indexing a repository typically results in a single shard, but zoekt limits shard files to be about 100MiB in size, so big repositories get more than one shard.

When you search repository contents (i.e. make a non-repo: query), are in fact searching repository shards, and so for a repository with more than one shard, you will see that the counts of shards and associated data in the table go down when you enter a query that matches content in only some of its shards.

Preferences

The preferences page allows you to persistently configure some aspects of the neogrok UI. Preferences configured here are stored in browser cookies and are updated instantly upon interacting with any of the inputs.