Search API in Drupal 8 is about to be stable!

TL; DR: After more than three years of development (and several blog posts), the Drupal 8 port of the Search API module is finally getting a stable release! The current Beta 5 release will be the last Beta, and there should be a release candidate and the final, stable release within the next two weeks. So, test it now, if you haven't already, to help eliminate any remaining (major) bugs!

The current status in detail

Last Saturday, April 1, I committed the fix for issue #2682369: Fix problems with overridden config entities, the last (and incredibly complicated – dedicated blog post upcoming) release blocker^[1] for the stable release. The day after, I created the final Beta 5 release to give everyone still a few more days for testing.

Unless testing of this new Beta release discovers a serious enough problem, the first release candidate (RC) will be released on the coming weekend, April 8 or 9. Finally, if that doesn't uncover any major problems, the first stable release, Search API 8.x-1.0, will follow the weekend after that, on April 15 or 16. And then: rejoicing, parties and a little shield icon, to make everyone feel better.

To be clear, though, the Search API has been pretty usable for a long time (at least a year, I guess – possibly more, depending on your requirements), and we didn't have any major bugs or problems (unless you used config overrides) for some time now. The release blockers were mostly due to desired framework changes which would have been more difficult to do after a stable release, especially with other modules with stable releases already building on it. There were also several usability problems classified as release blockers, which was probably a mistake in hindsight. Having a clearer UI is nice, and helps avoid floods of bug reports due to confused users – but it shouldn't delay a stable release, and possibly dissuade site builders from using a basically working module. It also shouldn't prevent us from getting Security Team coverage, of course.

One plea regarding testing this last Beta: In case you are using config overrides for search indexes or servers in your site's setup or development workflow, it would be very helpful if you could verify that this last release fixes all associated problems you might have had – indexing/deleting using the wrong server (settings), overridden values being written back to the database, etc. This was a recent and pretty complex fix, so it would be great to feel more confident about that.

Also perhaps note-worthy: I've used the last few days of Dev Days Seville in March to finally update the module's documentation to Drupal 8. While it's still not quite finished, especially the documentation for developers, it should already provide a pretty good guide to the module's functionality and should help to answer most questions for first-time (or even advanced) users.

What's new

Compared to Drupal 7, the Search API in Drupal 8 is a massive step forwards – we didn't just spend those three years twiddling our thumbs, after all! Here's a run-down of the most important improvements we made:

Indexes can contain more than one type of item/entity

This might have been the single largest restriction in the Drupal 7 Search API: indexes could only be created for a single entity type (or custom item type). If you wanted a site search across entity types, you had to either use the Multi-Index Searches modules or, since August 2015, use the "Multiple types" item type, essentially a workaround-type backport of the D8 functionality.

In Drupal 8, however, creating an index for multiple item types (now called "datasources") is baked right into the framework, all parts of the Search API will support this natively. Furthermore, changing or re-configuring (think: changing the content types of nodes indexed) datasources is also supported right by the framework, and all such changes will be handled without any need to manually correct tracking data, or clear the index, etc.

Overhaul of the "Fields" tab and functionality

Also a continuing source of frustration, even if mostly just a usability annoyance, the "Fields" tab has seen a major revamp. Instead of seeing all available properties right on one page (killing the web server in some cases, for particularly large sites) and enabling the ones you want, we've transitioned to a UI inspired by Views: we only list the fields that are currently indexed and use a second page/dialog for discovery of available properties and adding the ones desired. This is also a lot more convenient than the old "Add related fields" box for indexing fields of related entities or data structures.

Furthermore, it's now possible to add the same property more than once, eliminating the need to use the "Aggregated fields" work-around of creating duplicates of a field. You can also freely change the human-readable name and the machine name of a field, and some processor-generated fields (like "Rendered HTML output") even allow per-field configuration (so you could have two "Rendered HTML output" fields, with each using different view modes, for different purposes). Also, like Views, changes made to the fields (but only to them) won't be effective immediately, but need to be explicitly saved afterwards. This avoids triggering a reindex numerous times when changing fields.

Improved language/translation support

Language support has also been vastly improved. While multilingual Drupal 7 sites needed the Search API Entity Translations contrib modules (at least when using the "entity translation" system, not "content translation") for getting proper translation support, which then didn't work natively with some other contrib modules, support for Drupal 8's new translation system is included right in the Search API itself. Automatically, all entities will be indexed once for every one of their translations, and each entity-based search item is identified not only by entity ID, but also by its translation language. Furthermore, additional internal improvements also make it easier for other modules (e.g., Search API Solr Multilingual) to add additional language-specific functionality.

Processors are more powerful than ever

This item is something that's probably only apparent as a developer, but processors have become a lot more powerful, being able to influence a lot of different areas of the search system. For non-developers, the most notable change is probably that the previous distinction between "processors" and "data alterations" has been removed – but not only have processors gained all the functionality previously held by data alterations, several other functions have been added, too.

For site builders, though, the main benefit here (in addition to more useful processors potentially becoming available) is a cleaner, less confusing user interface.

Objects everywhere!

Also not really anything that non-developers will notice, but internally the Search API has fully embraced Drupal 8's move to object-oriented programming (OOP). While its Drupal 7 version was already among the most OOP-using modules available (I'm pretty sure), this gained even more momentum in this new version. Not only did we, of course, make use of all the new OOP-based APIs in Core, like entities, plugins, services, etc. – we also converted a lot of the internal data structures (which, like Core, often used "magic" array structures in Drupal 7) to properly classed objects (e.g., indexed items, search results, fields) and moved most helper functions to methods on service classes.

Great test coverage

The last item, which no-one ever gets really excited about, but is actually also a huge step forward, is that we have improved test coverage throughout the module tremendously – aided a lot, of course, by the corresponding improvements in Drupal Core and on drupal.org. This (hopefully) means a lot less bugs that go undiscovered, a lot less regressions when introducing new features, simpler, more confident refactoring of old code and consequently less technical debt.

Other projects

Search API has always relied on additional contrib modules to provide powerful search solutions – with just the Search API module, you wouldn't get far. So, what is the current status of related projects in Drupal 8?

• First off, the Database Search module (which provides the "Database" backend for servers) has been moved back into the Search API project itself, so this is already completely stable and very thoroughly tested. So, if you have a smaller site and don't require any advanced functionality (as offered by Solr or Elasticsearch), by just downloading the Search API project you can already set up a pretty good database-based search.
• The Facets module (replacement for the Drupal 7 Facet API module) is still in Alpha as there are still some API changes and breaks planned. However, its basic functionality is already there and quite extensive, and it's already used on quite a few production sites. So, if you want to add facetting to your search, you should at least give it a try.
• The Search API Solr module is in a similar position: It's already used on a lot of production sites, but there's still some rough edges (especially regarding support of optional features) and the module is currently considered to be in Beta state. However, the test coverage is quite good, so there shouldn't be any larger bugs lurking under the surface and, if you want to use Solr for your site, you shouldn't hesitate to give it a try. (Just beware that, like all modules with no stable releases, it's currently not covered by the Drupal Security Team, so security issues are a larger risk than usual.)
• If you want to use Elasticsearch, things look a bit worse. There are (confusingly) several modules in Drupal 7, but only Elasticsearch Connector has a Drupal 8 version yet. And since even the Drupal 7 version is not yet beyond Alpha (and the Drupal 8 version has nothing more than a dev snapshot release), it may take some time until it becomes stable. I haven't evaluated the code or functionality, though, so it might actually already work quite well – if you have information, please leave a comment below!
• The Pages module has been ported to Drupal 8 (and is currently maintained) by swentel. It, too, is already used on some production sites and has most of its functionality working fine (though some adaptions might be needed for the lasted Search API changes). It's currently in Alpha state.
• Preliminary ports also exist for both the Autocomplete and the Location module, with at least some functionality already working. The former of those will probably be my next focus, since it was one of the most popular extension modules in Drupal 7 (apart from the mostly-ported ones mentioned above). And the latter is set to be the focus of an upcoming Google Summer of Code project by dbjpanda, so we'll probably have a full-fledged Drupal 8 version of this by the end of August.
• The Saved Searches module has not been ported yet. I plan to start porting it once the Autocomplete port has been finished and I get the time.

Credits

Once again, I have to state that this release wouldn't have been possible, and the development would have taken even longer, if it weren't for a lot of helping hands who did their part in moving this module port along. Chief among them are Acquia, who sponsored two months of my time a year ago to focus on this port, and Joris Vercammen (borisson_) who was an invaluable help just by discussing ideas and reviewing patches (and being probably the second-best informed regarding the Drupal 8 Search API module). Thanks to both of them, but also to everyone else who contributed – much too many to list them all!

^[1] To be accurate, there are still two open but postponed ones, but those are mere clean-up issues that will be committed right before RC creation.

Image credit: bayasaa