Skip to content

Major Elasticsearch module refactoring

Justin Lewis Salmon requested to merge refactor-es into master

This MR greatly simplifies the Elasticsearch module, removing a whole bunch of unnecessary code, bad tests, and generally slimming down and improving the whole thing.

General points to note:

  • When a new indexing period rolls over (e.g. a new month) all types for that index are created at the same time. This removes the need to cache a map of which types have been created for which index; it is now sufficient to simply cache only the index names. Furthermore, this prevents concurrency issues as the first server in the c2mon cluster to receive an update in the new indexing period will "win" and create the index and all its types. The other nodes will check if the index is already created (via an ES call) and will discover that it already exists and simply cache it locally.
  • The TagDocument (formerly EsTag) class that is indexed as JSON is no longer a full-blown POJO, but instead simply extends HashMap. It is unnecessary to have a full POJO as this class is never used outside the module and exists only for a very short period of time as an intermediate transport object.
  • There are no longer POJOs for mappings - these live as JSON resource files. This is much more readable and simple (see #122)
  • All tests have been rewritten (without sleep) as previously many of them were confused and/or useless.

Merge request reports