Elasticsearch Units
DeviceLock Discovery can effectively discover documents of interest in Elasticsearch - a distributed system that provides real-time indexing and search for a wide variety of data types. The Discovery Server requests a document search by the specified configurable parameters, and then applies the discovery rules and actions to documents received from Elasticsearch. Discovery rules are matched to data in the document fields selected in accordance with filter settings (see
Filter control dialog box for Elasticsearch). The rule triggers if it matches data in at least one of those fields.
Important: •DeviceLock supports document discovery in Elasticsearch version 6.8.12 or later. •Document discovery in Elasticsearch requires one DeviceLock Discovery license for each Elasticsearch index that will be searched for documents. •The DeviceLock Discovery agent is not installed on Elasticsearch nodes. Discovery is performed without the use of the agent. •Elasticsearch-related discovery actions are limited to logging events and sending alerts. The Discovery Server cannot change or delete documents in Elasticsearch. |
To interact with Elasticsearch, a discovery task must use a unit of the appropriate type: when creating such a unit, select Elasticsearсh nodes in the Unit type list. The following parameters are used to configure a unit of this type:
•Computers - A configurable list of computers running Elasticsearch nodes that are subject to discovery. Click the Edit button next to the Computers field, and then, in the dialog box that appears, view the current list, and add or remove computer names from this list as needed.
The names of computers running the desired Elasticsearch nodes are listed in the right pane of the dialog box. To add computer/s to the list, type their name/s or IP address/es in the left pane and click the
button. You can type the host name or fully qualified domain name (FQDN) of the computer. Press ENTER after typing each name. To remove computer/s, select their name/s in the right pane and click the
button.
When typing a computer name, you can specify the number of the network port used by Elasticsearch, in the format name:port. If the port is not specified, the discovery task will scan all ports until it detects Elasticsearch. To speed up port scanning, select the Smart port lookup check box. When this check box is selected, the discovery task will only scan ports that are typically used by Elasticsearch. As port search can be time consuming, it is advisable to specify the Elasticsearch port number explicitly.
•Set Credentials - Click this button to specify the name and password of a user account with sufficient rights to access the Elasticsearch nodes on the servers in this unit. A name and password must be specified if Elasticsearch requires authorized access. If no account name and password are specified, the Discovery Server accesses Elasticsearch anonymously.
Note: If using a database from another Discovery Server, you will need to re-enter the account name and password. Since these credentials are encrypted with a key securely stored on the server, they cannot be decrypted by another Discovery Server, so the name and password must be re-entered. |
•Include Filter(s) - Conditions for including indexes and documents in the discovery process. The search is conducted only by indexes and documents that match at least one of these filters. Use buttons beneath this field to add, edit, or delete include filters. When adding or editing a filter, the
Filter control dialog box for Elasticsearch is used.
•Exclude Filter(s) - Conditions for excluding indexes and documents from the discovery process. The search is not conducted by indexes and documents that match any of these filters. Use buttons beneath this field to add, edit, or delete exclude filters. When adding or editing a filter, the
Filter control dialog box for Elasticsearch is used.
•Query <number> documents - Select this check box to specify the maximum number of documents to be requested from Elasticsearch. During the discovery process, Elasticsearch will return no more than the specified number of documents that match the filters in effect. Clear this check box if you want Elasticsearch to return all documents that match the filters.
•Sorting - The sort order of the documents returned by Elasticsearch. Clear the Sort by check box if it does not matter in which order the documents arrive from Elasticsearch (default sorting). Select this check box to have documents arrive in ascending or descending order of values of a certain field in the document. Specify the name of that field in the Field box, and select the desired sort order (ascending or descending).
Note: The same field can be indexed in different ways for different purposes (so-called multi-field). For instance, a string field could be mapped as a text field for full-text search, and as a keyword field for sorting and aggregations. In this case, it is advisable to specify the field for sorting as fieldname.keyword. |
The fields that list the filters display the following conditions for each filter:
•Index - A list of index names. The filter matches documents from any of the listed indexes.
Index names allow the use of wildcards: an asterisk (*) stands for an arbitrary series characters, a question mark (?) stands for any single character. For instance, a dot followed by an asterisk (.*) denotes any index whose name begins with a dot.
The condition of All indicates that the filter matches documents from any index.
•Field : Value / Query - A list of field-value pairs or a search query. In this filter condition, “Field” stands for the name of the field in Elasticsearch documents and “Value” stands for the value to search for in the field specified. “Query” stands for a query string that complies with Elasticsearch query syntax.
If a list of field-value pairs is specified, the filter matches documents in which the specified fields have the specified values. If a query string is specified, the filter matches the documents returned by the respective search query.
In a field-value pair, <All values> indicates that the filter matches documents with any value in the field specified.
The <All> mark indicates that the filter matches any documents from the indexes specified.
Index names that begin with a dot normally denote system indexes (for example, .kibana). As such indexes hold configuration settings and other system data, it is advisable to exclude them from the discovery process. Therefore, the exclude filter has the following default conditions: Index = .*; Field : Value / Query = All, which excludes all documents in all indexes whose names begin with a dot.