Resolve "maximum shards open; [type=validation_exception]" in Chef Automate Dashboard

This blog explains how to fix Chef dashboard compliance failure due to exceeding shards limit, which may give you an error similar to below.

Issue: Chef automate compliance node connection failing with following error message.

invalid output Report processing error: rpc error: code = Internal desc = InsertInspecReport: elastic: Error 400 (Bad Request): Validation Failed: 1: this action would add [10] total shards, but this cluster currently has [998]/[1000] maximum shards open; [type=validation_exception]

Reason: Shrads hitting the maximum allowed shrads.

Fix: Increase shards limit.

Login to Chef Server

Add  <<filename>>.toml file with following content

[opensearch]

  [opensearch.v1]

    [opensearch.v1.sys]

      [opensearch.v1.sys.cluster]

         max_shards_per_node = <NUMBER_OF_SHARD>

Make sure to use correct alignments and give shards number without ""

Once added the code, save and close file. Then run following command to update opensearch config. 

sudo chef-automate config patch <<filename>>.toml

Now you would be able to add new nodes to Chef automate compliance without getting the shard error