Index Lifecycle Management

What is ILM?

Index Lifecycle Management (ILM) moves indices through phases automatically:

Hot → Warm → Cold → Delete

Creating an ILM Policy

Basic Policy

curl -X PUT "http://localhost:9200/_ilm/policy/logs-policy" \
  -H 'Content-Type: application/json' -d'
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_primary_shard_size": "50gb",
            "max_age": "1d"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "warm": {
        "min_age": "2d",
        "actions": {
          "shrink": {
            "number_of_shards": 1
          },
          "forcemerge": {
            "max_num_segments": 1
          },
          "set_priority": {
            "priority": 50
          }
        }
      },
      "cold": {
        "min_age": "30d",
        "actions": {
          "set_priority": {
            "priority": 0
          },
          "freeze": {}
        }
      },
      "delete": {
        "min_age": "90d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}'

Retention-Only Policy

For simple retention without tiering:

curl -X PUT "http://localhost:9200/_ilm/policy/simple-retention" \
  -H 'Content-Type: application/json' -d'
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_primary_shard_size": "30gb",
            "max_age": "7d"
          }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}'

Rollover

Rollover creates a new index when the current one meets size or age conditions. It requires an index alias.

Set Up Rollover

Step 1: Create an index template with ILM

curl -X PUT "http://localhost:9200/_index_template/logs-template" \
  -H 'Content-Type: application/json' -d'
{
  "index_patterns": ["logs-*"],
  "template": {
    "settings": {
      "number_of_shards": 3,
      "number_of_replicas": 1,
      "index.lifecycle.name": "logs-policy",
      "index.lifecycle.rollover_alias": "logs-write"
    }
  }
}'

Step 2: Bootstrap the first index

curl -X PUT "http://localhost:9200/logs-000001" \
  -H 'Content-Type: application/json' -d'
{
  "aliases": {
    "logs-write": {
      "is_write_index": true
    }
  }
}'

Now all writes go to the logs-write alias, and ILM handles rollover automatically.

Check Rollover Status

curl -s "http://localhost:9200/logs-write/_ilm/explain?pretty"

ILM Phase Actions Reference

Hot Phase Actions

Warm Phase Actions

Cold Phase Actions

Delete Phase Actions

Applying ILM to Existing Indices

For indices not created with a template:

curl -X PUT "http://localhost:9200/my-existing-index/_settings" \
  -H 'Content-Type: application/json' -d'
{
  "index.lifecycle.name": "logs-policy"
}'

Monitoring ILM

Check Policy Status

# View all policies
curl -s "http://localhost:9200/_ilm/policy?pretty"

# Check specific index ILM status
curl -s "http://localhost:9200/logs-000001/_ilm/explain?pretty"

# View ILM errors
curl -s "http://localhost:9200/_ilm/status?pretty"

Common ILM Issues

Export and Import ILM Policies

When migrating between clusters, export your ILM policies:

Export

# Export all policies
curl -s "http://localhost:9200/_ilm/policy?pretty" > ilm-policies.json

# Export specific policy
curl -s "http://localhost:9200/_ilm/policy/logs-policy?pretty" > logs-policy.json

Import

# Import policy to new cluster
curl -X PUT "http://new-cluster:9200/_ilm/policy/logs-policy" \
  -H 'Content-Type: application/json' \
  -d @logs-policy.json

ILM with Data Tiers

In a multi-node cluster, assign nodes to tiers:

# elasticsearch.yml for hot node
node.roles: [data_hot, master]

# elasticsearch.yml for warm node
node.roles: [data_warm]

# elasticsearch.yml for cold node
node.roles: [data_cold]

ILM automatically moves indices between tiers based on your policy.

Lab: Create a Complete ILM Pipeline

1. 1 Create an ILM policy with hot, warm, and delete phases
2. 2 Create an index template that uses the policy
3. 3 Bootstrap the first index with a write alias
4. 4 Insert test documents
5. 5 Verify the ILM status with _ilm/explain

Next Steps

• Analyzers & Tokenizers — understand text analysis
• Cluster Monitoring — monitor ILM in production