Serverless audit logging to Splunk and Azure Monitor

Audience: Security and platform engineering
Status: Testing in progress — Azure Central US (Logstash bridge); native external delivery tracked as a product enhancement

---

Product enhancement (roadmap)

Native external delivery of Serverless audit logs (customer SIEM / observability — e.g. Splunk today, Azure Monitor later) is tracked as a Platform enhancement request.

No customer-specific configuration detail is kept on this page — see the implementation sections below for the shared Logstash pattern.

---

Requirement

Security requires all audit logging forwarded into your domain:

Native Serverless-only audit destinations do not meet this policy for production. External delivery to customer SIEMs is a future enhancement and is not available for GA.

Current phase: Azure Central US is delivered. Logstash bridge testing in Central US (over Private Link) is in progress as part of the share-out dated 9/29. Package test results and this proposal for customer review. Native push to Splunk / Azure Monitor stays on the product roadmap (ER #27661).

---

Recommended approach

Near-term path: run Logstash (or equivalent) in your network or Azure subscription.

Elastic Serverless (audit events)
        │
        │  Elasticsearch API over Private Link
        ▼
Logstash (your VPC / Azure)
        │
        ├─► Splunk HEC          (primary today)
        └─► Azure Monitor / Event Hub  (later)

Why Logstash:

• Runs in your domain, so audit data stays under your control
• Proven path from Elasticsearch to HTTP / HEC destinations
• Can send to Splunk now and Azure Monitor later without re-platforming Serverless

Longer term, Elastic can deliver audit events directly to customer-owned destinations. That work should not block GA if the Logstash bridge below is accepted.

---

Step-by-step implementation (example)

Example configuration for the Azure Central US pilot. Replace placeholders (<serverless-endpoint>, API keys, HEC URLs) with project-specific values. Run Logstash in Azure in the same VNet as the Private Link endpoint.

Phase 1 — Enable Serverless audit log delivery (source)

Doc: Audit trail log delivery

Official Serverless audit trail configuration. It covers:

1. Enabling delivery from your Serverless Search project (Admin or Editor on the source project).
2. Configuring ignore filters to suppress high-volume events (data searches, successful sign-ins, routine IAM checks) — important for cost control at production scale.
3. Audit events land in the destination data stream logs-elastic_cloud.serverless_audit.otel-default (dataset elastic_cloud.serverless_audit, namespace default). Older names (serverless.audit / elastic_cloud namespace) were renamed — update any data views, dashboards, or Logstash index patterns that still reference the previous stream.
4. ES|QL example queries for failed sign-ins, access-denied events, config changes, and sensitive index searches.

Do this first so Logstash has a stable source before wiring outputs.

Phase 2 — Logstash input: read from Serverless over Private Link

Doc: Elasticsearch input plugin (v5.x)

Use logstash-input-elasticsearch to pull from the audit data stream. Do not use match_all on every poll — that re-reads the full stream. Use tracking_field so each run only fetches documents newer than the last seen value (see Tracking a field’s value across runs).

Example:

input {
  elasticsearch {
    hosts => ["https://<serverless-endpoint>"]
    api_key => "<read-only-api-key>"
    index => "logs-elastic_cloud.serverless_audit.otel-default"
    query => '{
      "query": {
        "range": {
          "@timestamp": { "gt": ":last_value" }
        }
      },
      "sort": [ ["@timestamp", "asc"] ]
    }'
    tracking_field => "@timestamp"
    schedule => "* * * * *"   # poll every minute
    docinfo => true
  }
}

Notes:

• Use a read-only API key scoped to the audit data stream.
• Point hosts at the Private Link hostname for the Serverless project, not the public endpoint.
• Run Logstash inside Azure (same VNet as the Private Link endpoint).
• Persist run state (last_run_metadata_path if you need a durable cursor across restarts) so replays do not duplicate after a Logstash bounce.
• Confirm @timestamp (or another monotonically increasing field) is present on audit events before locking the tracking field.

Phase 3 — Logstash output to Splunk HEC (primary path)

No dedicated Splunk output in Elastic docs; use logstash-output-http:

output {
  http {
    url => "https://<splunk-hec-endpoint>:8088/services/collector/event"
    http_method => "post"
    headers => { "Authorization" => "Splunk <hec-token>" }
    format => "json"
    content_type => "application/json"
  }
}

Agree index, sourcetype, and token ownership with security before the pilot. Static HEC tokens keep auth simpler than Azure AD for the first path.

Phase 3 alt — Logstash output to Azure Monitor

Doc: Http output plugin

There is no native azure_monitor Logstash output. Post to Azure Monitor’s Data Collection Endpoint (DCE) with HTTP:

output {
  http {
    url => "https://<dce-endpoint>/dataCollectionRules/<dcr-immutable-id>/streams/Custom-<table>?api-version=2023-01-01"
    http_method => "post"
    content_type => "application/json"
    headers => {
      "Authorization" => "Bearer %{[@metadata][azure_token]}"
    }
    format => "json_batch"
    pool_max => 50
    retry_non_idempotent => true
  }
}

Auth gap: OAuth2 client-credentials token acquisition must happen in a filter or a separate script — the HTTP output plugin does not handle Azure AD auth or token refresh natively.

Alternative: Azure Event Hubs → Log Analytics. Elastic documents the Azure Event Hubs input plugin for the reverse direction (Event Hub → Elastic); use it as a reference for Event Hub connection patterns if you prefer push-to-Event-Hub then route to Log Analytics outside Logstash.

Doc index

Recommendation for the pilot: Splunk HEC first (simpler auth — static token), then Azure Monitor via Event Hub or DCE when security is ready for the second output.

---

Security acceptance criteria

• All required audit event types present (authentication, authorization, config, data access as applicable)
• Freshness SLA agreed (for example under 5–15 minutes)
• Retention in Splunk or Azure matches policy
• Pipeline and credentials owned by you (or jointly operated under your IAM)
• Private networking only — no public audit egress
• Ignore filters tuned so high-volume noise does not dominate cost

---

Options compared

Recommendation: commit to option A for GA; design the pipeline so B is a configuration change; track D as the long-term simplification.

---

Discussion points with security

1. Audit must land in Splunk (and later Azure monitoring), not only inside Elastic Serverless.
2. Native push to an external SIEM is a future enhancement — not available for GA.
3. For GA, Logstash in your Azure environment pulls Serverless audit data over Private Link and forwards to Splunk HEC.
4. When native external delivery ships, the bridge can be retired or reduced.
5. Confirm event catalog, lag SLA, and Splunk index / sourcetype so a pilot can run before cutover.

---

Open questions

1. Exact audit event catalog on Serverless Search versus hosted deployments.
2. Roadmap timing for native external log delivery.
3. Preferred Splunk path (HEC versus indexer) and Azure path (Log Analytics direct versus Event Hub).
4. Who operates Logstash and who owns API key rotation.

---

Next steps

1. Share this implementation example with security for sign-off.
2. Drive native external-delivery roadmap placement for ER #27661 (Splunk + Azure Monitor).
3. Stand up Phase 1–3 against the Central US Serverless project over Private Link (Splunk HEC first).
4. Document the runbook and lag alerts; add Azure Monitor as a second output when security is ready.