This article describes the troubleshooting framework built into the documentation, giving you a consistent way of thinking about the system—not just a list of commands.
Setting the stage: Before diving into specifics, here's a quick snapshot.
"Kubernetes-based platform with 60+ microservices that collects telemetry from sensors, processes it through a streaming pipeline, and serves insights via web UI and APIs."
The healthy state in one sentence:
- All pods running ✓
- Data flows sensor-to-UI in ~10 seconds ✓
- Users can log in and see current metrics ✓
Three Questions Framework
The diagnostic starting point: Use these three questions to quickly isolate the problem domain.
| Question | What It Reveals | Quick Check |
|---|---|---|
| Can users log in? | Authentication health | Try the UI login |
| Is data appearing? | Data pipeline health | Check Kafka consumer lag |
| Are pods running? | Infrastructure health | kubectl get pods -n pca |
The logic:
- If login fails → Focus on Presentation Plane (Zitadel, PostgreSQL)
- If data is missing → Focus on Data Plane (Kafka, Ignite, Druid)
- If pods are down → Focus on Infrastructure (resources, PVCs, configs)
Three Planes Architecture
Organizing 60+ services: Services are organized into three logical planes to make the system comprehensible.
Presentation Plane (User-Facing)
This is what you and your users interact with directly. When a user reports they "can't access the system" or "the UI is broken," start here. These services handle authentication, serve the web interface, and expose APIs.
| Service | What It Does | Symptom When Broken |
|---|---|---|
| Gather | REST API gateway | API errors, timeouts |
| nginx | Serves UI | Can't access web interface |
| Zitadel | Authentication | Login failures |
| Grafana | Dashboards | Metrics dashboards unavailable |
Data Plane (Telemetry Processing)
This is where the real work happens. Sensor data flows through these services to be ingested, processed, and stored. When users report "missing data," "stale metrics," or "dashboards not updating," investigate this plane.
| Service | What It Does | Symptom When Broken |
|---|---|---|
| Sensor Collector | Ingests sensor data | Sensors not connecting |
| Fedex ⚠️ | WebSocket broker | No real-time updates |
| Ignite | Stream processing | Data delays |
| Kafka | Message queue | Everything downstream fails |
| Druid | Time-series storage | Historical queries fail |
Control Plane (Configuration)
These services manage how the platform is configured and how sensors are orchestrated. Issues here are often more subtle—sensors failing to register, configuration changes not taking effect, or tenant-specific problems. Check this plane when the system is "running" but not behaving as expected.
| Service | What It Does | Symptom When Broken |
|---|---|---|
| Sensor-Orchestrator | Manages sensors | Sensors not registering |
| Config-Service | Configuration mgmt | Config not applying |
| Tenant-Manager | Multi-tenancy | Tenant issues |
Data Flow Visualization
Tracing the happy path: The following diagram shows how data moves through the system, so you can trace backwards when something breaks.
Sensor → Sensor Collector → Fedex ⚠️ → Kafka → Ignite → Druid → Gather → UI
Key insight: When data is missing, trace backwards:
- Is Druid ingesting? (Check supervisor status)
- Is Kafka flowing? (Check consumer lag)
- Is Sensor Collector receiving? (Check pod logs)
- Is the sensor connected? (Check sensor status)
Expected latency: ~10 seconds end-to-end
Common Scenarios Mapping
Quick lookup: The following table maps common symptoms to likely causes and first checks.
| Scenario | Likely Plane | First Check |
|---|---|---|
| "Users can't log in" | Presentation | Zitadel pods, PostgreSQL |
| "No data in dashboards" | Data | Kafka lag, Druid supervisors |
| "Sensors not connecting" | Control | Certificates, Sensor Collector logs |
| "Everything is slow" | Data | Druid cache, resource usage |
| "Pods keep restarting" | Infrastructure | Pod logs, resource limits |
Reference: Full Documentation Set
The following guides provide detailed procedures for each area covered in this overview. Use them as your go-to references when you need step-by-step instructions or deeper technical detail.
| Document | Purpose |
|---|---|
| Post-Installation Health Verification | Systematic health checks |
| Architecture Overview | Component relationships |
| On-Prem Troubleshooting Runbook | Diagnostic procedures |
| CLI Reference Guide | Command reference |
| Port Matrix & Firewall Rules | Network configuration |
© 2026 Cisco and/or its affiliates. All rights reserved.
For more information about trademarks, please visit: Cisco trademarks
For more information about legal terms, please visit: Cisco legal terms